JOBS WORKING PAPER Issue No. 72 Cost-Effectiveness of Jobs Projects in Conflict and Forced Displacement Contexts Virginia Barberis, Laura Brouwer, Jan von der Goltz, Timothy Hobden, Mira Saidi, Kirsten Schuettler, and Karin Seyfert © 2022 International Bank for Reconstruction and Development / The World Bank. 1818 H Street NW, Washington, DC 20433, USA. Telephone: 202-473-1000; Internet: www.worldbank.org. Some rights reserved This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http://creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution —  Please cite the work as follows: Barberis, Virginia, Laura Brouwer, Jan von der Goltz, Timothy Hobden, Mira Saidi, Kirsten Schuettler, and Karin Seyfert. 2022. “Cost-Effectiveness of Jobs Projects in Conflict and Forced Displacement Contexts.” World Bank, Washington, DC. License: Creative Commons Attribution CC BY 3.0 IGO. Translations — If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations  —  If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third‑party content — The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to re-use a component of the work, it is your responsibility to determine whether permission is needed for that re-use and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. All queries on rights and licenses should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e‑mail: pubrights@worldbank.org. ACKNOWLEDGMENTS This work is part of the program ‘Building the Evidence on Protracted Forced Displacement: A Multi-Stakeholder Partnership.’ The program is funded by UK aid from the United Kingdom’s Foreign, Commonwealth and Development Office (FCDO). It is managed by the World Bank Group (WBG) and was established in partnership with the United Nations High Commissioner for Refugees (UNHCR). The scope of the program is to expand the global knowledge on forced displacement by funding quality research and disseminating results for the use of practitioners and policy makers. This work does not necessarily reflect the views of FCDO, the WBG, or UNHCR. This report was produced by Virginia Barberis, Laura Brouwer, Timothy Hobden, and Karin Seyfert of Oxford Policy Management and Jan von der Goltz (Task Team Leader), Mira Saidi, and Kirsten Schuettler (Task Team Leader) of the World Bank. It was supervised by Ian Walker (Manager, Jobs Group). The authors would like to thank all the individuals who have contributed to the undertaking of the research and to producing this report. These include Ozlem Akkurt, David Vanzetti, and Jonathan Mitchell for their valuable inputs; Tove Sahr for support in data compilation; Jim Coleman, Michael Morlok, and Atul Sanganeria for their support during the implementation of this research; and Sophie Grumelard and William Wiseman for helpful peer review comments. The authors especially thank UNHCR and the UK Department for International Development for their support and cooperation throughout the design phase and data collection. Special thanks are due to all the program implementers and donors’ staff who generously gave their time and shared opinions in interviews conducted for this research. iii TABLE OF CONTENTS ACKNOWLEDGMENTS iii ABSTRACT v ABBREVIATIONS vi EXECUTIVE SUMMARY 1 1. BACKGROUND AND PURPOSE 5 2. DATA AND METHODS 7 3. JOBS SUPPORT PROGRAMS ANALYZED IN THIS STUDY 10 4. COST-EFFICIENCY AND COST‑EFFECTIVENESS OF JOB PROJECTS IN CONFLICT AND FORCED DISPLACEMENT 16 4.1 COST-EFFICIENCY: WHAT ARE THE COSTS PER INDIVIDUAL BENEFICIARY OR FIRM? 16 4.2 COST-EFFECTIVENESS: WHAT ARE THE COSTS PER JOB OUTCOME? 24 5. POLICY IMPLICATIONS 31 REFERENCES 33 APPENDIX 1 35 iv ABSTRACT The need for jobs support in economies affected by forced displacement and conflict is high, with forced displacement at its highest level since the Second World War and poverty expected to be increasingly concentrated in economies affected by fragility, conflict, and violence (FCV). Developing impactful and cost-effective jobs support requires good data on program costs and benefits, but such information remains notoriously scarce in FCV and displacement situations. This study presents insights from a new dataset of cost and results in the jobs support project portfolios of Foreign, Commonwealth and Development Office (FCDO), United Nations High Commissioner for Refugees (UNHCR), and the World Bank in six low- and middle-income economies affected by conflict and displacement. It analyzes results on the cost-efficiency of jobs support to inform design and budget planning, as well as results on cost effectiveness, with a view to informing choice between different modalities while taking into account additionality and sustainability of outcomes achieved. v ABBREVIATIONS AECF African Enterprise Challenge Fund BPC Business Plan Competition BSS Business Support Services FCDO Foreign, Commonwealth and Development Office FCV Fragility, Conflict, and Violence IDP Internally Displaced Person ILO International Labour Organization KRI Kurdistan Region of Iraq MSMEs Micro, Small, and Medium Enterprises PPP Purchasing Power Parity SMEs Small and Medium Enterprises UNHCR United Nations High Commissioner for Refugees WBG World Bank Group vi EXECUTIVE SUMMARY INTRODUCTION 1. Data on the cost of interventions is pivotal to scaling jobs support in conflict and forced displacement settings, but it remains scarce. The need for jobs support in economies affected by forced displacement and conflict is high, with forced displacement at its highest level since the Second World War and poverty expected to be increasingly concentrated in economies affected by fragility, conflict, and violence (FCV). Developing impactful and cost-effective jobs support requires good data on program costs and benefits, but such information remains notoriously scarce in FCV and displacement situations. 2. This study presents insights from a new dataset of cost and results in the jobs support project portfolios of Foreign, Commonwealth and Development Office (FCDO), United Nations High Commissioner for Refugees (UNHCR), and the World Bank in six economies affected by conflict and displacement. Data was collected from all jobs support projects working with individuals or businesses, funded by the three agencies over 2009–2019 in Iraq, Jordan, Kenya, Lebanon, Mali, and South Sudan. Qualitative data was collected from 109 interventions (freestanding support modalities within projects), and cost data was available from 63 interventions. While data availability does pose challenges to analysis (discussed in detail in the main report), the information provided here for a range of support modalities and country contexts does expand upon prior studies of individual countries or types of support. 3. Jobs support modalities in forced displacement and conflict settings reflect the nature of the operating environment and needs of beneficiaries. In forced displacement settings where access to formal work and business activities is difficult for the displaced, jobs support tends to focus on training to build human capital or on capital support to simple informal business activities. In many instances, project design reflects the specific needs of displaced workers. For instance, training or job matching support includes language or legal support, and psychosocial support is a modality in projects based on training as well as those focused on capital support. In conflict-affected settings, support modalities tend to be simple; for instance, capital-focused interventions tend to work with individual entrepreneurs rather than businesses and financing modalities are straightforward. 4. This study presents results on the cost-efficiency of jobs support to inform design and budget planning, as well as results on cost effectiveness, with a view to informing choice between different modalities. Cost-efficiency analysis considers cost per output—in this study, cost per individual beneficiary or per firm supported. Such information can inform ex ante budget planning and design choices by illustrating the potential scale projects can aspire to and making the incremental cost of additional services visible. Cost‑effectiveness analysis, by way of contrast, considers the cost of achieving (self-reported) outcomes—in this study, the cost per job created and the cost per additional unit of income reported. This data can inform the choice between jobs support modalities that may seem equally apt to address a given jobs challenge. However, it must be put in the context of important considerations such as the additionality and sustainability of outcomes achieved. 1 COST-EFFICIENCY 5. There is a large range of cost per beneficiary, determined by the value of direct transfers, complexity of support provided, context, and ancillary objectives. The cost of support provided to individuals varies from US$20 in a project supporting agriculture productivity to more than US$3,200 in a training project; similarly, spending per beneficiary firm varies from US$3,300 to US$835,000. Among design features, the value of direct transfers to beneficiaries is an important cost driver, especially for projects that work with firms. The complexity of support provided also contributes substantially to cost: while the median capital support project spends US$135 per beneficiary, projects that also provide training spend of US$973 at the median. Ancillary objectives beyond jobs support (for instance, the provision of infrastructure) and implementation challenges in the environments included in this review also matter for cost. 6. In the countries studied, capital support, value chain support, or job matching to individuals had modest median cost (US$135–180), with higher spending in labor-intensive public works (US$390) and training programs (US$680). The median cost per beneficiary is US$135, with most of the cost arising from direct transfers to beneficiaries; none of the projects reviewed spent more than US$834. The moderate cost is explained by the nature of the support provided: in-kind agricultural inputs or modestly sized cash grants. Matching and brokerage interventions had a similar median cost (US$180) but provide no direct transfers. Value chain interventions also have a low median cost (US$188) but spend up to US$2,500 depending on the complexity of services. Public works programs that are mainly intended as part of a social safety net spend US$392 at the median, with wages accounting for a wide range of cost, from 20 to 80 percent. Training-only interventions have an elevated median cost per beneficiary of US$683, with a maximum cost of US$3,234. Technical skills training and delivery involving the private sector demand the most resources. 7. Programs targeting businesses spend per beneficiary firm 75 times as much as projects supporting individuals pay per beneficiary, at the median, with cost largely driven by the size of direct transfers. Programs supporting businesses commit about US$35,000 per firm at the median, 75 times the median amount of US$468 spent per beneficiary in the same countries by programs targeting individuals. The business-oriented programs provide capital, and spending per business depends strongly on maximum grant size (90 percent in logs) which in turn depends on the types of businesses supported, with the lowest costs in interventions that work with microbusinesses. The two complex value chain projects for firms in our sample spend per beneficiary about four times as much as programs that are focused on improving the performance of individual businesses. 8. Forced displacement and conflict settings favor simple capital support programs but drive complexity in training and matching, further widening the cost gap. Forced displacement and conflict contexts further widen the gap in cost between approaches that emphasize access to capital and those that rely on training. Access to capital interventions in these contexts tend to be simple in design and provide small amounts of financial support, resulting in lower overall costs. In contrast, expenditure per beneficiary in training or matching programs must absorb additional or customized training modules. Simplicity of design directly lowers cost, but qualitatively it is also associated with fewer logistical demands and appears to encounter fewer difficulties in implementation in challenging operating environments, with further implications for cost. COST-EFFECTIVENESS 9. In interpreting cost-effectiveness data, a key question is whether job creation and income increases reported by projects are additional and sustainable, as well as how productive jobs are. Focusing on the outcomes of ‘jobs created’ and ‘income increases’ facilitates comparisons of cost-effectiveness among projects that report a range of outcome indicators. However, a meaningful comparison must consider that support projects differ greatly in job quality and in the additionality and sustainability of outcomes. Thus, ‘job creation’ can refer to jobs of any quality, from low-revenue part-time activities to highly productive full-time work. Further, while project result reporting rarely allows for a clear assessment of whether the intervention caused additional job creation or income, impact evaluation shows that some support modalities are much more likely than others to have such an impact. Finally, the durability of impacts achieved by jobs support has 2 bearing upon cost-effectiveness, with clearly temporary job opportunities at one edge of the spectrum and full careers or permanent changes to business success at the other. 10. Public works have by far the lowest cost per job created, in line with the temporary nature of work in these programs, while job matching can place beneficiaries in jobs at an intermediate level of cost. In public works interventions, the number of workers employed is both the key output and outcome indicator, so that cost-efficiency and cost-effectiveness coincide. Their low cost per job (US$390) is in line with the ‘low-sustainability, low-wage’ jobs they provide as short-term livelihood support. Job matching reports a cost of about US$3,300 per job placement. Given that these interventions often facilitate access to existing or temporary jobs, rather than creating new opportunities, the cost number is best thought of as ‘the cost of providing access to jobs for a certain beneficiary population’ facing specific challenges on the labor market. 11. Start-up-oriented support to access to capital and training-based support report a similar median cost, but capital-based support is more likely to provide additional jobs. Start-up-oriented ‘access to finance +’ interventions in our sample have a median cost per job of about US$4,100, while training interventions report about US$4,700 at the median. While the cost per reported job is thus comparable, it is important to consider that the available impact evaluation evidence suggests that jobs reported in capital-based support are much more likely to be ‘additional’ than those reported in ‘skills-only’ training programs. Cost per additional job created due to the program is therefore likely to be substantially lower in capital-based approaches. Capital support programs can hope to break even if jobs are sustained for some five years, or sooner if jobs are somewhat more productive than typical job activities. 12. In line with a more indirect approach to job creation, cost per job in capital support to businesses is a multiple of cost in programs that work with individuals, and it is particularly elevated when grants are larger. At the median, programs working with businesses report a cost per job of nearly US$14,000, about four times the median cost in individual-level support. The discrepancy reflects the fact that business-oriented programs look at job creation as a less direct outcome that may be farther downstream of project activities geared toward innovation or productivity. A particularly clear pattern is that projects that offer larger grants do not necessarily create proportionally more jobs. They tend to work with larger firms and in more modern sectors; their cost-effectiveness will hinge upon whether such an ambition translates into higher productivity or continued job creation in the future (as well as whether such job creation is additional, due to the grant). Thus, business-oriented projects can typically hope to break even within five years if they succeed in generating productive employment but would require a longer period to recoup investment with less attractive jobs. If programs that provide loans rather than grants continue to operate beyond the reporting period, recycling of funds could lower cost per job by an estimated 20–30 percent. 13. Agriculture-oriented capital support projects can expect to break even during project duration, while value chain interventions may recoup cost within two to three years. Reported cost-effectiveness is high in simple capital support projects that provide agriculture inputs, given the singular focus of the intervention on providing smallholder farmers access to inputs. With spending of US$0.20–0.40 per dollar of additional income, projects can break even within implementation—although it is worth noting that such an accounting does not consider the cost of labor and other inputs potentially provided by beneficiaries. Value chain interventions with a more systemic ambition and additional objectives spend significantly more, but with a median cost of about US$2 per dollar of income, they can still hope to break even within a few years. RECOMMENDATIONS 14. The cost-efficiency and cost-effectiveness analysis conducted yields recommendations for jobs support in situations affected by forced displacement and conflict. This report presents a broader set of cost and results data on jobs support in conflict and displacement settings than has been offered before. However, collecting such data from project reporting remains surprisingly difficult, and this report discusses in detail limitations for analysis. Careful analysis of the data results in the following recommendations: 3 • Remove restrictions on labor market access for the displaced to raise cost-effectiveness of jobs interventions. Restrictions curtail work options for the displaced, thus limiting jobs outcomes that can be achieved. They also diminish the types of jobs support that can be implemented, and require additional services to work around constraints, hence raising cost. • In challenging FCV environments, consider simple jobs support designs to achieve cost-efficiency. Insecurity, macroeconomic instability, disrupted markets, and low capacity raise implementation costs in FCV. Keeping objectives for jobs support simple and using context-appropriate tried and tested designs may help keep down cost. • Assess expected cost-effectiveness ex ante by considering the likely productivity, additionality, and sustainability of jobs and income increases. Cost-effectiveness is critical to the impactful use of scarce resources. A simple but clear-eyed ex ante assessment should focus on the likely productivity, additionality, and sustainability of jobs expected to be created. • Consider the cost implications of combining jobs support modalities, and open the black box of how different components contribute to impacts and costs. When the obstacles to better jobs appear complex, it is natural to want to provide several types of support. However, such packages come at a substantial cost, and projects should carefully weigh it against marginal benefits. • In capital support to business activities, consider the merits and cost implications of working with firms of different sizes and capacities. Programs have far higher cost per job when they work with larger firms, and even those directed toward small businesses spend more than those with individual beneficiaries. Productivity and future hiring may warrant such investments, but given the size of the cost differential, assumptions should be carefully scrutinized, particularly in FCV settings. • Closely scrutinize the case for jobs support through training. Training projects spend more per beneficiary and per job than capital support interventions, with weaker evidence of additional job creation. Training can be an effective fit for some situations, where lack of skills is a key constraint, but prospective training programs should closely scrutinize whether there is a realistic chance of a cost‑effective intervention. • In monitoring and evaluation, keep clear track of cost per beneficiary and cost per outcome. It is surprisingly difficult to find useful information on project spending per beneficiary and more so, cost per outcome. Given regular financial reporting, such information should be made routinely available. . 4 1. BACKGROUND AND PURPOSE 1. The need for jobs support is high in conflict and forced displacement settings. By 2030, up to two‑thirds of the extreme poor will live in areas affected by fragility, conflict, and violence (FCV) (World Bank 2020). In recent years, the prevalence of violent conflict has increased to the highest levels observed over the past three decades (United Nations and World Bank 2018). Improving jobs outcomes in FCV environments is thus crucial to ending poverty and spreading prosperity. At the same time, the world’s displaced population is at a record high. The United Nations High Commissioner for Refugees (UNHCR) estimates that there are over 84 million forcibly displaced people worldwide, of which 48 million are internally displaced. Six in seven refugees are hosted by low- and middle-income countries. In addition, the average duration in exile is now more than ten years (Devictor and Do 2016); in recognition of this, policy to support the displaced has shifted from humanitarian to development interventions. 2. Developing impactful and cost-effective jobs support in FCV and displacement situations requires good data on program costs and benefits. The World Bank Group (WBG) has since the 18th replenishment of the International Development Association (IDA 18) greatly increased funds for economies affected by FCV and forced displacement. Additional funds have been made available for middle-income countries hosting large numbers of refugees through the Global Concessional Financing Facility. Support for jobs is an important purpose of this engagement (Schuettler 2020). In turn, effective jobs support requires a good understanding of the benefits of different support modalities as well as of their costs. While knowledge on the impact of jobs support in FCV and forced displacement settings remains limited, there has been progress in identifying effective approaches (von der Goltz and Mavridis, 2020; Schuettler and Caron 2020). However, despite its importance, there is very little systematic evidence on the cost-efficiency and cost-effectiveness of such programs. This lack of good quality cost data risks poorly designed interventions (Blattman and Ralston 2015). 3. This study aims to shed light on the cost-efficiency and cost-effectiveness of jobs support in forced displacement and conflict contexts. While important everywhere, cost-effectiveness matters especially in FCV and forced displacement contexts given enormous needs. This study is a first attempt to synthesize data on cost-efficiency and cost-effectiveness of job interventions across different settings in low- and middle-income countries and to provide insights that can help improve the design of job projects, albeit with data constraints. It builds on previous efforts which have looked at cost and cost-effectiveness across interventions passim (Blattman and Ralston 2015), for specific types of interventions, such as economic inclusion programs (Andrews et al. 2021; Banerjee et al. 2015; Paul, Dutta, and Chaudhary 2021; Sulaiman et al. 2016), or in one country (Gado et al. 2019). 4. The study focuses on interventions in six low- and middle-income countries, financed by three funders. Analysis considers projects funded by the UNHCR, the World Bank, and the former UK Department for International Development, now known as the Foreign, Commonwealth and Development Office (FCDO). The six countries of focus are Iraq, Jordan, Kenya, Lebanon, Mali, and South Sudan. The sample consists of interventions delivered between 2009 and 2019. A comprehensive literature review complemented the analysis of the sampled interventions. 5 5. The study considers the cost of jobs support in displacement and conflict settings relative to cost in other development settings. It compares across two different types of contexts: • Interventions in forced displacement contexts compared to interventions in non-forced displacement contexts. All the countries covered in the study host significant displaced populations (see Annex A); comparisons are therefore between projects that work with the displaced and their hosts and projects that do not work with these groups and instead focus on the general population. • Interventions implemented in FCV situations compared to those in more stable environments. Among the countries in sample, South Sudan, Mali, and Iraq were classified as conflict affected during the reference period whereas Kenya, Jordan, and Lebanon were classified as non-conflict affected. Lebanon was classified as having ‘high institutional and social fragility.’ 6. Following the 2013 World Development Report, the analysis defines ‘jobs’ broadly as any licit activity that generates income. For this study, a definition is appropriate that encompasses the complex and diverse nature of jobs, such as varying levels of formality, self-employment, and work for others, and different degrees of market orientation (World Bank 2013). In low-income countries especially, self-employment is the dominant source of income and most jobs are informal. Further, workers usually obtain income through several activities and their job can be considered a ‘jobs portfolio’ (Blattman and Ralston 2015). In line with this approach, the study considers not only interventions that create new full-time jobs but notably also interventions that aim to raise revenue from existing income-generating activities. It also includes projects that promote access to existing job opportunities for the forcibly displaced and other disadvantaged groups. 7. The report is structured as follows: Section 2 provides an overview of the methodology, and Section 3 describes the dataset of interventions. Section 4 discusses the key findings of the analysis in terms of cost‑efficiency and cost-effectiveness, while Section 5 draws out key conclusions. Section 6 concludes by offering some recommendations. Annexes to this study provide detail on displacement in the six case study countries, the different approaches to jobs support assessed, methodology and assumptions underpinning the study, and additional dimensions of cost-efficiency data. 6 2. DATA AND METHODS 8. This report draws upon a new dataset of cost and results data for individual-level jobs support projects. The study includes programs or interventions with a focus on jobs support as per the 2013 World Development Report’s broad definition, for the countries, funders, and period described earlier. Inclusion and exclusion criteria were applied to the list of all programs delivered by the three agencies in the focus countries. For example, programs with very long-term potential jobs impacts, such as education programs, were excluded as were programs solely focused on policy advocacy with no individual-level support. Jobs support within the scope of this study includes programs aimed at creating jobs, increasing the quality of jobs (productivity, income, and working conditions), and improving access to jobs for disadvantaged groups. For each project, the study reviewed detailed program documentation to extract a description of support activities conducted by the program and their cost and results. Key informant interviews provided further context and clarification. Figure 1 provides an overview. FIGURE 1 Process of sample development 130 Universe of programs—all programs by three funders in six case study countries 55 Selected programs for analysis (full sample) 109 Inverventions part of selected programs (full sample) 109 Inverventions included in qualitative review 63 Inverventions in cost-efficiency analysis Inverventions in 42 cost-effectiveness analysis 9. The unit of analysis for this study is the ‘intervention’ or the set of services a project provides to a distinct group of beneficiaries. There is a methodological challenge in that jobs programs commonly include different support elements. For example, a project might include both training and grant-making components that have different target groups and outcomes. To enable a comparative analysis, this study groups together as ‘interventions’ those components that target the same beneficiaries and separates them from support to other beneficiary groups. For example, if training and grants are delivered to the same group of beneficiaries, we consider this as one intervention. If the training and grants are delivered to different beneficiaries, we consider these as two separate interventions. By way of illustration, one program in Kenya focuses on enhancing the agricultural 7 productivity of smallholder farmers through multiple interventions. One component provides subsistence farmers with basic inputs, while a second component provides more established smallholder farmers with loans. These components target different groups of farmers and so are considered as two different interventions. This is illustrated in Figure 2. FIGURE 2 Distinction between interventions Agriculture Program for Smallholder Farmers Component 1 Component 2 Provision of crops, agricultural inputs Credit provision for smallholder and seeds for subsistence farmers farmers through commercial banks Intervention 1 Intervention 2 10. Cost-effectiveness and cost-efficiency analyses are the two main analytical streams of this study. • Cost-efficiency analysis discusses how well the interventions convert inputs into outputs. The cost-efficiency metric adopted for this study is the ‘cost per output’ ratio, based on cost reported in budgets and outputs recorded in results frameworks (DfID, 2019). For different jobs support projects, this could, for instance, be the cost per individual trained in entrepreneurship skills or the cost per business of providing finance. We use two generalized output indicators to allow for aggregation and comparison of results: the number of individual beneficiaries reached, and the number of beneficiary firms or organizations reached.1 • Cost-effectiveness looks at how spending relates to achievement of intended outcomes, including from a break-even perspective. The cost-effectiveness metric for this study is the ‘cost per outcome’ ratio, as reported by project monitoring. For different interventions, it could, for instance, measure the ‘cost per job’ reported in training programs or ‘cost per employment day’ in short-term employment programs. Outcomes refer to changes that are achieved with program outputs. For instance, training an individual is an output (analyzed under cost-efficiency); when a trainee then gets a job due to this training, this is a program outcome and analyzed under cost effectiveness. We use three generalized outcome indicators: (a) the number of jobs created; (b) the number of employment days provided and (c) the amount of increase in income. The analysis also seeks to consider under what assumptions an intervention would break even, in order to highlight issues around sustainability. 11. ‘Cost per outcome’ is a relatively simple measure of cost-effectiveness. ‘Cost per outcome’ is an uncomplicated measure of cost-effectiveness that can be computed with the limited data available. It is worth noting some conceptual limitations of the indicator (beyond issues relating to measurement, which we discuss below). Thus, on the benefit side it does not capture externalities, multiplier effects, or effects beyond a program’s reporting horizon; on the expenditure side, it does not capture supporting investments from beneficiaries, whether co-investments of businesses in matching grant or loan schemes, or investments of time and own resources by individual beneficiaries. However, ‘cost per outcome’ is a consistent measure of how projects translate resources 1 Throughout, the analysis only considers achieved outputs, not target values. 8 into desired outcomes that can be computed with available data and does facilitate a useful comparison of support modalities and contexts. Where the indicator may miss important aspects of cost-benefit analysis, we point it out—for instance in discussing the sustainability of outcomes through a break-even analysis. 12. A lack of good cost and results data poses a challenge to the analysis. Analysis of cost-effectiveness and cost-efficiency is hampered by limited availability of data on results—especially outcomes as opposed to outputs—as well as a dearth of detailed budget data (Table 1). The analysis in this report requires cost data and data on outputs and outcomes of interventions, and it requires that the two can be linked (that is, expenditures can be attributed to specific actions and their results). Such data was not available for all projects of interest (Figure 1). Consequently, disaggregation and comparison across different project types is at times difficult. It is also worth noting that the analysis shown here does not have the same depth as an assessment of an individual program’s financials and results might be able to achieve, for instance, in considering cost drivers. In addition, there is no way to discern whether there is any correlation between data availability and cost-efficiency and cost-effectiveness. This caution should be kept in mind when interpreting findings. 13. The data nonetheless allows for new and important observations to be made on cost-effectiveness. While limitations are clear, the dataset collected for this report does provide useful information on intervention costs and results across a range of support types and contexts. It makes a contribution to filling the gap on cost data for job interventions in general and in conflict-affected and forced displacement contexts in particular. The analysis carefully notes limitations and strives to use the data appropriately. TABLE 1 Overview of limitations Limitation How limitations were addressed Limitations to using In most cases, we work with results data that is self-reported in program documents that are used as an self‑reported results accountability mechanism either within an organization or between funder and implementer. We conducted key informant interviews with implementers to understand nuances in reported results and explicitly state the limitations that arise for analysis. Overall limited availability Of the total sample of interventions, only a subset had the data needed for cost analysis. We have thus of cost and indicator data focused our analysis on the 63 interventions (of 109) with cost and output data available and 42 for which we had cost and outcome data. Lack of standardized cost While the research team attempted to apply a consistent template to collect costs and results data, the and indicator data final dataset encompassed various formats. In consequence, we made several assumptions to allow for comparability, discussed below and in the annexes. Lack of disaggregated Cost is often not broken down by year and rarely by budget lines (management, fixed costs, and so on). budget data In some programs, it is also difficult to attribute cost to individual interventions. We discuss how these challenges should be considered in interpreting results. Non-attribution of Only three interventions have conducted an impact evaluation that is available. All other interventions report outcomes to the outcomes either as a before-after comparison (or net jobs), or after-only estimates (jobs ‘created’), without intervention showing a counterfactual. We therefore cannot determine whether outcomes are causally related to the intervention. Where possible, we compare data to the impact evaluation literature on similar programs. Question of The overall sample of 109 interventions is representative of the jobs support programs implemented by the representativeness three funders. Yet, it is possible that cost and outcomes data availability relates to whether interventions are cost-effective and cost-efficient. If so, generalization from the sample would not be possible. 9 3. JOBS SUPPORT PROGRAMS ANALYZED IN THIS STUDY 14. Jobs support can take many different forms, and the data collected reflects this diversity. The data collected for this study reflects the diversity of jobs support. Support can be provided on the demand side of the labor market, to businesses and the self-employed, or on the supply side, to workers employed by others. Interventions can aim at creating jobs, at increasing the quality of jobs, or at improving access to jobs for certain groups. Jobs can be a key or rather a secondary outcome of an intervention. The directness of the support varies, as does the emphasis on sustainability. 15. The sample used in this study consists of 55 jobs programs, including 109 separate job interventions. More than one-third of the interventions were implemented in Kenya (Table 2), while Mali accounts for only eight of the interventions. The remainder are nearly evenly split across Iraq, Jordan, Lebanon, and South Sudan. The majority of interventions were funded by the World Bank. Just over one-third were implemented in conflict‑affected contexts (38 interventions), and just under one-third provide services to forcibly displaced persons and host communities (36 interventions). TABLE 2 Overview of interventions by funder and country Iraq Jordan Kenya Lebanon Mali South Sudan All World Bank 7 8 29 4 5 9 62 FCDO 4 2 7 11 0 2 26 UNHCR 4 6 4 0 3 4 21 All 15 16 40 15 8 15 109 Interventions in conflict-affected and forced displacement contexts Conflict affected 15 0 0 0 8 15 38 Forcibly displaced persons 7 5 9 8 3 4 36 and host communities 16. To facilitate analysis, jobs interventions were grouped into six stylized support modalities based on the data collected. There is a rich range of jobs support modalities, and design details vary across programs. To facilitate analysis, this study groups modalities into a simplified typology of six approaches based on the data collected (Table 3). 10 17. These include training; job matching; short-term employment in public works; and capital support through grants, in-kind or through the promotion of access to finance (the latter are sometimes combined with business support services (BSS), more comprehensive ‘graduation’ packages, or support to value chain integration). Programs use these approaches either alone or in combination to target one specific group. While a typology is necessary for our analysis, it is important to recognize that there are differences among projects within each ‘type.’ Annex B provides additional detail. TABLE 3 Overview of jobs support modalities in the sample Approach Overview Variations of approach Training Focus on transferring skills and knowledge to participants. Different types of training: Interventions tend to assume job seekers do not have the skills to • Soft/life skills training be productive and see this as a major barrier to finding employment • Technical and vocational training or to doing well in self-employment and business. Training can focus • Business and entrepreneurship training on aspiring entrepreneurs, business owners, or job seekers. Job matching and The approach includes many different types of activities, designed Different activities: brokerage to connect job seekers and employers. Information asymmetries • Job fairs and job matching (platforms, brokers) between job seekers and employer are often seen as a major barrier • Career support services to accessing jobs. • Direct work placements • Subsidies for employers • Help with barriers such as childcare, transport, and permits Graduation/ The graduation or economic inclusion approach sequences services Most interventions with this approach are economic for beneficiaries to support them in exiting poverty. This usually designed in a similar way. There is variation in inclusion2 includes (a) consumption support, (b) savings support, (c) access level of vulnerability or poverty of beneficiaries. to a productive asset or support in starting an income-generating How services are delivered may vary. Some activity, and (d) technical and soft skills. The rationale is that people interventions deliver other elements as well such living in poverty first require basic (consumption) needs to be met as mentoring and coaching. before being able to increase productivity, build productive assets, and start saving. Capital support Interventions provide businesses, start-ups, farmers, community This approach can involve many different activities and access to groups, and others with access to capital (in-kind or vouchers) and and types of capital: finance funding (cash) as a grant or loan. The approach perceives a lack of • Cash grants capital as a major barrier to productivity or to starting or sustaining • In-kind grants income-generating activities. Interventions targeting firms often • Credit and microcredit also provide business support services (BSS). • (Agri-)input vouchers Public works This approach provides short-term employment to poor households Different types of public works, depending on the in building labor-intensive infrastructure or providing community primary aim: services. The primary objective can be to build large-scale • Safety net public works infrastructure (infrastructural public works) or to provide a • Infrastructure public works short‑term source of income (safety net public works). Projects • Skills-focused public works sometimes aim to improve participants’ skills for employability (skills-focused public works). Market systems/ The market system approach looks at improving the functioning Differences in beneficiaries: value chain of a specific market or value chain, based on the assumption that • Smallholder farmers and other self-employed productivity will rise if inefficiencies in a value chain are removed. • Businesses This approach is broad by definition: the strategies employed will vary widely depending on the identified market failure and nature of the value chain. 2 Data needed for a cost analysis is not available for interventions using the graduation/economic inclusion approach. Thus, the graduation/economic inclusion approach is not discussed in the following sections. 11 18. The support modalities differ in complexity, in the degree of focus on jobs as an outcome, the emphasis on sustainability, and the directness of support. To correctly interpret cost-efficiency and cost‑effectiveness data, it is important to understand differences between support approaches. Complexity varies: value chain interventions, for example, commonly include a broad set of coordinated actions to improve the performance of actors within the value chain. On the other hand, (at least some) public works programs provide straightforward support for a limited number of paid workdays. Programs further differ in whether they directly target job creation as an outcome (for instance, in grant programs to businesses) or aim to increase income (say, in an agricultural productivity program) or access to existing jobs (for example, in jobs matching programs). Sustainability is generally desired but is not equally central to all projects. For instance, ‘safety net’ public works programs tend to aim at providing short-term income opportunities, while graduation support aspires to a lasting impact. Finally, when targeting businesses, individuals become indirect beneficiaries, with the direct benefits going to firms. 19. The complexity and cost of support depends partly on the vulnerability of beneficiaries—but vulnerability can favor both simple and complex approaches, depending on context. The poverty and vulnerability of intended beneficiaries has bearing upon the design of support programs, even after the choice of support modality is made. Effective jobs support for vulnerable groups can call for more complex or thorough support which aims to address multiple constraints. For instance, skills training programs may need to provide additional training to help vulnerable beneficiaries with a lower level of education reach a given skill level; similarly, when cash grant programs work with very poor beneficiaries, they often first provide regular cash transfers to address consumption needs. Conversely, however, working with more vulnerable groups can sometimes favor simple designs or less thorough support. Thus, agriculture programs working with the rural poor tend to provide basic support to productivity through inputs, cash, or some training and tend to be less complex than value chain support targeted toward less poor rural groups. Similarly, public works are often designed to be self-targeted toward poorer workers and provide a modest level of support. The vulnerability displaced workers experience is a particularly relevant case in point, as we further discuss below. BOX 1. GENDER EQUITY CONSIDERATIONS VARY BY JOBS SUPPORT APPROACH Jobs support approaches differ in the share of women among beneficiaries as well as in the impact they might have on women’s labor market outcomes and occupational segregation. Gender disaggregated quantitative data is not always available in project reporting, and so is not systematically discussed in the following sections. This is especially the case for data on outcomes of job interventions, for example, in terms of jobs filled by women and benefits that accrue to women. Qualitative data collected suggests that women’s participation in training and job matching interventions is high, but often channeled toward sectors perceived to be ‘for women.’ In a similar way, women tend to participate less in public works interventions, where work opportunities are often in activities traditionally less open to women, such as construction. Women’s participation in value chain interventions depends on the specific value chains chosen, with high participation for example in agriculture and lower participation in other sectors such as solid waste management. Qualitatively, they appear to sometimes not to meet their gender targets. Access to finance interventions targeted to women primarily provide microfinance services and cash grants for small income-generating activities, while targets for woman beneficiaries are less common and less ambitious across interventions designed to promote business growth and entrepreneurship. This poses the potential risk of reinforcing occupational segregation or further limiting the ability of women to access better jobs and increase their earnings. In displacement or conflict-affected settings gender norms as well as the number of female-headed households might change, further facilitating or hampering female participation and positive impacts on their labor market outcomes compared to other settings. 12 20. Within the sample, capital support and access to finance is the most common support modality, followed by training—both often in combination with others. The sample comprehensively covers the jobs support provided by the relevant organizations within the countries of interest over the past ten years. Figure 3 shows how the 109 interventions in the sample map to the six approaches outlined. Many interventions (roughly one-third) combine approaches, so that beneficiaries receive a package of support.3 Capital support or access to finance is the most common approach in our sample and is used in about half of all interventions— either as stand-alone support (27 interventions) or in combination with other approaches (25). When bundled, it is most commonly combined with training (13 interventions) or business support services (10). Training is the second most prevalent approach and is used in about one in every three interventions (34). Eleven interventions provide only training, eight combine training with job matching and brokerage, and thirteen combine it with access to finance. FIGURE 3 Number of interventions mapped to jobs support approaches Number of programs: 55 Targeted Context beneficiaries Non-conflict Forcibly affected displaced Number of interventions: 109 Conflict (stable/ persons Nationals affected fragile) and host only Training 11 4 7 7 4 76 Job matching and brokerage 7 1 6 5 2 Single interventions Graduation approach 2 0 2 1 1 using one Access to finance and microcredit 27 8 19 7 20 job creation approach Public works 16 9 7 5 11 Market systems: value chain approach 13 3 10 2 11 Job matching and brokerage + training 8 3 5 2 6 33 Combined Access to finance + training 13 5 8 3 10 interventions Access to finance + business support services 10 3 7 3 7 using multiple job creation Access to finance + business support services + training 1 1 0 1 0 approaches Access to finance + training + public works 1 1 0 0 1 Totals: 38 71 36 73 3 In addition, though they are defined as a single intervention, graduation programs combine several approaches per definition. 13 JOBS SUPPORT APPROACHES IN FORCED DISPLACEMENT AND CONFLICT-AFFECTED CONTEXTS 21. A qualitative assessment of the sample highlights some notable features of jobs support in forced displacement and conflict-affected contexts. While the sample captures support provided by the three agencies in the six focus countries, it is not necessarily representative of broader patterns in FCV and forced displacement contexts. However, a qualitative assessment of project design reveals that, as might be expected, forced displacement and conflict-affected contexts favor the use of different types of approaches and designs. 22. Restrictions on refugees’ access to the labor market can influence the choice of support modality. Training interventions are a prominent choice in forced displacement contexts where refugees are often prohibited from formal employment or owning businesses. Where it is difficult to implement programs that directly support refugee employment or business ownership, even in the informal sector, training can still seek to build the beneficiaries’ human capital and potentially to improve their longer-term prospects. Similarly, refugees are often prevented by law from owning land, hence limiting the scope for support to activities in agriculture. By way of contrast, in Jordan, where the Jordan Compact includes provision of work permits to refugees, job matching support connects employers with the displaced to seize job opportunities. 23. Jobs support designed for the displaced and their hosts may have lower ambitions than support provided elsewhere. Capital support and access to finance interventions in forced displacement contexts tend to provide grants for small self-employed income-generating activities or start-up business activities. This is often because governments may prevent refugees from owning or starting a business, owning land, being legally employed or moving to where economic opportunities are, so that refugees’ activities are small scale and informal or limited to refugee settlements. Only very few interventions provide loans: providing credit instead of grants faces additional challenges, as refugee IDs are less likely to satisfy identification requirements and collateral required and perceived risks are higher.4 Similarly, none of the interventions targeting forcibly displaced persons in our sample provide loans or grants for the expansion of existing businesses—a constraint on the type of support that can be provided to both refugees and internally displaced persons (IDPs). Higher-end business support is thus more common in non-forced displacement settings. 24. Some interventions are tailored to address specific needs of those forcibly displaced. Those forcibly displaced face specific obstacles on the labor market compared to hosts and other migrants (Schuettler and Caron 2020). Besides the legal obstacles that mainly concern refugees, refugees and IDPs are likely to face among others: (a) loss of assets and separation from family members, (b) lack of skills (including language) required in the host labor market, (c) negative impacts of forced displacement on their physical health and socio-emotional well-being, and (d)lack of social networks and discrimination. Training can be further designed to meet specific needs of forcibly displaced persons, such as language courses and legal or psychosocial support. Such additional support is offered alongside or integrated into vocational and technical training in two interventions in Lebanon in our sample. Similarly, job matching and brokerage interventions for forcibly displaced persons and their host communities in our sample offer additional support to job seekers beyond just the matching (transport and childcare allowances, emergency cash support, sessions on time management and communication skills, legal counseling, and provision of referrals). The matching itself addresses the fact that refugees and IDPs are more likely than hosts to lack a social network that can help with finding a job. 25. Jobs support in conflict-affected contexts in our sample tends to emphasize simple designs and rapid impact over systemic or lasting change. Fragile and conflict-affected contexts pose unique challenges to economic activities and jobs support alike, due to insecurity and economic disruption, difficult implementing environments, and a particular pressure to rapidly improve outcomes. Across multiple intervention types, a tendency toward supporting informal and short-term activities is palpable: 4 There are examples in the literature showing that lending to refugees might be sustainable under some conditions (Schuettler and Caron 2020). However, our research does not provide evidence to contribute to these observations. 14 • Public works interventions tend to focus on providing a safety net in conflict settings. • Access to finance interventions are more likely to be geared toward providing small-scale financing or inputs to individuals or self-employed workers rather than to businesses. This speaks to constraints in working with firms in conflict contexts where, for example, even a simple due diligence process may become complex and where the business environment may be particularly difficult. Thus, when an intervention in South Sudan sought to address firms’ access to finance barriers by promoting access to early-stage investments and start-up grants, it struggled to recruit firms with the required maturity and faced significant reversals due to a resumption of conflict. • Market systems and value chain interventions are less common in our sample in conflict-affected contexts. Such operations require longer timelines, synchronized actions, advocacy work, and building relationships with market actors—all very difficult things to do in a conflict situation. The few market systems and value chain interventions in our sample focus on products in the agricultural sector, such as mango, potato, and shallots, reflecting the importance of agriculture in the recipient countries. 15 4. COST-EFFICIENCY AND COST‑EFFECTIVENESS OF JOB PROJECTS IN CONFLICT AND FORCED DISPLACEMENT 4.1 COST-EFFICIENCY: WHAT ARE THE COSTS PER INDIVIDUAL BENEFICIARY OR FIRM? 26. Cost-efficiency analysis shows how much programs spend on providing jobs support and can inform design and budget planning. The cost-efficiency analysis looks at cost per output. (Insights into whether that spending results in the desired employment outcomes are discussed in the cost-effectiveness section.) Such information can be useful in ex ante budget planning for jobs support and in design choices. It illustrates the potential scale projects can aspire to and makes incremental costs of additional services visible. While outputs differ across support modalities, this section draws comparisons by focusing on the indicator of ‘cost per beneficiary reached.’ It differentiates between projects working with (a) firms or organizations and (b) individuals, due to the distinct nature of services and ambitions in these projects. 27. Costs vary strongly with the type and complexity of support given but also with contextual factors and objectives beyond directly supporting jobs. Even when distinguishing between interventions aimed at firms and support to individuals, the range of spending per beneficiary across interventions is large. In our data, the cost per output for training interventions targeting individuals, for example, ranges from US$33 to US$3,234 (Table 5). The cost per output for interventions targeting firms varies even more within our sample, with a minimum cost per firm for capital support and access to finance at US$3,308 and a maximum cost per firm of US$835,038 (Table 7). While the data does not allow for a full statistical analysis of drivers of cost, a number of important factors emerge from a qualitative assessment: • The value of direct transfers to beneficiaries. In interventions that provide direct transfers to beneficiaries, overall costs relate strongly to decisions on how large these transfers will be. Consider for example the difference between an intervention that provides credit for the purchase of fertilizer or seeds to smallholder farmers and one that provides scale-up loans to small businesses. While it is obvious that such a discrepancy will arise, it is important to recognize the magnitude of the resulting differences in cost—and to consider that it must be weighed against the potential benefits of various approaches. • Type and complexity of support provided to each beneficiary. Clearly, interventions that are complex and provide multiple kinds of support to beneficiaries will tend to cost more.5 5 In this chapter, interventions combining multiple kinds of support (for example, access to finance and training) are flagged with a ‘+’. The cost-efficiency and cost-effectiveness analysis for these interventions refers to the combined approach and cannot assess the relative contribution of each approach. Approaches have been grouped based on their relative importance in terms of budget or contribution to the achievement of the project objectives, as outlined in the project log frame or theory of change. For example, interventions under capital support and access to finance + include capital support and access to finance as primary approach and provide beneficiaries with additional support in the form of training, job matching, or BSS. 16 • Additional objectives beyond jobs support. Some interventions focus on more than direct jobs support for beneficiaries. For example, in addition to providing temporary employment, infrastructural public works construct public assets that are of value in themselves. These nonemployment benefits are not accounted for in our analysis and so the total benefits of these interventions are underestimated. • Context. Difficult implementing environments, particular needs of beneficiary groups, and other contextual factors also drive costs, as more resources are required to provide the same type of support. 4.1.1 Interventions targeting individuals 28. Capital and access to finance support to individuals tends to be uncomplicated in FCV and forced displacement settings and has the lowest median cost, consisting mostly of direct transfers. The median cost per beneficiary is US$135, and none of the projects reviewed spent more than US$834. The moderate cost is explained by the nature of the support being provided. Interventions in our sample are often simple and provide in-kind agricultural inputs or modestly sized cash grants to smallholder farmers (Table 4). It is worth noting that most of the cost consists of direct transfers to beneficiaries. TABLE 4 Characteristics of capital support and access to finance interventions targeting individuals Country Context Target Type of support Cost per Total cost beneficiary (US$) (US$) South Sudan Conflict affected Non-forcibly displaced Agricultural inputs 37 8,096,957 Kenya Non-conflict Non-forcibly displaced and Grants for income-generating 77 8,924 affected forcibly displaced activities Kenya Non-conflict Non-forcibly displaced Agricultural inputs (voucher 101 4,750,000 affected scheme) Kenya Non-conflict Non-forcibly displaced and Grants for income-generating 114 68,271 affected forcibly displaced activities South Sudan Conflict affected Non-forcibly displaced Forgivable loans (through 135 3,662,069 micro-finance institutions) Kenya Non-conflict Non-forcibly displaced Community grants 194 24,900,000 affected Mali Conflict affected Non-forcibly displaced Community grants 471 4,710,000 Kenya Non-conflict Non-forcibly displaced Grants for inputs and services 517 1,506,447 affected in agriculture South Sudan Conflict affected Non-forcibly displaced Forgivable loans (through 834 3,168,167 micro-finance institutions) 29. Value chain interventions also have a low median cost, but a much wider range of spending, depending on the complexity of services provided. Value chain support has a similar median cost per beneficiary, at US$188. Yet, some projects invest over US$2,500 per beneficiary. Within our sample, value chain interventions mainly work in agriculture and target smallholder farmers in Kenya, Mali, and South Sudan, settings that favor moderate expenditure. Rather than engagement in different value chains, the high variation in costs (US$20–2,569) reflects different design choices: the more costly interventions provide a more complex package of support modalities and seek to address multiple constraints, for example, by working with financial institutions in addition to smallholder farmers. The least costly interventions focus more narrowly on providing inputs and extension services. 17 30. Matching and brokerage interventions had a median cost similar to those that support access to capital or value chains but provide no direct transfers. Job matching interventions have a median cost of US$180, with a maximum of US$500. Cost was lowest in interventions that built on existing matching platforms and career services, thereby avoiding large up-front establishment costs, while at the same time reaching many beneficiaries. In this support modality, the costs are being directed toward services being provided to beneficiaries and are not direct transfers. As shown in Table 5, our sample includes a few programs that provide significant additional support beyond advice and information, such as wage subsidies (we term these ‘job matching and brokerage +’). These programs are similar to more modest matching and brokerage support in that they share the goal of placing beneficiaries in jobs that are already available. However, their cost structure depends heavily on additional support, so that there is no direct comparison to more limited matching support. 31. Safety net public works programs spend a moderate US$392 at the median, while infrastructure public works employ one worker for every US$9,321 spent. Public works projects offer short-term employment, usually at relatively low wages which facilitate self-targeting. Safety net public works projects view temporary income creation as their key objective, similar to a direct cash transfer program. It is typical for programs to provide 60–100 days of work opportunities, for a wage on the order of US$2.50–4 per day. Hence, the value of the direct transfer to beneficiaries tends to range between US$150 and US$400, and with typical values of about US$250–300. Total spending per beneficiary is consequently moderate, with a median cost of US$392 per beneficiary. The programs spent between 21 percent and 78 percent of their total costs on wages. Programs that primarily seek to construct major infrastructure sometimes also use a public works approach to carry out labor-intensive construction steps, to generate temporary employment as a co-benefit. In these projects, cost unrelated to jobs support dominate: in the two interventions for which data was available, wage transfers only made up one and four percent of the total intervention cost, respectively.6 Because of their different nature, we do not include these two projects in the following analysis. For project design purposes, however, it is useful to observe that the two projects in our sample employed one worker temporarily for every US$9,321 of project spending. 32. Public works programs that provide light additional training do not record unusual cost per beneficiary. There are important unanswered questions as to the impact of complementary jobs support measures on the cost-effectiveness and cost-efficiency of public works programs (Gehrke and Hartwig 2018). Some limited research has been done on the impact of training on life skills or technical skills that is sometimes delivered alongside public works (Lombardini and Mager 2019) as well as on the combination of public works opportunities with cash grants. However, little information is available on how such additional services relate to cost, and hence to cost-effectiveness and efficiency. Regrettably, the data compiled for this report does not allow for a systematic comparison of public works programs that offer little additional support and those that do. The sample includes two projects with cost data that provided, respectively, some life skills and technical skills training. Qualitatively, we note that at US$292 and US$392 per beneficiary, neither recorded unusual high cost. Further, one of the two programs for which spending on wages can be identified allocated 78 percent of its spending to wages, an elevated level. Hence, the limited available data suggests that some additional training can be provided without large changes to the typical cost structure of public works programs. 33. Training-only interventions have a more elevated median cost per beneficiary of US$683 and a maximum cost of US$3,234, with training design choices driving costs. There is significant variation in costs depending on the type and intensity of training. Business training, training to promote self-employment, and training in ‘soft skills’ are less costly than technical skills training for wage employment. Besides the type of skills provided, the modality of delivery of training affects the intervention costs. Costs are more elevated when programs involve the private sector in training curricula or offer on-the-job training through apprenticeships. Using community coaches is less costly than certified trainers, and the duration and intensity of the training matters, as do daily stipends. 6 One intervention in Kenya with a total cost of US$8,887,010 and 19,026 labor days created, and one intervention in Jordan with a total cost of US$10,660,720 and 15,000 labor days created. 18 34. As is intuitive, interventions that include multiple forms of support cost more to deliver. Within our data, we observe two types of projects that combine different support modalities for individual beneficiaries. A first group consists of interventions combining access to finance with other services; they have a median cost per beneficiary of US$973. All of these interventions offer some training in addition to capital support (including in two cases BSS), one provides job matching services. Training is the major cost driver in these projects. In the second group of projects that combine job matching services with training, the median cost per beneficiary is US$1,221, driven in significant part by direct job placements offered to participants. TABLE 5 Cost per beneficiary by jobs support approach—individuals Approach N Median (US$) Mean (US$) Min (US$) Max (US$) Single support modality Training 15 683 1,136 33 3,234 Job matching and brokerage 4 180 223 35 499 Capital support and access to finance 9 135 275 37 834 Safety net or skills-focused public works 5 392 797 180 1,735 Infrastructure and community public works 2 9,321 9,321 905 17,737 Value chain interventions 8 188 546 20 2,569 Combined support modalities Job matching and brokerage + 3 1,221 1,482 937 2,288 Capital support and access to finance + 6 973 1,251 72 3,189 All 52 468 1,141 20 17,737 Note: All ‘capital support and access to finance +’ interventions combine access to finance and capital support with training. One intervention also provides job matching services and two interventions also provide BSS for young entrepreneurs receiving start-up funds. Out of three ‘job matching and brokerage + interventions’, two provide training and short-term job placement to participants. One intervention provides training and internships in formal and informal sector businesses as well as entrepreneurship grants to some beneficiaries. 4.1.2 Interventions targeting businesses 35. Programs targeting businesses spend per beneficiary firm 75 times as much as projects supporting individuals pay per beneficiary, at the median. Jobs support that works with businesses relies on different theories of change than those that work with individuals. Support to individuals either seeks to help them become more employable or to improve their success in self-employment. Programs that work with firms, on the other hand, generally seek to enhance the performance of firms, with the assumption that more successful businesses will be more likely to hire. Below, we consider outcomes achieved with each approach. It is, however, worth reiterating at the outset that costs between the two approaches are vastly different: programs supporting businesses commit about US$35,000 per firm at the median, 75 times the median amount of US$468 spent per beneficiary in the same countries by programs targeting individuals. 36. In interventions targeting businesses, cost is largely driven by the size of direct transfers in the forms of loans or grants. Spending per business varies by orders of magnitude between projects, chiefly because of the size of loans or grants. Thus, interventions providing only loans or grants have costs per beneficiary between US$5,770 and US$835,038 (Table 6). Maximum loan or grant size and overall cost per firm correlate 19 strongly (90 percent in logs). As expected, the lowest costs are for interventions that focus on small grants to microbusinesses. At the other end of the spectrum, the African Enterprise Challenge Fund (AECF) provides grants of between US$100,000 and US$1.5 million to established firms, implying considerable spending per beneficiary firm. 37. Cost figures are based on a program perspective and do not capture potentially continuing lending beyond an intervention’s reporting horizon. Some interventions in our sample provide grants to businesses but others provide loans, usually at highly concessional terms. Where loan programs are successful, they routinely recycle repaid loans into new loans for additional beneficiaries. At the same time, they also have higher administrative costs to monitor and handle repayments. This is an important difference to grant programs and has potentially important implications for cost-efficiency and cost-effectiveness. When such recycling takes place during a project’s reporting period, additional loans are reflected in results reporting, and hence reflected in our cost-efficiency and cost-effectiveness numbers. For instance, an intervention in Jordan in our sample lent US$1.90 over seven years for every US$1 of the fund it initially received to make loans and reported the outcomes associated with these loans. Lending may also continue beyond the reporting time horizon and is then not captured in our data. We look at this scenario as an important aspect of the notion that assumptions on sustainability matter for cost-effectiveness and discuss it further, below.7 38. In our sample, programs that combine financing and BSS tend to target smaller firms or start‑ups and therefore provide smaller grants than pure access to finance. By way of contrast with projects oriented toward individuals, business interventions that provide access to finance along with BSS tend to have lower costs than those providing finance alone, with the cost per firm supported ranging between US$3,308 and US$37,350. This is because the type of beneficiary firms targeted lend themselves to support at a more modest scale: BSS interventions are more likely to target smaller enterprises that are less sophisticated in their business practices, and hence both in need of business support, and able to absorb only smaller amount of financing. Conversely, interventions that provide grants or loans without additional support are more likely to target more mature businesses that can utilize larger capital amounts, hence resulting in a higher overall cost.8 TABLE 6 Characteristics of capital support and access to finance interventions targeting firms Country Context Target Type of support Grant or loan Firm type Cost per Total cost size (US$) firm (US$) (US$) South Sudan Conflict Non-forcibly Loans/grants through 1,000–20,000 MSMEs and 5,770 2,624,399 affected displaced a business plan start-ups competition (BPC) Jordan Non-conflict Non-forcibly Loans through MFIs n/a MSMEs 8,996 113,300,00 affected displaced Lebanon Non-conflict Non-forcibly Grants/co-equity Up to 50,000 9 MSMEs and 56,036 14,569,268 affected displaced investment start-ups South Sudan Conflict Non-forcibly Loans and grants About 400,000 SMEs 835,038 3,891,136 affected displaced through a challenge on average fund (AECF) 7 Another way of looking at the issue is to view the initial fund provided for lending as an asset that retains value at the end of the project’s life and may continue to generate benefits, much like, for instance, infrastructures constructed in value chain projects or public works projects in our sample. 8 It is of course also possible that in some cases, these grants are in fact used by businesses to pay for business development services themselves—for example, the AECF allows some grant funds to be spent on these types of services. There is therefore likely some overlap between the interventions providing access to finance alone and those also providing BSS. 9 Value of co-investment per firm is not available. 20 Country Context Target Type of support Grant or loan Firm type Cost per Total cost size (US$) firm (US$) (US$) Iraq Conflict Non-forcibly Micro loans/ 250 on average Start-ups 3,308 1,526,112 affected displaced provision of equipment + BSS Lebanon Non-conflict Non-forcibly Grants + BSS About 2,000 MSMEs 9,031 879,099 affected displaced on average and forcibly displaced 10 South Sudan Conflict Non-forcibly Loans + BSS through n/a 11 MSMEs and 19,216 770,000 affected displaced a BPC start-ups Lebanon Non-conflict Non-forcibly Grants + BSS Up to 22,000 MSMEs 35,236 4,019,056 affected displaced Lebanon Non-conflict Non-forcibly Loan + BSS 8,000–75,000 13 MSMEs 37,350 5,453,122 affected displaced (5 workers and forcibly on average) displaced12 Note: MSMEs = Micro, small, and medium enterprises; SMEs = Small and medium enterprises. 39. The two complex value chain projects for firms in our sample spend per beneficiary about four times as much as pure grants programs. The cost per business supported in the two interventions in our sample is US$122,450 in Lebanon and US$153,146 in Jordan, about four times the median cost of pure access to finance support in the same two countries (Table 7). It is not surprising that costs are higher than in capital support interventions: the two value chain interventions attempt to resolve multiple barriers to well-functioning markets and work with multiple market actors. In Jordan, for example, the value chain intervention focuses on the ecotourism, artisanal products, and service provision markets. In Lebanon, the intervention strengthens the solid waste management industry. TABLE 7 Cost per beneficiary by jobs support approach—businesses and organizations Approach N Median (US$) Mean (US$) Min (US$) Max (US$) Capital support and access to finance 4 32,516 226,460 5,770 835,038 Capital support and access to finance + BSS 5 19,216 20,828 3,308 37,350 Value chain interventions 2 137,798 137,798 122,450 153,146 All 11 35,236 116,871 3,308 835,038 10 The project targeted forcibly displaced people until 2018. 11 Loans with a 4 percent rate of interest. On the successful repayment of the principal and the interest, the BPC winners had the option of opting for another loan at the same rate of interest or the funds were to be made available as a grant. 12 The project targets firms hiring refugee workers. 13 MSMEs can access additional funds based on the number of jobs they create. Additional funds correspond to 15 percent of the loan value for one job and 30 percent for two jobs for a loan value below US$42,000. Above US$42,000, additional funds correspond to 15 percent of the loan for three jobs and 30 percent for four jobs. In addition, MSMEs can get additional grants (25 percent of the loan value) for purchasing assets (this based on need assessment and business case for growth). 21 4.1.3 Cost-efficiency across contexts 40. While there is limited data for a comparison between displacement, fragile, and other contexts, some indicative observations can be made. A key focus of this analysis is to understand cost-efficiencies in forced displacement and conflict-affected contexts. Unfortunately, there is a limited sample of interventions with good cost data, so that comparisons should be taken as indicative rather than definitive. 41. Spending per beneficiary is lower for capital support than training across all contexts. While there is not enough data to reliably compare the cost of all support modalities across contexts, it is notable that the ranking of certain types of support based on their spending per beneficiary is similar. Thus, capital support or access to finance interventions in forced displacement and conflict settings spend less per beneficiary than training interventions, much as they do in other contexts.14 42. Forced displacement and conflict settings favor simple and modest capital support programs but drive complexity in training, further widening the cost gap. Forced displacement and conflict contexts further widen the gap in cost between approaches that emphasize access to capital and those that rely on training. As noted earlier, access to finance interventions in these contexts tend to be relatively simple and provide small amounts of financial support, resulting in lower overall costs. In contrast, expenditure per beneficiary in training reflects additional or customized training modules to meet the specific needs of displaced populations. For example, soft skills trainings, psychosocial support, counseling, or legal advice may be required (sometimes in different languages than the existing material), which is not always the case in non-displaced contexts. 43. Similarly complexity and costs of job matching programs are higher in forced displacement settings. Job matching and brokerage interventions for forcibly displaced persons and their host communities in our sample are more costly than interventions targeting local communities only. This is because interventions offer a package of support to job seekers (transport and childcare allowances, emergency cash support, sessions on time management and communication skills, legal counseling, and provision of referrals) beyond just the matching. 44. In conflict-afflicted economies, the context can drive up implementation costs but simple designs lower costs. For instance, in Mali, a value chain project reported that the unstable macroeconomic and political environment affected the implementation pace, fund disbursement, and costs. Similarly, a public works project in South Sudan reported much higher operating costs than in other countries due to inflation, insecurity, and poor infrastructure (Box 2). Yet, as argued earlier, simple designs and low transfer values drive low cost. TABLE 8 Cost per beneficiary (individual) by jobs support approach—forced displacement contexts Approach N Median Mean Min Max Median non-forced (US$) (US$) (US$) (US$) displacement (US$) Training 6 954 1,456 88 3,234 683 Job matching and brokerage 2 398 398 297 499 49 Capital support and access to finance 2 95 95 77 114 194 Capital support and access to finance + 1 459 459 459 459 1,118 Safety net public works 1 1,735 1,735 1,735 1,735 344 All 12 462 993 77 3,234 n/a Note: Overall median in non-forced displacement contexts includes a different composition of approaches. Capital support and access to finance + combines access to finance, training, and job matching services. 14 There are only three programs in our sample that support businesses in forced displacement contexts, so that a comparison for these programs is not possible. Qualitatively, there is no obvious difference. 22 TABLE 9 Cost per firm or organization by jobs support approach—forced displacement contexts Approach N Median Mean Min Max Median non-forced (US$) (US$) (US$) (US$) displacement (US$) Capital support and access to finance 2 23,190 23,190 9,031 37,350 19,216 + BSS Value chain interventions 1 122,450 122,450 122,450 122,450 153,146 All 3 37,350 56,277 9,031 122,450 n/a Note: Overall median in non-forced displacement contexts includes a different composition of approaches. BOX 2. HIGHER IMPLEMENTATION COSTS IN FCV SETTINGS: THE EXAMPLE OF SAFETY NET PUBLIC WORKS IN SOUTH SUDAN In November 2014, the World Bank launched a safety net and skills development program in South Sudan to provide access to income opportunities and temporary employment to the poor and vulnerable. The program had an overall budget of US$21 million, with US$15.5 million allocated to a public works inter- vention. This public works component supported 53,136 households, with participants receiving between US$2.4 and US$3 per day for 10–20 workdays per month in one year, depending on the location. The economic analysis of the program reports a cost per US$1 transfer of US$2.36. This is higher than other similar safety net public works implemented in Liberia, Ethiopia, and Bangladesh. In the case of South Sudan, several factors affected costs of implementation: • Rapid inflation and devaluation of local currency drove up costs of local goods. • Lack of access to goods (including fuel) and services in country made costly imports necessary. • Transport cost was high due to poor infrastructure and weather-related disruptions, requiring airlifts in some areas. • Due to the high level of insecurity, most service providers charged a security premium. • The public work projects required an intense level of effort and time in community engagement due to low capacity and lack of institutional coordination on the ground. Program implementers argued that “These aspects make it extremely expensive to maintain a scale and quality of activities […] In general, operating costs in South Sudan are much higher than other countries in the region and constitute about 30–35% of total cost.” 23 TABLE 10 Cost per beneficiary (individual) by jobs support approach—conflict-affected contexts Approach N Median Mean Min Max Median non-forced (US$) (US$) (US$) (US$) displacement (US$) Training 6 574 1,017 337 3,042 788 Job matching and brokerage + 1 937 937 937 937 1,754 Capital support and Access to finance 4 303 369 37 834 114 Capital support and Access to finance + 1 3,189 3,189 3,189 3,189 829 Public work (Safety Net and Skills 4 344 563 180 1,384 1,735 development) Value chain interventions 3 142 910 20 2,569 234 All 21 471 1,682 20 17,737 n/a Note: Overall median in non-conflict-affected contexts includes a different composition of approaches. Job matching and brokerage + combines job matching services and training. Capital support and access to finance + combines capital support and access to finance with training and BSS. Infrastructure-focused public works were omitted, as our sample does not include this type of intervention in non‑conflict settings. TABLE 11 Cost per beneficiary (firm) by jobs support approach—conflict-affected contexts Approach N Median Mean Min Max Median non-forced (US$) (US$) (US$) (US$) displacement (US$) Capital support and access to finance 4 12,493 215,833 3,308 835,038 35,236 (single or combined approach with BSS) 4.2 COST-EFFECTIVENESS: WHAT ARE THE COSTS PER JOB OUTCOME? 45. Cost-effectiveness analysis discusses how program spending relates to reported jobs outcomes. While cost-efficiency analysis can inform budget planning, it does not provide enough information to compare the performance of different support modalities for jobs in conflict and forced displacement. Such an assessment requires data on cost-effectiveness—that is, the relationship between program spending and results in terms of the desired employment outcomes. Data availability limits the analysis we can present; in particular, sample size is too small to allow for a comparison across contexts affected and unaffected by conflict and displacement. However, the information presented nonetheless provides some insight into whether some modalities are more likely than others to achieve objectives at lower cost. 46. To compare across interventions and settings, we focus on the cost of supporting two key jobs outcomes, job creation, and increasing incomes. Projects report a range of jobs-related outcomes. The cost‑effectiveness analysis in this study focuses on the creation of new jobs (whether clearly temporary or potentially permanent) and increases in income (or related variables, such as agricultural output). These are the two outcomes most consistently reported, and they correspond to the broad definition of ‘job’ applied in this study. 47. The analysis maintains the distinction between interventions that work with individuals and those that work with businesses. As noted, cost structures are quite different between these two program types. In addition, they diverge in the theories of change they offer and that underlie how ‘job creation’ will be conceived of. For instance, in interventions targeting individuals, ‘number of jobs created’ may refer to new 24 self-employed activities but also to access to existing wage jobs. In interventions supporting businesses, ‘jobs created’ will generally refer to additional hiring. For meaningful comparisons, it is thus useful to maintain the distinction between the two program types. 48. The following discussion recognizes that interventions differ in whether job and income increases are additional and sustainable. A focus on job creation and income increases imposes some consistency on otherwise divergent reported outcomes. However, to meaningfully compare cost-effectiveness across support modalities, it is crucial to recall that programs differ greatly in the additionality and sustainability of even these seemingly uniform outcomes. First, while the evaluation frameworks of projects in our sample rarely allow for a clear assessment of whether the intervention caused additional job creation or income, impact evaluation shows that some support modalities are much more likely to have such an impact than others. This literature informs the following cost-effectiveness discussion. Second, job creation outcomes may refer to temporary jobs and more permanent jobs. Increases in income might also persist for different periods of time—but project evaluation generally tracks only short-run results. This study offers a break-even analysis to show how different assumptions on sustainability influence cost-effectiveness. 49. A break-even analysis emphasizes how long jobs outcomes would have to be sustained in different programs to recoup investments. The break-even analysis asks how long job-related benefits would need to be sustained for the benefits generated to equal the cost incurred.15 To conduct such an analysis, we use assumptions to translate reported outcomes into monetary benefits and compare these to the costs of implementing the interventions.16 In interpreting results, it is important to note that this analysis values only job-related outcomes and does not include any other benefits generated, for example, the benefits of public infrastructure constructed in public works interventions or the spillover benefits of strengthening value chains. 4.2.1 Cost per job Interventions targeting individuals 50. Public works interventions report the lowest cost per job created—unsurprisingly, given that jobs are explicitly temporary. Public works interventions present the same values of cost-efficiency and cost-effectiveness because output (that is, number of workers employed) and outcome indicators (number of jobs created) coincide. As shown in Section 4.1.1, public works interventions have the lowest median cost per beneficiary reached and thus per job created. Given that the programs in our sample offer between 10 and 90 days of employment at moderate wages, these low costs are to be expected. The jobs created by these interventions are therefore ‘low-sustainability, low-wage’ jobs. However, this is largely in line with the goal of these programs: providing vulnerable individuals with short-term livelihood support. 51. Safety net-focused labor-intensive public works spend between 21 percent and 78 percent of their total cost on workers’ wages, with important implications for cost-effectiveness. Since public works programs function chiefly as income transfers, it is instructive, for cost-effectiveness, to consider how much of their expenditure goes to beneficiaries as wages. In four ‘safety nets’ public works programs in our sample, we can compare spending on wages to other costs. Two of those programs (in Lebanon and Mali) spent about 20 percent on wages, while another program in Lebanon spent 30 percent. A large emergency public works program in South Sudan transferred nearly US$4 in every US$5 spent through wages (78 percent). Prima facie, programs with a higher direct transfer may have an easier case for cost-effectiveness. However, public works programs differ from unconditional cash transfers: they potentially include easier targeting owing 15 To be able to conduct a comparative analysis, only intervention types where we had at least four observations with the required data are included. 16 The following key assumptions are used to impute monetary benefits for the number of jobs created and increases in income: (a) to value ‘jobs created’, we use wages reported by the intervention where available and elsewhere, a proxy of either minimum wage or median incomes based on the project context and reported characteristics of jobs; (b) To value increases in income we rely mainly on data reported by the interventions. Where such information was not readily available but outcomes were provided in terms of a relative increase in output, we relied on price data from market surveys to value such increases. We further assume that benefits neither continue to grow nor decline after the end of the intervention. 25 to self-selection, higher social acceptability, and social benefits (such as building networks or the feeling of being productive). It has also been argued that labor-intensive public works have ancillary benefits in the construction of infrastructure (Gehrke and Hartwig 2018) and may lead to continued employment (Beierl and Grimm 2019). Impact evaluations do, however, suggest that labor-intensive public works do not reliably succeed in achieving these goals (World Bank 2020). 52. Job matching reports a cost of about US$3,300 per job with similar cost in programs with narrow and broader benefits packages, due to different placement rates. The median cost per job is US$3,340 for programs that provide job matching, aggregating the two freestanding matching projects listed in Table 12 with the two projects that offer additional support. At first blush, this relatively high cost per jobs is, perhaps, surprising, given that programs that provide only job matching services have a very modest level of spending per beneficiary (US$180 at the median). However, what accounts for a high cost per job is the fact that a relatively low share of beneficiaries report finding or maintaining jobs after counseling—eight percent in the two projects that provide only counseling and information. For instance, one program working with refugees and hosts in Lebanon reports that about 2,500 beneficiaries received jobs counseling, while about 300 were employed three months after counseling. Conversely, two ‘jobs matching +’ interventions that funded subsidized work experience reported a much higher placement rate (57 percent) but also had far higher cost, as shown in Table 5. In consequence, narrowly designed matching support and broader ‘matching +’ designs have similar cost per job. 53. Often, matching only facilitates access to existing or temporary jobs, rather than creating new opportunities. In matching interventions, the goal is to place beneficiaries in employment, often in existing jobs. It is therefore not clear to what extent they create additional employment (or to what degree employment is sustained). The existing evidence in the literature is not encouraging and suggests that the impact of job matching services on employment may not be significant and observable only in the short term (McKenzie 2017) and that they may lead to job displacement rather than an increase in jobs due to reduced search costs (Fox and Kaul 2018). Similarly, preliminary findings from the impact evaluation of a voucher scheme in our sample found that the intervention did not create additional jobs, while vouchers were used by employers to minimize labor costs in the short term. However, job matching and wage subsidies can help disadvantaged groups such as youth, women, or those forcibly displaced improve their employability and enter the labor market (Almeida, Orr, and Robalino 2014). The cost numbers are therefore best thought of as ‘the cost of providing access to jobs for a certain beneficiary population.’ 54. Start-up-oriented ‘access to finance +’ interventions have a median cost per job of about US$4,100, with a stronger presumption of additionality and permanence. All three access to finance + interventions in our sample that have data on ‘jobs created’ provide modest amounts of start-up financing and support to youth, through entrepreneurship programs and BPCs. (Cost-effectiveness of low-cost individual access to finance support through input and small grants is discussed below, in terms of the cost of raising incomes.) Available evidence on the impact and sustainability of cash grant programs for employment through entrepreneurship, with several examples from low and middle-income countries showing positive impact on employment and self-employment (Fafchamps and Quinn 2017; McKenzie 2017, Cho and Honorati 2014). 55. Start-up support will tend to break even if jobs are sustained for some five years, or sooner if jobs are somewhat more productive than typical job activities. Cost per job in the access to finance + interventions is equivalent to four to eight times the median income or minimum wage in the respective host countries.17 That is, it is a reasonable rule of thumb that start-up access to finance + projects may expect to break even if beneficiaries sustain their job activities for some five years (and would not otherwise have had a job). Program would recoup cost more quickly if they provided access to incomes above the median. Both assumptions—on sustainability and perhaps somewhat higher-than-typical incomes—are in line with the ambitions and theories of change of these projects, although the absence of longer-term impact evaluations means that there is no direct evidence on whether these hopes have been fulfilled. 17 Comparisons are made to median income for projects that seem most likely to promote lower-productivity and informal employment and to the minimum wage for those that seem more likely to yield more productive or formal employment. 26 56. Training interventions report a median cost per job of US$4,653, in the context of weaker evidence of additionality. Interventions in our sample report that while participants learned new skills, they were only modestly successful in finding employment. This is perhaps to be expected: these interventions were implemented in difficult economic and security environments, where demand even for qualified workers is low. In our sample, for every 100 individuals trained, an average of 46 go on to find a job (with no counterfactual). Further, impact evaluations of ‘skills-only’ interventions suggest that many of those who find a job would have also succeeded in the absence of additional training (evidence summarized in von der Goltz and Mavridis 2020, Blattman and Ralston 2015; McKenzie 2017). Given this poor track record of additionality, it must be assumed that the cost per new job created through training is a multiple of the median cost per employed trainee reported in our data. At the same time, it is worth noting that skills training programs generally do not purely aim to provide access to jobs but access to better jobs—jobs with higher skill requirements and higher productivity. This consideration is potentially important to cost-efficiency: for instance, if the training programs in our sample succeeded in providing beneficiaries with access to jobs that pay the (relatively elevated) legal minimum wage in the respective countries, the revenue from these jobs would be double the median income. If projects measure increases in income in addition to jobs provided, these impacts would be reflected in the cost-efficiency analysis. TABLE 12 Cost per job—interventions targeting individuals Approach N Median Mean Min Max Median non-forced (US$) (US$) (US$) (US$) displacement (US$) Training 4 4,653 4,009 774 5,955 3,934 Job matching and brokerage 2 4,913 4,913 2,550 7,276 819 Job matching and brokerage + 2 3,340 3,340 3,077 3,603 8,424 Capital support and Access to finance + 3 4,103 4,403 2,862 6,243 9,908 Public works 5 392 797 180 1,735 49,155 Safety Net and Skills development Public works Infrastructure focus 2 9,321 9,321 905 17,737 28,775 Value chain interventions 1 5,490 5,490 5,490 5,490 1,247 All 19 3,077 3,888 180 17,737 102,262 Note: Capital support and access to finance + indicates that the interventions combine training and access to finance. Two interventions also offer BSS. Job matching and brokerage + includes intervention combining job matching services and training. Interventions targeting businesses 57. Cost per job in interventions targeting businesses is a multiple of cost in programs working with individuals, reflecting a more indirect approach to job creation. At the median, programs working with businesses report a cost per job of US$13,561, about four times the median cost in individual-level support. To understand this discrepancy, it is worth recalling that programs that work with businesses look at job creation as a less direct outcome that may be farther downstream of project activities and may take time to materialize. For example, the interventions include challenge funds and concept development grants which seek to create jobs through business innovation and growth. Others start further upstream and provide subsidies to lending institutions to make loans that are intended to support business growth and hiring. Because the link to jobs is less direct, more attention needs to be paid to assumptions about the sustainability of support and future growth. 58. In our sample, interventions that provide larger loans or grants and spend more per firm have higher cost per job. Data on loan or grant size and cost per job is only available for four projects, but there 27 is slightly more information on the relationship between loan or grant size and cost per firm on the one hand and cost per firm and cost per job on the other. Both of these relationships are tight: projects that offer larger grants (almost trivially) spend more per beneficiary firm but cost per job also rises steeply with spending per firm (both correlate by about 90 percent in logs) (Figure 4). That is, projects that offer larger grants do not necessarily create proportionally more jobs. They tend to work with larger firms and in more modern sectors; their cost‑effectiveness will hinge upon whether such an ambition translates into higher productivity or continued job creation in the future.18 FIGURE 4 Loan or grant amounts correlate with spending per firm and cost per job 13 15 14 12 13 11 Maximum grant amount (log) 12 10 Cost per job (log) 11 9 10 9 8 8 7 7 6 6 5 5 7 8 9 10 11 12 13 14 15 Cost per beneficiary firm (log) Cost per job (log) Maximum grant amount (log) 59. Business access to finance programs can expect to break even within about five years if they succeed in generating productive employment. With high cost per firm, breaking even with a business‑oriented access to finance program is most plausible if beneficiary firms hire worker into sustainable jobs and relatively productive activities (and if such jobs are additional, again in the sense that they would not have been created without the grant). At minimum wage, projects in our sample tend to break even within two to five years, given the number of jobs they report having created. Breaking even is a much harder proposition for projects that are likely to offer less productive or casual employment. These interventions would require more than a decade to recoup cost with the level of new employment recorded at closure. That is not to say, of course, that cost may not be recovered through additional employment growth or ancillary benefits such as value chain improvement. However, the data does suggest that loan or grant programs to businesses that aim to create lower-productivity employment should carefully consider the potential for such less-predictable benefits and the merits of alternative investments into individual-level support. 18 Projects that provide additional support beyond grants have a lower median cost per job in our sample, as show in Table 13. Yet, the range of cost is similar, with the exception of one pure access to finance project that offers large grants, so the discrepancy should not be overinterpreted. 28 60. In concessional loans programs, cost-effectiveness may be higher than reported if the lending programs are sustained beyond the reporting time horizon. Many access to finance programs provide (often concessional) loans and can on-lend resources provided to them. The effect of such on-lending is reflected in our cost-efficiency data if it takes place during the reporting period but not beyond this time frame. Sustaining lending programs is challenging in many forced displacement and FCV contexts. However, it is useful to consider the effect of more longer-lived efforts on cost-effectiveness. If we assume that a fund remained active after program support ceases for as long as it has been operating (4–8 years in our sample), then with parameters in line with projects in sample,19 it may hope to make additional loans equivalent to 40–80 percent of the lending it did during the reporting period, just with its original endowment. If these further loans continue to create jobs at the same rate as during the reporting period, and we assume that management overhead accounts for one-third of initial cost, then cost per job would be about 20–30 percent lower than we report here. TABLE 13 Cost per job—interventions targeting businesses or organizations Approach N Median Mean Min Max Number of jobs (US$) (US$) (US$) (US$) created Capital support and access to finance 5 15,537 29,113 2,459 85,208 9,400 Capital support and access to finance 6 10,297 12,298 2,257 23,626 2,335 + BSS Value chain interventions 2 28,682 28,682 13,561 43,803 5,371 All 13 13,561 21,286 2,257 85,208 17,106 4.2.2 Cost per increased income 61. Interventions that provide in-kind support to smallholder farmers in Kenya and South Sudan spend between US$0.19 and US$0.40 per dollar of additional income. Data for income increases is available for three projects that provide in-kind agriculture inputs (Table 14). Cost-effectiveness is high in these simple capital support projects, given the singular focus of the intervention on providing smallholder farmers access to inputs.20 Projects oriented toward this kind of support can hence hope to break even during implementation. It is worth noting, however, that the projects do not consider the cost of labor and other inputs potentially provided by beneficiaries; overall cost-effectiveness may therefore be somewhat overestimated. 62. Value chain interventions with a more systemic ambition and additional objectives spend significantly more—about US$2 per dollar of income at the median. By way of contrast with simple projects providing inputs, value chain interventions seek to address multiple constraints. They have ancillary objectives less directly related to jobs that concern the overall functioning of the value chain and seek to generate positive spillover effects in the local economy. This wider remit is reflected in higher cost.21 Increased income would need to be sustained for a further two to three years after the intervention has ended. Whether this is likely or not depends on factors that are not easy to assess, such as the functioning of farmer cooperatives, permanence of reductions in barriers to accessing markets, or farmers’ ability and desire to take loans beyond the lifetime of the project. 19 Assuming lending maturities of 2.5–4 years, concessional interest rates of 0–4 percent, inflation of 6–10 percent, and a non‑performing loans share of 4–10 percent. 20 Our data includes only a single example of income increases through support to firms, an access to finance intervention targeting SMEs and rural community groups. It had a cost of US$31 for a US$1 increase in income. 21 A project in Kenya with significantly lower spending per dollar of incremental spending scales established programs, avoiding some of the difficulties typically faced by value chain projects. 29 TABLE 14 Cost per dollar of income increase Approach Beneficiaries Cost per one dollar of income increase (US$) Capital support and access to finance Individuals 0.40 Capital support and access to finance Individuals 0.30 Capital support and access to finance Businesses and organizations 31.11 Capital support and access to finance + Individuals 0.19 Value chain interventions Individuals 0.14 Value chain interventions Individuals 2.09 Value chain interventions Individuals 2.03 Note: Capital support and access to finance + combines capital support and access to finance and training. 30 5. POLICY IMPLICATIONS 63. Despite its limitations, the data presented in this report yields policy recommendations on the design of jobs support in FCV, forced displacement, and other settings. To our knowledge, the analysis of the cost of jobs support presented here takes a broader view of the issue than previous studies. While data quality and availability impose significant limitations, the analysis does have policy implications for the design and implementation of jobs support. IMPLICATIONS FOR JOBS SUPPORT IN SITUATIONS OF FORCED DISPLACEMENT AND FCV 64. Removing restrictions on labor market access for the displaced can raise cost-effectiveness of jobs interventions. Refugees often face legal obstacles to integrate the labor market, such as restrictions on their right to work, create a business, own land, move and settle freely, or use financial services. As citizens, IDPs usually do not face these legal constraints but there are exceptions. These legal restrictions limit the type of work refugees can do in the formal but also in the informal sector. At the same time, the restrictions limit the types of jobs support that can be implemented and require additional services to work around constraints (such as legal support to access work permits). It also limits the outcomes that can be achieved. This raises cost per beneficiary in jobs support to the displaced and lowers cost-effectiveness. While important in itself, progress in allowing labor market access for the displaced thus can help promote cost-effectiveness of jobs support. 65. In challenging FCV environments, simple jobs support designs may best help achieve cost-efficiency. FCV economies are characterized by insecurity, macroeconomic instability, disrupted markets, and low capacity. These obstacles increase implementation costs. Keeping objectives for jobs support simple and using context- appropriate tried and tested designs may help keep down cost. For instance, in displacement and FCV environments, the cost gap between simple cash-based support and training-based approaches is wider. IMPLICATIONS FOR JOBS SUPPORT IN ANY ECONOMY 66. Assess expected cost-effectiveness ex ante by considering the likely productivity, additionality, and sustainability of jobs and income increases. Cost-effectiveness is critical to the impactful use of scare resources. It need not come as a surprise as long as teams are realistic in thinking through what assumptions are needed to make a project cost-effective and how likely they are to be realized. Too few projects conduct such an investigation, and those that do, too often gloss over difficult choices. A simple but clear-eyed investigation should focus on the likely productivity, additionality, and sustainability of jobs expected to be created to understand the chances that the project will break even. 67. Consider the cost implications of combining jobs support modalities, and open the black box of how different components contribute to impacts and costs. Combining different types of jobs support increases costs per beneficiary substantially. This may seem a trivial point, but it should receive far more attention in project design. When the obstacles to better jobs appear complex, it is an understandable instinct to want to provide several types of support. However, it is often far from clear that it is preferable to add an additional support modality that (say) doubles cost per beneficiary rather than providing a single type of support to double 31 the number of beneficiaries. A clear consideration of this question is not helped by the fact that the impact evaluations have only recently begun to untangle the effect of different interventions in a bundle, and more work is needed. 68. In capital support to business activities, consider the merits and cost implications of working with firms of different size and capacity. In programs offering financial support to firms and individual entrepreneurs, cost varies greatly with the size and capacity of beneficiary firms. Programs working with larger and more established firms tend to provide far more financial support per beneficiary. Such investments are usually made in the context of hopes that such businesses will hire significant numbers of workers and that employment in them will be productive; where support comes through lending institutions, there may also be a good case for believing that they may be sustained. However, the difference in cost is such that these assumptions should be carefully scrutinized. For instance, cost per job is 2.5 times higher in capital support programs than in capital support to individual business activities. 69. Closely scrutinize the case for jobs support through training. Across all contexts, training projects spend more per beneficiary than capital support and access to finance interventions. Costs per job, at least in the short term, are also higher even without considering the question of additionality. In turn, the impact evaluation literature suggests that pure training programs typically have little impact, so that cost per new additional job may be substantially higher than reflected in our data. Training can still be effective as jobs support in some situations, notably in the longer term. But the preponderance of the evidence is that prospective training programs should closely scrutinize whether there is a realistic chance of a cost-effective intervention. 70. In monitoring and evaluation, keep clear track of cost per beneficiary and cost per output. It is surprisingly difficult to find clear information on project spending per beneficiary, and more so, cost per output. Given the financial reporting projects routinely carry out, reporting such information is not a big ask—and routinely making it available can foster sound consideration of cost in future project design. The effort should be made. 32 REFERENCES Almeida, Rita, Larry Orr, and David Robalino. 2014. “Wage Subsidies in Developing Countries as a Tool to Build Human Capital: Design and Implementation Issues.” IZA Journal of Labor Policy 3 (1): 1–24. Andrews, Colin, Aude de Montesquiou, Ines Arevalo Sanchez, Puja Vasudeva Dutta, Boban Varghese Paul, Sadna Samaranayake, Janet Heisey, Timothy Clay, and Sarang Chaudhary. 2021. The State of Economic Inclusion Report 2021: The Potential to Scale. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/34917. Banerjee, Abhijit, Esther Duflo, Nathanael Goldberg, Dean Karlan, Robert Osei, William Parienté, Jeremy Shapiro, Bram Thuysbaert, and Christopher Udry. 2015. “A Multifaceted Program Causes Lasting Progress for the Very Poor: Evidence from Six Countries.” Science 348 (6236): 1260799. Beierl, Stefan, and Michael Grimm. 2019. Do Public Works Programs Work? A Systematic Review of the Evidence from Programs in Low and Lower-Middle Income Countries in Africa and the MENA Region. Bonn/Eschborn: GIZ. Blattman, Christopher, and Laura Ralston. 2015. “Generating Employment in Poor and Fragile States: Evidence from Labor Market and Entrepreneurship Programs.” SSRN Scholarly Paper ID 2622220. Social Science Research Network, Rochester, NY. https://papers.ssrn.com/abstract=2622220. Cho, Yoonyoung, and Maddalena Honorati. 2014. “Entrepreneurship Programs in Developing Countries: A Meta Regression Analysis.” Labour Economics 28 (C): 110–130. Devictor, Xavier, and Quy-Toan Do. 2016. “How Many Years Have Refugees Been in Exile?” Policy Research Working Paper 7810. Washington, DC, World Bank. https://openknowledge.worldbank.org/handle/10986/25056. DFID. 2019. DFID’s Approach to Value for Money (VfM). London: Department for International Development. Fafchamps, Marcel, and Simon Quinn. 2017. “Aspire.” Journal of Development Studies 53 (10): 1615–1633. doi: 10.1080/00220388.2016.1251584. Fox, Louise, and Upaasna Kaul. 2018. “The Evidence Is In: How Should Youth Employment Programs In Low-Income Countries Be Designed?” Policy Research Working Papers, World Bank. https://doi.org/10.1596/1813-9450-8500. Gado, Boureima, Jan von der Goltz, Mira Saidi, and Abdoulaye Sambo Soumaila. 2019. “Support to Jobs for Youth in Niger: A Retrospective Evaluation of Jobs Projects 2007–2018.” Jobs Working Paper 34, World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/31808. Gehrke, Esther, and Renate Hartwig. 2018. “Productive Effects of Public Works Programs: What Do We Know? What Should We Know?” World Development 107: 111–24. Lombardini, Simone, and Franziska Mager. 2019. Livelihoods in the Za’atari Camp: Impact Evaluation of Oxfam’s Cash for Work Activities in the Za’atari Camp (Jordan). Effectiveness Review Series 2017/2018. Oxfam: Oxford, UK. McKenzie, David. 2017. “How Effective Are Active Labor Market Policies in Developing Countries? A Critical Review of Recent Evidence.” Discussion Paper Series 10665. IZA Institute of Labor Economics, Bonn, Germany. http://ftp.iza.org/dp10655.pdf. Paul, Boban Varghese, Puja Vasudeva Dutta, and Sarang Chaudhary. 2021. “Assessing the Impact and Cost of Economic Inclusion Programs. A Synthesis of Evidence.” World Bank Policy Research Working Paper 9536. Sulaiman, Munshi, Nathanael Goldberg, Dean Karlan, and Aude de Montesquiou. 2016. “liminating Extreme Poverty: Comparing the Cost-Effectiveness of Livelihood, Cash Transfer, and Graduation Approaches. Washington, DC: CGAP. Schuettler, Kirsten. 2020. “Jobs Interventions for Refugees and Internally Displaced Persons.” Jobs Notes 13, World Bank, Washington, DC. Schuettler, Kirsten, and Laura Caron. 2020. “Jobs Interventions for Refugees and Internally Displaced Persons.” Jobs Working Paper 47. World Bank, Washington, DC. UNHCR. 2022. “UNHCR Global Trends 2020.” UNHCR. https://www.unhcr.org/refugee-statistics United Nations, and World Bank. 2018. Pathways for Peace: Inclusive Approaches to Preventing Violent Conflict. Washington,DC: World Bank. 33 von der Goltz, Jan, and Dimitris Mavridis. 2020. “Supporting Jobs in Fragility, Conflict, and Violence Situations.” Jobs Notes 11, World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/33692 World Bank. 2013. World Development Report 2013 : Jobs. World Bank. https://openknowledge.worldbank.org/ handle/10986/11843. World Bank. 2020. Fragility and Conflict: On the Front Lines of the Fight against Poverty. Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/33324. 34 APPENDIX 1 A TEMPLATE FOR TRACKING COST IN MONITORING AND EVALUATION FRAMEWORKS Purpose. Monitoring and evaluation or results monitoring frameworks routinely track data on inputs provided under programs, outputs delivered by them, and outcomes achieved under them (shortcomings in tracking outcomes that can causally be attributed to program support are well known and discussed elsewhere). However, more rarely they track program expenditures in a way that allows for them to be linked to results, and hence to inform cost-benefit assessments. Yet, such information is in principle easily available and collecting it in a usable format need not be difficult. This appendix provides a template for such a tracking of expenditures. With a view to limit the reporting burden, it emphasizes simplicity and focuses on the most important cost information only. • Distinguish interventions. To track cost, clearly distinguish between interventions and sets of services provided by a program to distinct groups of beneficiaries. • Track expenditure data. Keep a record of program spending, broken down by • Intervention; • Time period: the key point is that this be aligned with results reporting periods, it may be the year, fiscal year, or other period; and • Basic cost categories: in the interest of simplicity, these can be kept to a minimum: - Direct transfers to beneficiaries (cash and in-kind) - Project management overhead - Other cost • Adjust inflation. Keep clear track of currencies, exchange rates, and dates. • Collect ancillary output and outcome information to facilitate analysis. Under each intervention, consider indicators defined in the results monitoring framework and how do they fit with generalized indicators such as (a) on the level of outputs, the number of individual beneficiaries and the number of beneficiary firms and organizations and (b the number of jobs created, the number of employment days provided, and the amount of increase in income. Where several interventions have the same outcome indicators, consider what ancillary information can be collected as part of monitoring and evaluation to illustrate important distinctions, for instance, related to the quality of jobs created, expected revenues, and sustainability. Also collect assumptions. 35 Most Recent Jobs Working Papers: 71. Towards a Just Coal Transition Labor Market Challenges and People’s Perspectives from Wielkopolska. (2022) Luc Christiaensen, Céline Ferré, Maddalena Honorati, Tomasz Janusz Gajderowicz, and Sylwia Michalina Wrona. 70. Towards a Just Coal Transition Labor Market Challenges and People’s Perspectives from Silesia. (2022) Luc Christiaensen, Céline Ferré, Tomasz Gajderowicz, Elizabeth Ruppert Bulmer, and Sylwia Wrona. 69. Towards a Just Coal Transition Labor Market Challenges and People’s Perspectives from Lower Silesia. (2022) Luc Christiaensen, Céline Ferré, Tomasz Gajderowicz, and Sylwia Wrona. 68. A Tale of Two Countries: Labor Market Profiles of Youth in Urban and Rural Cameroon. (2022) Ioana Botea and Mitja Del Bono. 67. Improving Smallholders’ Jobs Through Agribusiness Linkages : Findings of the Mozambique Agricultural Aggregator Pilot (MAAP). (2022) Michael Baxter, Christopher Brian Delgado, Jose Romero and David Ian Walker. 66. Rural Employment in Africa: Trends and Challenges. (2022) Luc Christiaensen and Miet Maertens. 65. Understanding and Predicting Job Losses Due to Covid-19: Empirical Evidence from Middle Income Countries. (2021) Maho Hatayama, Yiruo Li, and Theresa Osborne. 64. Revisiting Labor Market Regulations in The Middle East and North Africa. (2021) Maho Hatayama. 63. Migrants, Markets and Mayors Rising above the Employment Challenge in Africa’s Secondary Cities— Key Insights. (2021) Luc Christiaensen and Nancy Lozano Gracia. 62. What was the Impact of Creating Better Jobs for more People in China’s Economic Transformation? What we know and Questions for Further Investigation. (2021) Dino Merotto and Hanchen Jiang. Click here for the full Jobs Paper Series Address: 1776 G St, NW, Washington, DC 20006 Website: http://www.worldbank.org/en/topic/jobsanddevelopment Twitter: @WBG_Jobs Blog: https://blogs.worldbank.org/jobs/