100439 Lessons from The World Bank’s Adolescent Girls Initiative This report is a product of the Jobs and the Gender Cross-Cutting Solutions Areas, and the Social Protection and Labor Global Practice at the World Bank Group, and was prepared by a team directed by Mattias Lundberg and Sarah Nedolast, including Kelly Cassaday, Shubha Chakravarty, Louise Fox, and Sarah Haddock. Excellent support was provided by Bilal Balawny, Marlene Justsen, Veronica Lopez, Zaineb Majoka, Michelle May, Carolina Romero Robayo, and Afia Tasneem. Report layout and design was done by Will Kemp. Numerous World Bank staff provided comments on drafts, and supplied additional information on the pilot projects. The report team is especially grateful to the task teams of the AGI pilots for all their hard work in making this program a success. Any errors or omissions are the responsibility of the report team. The team is grateful to the Economic Empowerment of Adolescent Girls and Young Women (EPAG) project within the Ministry of Gender, Children and Social Protection of Liberia for the use of photos of EPAG participants. The team would like to acknowledge the contributions and support of the AGI partners, including the Nike Foundation and the governments of Afghanistan, Australia, Denmark, Jordan, Lao People’s Democratic Republic, Liberia, Nepal, Norway, Rwanda, Southern Sudan, Sweden, and the United Kingdom. Attribution Please cite this work as follows: World Bank, 2015. The Spirit of Boldness: Lessons from The World Bank’s Adolescent Girls Initiative. Washington DC: World Bank. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior permission of the World Bank. First printing February, 2015 This volume is a product of the staff of the World Bank, and the judgments made herein do not necessarily reflect the views of its Board of Executive Directors or the countries they represent. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility whatsoever for any consequence of their use. The boundaries, colors, denominations, and other information shown on any map in this volume do not imply on the part of the World Bank any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. Contents THE EMPLOYMENT CHALLENGE FOR YOUNG WOMEN AND GIRLS. . . . . . . . . . . . . . . . . . . 1 THE ADOLESCENT GIRLS INITIATIVE—TESTING APPROACHES FOR YOUNG WOMEN TO ENTER PRODUCTIVE EMPLOYMENT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 The pilot projects had multiple objectives.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 The pilots followed multiple models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 The pilots used multiple strategies to engage the private sector and tailor programs to local labor markets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Features to sustain participation and achieve other goals.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Monitoring and evaluating the pilot projects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 ACHIEVEMENTS OF THE AGI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Demand for programs included in the AGI pilots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Economic empowerment outcomes of AGI pilots and similar programs. . . . . . . . . . . . . . . . 18 Agency and economic power impacts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Cost-effectiveness. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 LESSONS AND CONCLUSIONS.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 iv THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE “I have now gotten the spirit of boldness.” —Participant, Liberia  1 The Employment Challenge for Young Women and Girls Y oung people today make up about one-quarter Elsewhere in the developing world, the youth popula- of the world’s population, the majority living in tion will contract or remain stable. Across all developing Asia. Countries in Sub- countries, youth will represent over Saharan Africa and South 40 percent of the population and Asia are in the midst of a demo- “My boyfriend and others thought over 30 percent of the labor force.1 graphic transition that will rapidly it was ridiculous for me to be The importance of this age group increase the share of youth in the doing masonry. They said I was extends beyond its numbers. The total population and the labor force. getting darker under the sun transitions of youth have long-lasting In Sub-Saharan Africa, for instance, and I couldn’t put on long consequences. Adolescence is when the youth population will grow from nails anymore. But I said to my individuals continue to gain skills 195 million in 2015 to 284 million in boyfriend, ‘We will have to see and develop agency, the ability to 2030; it will also grow by more than which one of us advances faster formulate and attain goals indepen- 15 percent over the next 20 years in after my internship.’” dently of their parents. At this time the Middle East and North Africa. —Participant, Haiti they establish habits that they will BOX 1 What is a job? In developing countries, the definition of a job “With this job I can become a different kind of encompasses more than “a wage or salaried posi- woman; an independent woman.” tion with an employer.” In reality, a great many indi- —Participant, Haiti viduals hold jobs that are more realistically defined as “activities that generate actual or imputed “You can tell anyone that I am now a professional.” income, monetary or in kind, formal or informal.” —Participant, Liberia A job also confers more than an income. It “Just look at me. I used to be afraid of electricity develops a person’s sense of identity, status, self- and now I’m an electrician!” confidence, connections to others in the community, —Participant, Haiti and overall satisfaction with life. The distribution of jobs within society and the perceptions about who Not all jobs contribute to an individual’s well- has access to opportunities, and why, often shapes being. The type of job, working conditions, contract, young people’s expectations and aspirations for the benefits, and safety and security at work all matter. future, their sense of having a stake in society, and Some forms of work cannot even be considered their perceptions of fairness. These considerations jobs—for example, activities that are performed are especially important for young females transi- against the will of the worker or that violate basic tioning from school to work, who tend to lack the human rights. Girls and women may be espe- opportunities, confidence, and resources available cially vulnerable to these kinds of coercive work to young males. conditions. Source: World Bank 2012b; Filmer and Fox 2014. 1 US Census Bureau International Database. 2 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE BOX 2 Characteristics of labor markets in low- and middle-income countries Labor markets in low- and middle-income coun- little support is available to promote self-employ- tries exhibit many significant differences, yet they ment and entrepreneurship (for example, through share several general characteristics that shape training in business skills or the provision of financial young people’s transition from school to employ- services). ment. A marked feature of labor markets in low- and Agriculture continues to claim a significant share middle-income countries is that they remain divided of the labor force, especially in the poorest coun- along gender lines, even though girls’ schooling tries, where it often provides the first (sometimes has improved dramatically over the past generation only) income-earning opportunity for young people. in many countries. In addition to doing household For example, agriculture occupies more than chores, females do most of the unpaid work; when 70 percent of the labor force in Sub-Saharan Africa’s they engage in paid employment, it tends to be low-income countries and more than 50 percent in in the informal sector and poorly remunerated. its lower-middle-income countries. The other major Females often are discouraged from seeking wage source of employment in low- and middle-income employment, have little information about their countries is the modern wage sector; it encom- rights as workers, and remain under-represented in passes small, medium, and large firms that continu- the ranks of entrepreneurs and senior staff. ously employ five or more workers, and it includes A telling characteristic of labor markets in low- the public sector, which remains a large share of the and middle-income countries is that a large and modern wage sector in some countries. expanding share of the labor force works in house- Workers in low- and middle-income countries hold enterprises (HEs)—unincorporated, non-farm are highly vulnerable to income shocks, because businesses owned by households. HEs include self-employment is the most common type of work, self-employed individuals operating a business that and even among wage workers, social insurance may employ their family members without pay, or program coverage is low. While unemployment is self-employed people operating a business that still rare in low-income countries—because few can employs a small number of non-family workers on afford to be out of work—rates of youth unemploy- a casual basis. More than 15 percent of workers in ment and underemployment tend to be high in Sub-Saharan Africa and more than half in South Asia low- and middle-income countries. On average, operate HEs. Many individuals may lack the educa- for example, more than one-quarter of the young tion for a wage job but can pursue the economic people in the Middle East and North Africa are opportunities that HEs offer. Despite the critical role unemployed, 28 percent in Indonesia, and 40 per- of HEs in many low- and middle-income countries, cent in South Africa. Source: World Bank. 2012b; Elborgh-Woytek et al. 2013; Filmer and Fox 2014. express for the rest of their lives. Above all, for many it is it can determine the kind of job a person will have for a time of moving definitively into the world of work and the rest of his or her working life. An initial period of consolidating an identity that is at least partially derived unemployment or a first job lacking learning opportuni- from that work. ties can limit an individual’s future productivity. Once Transitioning from school to productive employ- young people start working in a sector—whether in a ment presents an array of challenges for young people household enterprise, agriculture, or a wage job—they in general and adolescent females in particular. “Jobs” may well remain there, although moving across sectors is take forms beyond the typical notion of paid employ- somewhat easier in urban settings than in rural settings. ment in a formal workplace (Box 1). The characteristics Migration may substantially increase earnings, but it is of labor markets in low-income countries (Box 2) mean not an option for the majority. that many young people are either shunted into their Ideally, for both girls and boys, adolescence is a parents’ activities (e.g. farming) or of necessity create time of expanding opportunities, when they can begin their own jobs based on the skills and other resources to realize their ambitions, yet for girls it can become a at their disposal, however limited. Females tend to have time of narrowing options and diminished freedom to trouble getting access to paid employment in the formal exercise choice. Moving through the transition from sector. For many, that first job remains the key, because school to work, girls begin to face specific expectations THE EMPLOYMENT CHALLENGE FOR YOUNG WOMEN AND GIRLS 3 FIGURE 1: Females participate less especially once males finish secondary school 100 90 Labor force participation, % 80 70 60 50 40 30 20 10 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Age (years) LMIC - Females LMIC - Males LIC - Females LIC - Males Source: Authors. Note: LIC = low-income countries; LMIC = low- and middle-income countries. and constraints to their development that boys do not FIGURE 2: Females are pushed to form families earlier face, and that often force girls to make decisions with SAR the potential for negative, irreversible consequences. 100% % of age cohort For instance: 80% • Families often perceive that the returns to investing 60% in girls are lower than returns to investing in boys, 40% so girls’ schooling, health care, and nutrition may 20% receive less attention than boys’. For this reason, 0% especially in low-income countries, they enter the 15 20 25 30 35 Age (years) labor force earlier than males, resulting in a higher labor force participation rate than males on average SSA 100% (Figure 1). % of age cohort 80% • Adolescent girls typically have more household 60% responsibilities than boys do, allowing them less time for school or work outside the home. Girls 40% often must care for siblings or even children of their 20% own (Figure 2). Girls’ movements outside the family 0% home are likely to be more restricted and scrutinized 15 20 25 30 35 Age (years) than boys’. This means that even though they are engaged some hours of the day in economic activity, LAC 100% it tends to be home based and less remunerative. % of age cohort 80% • Adolescent girls are often less qualified for jobs. Girls still have lower rates of secondary school 60% enrollment and completion in many parts of the 40% world (Figure 3), and they may not have a chance to 20% develop the right skills for higher-earning wage jobs. 0% • Young women struggle to find jobs because they are 15 20 25 30 35 Age (years) more socially isolated, with fewer contacts to help them in their search. They often lack the confidence Ever married - Females Ever had a child - Females Ever married - Males Ever had a child - Males to go out and find a job. • Social norms make it harder for young women Source: Authors, based on data from DHS (various years). Note: SAR= Sout Asia Region; SSA = Sub-Saharan Africa; and LAC = to earn a living. Their families, male partners, Latin America and the Caribbean. 4 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE FIGURE 3: Females still get less education FIGURE 4: Social norms restrict female mobility 120 50 she goes out without telling him (%) Female-Male enrollment ratios Women who believe a husband is justified in beating his wife when 100 45 40 80 35 30 60 25 40 20 15 20 10 0 5 LIC LMIC 0 SAR EAP LAC SSA Primary Secondary Tertiary Source: WDI. Source: WDI. Note: LIC = low-income countries; LMIC = low- and middle-income Note: SSA = Sub-Saharan Africa; EAP = East Asia and the Pacific; LAC = countries. Latin America and the Caribbean; and SAR = South Asia Region. employers, or even they themselves may think that married (Figure 4). In many communities throughout certain courses of study or certain jobs are only for the world, both men and women still believe that if a men and not for women. wife leaves the house without first getting her husband’s permission, the husband is justified in beating her. These constraints, and the decisions they engender, These attitudes not only restrict adult females’ choices; such as leaving school, marrying and bearing children they also restrict the choices made by parents for their early, and entering less productive and remunerative daughters. lines of work, have consequences not merely for girls Even in areas with less restrictive social norms, the themselves but for subsequent generations. Early mar- prevalence of gender-based violence and threats to riage and early childbearing are still the norm among safety can prevent adolescent girls and young women adolescent girls in poorer, more traditional societies. from pursuing opportunities. For example, in South In societies where established paths to adulthood and Africa a GPS-enabled analysis of boys and girls age social networks have been lost to civil conflict, natural 10–11 in rural and urban areas showed that both sexes disasters, epidemics, or economic upheaval, girls and had about the same mobility, in terms of area covered women also lose opportunities to acquire schooling and in a single week. By the time the girls reached age life skills just when they need those skills most to pursue 14, however, the area that they covered in a normal a livelihood. week had shrunk by more than 50 percent, while that Social norms discourage females from develop- covered by boys had widened, providing more social ing economic agency and empowerment by restricting contacts and opportunities to learn life skills through mobility and decision making, especially once they are experience.2 2 Hallman et al. (2013). THE EMPLOYMENT CHALLENGE FOR YOUNG WOMEN AND GIRLS 5 6 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE  7 The Adolescent Girls Initiative—Testing Approaches for Young Women to Enter Productive Employment I n October 2008, the World Bank launched the projects in eight countries: Afghanistan, Haiti, Jordan, Adolescent Girls Initiative (AGI), a public-private Lao PDR, Liberia, Nepal, Rwanda, and South Sudan. partnership to promote the transition of adolescent Most of these settings present difficult environments girls and young women to productive employment. for females to enter adulthood. Rwanda is a poor but The AGI provided a unique opportunity to experiment stable country. Nepal is poor and economically stable, in diverse settings, take risks, and find effective, female- although with periodic eruptions of internecine conflict. friendly approaches to vocational training and youth Liberia, a poor country, is still emerging from years of employment programs. The initiative supported pilot conflict. Afghanistan and South Sudan are not only poor TABLE 1: AGI pilot projects: Locations, timelines, and participants Projects in low-income countries Afghanistan Nepal Haiti Project Female Youth Employment Adolescent Girls Employment Adolescent Girls Initiative (AGI) Initiative (FYEI) Initiative (AGEI) Timeline November 2013–December 2014 2010–12 May 2012–December 2014 (3 cohorts: 2010, 2011, 2012) Who participated? • Females age 18–30. • Females age 16–24. • Females age 17–20. • High school completed or • High school not completed. • Out of formal schooling for at nearly completed. • Socially and economically least 1 year. disadvantaged. • Basic literacy in Creole. How many 1,300 4,410 1,000 participated? Where were they Urban and semi-urban areas: City Urban and semi-urban areas: 50+ Urban area: Port-au-Prince. located? of Mazar-i-Sharif and four peri- districts nationwide. urban districts of Balkh Province. How were they Social mobilization through Outreach and communication Through a network of local recruited? school management shuras campaign. NGOs. (councils) in every secondary school in the project districts. Liberia Rwanda South Sudan Project Economic Empowerment of Adolescent Girls Initiative (AGI) Adolescent Girls Initiative (AGI) Adolescent Girls and Young Women (EPAG) Timeline March 2010–February 2011 June 2013–December 2014 2010–13 (Round 1) (3 cohorts) June 2011–May 2012 (Round 2) Who participated? • Females age 16–24 (for job skills • Females age 15–24. • Females age 15–24. training) and 18–24 (for business • Out of formal school for at least • Resided permanently in a village skills training, which also 1 year. where a youth club was located. required previous experience). • Basic literacy. • Out of school for the past year. • Socially and economically • Basic literacy. vulnerable (based on national • Resided in a community social protection classification). targeted by the project. 8 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE How many 2,500 2,000 3,000 participated? Where were they Urban and semi-urban areas: 8 2 urban and 2 rural districts. Urban and semi-urban areas 100 located? communities in greater Monrovia communities across 4 states. and Kakata City. How were they Through widely advertised Public campaign to raise Door-to-door census to identify recruited? community events. widespread awareness of the potential participants. project in the 4 districts. Projects in middle-income countries Jordan Lao PDR Project New Work Opportunities for Adolescent Girls Initiative/ Women (NOW) Supporting Talent, Entrepreneurial Potential, and Success (AGI/STEPS) Timeline December 2010–August 2011 December 2010–December 2013 Who participated? Community college final year • Entrepreneurs under 35 years, students who had passed at least half of whom were graduation exams. female, who participated in the Marketplace Competition. • University and technical/ vocational school students, at least half of whom were female. How many 900 • 10 entrepreneurs. participated? • 488 students received training; 5,000+ participated in job fair. Where were they Urban and semi-urban areas: Urban areas: 3 provincial capitals. located? 8 community colleges across Jordan. How were they By advertising to firms in the areas • Through the call for proposals recruited? where the project operated. for the competition, as well as outreach at schools, business associations, and other venues. • Through Career Counseling Office outreach to students. Source: Authors. but fragile, conflict-affected countries where security The AGI pilot projects varied in duration, rural-urban remains a serious concern. Haiti is a poor, fragile state focus, and the size and heterogeneity of the groups they emerging from disasters and suffering the consequences assisted (Table 1). The interventions used in each pilot of decades of corrupt mismanagement. The exceptions (Table 2) reflected where governments wanted to experi- are Lao PDR, a stable, lower-middle-income country ment, local labor market opportunities, and specific local growing rapidly through natural resource exports, and constraints faced by girls and young women. All projects Jordan, considered a stable, middle-income state. included life skills or employability training, generally Apart from South Sudan, which is unranked, the combined with an array of technical, vocational, and poorer AGI countries rank among the bottom third of business development skills. Training developers and the United Nations Human Development Index. For providers included local and international NGOs as example, literacy in the poorer AGI countries is low, and well as private service providers, government agencies, it is lower for women than for men. In Haiti, 45 percent and university/college staff. Arrangements for imple- of women age 15 and over are literate; in Afghanistan, menting the pilots varied depending on the setting fewer than 20 percent of women are literate; and in but involved government ministries, the World Bank, Liberia, fewer than 30 percent of women are liter- and local implementing partners. To gain evidence and ate.3 Happily, literacy is improving: young people are build an understanding of which approaches succeed in more likely to be literate than older people in all AGI achieving particular objectives for specific groups of girls countries. and young women, most pilots incorporated rigorous evaluations. 3 WDI. THE ADOLESCENT GIRLS INITIATIVE—TESTING APPROACHES FOR YOUNG WOMEN 9 TABLE 2: Content and duration of training provided under AGI pilots Business skills Duration and intensity of Technical/vocational training training Life skills training training Afghanistan: Technical and life skills Office skills, including computer skills No (although note Personal awareness and management; 8 months, 6 days per week, (typing, Windows Operating System, overlap with parts building healthy relationships; decision 4 hours per day, with Office Suite, Quick Book, video editing, of the technical making; resisting peer pressure; job morning and afternoon internet); general management; training program). search and interviewing skills; nutrition sessions. administration; communication; during particular times in the life financial accounting and bookkeeping; cycle (pregnancy, lactation, infancy, marketing. Additional training in childhood, and adolescence); and information and communication strategies to promote nutrition in a technology for a subset of trainees. community. Nepal: Technical, business, and life skills Technical training spanning more Business Negotiation skills, dealing with 3 months of technical than 40 occupations, including non- development discrimination, workers’ rights training, based on the traditional occupations such as mobile skills and financial education, and sexual and trade; 40 hours of business phone repair, aluminum work, arc management. reproductive health. skills training, followed by welding, carpentry, electrical wiring, 40 hours of life skills training. and radio and TV repair. Haiti: Technical and life skills Training in auto repair, refrigerator No business skills Psycho-social education; civic 4–6 months of technical and air conditioning equipment training. engagement and leadership; sex, training (weekdays, 4 hours repair, electrical work, construction, gender, and violence; sexual and per day). Life skills training information technology essentials reproductive health; preparing for held on the weekend to (computer hardware and software), work; reducing risks related to natural avoid overlapping with hospitality services, masonry, carpentry, disasters; financial literacy. technical training. plumbing, and heavy machinery repair and operation. Liberia: Technical, business, and life skills Job skills in house and office painting, Entrepreneurship Preparing for the world of work Training lasted 3–4 hours per hospitality services, professional principles, market (workplace conduct, career day, 3–4 days per week, over driving, office and computer skills, analysis, business development, decision-making, and 6 months. security guard services, professional management, teamwork); sexual and reproductive cleaning, waste management. Job customer health; family skills; healthy living; skills trainees also received “light” service, money preventing and responding to business skills training covering management, and sexual and gender-based violence; entrepreneurship principles, business record-keeping. communication, self-esteem, and management, and financial literacy. leadership; “know your rights”; and community service. Rwanda: Technical, business, and life skills Technical training in one of the Yes. Trust, problem solving, team 1 week of life skills and 1 following: arts and crafts; culinary arts; building, and setting personal goals; week of business skills, agri-business (nursery beds and bee- self-awareness, self-esteem, and followed by 5 months of keeping); and food processing. leadership; family skills; workplace technical skills training. communication; managing stress, anger, and conflict; personal hygiene, nutrition, and healthy lifestyles; sexual and reproductive health; sex, gender, and violence. South Sudan: Technical, business, and life skills training through female youth clubs in project communities Livelihood training offered based Budgeting, Making effective decisions; knowing Each livelihood training on the interests of club members savings, and living with others; knowing lasted from 1 week to 3 and demand in the local market: accounting, and living with oneself; sexual and months. Life skills training tailoring, hairdressing, agriculture, pricing and reproductive health (menstruation, (20 hourly sessions) was poultry farming, goat rearing, catering, marketing, and early pregnancy, preventing sexually conducted in separate carpentry, and running a small customer service. transmitted diseases, and family sessions over 5 months. business. planning, among other topics); leadership; gender and bride price; and rape (definition, prevention, responding, and coping). 10 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE Jordan: Employment vouchers and employability/life skills No technical skills training. A combination Effective communication and business 9 days (5 hours per day). of life skills and writing skills (making a presentation, employability writing business reports, different types skills (see the next of correspondence), team-building column). and teamwork skills (characteristics of a successful team, how to work in different roles within a team); time management; positive thinking (in a business context); customer service; and CV development and interview skills. Lao PDR: Business and career counseling No technical skills training. Shortlisted Students trained in teamwork skills, 10 days business training to entrepreneurs negotiation skills, problem solving, CV 10 shortlisted entrepreneurs. trained in development, and interview skills. 1 day work readiness training business skills. (co-ed classes) provided to students by counseling centers. Source: Authors. The pilot projects had multiple • Build girls’ and young women’s assets for entre- objectives preneurial success. The AGI pilots are building participants’ assets—human, social, and finan- cial—and providing a foundation for venturing into The pilots’ multiple objectives reflected the specific self-employment. Many pilots provided instruction populations and interventions they supported: in financial management and business development • Reach vulnerable girls and young women. skills, along with an opportunity to develop savings. Experience has shown that often • Bolster girls’ and young women’s the most vulnerable girls and personal agency. Through the young women cannot participate “Should only men be allowed to be builders, heavy machinery delivery of life skills training, the AGI in training due to prohibitive drivers, or electricians? No—I want pilots provided adolescent girls and time or financial costs.4 The first to be able to do these jobs, too.” young women with the tools and step toward reaching them is —Participant, Haiti confidence to take advantage of new to understand their needs and economic opportunities. Life skills constraints and design a program training focused on developing non- that is more accessible to them. cognitive skills in multiple domains— • Challenge gender norms in the labor market. Social in the family, workplace, and society; emotional and norms often relegate young females to traditional behavioral awareness; and long-term thinking and trades that are typically defined along gender lines planning. Table 2 lists the array of topics covered. and pay little. The job skills taught in Liberia, Haiti, and Nepal, in contrast, focused on more lucrative occupations that were new to many women, such as house painting, auto repair, the operation and The pilots followed multiple maintenance of construction equipment, and mobile models phone repair. • Bridge the gap to the labor market. Because Each pilot followed one of three general models—two many adolescent girls and young women lack in the low-income group of countries and one in the the social contacts to find a job, especially one middle-income countries (Jordan and Lao PDR). Models with better prospects than the jobs held by most used in low-income countries were designed to reach females, most pilots focused on expanding young disadvantaged females and enable them to enter women’s networks and linking them to employment either self-employment or wage employment. Except opportunities. for Afghanistan, pilots in low-income countries focused on females who had not completed (or even started) 4 Levine et al. (2009). THE ADOLESCENT GIRLS INITIATIVE—TESTING APPROACHES FOR YOUNG WOMEN 11 secondary education. As Table 2 summarizes in detail, access to job opportunities would benefit their families they featured: and communities. A survey of employers identified • Traditional classroom vocational skills training, with labor market opportunities and obstacles facing female supplementary programs to instill life skills, teach jobseekers before the training began, and private sec- practical business skills, and provide health and tor employers continued to provide advice during the other information (Afghanistan, Haiti, Nepal, Liberia, training. Upon graduation, young women had access and Rwanda). These programs were primarily for to six months of employment assistance at nearby job females who had left school. search centers, where they were helped to develop their CVs and interview skills. A subset of • Female youth clubs, which graduates received internships. focused on social support and “Life skills was really helpful in The private training provid- mentoring, supplemented by the community because things ers in Nepal’s Adolescent Girls short training modules in liveli- that happened before, like early Employment Initiative (AGEI) hood skills, life skills, health, and marriage…[are] not happening conducted rapid labor market assess- other topics (South Sudan). The that much anymore.” ments following a standard method- clubs reached adolescent females –Participant, South Sudan ology developed by the Employment who might still be in school, in Fund. The technical training provid- hopes of preventing dropouts. ers received incentives (detailed Females in middle-income countries had access to later) to deliver market-relevant skills to trainees and education but faced barriers to using their education in place them in jobs once training had concluded. the labor force. The model for pilots in those countries A survey of employers in Haiti’s AGI pilot identified featured: labor market opportunities before training started, built • Programs to help females with more advanced edu- interest in the pilot, and revealed potential barriers to cation enter the labor force, either through wage females’ employment. Following their training, young employment or entrepreneurship. women worked as interns (generally unpaid) in local companies to gain the practical experience to obtain a The Nepalese and Lao PDR programs built on job in their new line of work. existing or planned public initiatives, and included both In Liberia’s pilot (Economic Empowerment of males and females. The pilot in Nepal expanded access Adolescent Girls and Young Women, EPAG), the to job training by partnering with the Employment Fund, training programs reflected a labor market assess- a job training and placement initiative operated by an ment as well as a girls’ vulnerability assessment, which international NGO under supervision of the Government identified constraints to labor market participation. An of Nepal;5 the pilot also added business and life skills Employer Advisory Council helped to ensure that train- training and an impact evaluation. The Lao PDR program ing remained relevant, and several approaches helped implemented a minimum quota of 50 percent female to maintain participants’ interest, including business participants to encourage providers to reach females. plan competitions, mentoring, and guest speakers in the classroom. For six months after training, graduates received job placement and business advisory services, The pilots used multiple mentoring, and support to initiate a business; they also participated in job fairs. strategies to engage the private A girls’ vulnerability assessment for Rwanda’s AGI sector and tailor programs to pilot helped to identify constraints to labor market local labor markets participation before designing and initiating train- ing. Graduates received support of various kinds from Among the traditional classroom programs, trainers, school managers, and business mentors. Some Afghanistan’s Female Youth Employment Initiative received support to find jobs or an internship in the (FYEI) was unique in focusing on young women who private sector; others were helped to form cooperatives had completed or nearly completed secondary school. and obtain credit to open small businesses. The intention was to help these young women to obtain BRAC, an NGO, had successfully helped young jobs and to indicate how girls’ continuing education and women to become self-employed in Uganda. That expe- rience (detailed later) led South Sudan’s AGI pilot to contract with BRAC to adapt its model to South Sudan, 5 The NGO is Helvetas; the Employment Fund is supported by the Swiss Development Council, the United Kingdom Department for where opportunities for self-employment predominated. International Development, and (from 2010–12) the World Bank. BRAC formed female-only youth clubs, which offered 12 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE livelihood and life skills training. Participants formed A female-friendly training environment savings groups to become accustomed to saving and Pilots took explicit account of prevailing social norms amass the funds to begin self-employment. Following while challenging some restrictions. Several pilots the training, BRAC encouraged qualified club members focused on providing a safe space for women to learn to seek out microcredit services to support their busi- and develop, often with the support, assent, and implied ness development plans. protection of the local community. Afghanistan, Haiti, As noted, pilots in Jordan and Lao PDR focused Liberia, and South Sudan all offer examples of this less on preliminary training for the workplace and more approach, but Afghanistan is notable for building a on creating the conditions to obtain a first job. Jordan’s network of community support for trainees before and New Work Opportunities for Women (NOW) featured throughout their training. “Social mobilizers” worked employability training and/or job vouchers (covering six extensively in each community to gain agreement months’ wages). The training was developed based on among families, schools, and private and public sector employers’ assessments of skills they required in recent players about the importance of allowing young women graduates. The purpose of the job vouchers was to to work, the goals of the pilot employment project, and encourage firms to hire new graduates with no experi- the safety of the training and job recruitment environ- ence and overcome stereotypes discouraging women’s ment. In Jordan, training was held during daylight hours employment. The AGI pilot in Lao PDR focused on at locally known and trusted institutions, such as the prospective entrepreneurs already in the labor force and Chambers of Industry and local universities. Female- on university and technical college students who would only classrooms (or clubs) were adopted in Afghanistan, need jobs upon graduation. The prospective entre- Liberia, Rwanda, and South Sudan. preneurs (the group with the widest age range in the AGI pilots) were invited to submit business proposals for funding under a Marketplace Competition imple- Strategies to promote inclusion mented in partnership with a local business association. Aside from promoting safety, locating training within the The finalists received training in developing a business community fostered participation by reducing partici- plan and starting a business. Winners received grants pants’ travel time—an important consideration, given and mentoring to start their businesses, and the other that girls and young women have many claims on their participants were eligible to apply for loans and services time. The pilots in Afghanistan, Liberia, and Rwanda from commercial and private banks. For the students, held morning and afternoon training the AGI established career counseling sessions to accommodate partici- offices that provided information on “Having such care for the babies pants’ other chores and activities; for current job vacancies in sectors and encouraged the mothers to relax the same reason, the youth clubs in trades with a high demand for labor, and focus on the training.” South Sudan were open after school. as well as job search assistance. –Participant, Liberia In Haiti, Liberia, and Rwanda, the AGI provided a monthly stipend to cover transportation and food, and Afghan Features to sustain trainees received free transport. The Rwandan gov- participation and achieve ernment funded nine training centers that will remain other goals available for the exclusive use of girls and women, and the pilot supported attendance by making childcare and sanitary products available for trainees. Liberia’s EPAG All pilots incorporated special features to encour- pilot used complementary strategies to encourage and age and sustain females’ participation and success sustain participation: free childcare at each training site (Table 3). These approaches, touched upon earlier, are (about 70 percent of trainees had one or more children); discussed in greater detail here because they reveal literacy training (the greatest demand for training came both the diversity and the comprehensiveness of the from less literate girls and women); attendance prizes; pilots. Learning exchanges among the project teams contests; and a completion bonus. Haiti and Rwanda led some pilots to adopt approaches valued by others.6 offered psycho-social counseling as needed. In Nepal, Experience with safe spaces and childcare in Liberia, for the Employment Fund and private training providers example, led Rwanda’s AGI team to incorporate those conducted an outreach campaign; as part of that effort, features. community-based organizations received an incentive (about US$ 1 per person) when they referred individuals 6 For information on the AGI’s South-South learning exchanges, see World Bank (2014a). THE ADOLESCENT GIRLS INITIATIVE—TESTING APPROACHES FOR YOUNG WOMEN 13 TABLE 3: AGI pilots: Features to foster participation of adolescent girls and young women Country Feature Afghanistan • Community social mobilization throughout the pilot to build a supportive environment for young women’s participation in training and employment. • Morning and afternoon sessions to allow participants to accommodate competing demands on their time. • Female-only classrooms. • Mostly female trainers. • Life skills curriculum tailored to female participants’ needs. • Transport provided to and from training, free of charge. • Training and job search services located close to each other for ease of access. • In-depth consultation with guardians on job placement (permissible work options for participants). Nepal • Incentive pricing scheme to encourage training providers to recruit young female participants. • Life skills curriculum tailored to female participants’ needs. • Working with women’s empowerment organizations as outreach partners. Haiti • Safe training venues within communities (which also reduced travel time). • Life skills curriculum tailored to female participants’ needs. Liberia • Morning and afternoon sessions to allow participants to accommodate competing demands on their time. • Safe training venues within communities (which also reduced travel time). • Female-only classrooms. • Life skills curriculum tailored to female participants’ needs. • Code of Conduct for trainers. Rwanda • Morning and afternoon sessions to allow participants to accommodate competing demands on their time. • Female-only classrooms. • Life skills curriculum tailored to female participants’ needs. South Sudan • Female-only youth clubs. • Clubs located in villages to limit participants’ travel time. • Clubs open in the late afternoon to allow members to attend school and accommodate competing demands on their time. • Life skills curriculum tailored to female participants’ needs. • All training delivered within club premises, with the exception of training for a few technical trades. Lao PDR • Quota of at least 50% female participation in the Marketplace Competition. • Quota of at least 50% female participation for career counseling. Jordan • Sessions held during daylight hours at locally known and trusted institutions. • Life skills curriculum tailored to female participants’ needs. Source: Authors. from vulnerable groups for training, and those individu- based on pre-defined vulnerability criteria;7 the highest als were accepted. incentive was awarded for training and placing the most The NOW pilot in Jordan was directed entirely at disadvantaged females, such as widows, ex-combatants, fostering female inclusion in the workplace. The voucher and disabled women. The combination of a results- program essentially paid employers who hired female based system with a progressive incentive scheme, university graduates. The AGI pilot in Lao PDR was under which training providers receive higher payments co-ed, but it ensured female inclusion through a quota, for graduates who are employed, was intended to requiring that at least half of the entrepreneurs and encourage training providers with the capacity to work students assisted through the pilot were female. with vulnerable groups to do so. Note that although training providers could receive a higher payment for catering to higher-priority groups, they also had a Performance-based contracting to benefit higher risk of failing to achieve the outcome (gainful marginalized groups and improve the employment). accountability of service providers Liberia’s EPAG pilot used a withheld incentive pay- In Nepal, the Employment Fund’s contracts with private ment under which 10 percent of the value of a train- training providers promoted inclusion by offering ing contract was withheld until six months after the financial incentives to train disadvantaged females and classroom training ended; from the withheld funds, the successfully place them in jobs. The incentives were 7 For details on the differential pricing scheme for vulnerable groups and the outcome-based payment system, see World Bank (2012a). 14 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE training provider received an amount proportional to the Mentoring in South Sudan‘s AGI was done through number of graduates employed, relative to a previously Adolescent Leaders (one per youth club), who were set target (for example, a provider meeting 90 per- female community residents, age 19–22, with leader- cent of the target received 90 percent of the withheld ship skills and basic literacy. For a small honorarium and amount). the opportunity to attend the technical, leadership, and skills training, the leaders managed the club activities and facilitated life skills training. Strategies to build The pilots in Rwanda and Lao participants’ financial and PDR offered mentoring to start and “I learned how I can be an social assets entrepreneur. I learned how to sustain a business. In Rwanda, men- To build financial assets and develop apply for a loan if I don’t have tors not only helped participants to the habit of saving, trainees in Liberia capital so I can start from that form cooperatives (including articles opened savings accounts and were loan.” of incorporation and registration) encouraged to save part of their –Participant, Rwanda but assisted them to identify market monthly food and transport stipend. opportunities and develop a business They built social capital by working in plan, provided support for saving pairs (one teamwork strategy paired more and less liter- and access to finance, and liaised with local authorities. ate trainees). Rwandan trainees obtained group savings In Lao PDR, finalists in the Marketplace Competition accounts through a savings and credit cooperative orga- were assigned a mentor (a volunteer, either a member of nization. South Sudan’s youth clubs enabled members the Young Entrepreneurs Association of Laos, a success- to build social capital and networks by providing a safe ful female entrepreneur, or trusted adult) to support the space to learn, solve problems, and socialize (including startup and growth of their businesses. through sports and games) and by helping members to form savings groups to develop and sustain their future economic activities once the program ended. Monitoring and evaluating Mentoring the pilot projects Several AGI pilots employed mentoring to achieve their Consistent with the AGI’s learning goals, all of the pilot goals. Some mentors were hired to assist with training projects included process and endpoint monitoring and and to support girls at risk of dropping out of training. evaluation. Table 4 summarizes monitoring and evalu- In Haiti’s AGI, for example, each life skills class of 25 ation arrangements for the pilots. Surveys covering individuals was taught by a mentor—a young, educated participant’s characteristics, skills, current and past eco- woman with experience working in the community, who nomic activity and earnings, and attitudes were admin- also provided individual psycho-social counseling to istered at the beginning of the pilots and some months participants. The mentor’s role was to help participants after completion. Participants and stakeholders also make informed decisions about which type of train- responded to qualitative surveys designed to capture ing and internship to pursue; act as a role model; and their views about the pilot and its progress. motivate those who did not attend classes regularly or Five of the eight projects had so many applicants dropped out. Liberia’s EPAG pilot experimented with that they were able to identify a control group—indi- mentoring models. In the first round of EPAG, one viduals who either did not participate in the program or mentor (a respected woman from the community) was who participated at a later stage. By collecting data on employed for every seven trainees. The mentor pro- participants and the control group before and after the vided support and guidance for trainees’ businesses pilot, evaluators were able to isolate the improvements or jobs in 56 hours of group and one-on-one sessions, brought about solely by participation in the pilot (as mostly outside the classroom, for a small stipend (about opposed to those brought about by time or improve- US$ 168). In the second round, one coach per class ments in the economy, for example). Three of these eval- of about 25 trainees acted as an assistant trainer and uations are now completed (Liberia, Nepal, and Jordan), class “mother” in addition to following up with absent and their results are discussed in detail later. To provide or troubled trainees and with graduates at their jobs some context for that discussion, Box 3 reviews some of or businesses. The coaches also received training, a the techniques, issues, and options involved in design- monthly stipend of US$ 50, a monthly US$ 10 calling ing and evaluating experiments based on interventions card, and a savings account (with a US$ 5 deposit). of this kind. THE ADOLESCENT GIRLS INITIATIVE—TESTING APPROACHES FOR YOUNG WOMEN 15 TABLE 4: Monitoring and evaluation arrangements for the AGI pilots Quality control and monitoring Monitoring after classroom Evaluation Country (classroom phase) phase Evaluation design status Afghanistan Spot checks/site visits by ministry/ Ministry staff routinely checked Pre/post evaluation of In progress administrative staff; tracking of in with employers to track intern trainees dropouts; government testing/ performance, followed up with certification of skills family in case of any issues Nepal Spot checks/site visits by Verification of job/business Quasi-experimental In progress Employment Fund staff; engagement; web-based design with matching government testing/certification database of graduates and of skills; grievance mechanism applicants (hotline) for complaints Haiti Spot checks by field coordinator; Internship supervision; Randomized pipeline In progress tracking of dropouts: grievance web-based monitoring and mechanism (hotline) for complaints; evaluation platform; feedback government certification of skills from community organizations Liberia Spot checks/site visits by ministry/ Verification of job/business Randomized pipeline Completed administrative staff; computerized engagement attendance records; tracking of dropouts; short interviews with 2 trainees during every site visit Rwanda Spot checks/site visits by ministry/ – Pre/post evaluation of In progress administrative staff trainees South Sudan Spot checks/site visits by ministry/ – Randomized design at In progress administrative staff village level Lao PDR – – Pre/post evaluation of Completed participants Jordan – – Randomized Completed assignment to intervention with control Source: Authors. BOX 3 Evaluation techniques: Choices and challenges Most discussions about evaluation techniques the world around them. For example, if the general concern the best way to attribute changes among economic environment improves during the treat- a population to the program in which they par- ment period, changes observed in the treated group ticipated. Such attribution requires other possible may have arisen from the improving economy and causes for the observed changes to be excluded, have nothing to do with the program. The second and it requires a good idea of what participants approach (comparing the treated and untreated would have achieved if the program had never been group after treatment) presents the risk of conflat- available. A number of evaluation techniques seek ing the treatment effect with the impact of underly- to achieve these two objectives. They can be crudely ing differences in the pre-existing characteristics of described as non-experimental, experimental, and the two groups. The treatment group may already quasi-experimental techniques. be smarter, better motivated, better educated, or Non-experimental techniques include observing have other attributes (observable or unobservable) only the group that received an intervention before that would make them more likely to succeed even and after treatment, or looking at both a treated and without the program. For example, it is unclear that untreated group, but only once, after the treatment. the higher salaries received by graduates of a top Both approaches can yield results with consider- university compared with graduates of a lower-qual- able bias. The first approach (observing only the ity university can be attributed to the top university’s treated group over time) was used in the pilots in superior education program. Most likely, the stu- Afghanistan, Rwanda, and Lao PDR. It runs the risk of dents accepted to the top university were smarter, attributing to the treatment any changes in partici- better motivated, better prepared, and would have pants that could have been caused by changes in done well even if they attended the lesser university. 16 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE Experimental techniques were developed to score just below the cut-off point. Imagine a ranking correct for such problems. After a target group is exercise that gives applicants scores between 1 and identified, some members of the group are selected 100. Those who receive a score of 71 are not that randomly to receive treatment and the remainder different from those who score 69. If the cut-off point form the comparison group. If the random assign- is 70, those who score 69, and are not eligible to par- ment is done correctly—and the sample is large ticipate, can provide a good comparison group for enough—then the two groups are identical on those who score 71, and are eligible to participate. average, except for the treatment offered exclusively This approach was used in the evaluation of Nepal’s to one group. With this purely randomized assign- pilot program. ment, evaluators can be fairly confident that any When the criteria for selecting participants in differences observed between the two groups arose a program are unclear, the evaluation results can exclusively from the treatment. be biased if the group selected for treatment was Experimental evaluation methods include pure already more successful than those who were not randomized controlled trials (RCTs) and randomized selected. The “treatment” group would be doing phase-in or pipeline design methods. In RCTs, the better than the “comparison” group even if the pro- unit of randomization is most commonly the indi- gram had never existed, and it would be a mistake vidual or household, but it may be schools, or health to attribute the treatment group’s success to the facilities, or entire villages (referred to as cluster ran- program. domization). Uganda’s Employment and Livelihoods External validity can be a problem for all of for Adolescents program is a cluster village-level these evaluation techniques, meaning that the RCT. Phase-in (pipeline) evaluations also feature ran- group included in the experiment is somehow dom assignment to treatment and control groups, different from the general population. Such differ- but the control group is informed that they will ences can be both an advantage and an intended receive the treatment at a later date. In this case, the feature of the program. Programs seeking to comparison is not strictly between those who receive encourage entrepreneurship, for example, are the treatment and those who do not; the compari- more effective when participants have a particular son is between those who receive the treatment entrepreneurial ability or mind-set. It makes sense now and those who will receive the treatment later. to see how programs promoting entrepreneurship Liberia’s pilot program uses a randomized pipeline work among people with characteristics correlated design. This design helped to ensure not only that with entrepreneurial success, but it is not possible participants were committed to the program (a high to conclude that the average person would benefit number of dropouts is both costly and can bias the from an entrepreneurship program. The Northern evaluation) but that the treated and control groups Uganda Social Action Fund project, for example, were as alike as possible. asked community leaders to select young people Quasi-experimental techniques enable treat- who were the most motivated (and likely to succeed) ment impacts to be estimated when some members to participate. Less motivated individuals with fewer of a group have received a treatment and others resources may not have had the same success.a have not, but random assignment is or was not Ultimately, resources are better spent among those possible. This approach involves identifying non- for whom the training is likely to be more effective. participants who are as similar as possible to those In sum, even among ostensibly untargeted receiving treatment. Many programs determine who programs, it is important to understand how the is eligible to participate by ranking potential benefi- beneficiary population may differ from the general ciaries according to specific criteria. Applicants to population. A program that randomly assigns train- programs may be given scores based on objective ing to half of those who apply for training is follow- criteria. Those who score above some minimum ing best practice for conducting an evaluation, yet threshold are admitted to the program, and the the evidence from that evaluation is most relevant remaining applicants are not. Discontinuity designs for the population of “those who are most likely to rely on the assumption that those who score just apply for training.” above the cut-off point are similar to those who Source: Authors. Note: For a more in-depth treatment of all of these methods, see Gertler et al. 2011 and Hempel and Fiala 2011. a Blattman, Fiala, and Martinez 2014.  17 Achievements of the AGI T he AGI was motivated by the idea that vocational made in low-income countries, females are keen on training and youth employment programs tailored training. to the needs of girls and young women could Participants may need incentives beyond the train- improve their economic empowerment and ing itself, because participation may imply some kind agency. By putting that idea into practice in a number of sacrifice (time, income) on the part of the participant of ways, the AGI pilots are making it possible to learn or her family. For example, the pilot in Liberia offered about the demand for such programs and whether in childcare; in Haiti, participants received a monthly their current form they are a feasible and (in some cases) stipend to cover the costs of food and transport; and cost-effective means of meeting their objectives. in Afghanistan, where gender norms can constrain women’s access to public space and safety is a major concern, the pilot arranged for a van service to bring participants to and from the training sessions. Demand for programs In the lower-income countries, once females entered included in the AGI pilots the training programs, they tended to complete them. Performance-based incentives for the training providers The pilots in most instances confirm that girls and young obviously helped, as the providers had a strong motiva- women want training and employment programs. They tion to identify bottlenecks and work with participants also confirm that young women not only valued the to solve them. In some pilots, intensive efforts to track acquisition of technical and vocational skills; they valued dropouts and follow up with them and their families as life skills training and mentoring when those features soon as young women started missing class undoubt- were available. edly contributed to low dropout rates. But in Jordan, the pilot country where incomes were highest, dropout rates were fairly high—about 40 percent. Females want to participate The Lao PDR program also showed that females Females, especially from low-income countries, want wanted to start businesses. In the pre-program inter- to participate in programs to support their economic views, female entrepreneurial candidates cited many advancement, but they face barriers to expressing that more obstacles to starting a business than males did, demand. Girls and young women are often excluded and they were particularly more likely to cite the lack from youth programs that are not engendered. In many of credit or startup capital as a problem (males found it parts of the world, girls become less visible to the out- easier to get financial support from their families). Yet side world during adolescence. Recruiting adolescent the female winners of the competition were as likely as girls to participate in youth programs, especially disad- males to start a business and keep it going, including vantaged girls, may require more time and resources hiring employees. than recruiting other types of participants. Adolescent girls often do not have sole authority to decide on their Participants value training in life skills own actions, and multiple stakeholders and even entire communities may need to be engaged, including male Girls and young women, especially those at the social leaders who will support the program. The policy of the and economic margins of their communities, may have pilot in Nepal—paying a premium to training institutes few or no opportunities to acquire the key non-cognitive that get more disadvantaged groups to participate in skills that bring success in the labor force and adult- and complete training—clearly shows that if efforts are hood more generally. Gender norms may also inhibit 18 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE the development of these skills. All AGI pilots eventu- Economic empowerment ally included some training and support in life skills that varied in length (as shown previously in Table 2). The life outcomes of AGI pilots and skills offered through the pilots were remarkably diverse, similar programs including (but not limited to) negotiation skills, effective communication and business writing, team-building The AGI pilots are providing new evidence on the out- and teamwork, workers’ rights, financial literacy, civic comes of the various approaches that they have used to engagement and leadership, dealing with discrimina- foster the economic empowerment of girls and young tion, reducing risks related to natural disasters, making women. To place the AGI outcomes in perspective, effective decisions, family skills, knowing and living with emerging evidence from similar efforts in other countries others, sexual and reproductive health, bride price, and is reviewed first. gender and violence. Qualitative surveys indicated that most participants valued this training a lot. Notably, Early evidence from developed and middle- Nepal’s AGEI pilot did not offer life skills in its first round income countries of training but added them in the second. The females in this cohort had higher earnings and reported more Prior to 2000, very little experimental evidence existed control over their earnings (in other words, greater on the effectiveness of programs in developing coun- economic power and agency), but it tries to support young people’s is hard to know what, if anything, was economic empowerment. Such evi- the contribution of the life skills train- “She [mentor] was my inspiration dence was available from programs ing, as other factors may have been to pursue business; I can see in developed countries, however, at play as well. In Lao PDR, on the myself in her being so successful where the results were mixed. Some other hand, where life skills training and it motivates me.” skill development interventions took the form of career counseling, —Participant, Lao PDR showed results for more disadvan- fewer than 20 percent of participants taged youths; for the average youth credited the training with success in population in developed countries, getting a job. The wide variations in life skills training most skill development programs had little or no impact, and in the perceptions of its usefulness suggest that meaning that they were not a cost-effective use of funds. more experimentation is needed to identify the core Between 2000 and 2010, a number of studies (most valuable) skills required in each context.8 examined the effectiveness of training and other types of active labor market programs implemented in Latin America for young people who were out of school and Young women value and need support and seeking to enter the labor market. Consistent with the mentoring to push the boundaries of gender countries’ middle-income status and well-developed norms labor markets, the programs were designed primarily to A number of programs included mentoring compo- support entrance into wage jobs. In addition to training, nents. Sometimes the mentors were volunteers, but some included support for on-the-job (OTJ) training, the pilots in low-income countries were likely to offer such as short-term wage subsidies or internships. some form of remuneration to mentors. One key find- Reviews of these studies11 show modest effects for ing of the pilot in Lao PDR is that females were more skills training programs for young workers. Better results likely than males to interact with, and benefit from, the were found when programs were combined with OTJ mentors (mentors for female participants were them- training, and one study12 (Argentina) found that only OTJ selves successful female businesswomen).9 Liberia’s training was effective. Some programs, such as those in EPAG project switched to a classroom-based coaching Colombia, Dominican Republic, and Peru, used a rigor- model in Round 2 based on feedback from trainees who ous impact evaluation design and demonstrated good wanted more frequent interactions with mentors. Some results with respect to higher earnings for the partici- participants in Nepal reported that the life skills training pants.13 Regardless of their content, most programs helped them to find mentors on their own.10 showed a much stronger effect on females, especially in terms of employment outcomes (overall participa- tion and hours worked). In general, results from these 8 World Bank (2013). 9 Overall, about half of participants assigned mentors did not utilize their help, for a variety of reasons including scheduling and perceived 11 Card et al. (2010), Kluve (2012), Tripney et al. (2013), and others. availability of mentors. See Knowles (2013). 12 Galasso, Ravallion, and Salvia (2001). 10 Learning from Practice Note on mentoring; see World Bank (2014b). 13 Tripney et al. (2013). ACHIEVEMENTS OF THE AGI 19 middle-income countries surpassed results from similar 5,000 young Ugandan women participating in the ELA programs in more developed countries, suggesting that program in 100 treatment communities experienced some of the hundreds of programs promoting youth significant improvements in employment, particularly empowerment in other parts of the world might show self-employment, after two years; those in treatment results if evaluated experimentally as well. An important communities were 7 percentage points more likely to element missing from most of these studies is informa- be engaged in income-generating activities than those tion on the cost-effectiveness of components of the in control communities.16 Notably, the cost of these pro- various programs. grams (less than US$ 100 per participant) is significantly lower than the unit cost of most youth training programs implemented in low- or middle-income countries. Emerging evidence from low-income countries A vocational training and business startup program In 2008, when the AGI started, no experimental results in northern Uganda. In 2006, motivated young adults were available from programs in low-income countries in Uganda’s conflict-affected north were invited to form to support youth economic empowerment. Since then, a groups and submit grant proposals for vocational train- few studies have delivered results. ing and business startup. These individuals (age 16–34) had a primary education or higher. Government funding Disadvantaged urban communities in India. A train- (from a World Bank credit) was randomly assigned ing program in stitching and tailoring offered to young among eligible groups. Treatment groups (average size women in poor slum communities of New Delhi had a 22 people, with 80 percent of groups having mixed male unique feature, introduced to increase commitment and and female membership) received unsupervised grants encourage regular attendance. The program required of about US$ 382 per member. Grant recipients invested selected participants to deposit 50 rupees (Rs) per the money in skills training, tools, and materials, and month for the duration of the training, or Rs 300 in total. they spent a small amount on living expenses while they Those who completed the program would be repaid were getting their businesses going. After four years, Rs 350. The amount of Rs 50 per month was around 1 half had established businesses using the skills they percent of the average household income for the popu- learned (such as brickmaking, carpentry, sewing, shoe lation. An evaluation of the program’s impact14 found repair, and hairdressing). Others folded their businesses that it increased the likelihood of casual or permanent and went back to agriculture full time, found other jobs, wage employment by more than 5 percentage points, or migrated. Relative to the control group, the program self-employment by almost 4 percentage points, and increased business assets by 57 percent, work hours any employment by 6 percentage points. The program by 17 percent (both in agriculture and non-agricultural increased hours worked in the post-training period activities), and earnings by 38 percent. Control group by around 2.5 hours, and in the short run it increased men also started businesses over the four-year period, monthly earnings by approximately Rs 150. The program but they tended to be petty trading businesses rather also had a positive effect on ownership of capital goods than businesses requiring skills and capital. Women in for entrepreneurship. In the medium term, women who the control group rarely started businesses, and even received the training were 13 percentage points more in the treatment group they took longer to realize an likely to own a sewing machine. effect on their earnings than men did, possibly because they opted for training in traditional female skills such Ten countries in Asia and Africa. The Adolescent as hairdressing and sewing, which also tend to be less Development Program (ADP) and Employment and lucrative.17 Livelihoods for Adolescents (ELA) programs imple- mented by BRAC in 10 countries in Asia and Sub- A poverty alleviation program in northern Uganda. Saharan Africa have had considerable success in In the same conflict-affected area of Uganda, a poverty improving young women’s economic awareness and alleviation program operated by an NGO provided cash attitudes. Between 1993 and 2013, BRAC started almost grants of about US$ 150 and business skills training to 40,000 clubs that have so far reached one million ado- rural females in poor households. Most recipients were lescent girls, including 1,200 clubs in Uganda that reach under the age of 30, with almost no education and no 50,000 girls. BRAC has also started the ELA program work experience outside of agriculture. Using a random- in Sierra Leone, South Sudan, and Tanzania.15 Nearly ized pipeline design, the effects on the first group of 14 Maitra and Mani (2014). 16 Bandiera et al. (2014). 15 The BRAC program in South Sudan is one of the AGI pilot projects. 17 Blattman, Fiala, and Martinez (2014). 20 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE participants were evaluated after 18 months relative to been quite different. Their families may already have had the control group. Not surprisingly, most women in the contacts to help them start a business or get a job, or treatment group continued to farm, but they were also they may have possessed other, unmeasurable charac- able to start trading businesses on the side, while the ter traits that helped them to succeed. This problem is control group women did not. As a result, the treat- called “selectivity,” and it is the reason why an experi- ment group doubled their cash earnings, increased their mental approach is preferred (see Box 3). Of course, it is savings, and their household consumption went up, possible that the later survey (following up with voucher reducing poverty.18 recipients 18 months after they finished training) will show returns more similar to those estimated from the A vouchers-for-training program in Kenya. In Busia survey data on people who chose and financed training District of Western Kenya, an NGO tried to improve on their own. access to post-school vocational training for poor and vulnerable youths by subsidizing the cost through Emerging evidence on the impacts of the vouchers. The voucher program was based on an analy- AGI pilot projects sis of data from the area, which showed high returns to vocational training, especially in male-dominated trades About half of the AGI pilots are still collecting outcome (construction, mechanics, driving, and computers, for results, so results from only three experiments—in example); returns in female-dominated trades (sewing, Nepal, Liberia, and Jordan—are available now. The beauty) were lower. At a recruiting meeting, prospective results are summarized in Table 5. participants received information on public and private vocational Nepal AGEI. Approximately two training centers located in the area, “Nowadays I am employed at a years into the AGEI, the Employment and a subset of participants received mobile [phone] repair shop where Fund program had significantly information on the different returns I earn 8–10,000 rupees per month. improved employment and earnings; to training in male-dominated trades This has changed my life and most impacts appeared to be driven compared to female-dominated gave me a lot of dignity.” by female participants.19 Because trades. Participants were divided —Participant, Nepal Nepal’s program included both male into a treatment and control group, and female participants, the impact and the treatment group was given analysis was done for the pooled vouchers to attend the vocational sample of males and females, and for training program they had selected. The average dura- males and females separately. Analysis of the gender- tion of the training programs was 15 months (spread out, disaggregated sample found that the program had no for example, in 5 terms of 3 months), including intern- discernable effect on outcomes for males. This result ships or other OTJ components. The initial survey, which can be attributed partly to the smaller male sample (over took place 3–9 months after completion of the training, 60 percent of participants were female), yet all of the showed very limited results. No increase in labor force regression results were lower for males than females. In participation was recorded, nor was a shift found from some cases the confidence intervals overlapped, how- agriculture into non-farm economic activities. Only ever, meaning that the estimated differences between individuals working for a wage showed an increase in the male and female participants were not very large, earnings over their counterparts working for a wage even if there was a discernable effect on outcomes for in the control group. It is possible that those engaged females. in self-employment are still getting their businesses The Nepal program significantly increased non-farm started, so their earnings are not higher than the earn- employment. Because the program focused on rural ings of self-employed individuals who are in the control populations, most participants had some economic group. The results were not disaggregated by gender. activity, generally in agriculture, prior to the training. The This study demonstrates the problems of extrapolat- training specifically sought to increase employment in ing estimates of returns to a particular type of training non-farm activities and reduce the chronic underemploy- to a broader population. Compared to the voucher ment of those engaged only in agriculture. The program recipients, the individuals from the area who chose succeeded: Non-farm employment among participants (and financed) training on their own—and who were increased by one-third. Participants were largely able the source of the data on returns to vocational training to find employment in the trades in which they had that inspired the voucher training program—may have received training. As a result, hours worked per month 18 Blattman et al. (2013). 19 For the detailed analysis on Nepal, see Ahmed et al. (2014). ACHIEVEMENTS OF THE AGI 21 TABLE 5: Evidence on the impact of AGI pilot projects in Nepal, Liberia, and Jordan Nepal Liberia Jordan Source report Ahmed et al. 2014 Adoho et al. 2014 Groh et al. 2012 Evaluation coverage and • 2 cohorts (2010, 2011), females • 2 cohorts (2010, 2011), females • 1 cohort, 4 groups (non- period and males. only. cognitive employability skills • Endline survey 9–11 months • Midline survey 6 months training, 6-month wage after training. after cohort 1 completed subsidy voucher, training and training; endline survey 1 voucher, control); females only. year and 6 months after • Endline survey 6 and cohort 1 completed training 14 months after start of and 6 months after cohort 2 intervention. completed training. Experimental design Quasi-experimental: matching Randomized pipeline. Randomized assignment to a of selected participants with group. potential (but not selected) participants in each cohort. Economic advancement • Non-farm employment • Employment increased 18 • In the first few months, outcomes increased 15–17 percentage percentage points, equivalent vouchers increased points for 2010 cohort, 12–14 to a 47% increase over the employment among users by percentage points for 2011 average baseline of 38%. 25% in Amman, 50% outside (driven almost entirely by • 70% of business skills Amman. gains in female non-farm participants had at least one • No employment gains among employment). income-generating activity; soft skills training participants. • Participants increased their 60% of vocational (job) skills • Voucher take-up rate was high, employment in trades trainees did. but dropout rate from the soft- for which they trained by • Most became self-employed, skills training was substantial. 20 percentage points, regardless of type of training compared to those who never (business or vocational skills). participated. This result was • By endline, 65% of both also driven by gains among cohorts receiving vocational females. skills training were self- • Underemployment fell: In the employed (mostly petty trade). 2010 cohort, hours employed • Average weekly income increased about 20% for increased by 80% in treatment males and females; in the versus control group. 2011 cohort, they increased by • At midline, the treatment about the same amount (with a group was 50 percentage stronger result for females). points more likely to be saving; • Monthly earnings increased by saved US$ 35 on average more 45–67% for treatment versus than the control group. control groups in 2010, and • By endline, when everyone had 67–82% for 2011. been trained, 81% had savings, • The percentage earning above on average US$ 129. 3,000 Nepal rupees per month • Source of most savings was increased by 12–14 percentage income-earning activities. points in 2010. Power and agency Yes, limited. Only control over Yes. At midline, cohort 1 Yes. Skills training group outcomes? earnings, and only for those participants reported less worry reported improved life outlook, females who received life skills compared to cohort 2, higher reduced depression. training in 2011. subjective well-being, and more confidence in business skills. Did the impact persist? To be determined. Survey of Yes. Increase in employment No. Employment gains were not 2012 participants included a persisted through endline sustained. No gains reported resurvey of previous cohorts; survey. in Amman in the final survey analysis pending. (after the wage subsidy had run out). Very limited gains reported outside of Amman. Program cost per 295 1,221 (business skills), 1,678 400 participant (US$) (vocational skills). Source: Groh et al. 2012; Adoho et al. 2014; Ahmed et al. 2014. 22 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE increased in the treatment group, and levels of under- the economic environment, yet it is possible that less employment fell. robust results might be achieved during an economic A higher percentage of females gained access to downturn. non-farm economic activities, in part because fewer had One interesting feature of the Nepal program is access before the training. Many more females than that life skills training was added only for females, and males in the treatment groups reported no income- only in 2011. That 2011 female cohort was the only one generating activity at all in the month before training in which any gains in empowerment were detected started; among females, there was even an overall compared to the control groups (the specific gain in participation effect. empowerment was control over one’s earnings). It is The program had strong impacts on monthly earn- impossible to know if differences between the 2010 and ings. Participation in the Employment Fund yielded a 2011 results for females were related to the added life statistically significant 45–67 percent increase in monthly skills training, to other changes in training providers, or earnings. Females were likely to have lower earnings at to the overall economic environment. Data for the 2012 the start, so their earnings effect was stronger, as was cohort are being analyzed; comparisons of the females the poverty reduction effect. in this cohort and the 2011 cohort with the 2010 cohort No impacts were detected on savings or access to may yield interesting results (although only in the form credit. The Employment Fund program had no savings of correlations). or microfinance component, although the evaluation measured savings and borrowing outcomes. About Liberia EPAG. The EPAG program of skill develop- 60 percent of participants reported ment, mentoring, and savings had some savings at the start of the considerable success in increasing “I want to become a nurse in ten program. Neither cohort showed both economic activity and earnings years’ time. I will start off from the an increase in borrowing; the 2011 among participants.20 Skills included training I received at EPAG as a cohort showed a small increase in vocational (job) skills, business skills, professional housekeeper and will total savings even if the number of and life skills. find a job that will enable me to participants saving did not change. Those receiving business skills go to school in two years’ time. Overall, the evaluation shows training were twice as likely to find After that I will…work towards that the Employment Fund program, employment as those who opted attending a nursing school. The which had focused on females and for vocational skills training. The money I received from work will had majority female participation project trained participants living in help me to save and then [I] will even before it was used to imple- urban areas in either business skills get married and in four years’ ment and evaluate training under the (in preparation for self-employment) time will get pregnant and obtain AGEI, was effective. Many features of or job-specific vocational skills (in my degree.” a good skills training program—the preparation for a wage job) plus —Participant, Liberia preliminary labor market assess- minimal business skills training. ment, performance-based payments, Participants were strongly advised incentives for providers to cater to choose the business skills track to disadvantaged and vulnerable youths, including because wage employment is still scarce in Liberia, females—were in place by the time the AGEI was devel- even in urban areas. Not surprisingly, by the time of the oped. The evaluation demonstrates just how important endline survey, most participants were self-employed, those features are, as females started from a lower base regardless of which training track they chose, and those than males for most of the variables that were measured, who chose business skills training were more likely to be and they gained a lot. The evaluation also confirms employed. This result supports the program’s decision results seen in other countries—specifically, that focus- to offer basic training in business skills to the groups that ing public subsidies for post-school training and skill chose to learn vocational skills. building on the most vulnerable is most likely to pro- The project increased trainees’ income by 47 per- duce results. Youths with some financing and agency will cent, compared to incomes of individuals in the control find their way into skill development programs and into group. Both treatment and control groups improved gainful employment, but the disadvantaged may not. In their incomes by the midline, but the change was signifi- South Asia, this group includes most rural females. cantly larger among the treatment group. The change The program and the evaluation were undertaken at was largest for those with more education; no discern- a time of good economic growth in Nepal. Given that ible impact was seen among the few participants with the control groups benefitted from this growth as well, the program’s superior results cannot be attributed to 20 For the detailed analysis on Liberia, see Adoho et al. (2014). ACHIEVEMENTS OF THE AGI 23 no education. EPAG improved earnings through two desirable, given the limited size of the private enter- channels: first, by bringing more people into employ- prise sector in Liberia) to help trainees find stable wage ment, and second, by increasing the productivity of jobs. Disadvantaged females can advance economically those who were already engaged in through self-employment—even in income-generating activities. These petty trading, an arena of low barriers “I used to braid hair and just effects are significant, both statisti- to entry and fierce competition— eat the money without thinking cally and economically, and much through skill development, mentor- about the future, but now I larger in magnitude compared to ing, and support for savings to make consider…that saving helps me to other vocational training projects that startup capital available. have some money when things have been rigorously evaluated. As in Nepal, in Liberia the pro- become tough.” EPAG significantly increased girls’ grams were implemented at a time —Participant, Liberia savings. At midline, the treatment of steady economic progress, even group was nearly 50 percentage if that progress was coming from a points more likely to have savings low base. It is hoped that participants than the control group, and on average the treatment will have gained enough resilience to cope with the set- group was saving 2,500 Liberian dollars (nearly US$ 35) backs Liberia is experiencing economically and socially more than the control group. The increase in savings was from the 2014–15 Ebola disaster. greater than the project completion bonus of US$ 20. In addition to saving in banks, the treatment group also Jordan NOW. The NOW program, designed to help reported having increased savings in informal savings female graduates of community college find a job, clubs. Participants were also more likely to obtain credit yielded rather disappointing results.21 Some positive from banks after the program. differences were observed in the short term but had EPAG’s economic impact appears to be long-lasting. largely disappeared after the vouchers had expired and Among Round 1 trainees (the treatment group), endline the endline survey was complete. The evaluation report survey data (collected 1.3 years after training ended) speculates that the program could not overcome social showed no significant decline in the likelihood of work- constraints on hiring young Jordanian women. ing compared to the midline results (collected 6 months At midline, the job vouchers had an encouraging, after training ended). Meanwhile, endline data con- large, and strongly significant impact on labor force firmed that employment among the participation. The 40 percentage point control group (the Round 2 trainees, increase in employment more than “Before I went into the program, who by this point had received train- tripled the employment rate of 18 per- I didn’t know what to expect. I ing) had caught up with, and even cent in the control group. In contrast, didn’t know how to communicate slightly surpassed, that of the treat- the employability/life skills training well. I didn’t know how a CV is ment group. Savings effects were had only a small (3 percentage point) done. Now, I know how to meet also maintained in the treatment and statistically insignificant impact on with Human Resources and I know group (first cohort) when surveyed employment, and the analysis found how to negotiate.” at endline. no effects of an interaction between —Participant, Jordan EPAG improved participants’ the training and job vouchers. agency. Confidence, outlook on life, By the endline survey, the increase belief in entrepreneurial capabilities, in labor force participation among and subjective well-being increased in the treatment the voucher participants had dissipated, and the impact group compared with the control group. The treatment of training was still not significant. The control group’s group had already reported having a high degree of labor force participation had fallen to 48 percent by control over their earnings or other resources, so it is not then, reflecting the fact that they had stopped actively surprising that this group had no apparent improvement looking for work. The job voucher group was only in this outcome. The baseline level of gender-based 2.8 percentage points more likely to be employed at violence experienced by participants was already high endline than the control group—the vouchers had no and did not fall, suggesting that this social norm did significant impact on formal employment. not change solely because participants’ empowerment Hours worked and incomes were higher for the job changed. voucher group at midline but not significantly different Liberia’s experience shows that it is possible to help at endline. At midline, the job voucher group earned 64 disadvantaged females, who lost opportunities for edu- more Jordanian dinar (JD) per month than the control cation and skill development during a period of violent conflict, to gain economically. It is not necessary (or even 21 For the detailed analysis on Jordan, see Groh et al. (2012). 24 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE group—in other words, they were earning more than enrollment and attainment of adolescents in these triple the monthly wages of the control group (JD 25). By villages, including enrollment in private for-fee train- endline the difference had fallen to only JD 6 per month ing institutes, and led to reduced malnutrition among and was not statistically significant. all school-aged females. Finally, it led to an increase in The Jordanian program was especially disappointing the average age of marriage and childbirth. Likewise in because 93 percent of female community college gradu- Uganda, two years of participation in Adolescent Girls ates said that they wanted to work after graduation. It Clubs decreased the likelihood of teen pregnancy by 26 appears that this relatively modest program could not percent and reduced early entry into marriage/cohabi- overcome the barriers they faced, yet it is not clear what tation by 58 percent. The share of girls reporting sex type of program would succeed in its place. If employ- against their will dropped from 14 percent to almost half ers did not know the value of female workers, they had that level, and preferred ages of marriage and childbear- a chance to find out during the voucher period, yet they ing both moved forward.25 still did not retain those employees. In contrast, an evaluation of the poverty alleviation It should be noted that wage subsidy programs are program in northern Uganda (discussed earlier) found popular with economists, but their track record is not no evidence that women’s increased cash earnings trans- good.22 Where they have been effective, their impact lated into social empowerment.26 No impact was found has occurred largely in informal or temporary jobs and on women’s self-reported anxiety, status in the village, not in the longer-term, steady wage employment sought influence on household decisions, or independence. by most individuals. Similarly, an evaluation of the Youth Opportunities Program in northern Uganda (where the majority of the participants were male) found no evidence of an Agency and economic income-earning effect on antisocial behavior, trust in government, political participation, or social cohesion.27 power impacts It is often argued that having cash “Now decision making on use of Emerging evidence from AGI income (economic advancement) the income also has become The AGI pilots differed somewhat leads to more equitable economic different. She brings in money and from the projects just discussed, and social relationships within the we see what we do. When she except for the ELA programs, in that household and in the community. comes with money it also gives most of the pilots explicitly included Indeed, in developed countries, her power.” life skills components designed wider economic opportunities —Husband of participant, to increase agency, aspirations, for women were correlated with South Sudan self-control, optimism, and overall increased economic and social well-being. In Liberia and Nepal, empowerment, including increased “I don’t have to ask anyone for the combination of life skills training political participation, more control money. In fact, I can even help and the other program components over resources, and less discrimina- my family out with expenses.” did result in discernable changes in tory institutions.23 To what extent —Participant, Nepal some of these variables. Importantly, have increased economic opportuni- in Nepal, which introduced life skills ties for women in the AGI programs training only in 2011, it was the 2011 and similar programs in developing countries led to cohort that registered improvement on the empower- increased agency and social empowerment? ment scales, although it was only control over earnings that improved. The 2011 cohort also had a greater Emerging evidence from other studies increase in average earnings, so it is impossible to tell what triggered the change. In Liberia, multiple interven- Evidence from other studies is contradictory. In rural tions were designed to enhance agency and well-being, India, a program connecting village women with recruit- and they yielded results. Given the contrasting experi- ers for Business Processing Outsourcing jobs increased ence in Uganda (discussed above), where economic the number of women in the village working at any kind advancement did not lead to improved agency or status, of job for pay outside the home, and it shifted their aspi- it is probable that at least some part of the Liberia rations toward work.24 It also increased the educational 22 Groh et al. (2012). 25 Bandiera et al. (2014). 23 World Bank (2012c). 26 Blattman et al. (2013). 24 Jensen (2012). 27 Blattman, Fiala, and Martinez (2014). ACHIEVEMENTS OF THE AGI 25 program that did not involve vocational or business skills TABLE 6: Costs of training in AGI pilots and training (the life skills training or mentoring, for example) comparator programs helped to enhance agency and well-being—but it is difficult to disentangle the various components contrib- Training cost per graduate Country and program (US$) uting to this result. Finally, the fact that participants in Uganda (ELA) 87 Jordan who received the combined employability and life skills training reported improvements in life outlook Kenya (vocational training vouchers)a 250 and reduced depression suggests that such training Nepal (AGEI)a 295 does have intrinsic value for young females trying to find Uganda (NUSAF) a 382 their way in the world. Jordan (NOW) 400 Rwanda (AGI) 1,013 Cost-effectiveness Afghanistan (AGI) 1,158 Liberia (EPAG, self- employment skills) 1,221 Assessing cost-effectiveness in a pilot program is dif- Liberia (EPAG, vocational ficult, as pilot programs are usually small and unlikely to skills) 1,678 realize economies of scale. At the same time, the ques- Source: Authors. tion of whether the pilot program should be scaled up Note: ELA = Employment and Livelihoods for Adolescents; AGEI = arises if the costs per participant are well above those for Adolescent Girls Employment Initiative; NUSAF = Northern Uganda alternatives uses of public funds that are known to have Social Action Fund; NOW = New Work Opportunities for Women); AGI = Adolescent Girls Initiative; EPAG = Economic Empowerment of a high payoff—such as programs that improve access to Adolescent Girls and Young Women). a Does not include project overhead. and the quality of primary education or healthcare. Costs of training in AGI pilots and about US$ 80 per year (median), suggesting that rela- comparator programs tive to the opportunity cost of public funds, it might be Vocational training programs are known to be expen- cost-effective. sive, primarily because of the high cost of facilities, The Liberia program was the most expensive. The equipment, and materials. For example, in Sub-Saharan costs for Liberia, Afghanistan, and Rwanda include the Africa, vocational training is estimated to cost 4–10 times usual pilot startup ones, which necessarily raise cost as much as general secondary school.28 In the experi- per participant in comparison with Nepal. Nonetheless, ment with vocational education vouchers in Kenya, the given Liberia’s low per capita income and high level of cost per participant was about US$ poverty and deprivation, it would 250 (Table 6).29 Each participant in seem that despite EPAG’s obvious ”The life-skills training was crucial the Northern Uganda Social Action successes, the cost per participant to transform my internship…into a Fund (NUSAF) project received a would need to be lowered to scale full-time job. It taught me how to cash transfer of just under US$ 400 up this intervention. The programs interact with colleagues and build to spend on training and business in Afghanistan and Rwanda might new collegial relationships.” startup costs,30 whereas the Jóvenes face a similar dilemma. But these —Participant, Afghanistan projects in Latin America (in countries pilots could reduce costs dramati- more similar to Jordan) cost US$ cally only by cutting out important 600–2,000 per participant.31 Overall, features. Such an effort should have as Table 6 shows, the costs of the AGI pilots were a new impact evaluation to be sure that results were not roughly in this range. The Nepal pilot, which used the compromised. infrastructure of a nationwide program that had already been in operation for several years, recorded the lowest Alternatives to reduce costs costs, providing some indication of the economies of scale. This program increased participants’ incomes by The AGI pilots and other programs reviewed here offer some possibilities for reducing costs. Two alternatives 28 Fox and Filmer (2014). that may be worth testing are voucher programs for 29 Hicks et al. (2013). Note that thus far, the training has not yielded any increased earnings in the treatment group, which does call the program’s vocational training and programs that focus on life skills cost-effectiveness into question. Benefits may accrue later. training alone. 30 Blattman, Fiala, and Martinez (2014). 31 Figure cited in Bandiera et al. (2014:21). 26 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE Just provide vouchers? The pilots as well as the other Additional considerations. Overall, the main problem projects cited here clearly demonstrate that many with evaluating an extensive and comprehensive model low-income countries have training providers that, such as Liberia’s EPAG program is that the value of each if encouraged, could meet the needs of adolescent component is unknown. Nor can the results be com- females, especially if financing can be provided. The pared easily with the results from Uganda’s ELA pro- Ugandan, Kenyan, and Nepalese programs all suggest gram. It is not clear that the situation of older females in that publicly financed voucher programs might be the urban Liberia, who have formed families and who had most cost-effective approach; they cost much less than no access to education because of the civil war, is similar the alternatives. Such programs would provide vouchers to that of adolescent females in rural Uganda, who may that recipients can use to pay for either public or private have access to education, who are not married, and who training programs, and providers would be required to do not yet have children (only 10 percent have children). meet performance targets. The adolescent component The conclusion remains that it is still not clear (i) which would be to make the vouchers available only to vulner- barriers to economic empowerment are most important able young females, or the providers would have to be for which females, (ii) what interventions work best for encouraged or required to recruit females. which barriers, and (iii) in what situations and when in the adolescent life cycle should interventions be offered Focus on life skills only? Uganda’s ELA model, now to have the greatest effectiveness. As a result, it is still under evaluation in South Sudan as well, is a much difficult to choose which intervention will benefit the cheaper option, costing less than US$ 100 per partici- greatest number of females at the lowest cost. pant in Uganda. ELA did not have an extensive voca- While this report has focused on the economic tional or business training program; only 57 percent advancement of individuals—which was the overarch- of the participants took advantage of the vocational ing goal of the AGI pilots—the achievement of other training.32 Instead, ELA focused strongly on life skills, outcomes, including changing gender norms to widen savings, and financial inclusion. It also engaged with opportunities for women, reducing the tendency toward younger females, many of whom were still living at early marriage and fertility, and increasing agency and home. Nonetheless, ELA increased labor force participa- self-esteem, are quite important to such long-term tion, earnings, and household consumption. Whether outcomes as human capital formation and economic this program will have the same effects in the long run as growth. It is difficult to place a monetary value on these the more extensive training programs offered elsewhere outcomes, and to gauge the cost-effectiveness of pro- remains to be seen. Nonetheless, for poor countries, grams in reaching these goals. where most employment opportunities are found in household farms and firms, the ELA model is clearly an option that policy makers could consider. 32 Bandiera et al. (2014).  27 Lessons and Conclusions Adolescent females in lower-income countries face a The AGI pilots in low-income countries economically difficult environment in their path toward economic empowered girls and young women. The strongest empowerment, a critical dimension of adulthood. evidence comes from the two evaluations of experi- Youth in low-income countries rarely see a well-marked mental results that have been completed to date. The path from school into a stable job; they often need to Employment Fund program in Nepal, which targeted make their own livelihood. For young females, con- females (including females from poor and marginal straints such as household responsibilities, lack of access groups) and had majority female participation, signifi- to education and health services, social norms, and lack cantly improved employment and earnings, especially of public safety can lead to more isolation and less of a non-farm employment. The majority had never worked chance to develop the aspirations, skills, networks, and outside of agriculture, and their vocational training confidence needed to make a livelihood. The alternative options spanned professions that were less traditional to economic advancement—early family formation— for women but more remunerative. Liberia’s EPAG pro- reduces opportunities later in life, perpetuating the gram increased participants’ economic activity and earn- cycle of vulnerability in their own families and reducing ings, often significantly, as well as their savings (potential development opportunities for their community and startup capital), and its impacts endured after the train- nation. The case for interventions to break this cycle is ing ended. The incorporation of business skills (even strong. in the vocational training) appeared to have played a But the question remains: What to do, and how? significant role in enhancing participants’ prospects Obviously, the answer will differ depending on the con- for self-employment, the sector where employment is text—labor market and economic opportunities, culture, expanding most rapidly in Liberia. Most notably, the social norms, institutions, household and community participants gained economically even though they were resources, and so forth. The World Bank, its partners, (for the most part) very young mothers, often margin- and others in the development landscape have been ally literate, who had come of age following a period of trying to figure out what works, for whom, how, and how violent conflict. interventions can be scaled up cost-effectively. Thanks to multiple pilots and more rigorous evaluation, a number The only pilot in a middle-income country (Jordan) of lessons (and potentially some answers) are emerg- was not a success. Although 93 percent of female com- ing. This report has primarily focused on the lessons munity college graduates said that they wanted to work from AGI, a World Bank–sponsored initiative, but it has after graduation, increases in labor force participation, brought in other lessons as well. earnings, and hours worked were minimal and insignifi- cant by the time of the endline survey. The reasons for Females, especially from low-income countries, want this failure are unclear and likely to be complex, and to participate in programs to support their economic it remains uncertain what type of program might have empowerment. Girls and young women may be less vis- succeeded. ible members of the community, and they may have few opportunities to express their needs. Even so, they want Effective programs shared certain features that made to overcome the barriers to achieving their economic it possible for them to reach adolescent girls and goals—including their lack of skills, insufficient financial young women and successfully assess and impart the and social assets, the residual effects of conflict, and skills that they needed: social and gender norms. But evidence has shown that unless programs are tailored to their needs, they will not • Effective programs used recruitment strategies be able to participate. that reached the groups they wanted to help. In 28 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE Afghanistan, it is doubtful that “For me, it [training] wouldn’t be provided the tools and confidence the pilot would have succeeded possible [without the stipend]…. for adolescent girls and young without extensive prior consul- Even though I live with my brother, women to take advantage of new tation in the communities to I am an orphan. There is no one economic opportunities. Interviews discuss the pilot and learn how else to help…. The stipend helped with training participants (in South it could be implemented with me to study in good conditions.” Sudan, Jordan, Rwanda, and Haiti, due respect for local sensitivities. —Participant, Rwanda for example) showed that they val- Similar door-to-door commu- ued these skills, although questions nity groundwork was essential remain about which skills contribute in South Sudan. Another approach was taken in most to success in which contexts. Nepal, where technical training providers received • Building social and financial capital. Many pilots incentives to recruit very poor, disabled, and oth- paired instruction in financial management and erwise marginalized young women for the training business development skills with an opportunity programs. to develop savings. To build financial assets— • Effective programs got the incentives right for the which in some cases provided essential startup community, the participants, the training providers, funds that girls and women could protect from and the employers. To allay concerns over safety, their families—trainees in Liberia opened savings the Afghan trainees had dedicated transport to and accounts, Rwandan trainees joined a savings from the training sites, and participants interacted and credit cooperative, and some youth clubs in only with committed employers. Other programs South Sudan formed savings groups. In Lao PDR, offered benefits such as childcare, food and trans- entrepreneurs who did not receive startup grants port allowances, and literacy training. They adjusted were eligible to apply for loans and services from their hours to accommodate trainees’ other com- commercial and private banks. The contribution mitments, including schooling and chores. Technical of savings to young women’s economic agency training providers in Nepal received incentives cannot be understated. The pilots offered safe to deliver marketable skills to places for girls and young women trainees and place them in jobs to expand their social (South Sudan) “I now recognize I am equal to a once training had concluded. and also professional (Afghanistan) man. I now understand that work In Liberia, training providers networks. Lao PDR’s marketplace that was once considered only for received a premium based on competition brought prospective men can be for me too.” job placement results; they also entrepreneurs together and facili- —Participant, Haiti added extra help for the most tated interactions with mentors and educationally disadvantaged financial institutions. participants. These results indicate a way forward but also • Effective programs conducted the preliminary labor demonstrate that much remains to be learned. Many market assessment that allowed them to identify programs are expensive—too expensive to scale up the most promising training opportunities, including with public funds to all needy adolescents. Yet with the non-traditional jobs. The identification of oppor- exception of Jordan, programs were usually adminis- tunities for training in non-traditional occupations tered as a package combining skills training, mentor- proved very successful in some contexts (Haiti, ing, social support, and other features. As a result, it Nepal, Liberia) and less so in others (in South Sudan, is hard to know which components contributed the women who learned to be drivers, for example, most to successful outcomes and for whom. This could not continue to practice their skills if they issue is relevant not only to the AGI programs but to could not find employment). An advantage of the many others, including NUSAF and the Kenya voucher program in Liberia is that it recognized the limita- program. Indeed, a review of the landscape done in tions of private wage employment and provided 2013 concluded that “there is surprisingly little rigorous business development skills even for job skill train- evidence to guide policy makers. This lack of evidence is ees—who thus gained an advantage in starting their especially noticeable in developing countries, where the own businesses. need for effective youth programing is the greatest.”33 • Effective programs made their efforts more success- To tease out these heterogeneous results, and to test ful and strengthened trainees’ resilience by: separately the impact of each component, is a costly • Offering appropriate supplementary skills (busi- ness skills, life skills, or both). Life skills training 33 J-PAL (2013:11). LESSONS AND CONCLUSIONS 29 and time-consuming process. Nonetheless, it is hard not has yet to prove successful elsewhere in Africa, as the to conclude that more research of this type is needed. results from the South Sudan pilot are still to come. The example of BRAC’s ELA One reason why this model may be programs in Uganda is an interest- cheaper is that it reaches females “Before the AGI training I was a ing one. It did not reach all eligible earlier—before they have lost so nobody, I had no confidence, participants in the villages where it many opportunities. This may turn and I did not believe I had was tested; indeed it only reached out to be a lesson. As with most anything to contribute to my 21 percent of those who were aspects of human development, an community. Today I see myself eligible. Nonetheless, for a very low earlier intervention may be cheaper as a valuable member of my cost, it achieved major outcomes. It and better. community with a lot to give.” —Participant, Rwanda 30 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE  31 References Adoho, Franck, Shubha Chakravarty, Dala T. Korkoyah, Mattias Lundberg, and Afia Tasneem. 2014. “The Impact of an Adolescent Girls Employment Program: The EPAG Project in Liberia.” Policy Research Working Paper No. 6832. World Bank, Washington, DC. https://openknowledge.worldbank.org/ handle/10986/17718. Ahmed, Ali , Shubha Chakravarty, Mattias Lundberg, and Plamen Nikolov . Forthcoming. “The Role of Training Programs for Youth Employment in Nepal: Impact Evaluation Report on the Employment Fund.” World Bank, Washington, DC. Bandiera, Oriana, Niklas Buehren, Robin Burgess, Markus Goldstein, Selim Gulesci, Imran Rasul and Munshi Sulaiman. 2014. “Women’s Empowerment in Action: Evidence from a Randomized Control Trial in Africa.” CSAE Working Paper WPS/2014-30. Oxford: Centre for the Study of African Economies (CSAE). Blattman, Christopher, Eric Green, Jeannie Annan, and Julian Jamison. 2013. “Building Women’s Economic and Social Empowerment through Enterprise: An Experimental Assessment of the Women’s Income Generating Support (WINGS) Program in Uganda.” Innovations for Poverty Action, Washington, DC. http://www.poverty-action.org/sites/default/files/wings_full_policy_report_0.pdf. Blattman, Christopher, Nathan Fiala, and Sebastian Martinez. 2014. “Generating Skilled Self-Employment in Developing Countries: Experimental Evidence from Uganda.” Quarterly Journal of Economics 129(2): 697–752. doi: 10.1093/qje/qjt057. Card, D., J. Kluve, and A. Weber. 2010. “Active Labor Market Policy Evaluations: A Meta-Analysis.” National Bureau of Economic Research Working Paper No. 16173. Cambridge, MA. DHS (Demographic and Health Surveys). Various years. Database. ICF International, Rockville, MD. http:// www.measuredhs.com/ Ehlert, C., J. Kluve, and S. Schaffner. 2012. “Temporary Work as an Active Labor Market Policy: Evaluating an Innovative Program for Disadvantaged Youths.” IZA Discussion Paper No. 6670. Institute for the Study of Labor (IZA), Bonn. Elborgh-Woytek, Katrin, Monique Newiak, Kalpana Kochhar, Stefania Fabrizio, Kangni Kpodar, Philippe Wingender, Benedict Clements, and Gerd Schwartz. 2013. “Women, Work, and the Economy: Macroeconomic Gains From Gender Equity.” IMF Staff Discussion Note No. 13/10. International Monetary Fund (IMF), Washington, DC. https://www.imf.org/external/pubs/ft/sdn/2013/sdn1310.pdf. Filmer, Deon and Louise Fox. 2014. Youth Employment in Sub-Saharan Africa. Africa Development Series. Washington, DC: World Bank. doi:10.1596/978-1-4648-0107-5. Galasso, E., M. Ravallion, and A. Salvia. 2001. “Assisting the Transition from Workfare to Work: Argentina’s Proempleo Experiment.” World Bank, Washington, DC. Gertler, Paul J., Sebastian Martinez, Patrick Premand, Laura B. Rawlings, and Christel M. J. Vermeersch. 2011. Impact Evaluation in Practice. Washington, DC: World Bank. Groh, M., N. Krishnan, D. McKenzie, and T. Vishwanath. 2012. “Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment Programs in Jordan.” Policy Research Working Paper No. 6141. World Bank, Washington, DC. Hallman, K., N. Kenworthy, J. Diers, N. Swan, and B. Devnarain. 2013. “The Contracting World of Girls at Puberty: Violence and Gender-Divergent Access to the Public Sphere among Adolescents in South Africa.” Poverty, Gender, and Youth Working Paper No. 25. The Population Council, New York. Hempel, Kevin, and Nathan Fiala. 2011. Measuring Success of Youth Livelihood Interventions: A Practical Guide to Monitoring and Evaluation. Washington, DC: Global Partnership for Youth Employment, World Bank. Hicks, Joan Hamory, Michael Kremer, Isaac Mbiti, and Edward Miguel. 2013. Vocational education in Kenya: Evidence from a randomized evaluation among youth. Processed. Jensen, Robert. 2012. “Do Labor Market Opportunities Affect Young Women’s Work and Family Decisions? Experimental Evidence from India.” Quarterly Journal of Economics 127(2): 535–586. doi:10.1093/qje/ qjs002. 32 THE SPIRIT OF BOLDNESS: LESSONS FROM THE WORLD BANK’S ADOLESCENT GIRLS INITIATIVE J-PAL. 2013. “J-PAL Youth Initiative Review Paper.” Abdul Latif Jameel Poverty Action Lab, Cambridge, MA. Knowles, James C. 2013. “Final Evaluation Report (Phase 1): AGI/STEPS Program.” Report to the EAS Country Units, World Bank. Processed. Levine, Ruth, Cynthia B. Lloyd, Margaret Greene, and Caren Grown. 2009. Girls Count: A Global Investment and Action Agenda. 2nd ed. Washington, DC: Center for Global Development. Maitra, Pushkar, and Subha Mani. 2014. “Learning and Earning: Evidence from a Randomized Evaluation in India.” GCC Working Paper Series, GCC 14-05. http://repository.upenn.edu/gcc_economic_returns/14/. Tripney J., J. Hombrados, M. Newman, K. Hovish, C. Brown, K. Steinka-Fry, and E. Wilkey. 2013. “Technical and Vocational Education and Training (TVET) Interventions to Improve the Employability and Employment of Young People in Low- and Middle-Income Countries: A Systematic Review.” Campbell Systematic Reviews 2013:9. DOI: 10.4073/csr.2013.9. US Census Bureau. International database. www.census.gov. Washington, DC. WDI (World Development Indicators). Database. World Bank, Washington, DC. http://data.worldbank.org/ data-catalog/world-development-indicators. World Bank. 2012a. “Adolescent Girls Employment Initiative: Nepal Baseline Summary Report 2010- 2012.” Washington, DC. http://documents.worldbank.org/curated/en/2012/04/20134117/ adolescent-girls-employment-initiative-nepal-baseline-summary-report-2010-2012. . 2012b. “Labor Markets in Low- and Middle-Income Countries: Trends and Implications for Social Protection and Labor Policies.” Social Protection and Labor Policy Note 7. Washington, DC. http://sitere- sources.worldbank.org/SOCIALPROTECTION/Resources/280558-1334441996287/SPL_Policy_Note_7. pdf. . 2012c. World Development Report 2012: Gender Equality and Development. Washington, DC. https://openknowledge.worldbank.org/handle/10986/4391. . 2013. “Promoting Youth Entrepreneurship and Employment: The Case of Lao PDR.” Adolescent Girls Initiative (AGI) results series. Washington, DC. http://documents.worldbank.org/curated/ en/2013/07/18397990/promoting-youth-entrepreneurship-employment-case-lao-pdr. . 2014a. “In Brief: AGI South-South Learning Exchanges.” Washington, DC. http://www.worldbank.org/ content/dam/Worldbank/document/Gender/AGI%20South-South%20Learning%20Brief_March2014.pdf. . 2014b. “Mentoring for Success in the Labor Market.” Adolescent Girls Initiative (AGI) learning from practice series. Washington, DC. http://documents.worldbank.org/curated/en/2014/09/20250166/ mentoring-success-labor-market.