Case Study September 2023 PHONE-BASED FINANCIAL This case study is a product HEURISTICS TRAINING FOR of the World Bank Africa Region Gender Innovation FEMALE RETAILERS IN ETHIOPIA Lab’s Innovations in Financing Women Entrepreneurs (IFWE) initiative, in partnership with Authors: Girum Abebe, Tsedey Asheber, Adiam Hagos Hailemicheal, Toni Weis the Digital Opportunity Trust. The Africa Gender Innovation SUMMARY Lab conducts impact A recent review of evidence on the impacts of business training indicates the need evaluations and rigorous to develop alternatives to formal classroom training, simpler rules-of-thumb-type research to figure out what trainings for less sophisticated firms, and business trainings that can be scaled up works and what does not to affordably.1 In partnership with ideas42, the World Bank’s Africa Gender Innovation improve gender equality and uses this evidence to shape Lab (GIL) piloted a heuristics (rules-of-thumb)-based business training in Ethiopia policy. Funded by Global that delivers easy-to-remember and -implement financial and business practices. Affairs Canada, IFWE works in The training was designed for small-scale female retailers and is delivered by phone partnership with World Bank using prerecorded messages and interactive voice response technology. A key operations to pilot, scale up, motivation was to make the program accessible to women who are busy tending and evaluate new approaches their families and shops, and who may have limited ability to attend classroom- to empowering Ethiopia’s based trainings. female entrepreneurs. IFWE works across three The financial heuristics training curriculum was previously developed by ideas42 and components: business skills tested in the Dominican Republic, India, and the Philippines.2 To better adapt the and services, access to training content to the Ethiopian context, the materials were refined through an iterative finance, and policy evidence. process involving extensive qualitative diagnostic interviews with a sample of female entrepreneurs in urban areas in Ethiopia. Following the curriculum development, the training was piloted with more than 3,000 female entrepreneurs across five Ethiopian cities. Engagement rates were consistently high over the five-month period, with two-thirds of participants receiving completion certificates after completing at least 14 of 21 lessons. Participation was high in all locations, and women were able to complete the training at a time when the COVID-19 pandemic had shut down most in-person programs. Results from a randomized controlled trial are forthcoming. https://www.worldbank.org/en/programs/africa-gender-innovation-lab MOTIVATION: WHAT IS THE PROBLEM? orientation of the information provided. The program offers Corner shops are a mainstay of African economies. Kenya’s entrepreneurs easy-to-remember strategies on how to dukas, South Africa’s tuck shops, and Ethiopia’s suks not only overcome the problems retail traders face daily, focusing serve their neighborhoods with the necessities of everyday especially on the areas of cash management, customer life; they also provide entrepreneurial opportunities for women. credit, inventory management, and supplier management Recent GIL research shows that retail trade is the sector with (see figure 1). Each of these four modules is further divided the greatest concentration of female owners, many of whom into multiple units (each lasting 3–5 minutes), for a total of lack the capital necessary to start a business in other, more 21 lessons. The program is narrated by a main character, profitable sectors.3 Samrawit, a retail business owner whose language and experiences are relatable to her audience, and who is voiced Traditional, classroom-based business trainings often fail by a professional voice actor. to serve the needs of these women, for three reasons. First, traditional business-training curriculums, which tend to focus The training content is delivered to participants using heavily on finance and accounting, are often overly complex interactive voice response (IVR) messages. Trainees receive and of limited relevance to small-scale shopkeepers. Second, phone calls with the prerecorded lessons and navigate menu delivering in-person training is expensive—prohibitively so, options or answer questions using their phone’s keypad. given the limited size of the average firm. Third, and just as For greater flexibility, participants are able to preselect the important, retail traders typically find it difficult to close their times at which they receive training calls and can use a shops for extended periods of time to attend offsite training. call-back feature to listen to or revisit previous messages. This constraint is especially severe for women, who tend to face IVR delivery significantly increases the possible reach of the competing demands on their time from family and household program compared with in-person training and e-learning duties. platforms, both of which require the use of a smartphone or personal computer with internet connectivity. It also raises Unsurprisingly, the results of business-training interventions the potential to scale up the training, as a basic-feature are often underwhelming. Evidence on the impact of phone is the only equipment required and the marginal cost entrepreneurship education in general is mixed, with limited for additional participants is low. effects overall and strong variation across approaches and target audiences.4 Programs focusing on financial skills fare little better. A meta-analysis of 201 studies showed that FIGURE 1: KEY TOPICS IN THE FINANCIAL efforts to improve financial literacy explain only 0.1 percent HEURISTICS CURRICULUM of the variance in the financial practices of participants, with even more limited impact for low-income samples.5 Such low Cash separation effects are often driven by limited take-up and high numbers of dropouts, which is especially problematic given the high cost • Maintaining separation between household and business cash of training per student. Moreover, a GIL study from Ethiopia suggests that participation in time-intensive training programs is • Setting business salary disproportionally driven by less-effective female entrepreneurs, • Measuring business profit with growth-oriented business owners preferring to spend their time improving their business.6 Customer credit • Identifying when to give customer credit FINDING A SOLUTION • Recording customer credit GIL partnered with behavioral design experts at ideas42 to • Tracking and updating decisions on customer develop a training that better serves the needs of female credit retail traders in Ethiopia. The result is a financial literacy program that simplifies business and financial management Inventory management lessons into concise and actionable heuristics (or rules of • Choosing items to stock thumb) and that can be delivered at the clients’ convenience through their mobile phones. Adapted from content created • Determining frequency of stocking items by ideas42 and its academic partners at the Massachusetts • Visiting and tracking competitors Institute of Technology and Harvard Business School, the Ethiopia financial heuristics program builds on previous trials Supplier management in the Dominican Republic, India, and the Philippines. • Comparing options Both the curriculum and its delivery mechanism set the • Negotiating prices program apart from traditional entrepreneurship training. • Financing supply purchases Behavioral science shows that heuristics are easier to act on than abstract concepts, given the simplicity and action IMPLEMENTATION texts afterwards. Participants were given a choice of different time slots—including times outside normal working hours—to GIL and ideas42 piloted the financial heuristics training with accommodate their schedule. Viamo implemented a call retry 3,000 women from five Ethiopian cities between March and pattern, which ensured that participants who did not answer a September 2021.7 To evaluate the impact of the training, an call would receive two more call attempts; the service provider additional 3,000 women were randomly assigned to a control also provided a callback feature for those who missed the call group.8 Participants were recruited from among the registered or wanted to listen to the week’s lesson again. In addition, the members of the Women Entrepreneurship Development WEDP program office offered a helpline for participants, and Program (WEDP), a longstanding government project financed staff were given a question-and-answer guide to help resolve by the World Bank that provides access to loans and business issues. Lastly, participants were told they could earn a WEDP development. In addition to shopkeepers, participants included training certificate by completing more than two-thirds of the owners of small service businesses such as cafes and beauty training lessons. salons. The average participant was 38 years old, had 10 years of experience operating a business, and employed one nonfamily staff member. About half described themselves as “necessity entrepreneurs,” the other half as opportunity- KEY TAKEAWAYS oriented business owners. Completion rates for phone-based training can To tailor the training to the Ethiopian context, ideas42 staff be as high as for in-person training—or higher. conducted interviews with 50 entrepreneurs in the retail sector. With the results, they identified common challenges, defined Across the 21 lessons, an average 82 percent of participants training needs, and integrated local language and business- picked up the scheduled calls. Pick-up rates in Ethiopia were related content into the lesson scripts. The interviews also slightly higher than during previous iterations of the program provided an assurance that mobile training delivery through in India (79 percent) and the Philippines (73 percent). Among IVR would be feasible, as all respondents owned phones and those who answered a call, 84 percent listened to the lesson used them for business purposes. The curriculum was tested in its entirety and 63 percent completed the quiz. Engagement by 40 entrepreneurs and further refined by incorporating their was at its highest during module one and predictably dropped feedback. An additional training trial with 100 business owners over time; however, the average pick-up rate never fell below 75 showed high participation rates, indicating the effectiveness of percent and completion rates hovered between 60 percent and IVR messages. 80 percent during modules two through four (the four modules comprise 21 lessons, delivered over the course of 21 weeks). Prior to launching the training, individual onboarding sessions At the end of the five-month period, two-thirds of training were conducted over the phone to clarify training expectations, participants had completed at least 14 of the 21 lessons, making explain IVR functionality, note the preferred call times, and them eligible for a WEDP training certificate. This completion provide access to support resources. The training was rate is at the upper end for in-person training programs offered implemented using the mobile service provider Viamo. Given the women’s preexisting relationship with WEDP (all had previously under WEDP; the typical range is 40–52 percent. registered to receive WEDP services at a local WEDP office), all communication with the program participants stated that Participants did not use the interactive quiz feature the training was being offered on behalf of WEDP, to leverage much. this preestablished trust and credibility. Almost all (98 percent) Despite its name, interactive voice response functionality in a participants attended at least one lesson. Approximately 90 phone-based training program offers limited opportunities for percent of participants answered the first call. Minimal drop- actively engaging participants. The Ethiopia financial heuristics off occurred between the first and second lessons, indicating training introduced a multiple-choice comprehension question that trainees grasped how to interact with the digital training. Detailed monitoring data on participant engagement, including at the end of each lesson that participants could answer by pick-up rates, lesson completion, quiz participation, and correct typing on their number pad. However, there was significant answers, was collected and analyzed through an engagement drop-off before the comprehension question, with only 63 monitoring dashboard. percent of participants who completed the lesson choosing to submit a response. The results indicate some knowledge The program team introduced several features to increase retention, as four out of five participants who answered the engagement and completion, some of which had not been comprehension question gave the correct response. However, part of ideas42’s previous iterations of the program in the they also suggest that IVR-based training has limitations where Dominican Republic, India, and the Philippines. Entrepreneurs participant interaction or critical engagement with the training received text reminders before each lesson, as well as summary content are key to learning outcomes. Training delivered by phone can reach women who find it difficult to attend classroom-based programs. An initial screening of eligible WEDP entrepreneurs showed that all the women owned either a basic-feature phone or a smartphone. In contrast to previous iterations of the program, all aspects of the Ethiopia financial heuristics training—recruitment, onboarding, baseline survey, content delivery, and technical support—were delivered by phone, making participation available to everyone regardless of location. Participation was high outside the capital, where resources to support entrepreneurs tend to be concentrated; completion rates were highest among women in Dire Dawa. Results showed consistent engagement across sectors, ranging from retail traders to restaurants and other service businesses. Crucially, phone-based delivery allowed the program to go ahead during the COVID-19 pandemic, when classroom-based training was suspended because of social distancing rules. Building in redundancy is key to delivering a 21-week program consistently. The low-income, low-technology environments in which phone-based training is most appealing are also the ones most prone to disruption. This was certainly the case in Ethiopia, where in addition to a global pandemic the project faced political conflict and network disruptions. As a result, participants did not always receive calls at the expected time or on the scheduled day. The call-back option thus proved useful and highly popular: 99 percent of participants used it at least once. When possible, the project team tried to anticipate service outages and alert participants via text message to adjustments in the training schedule. Monitoring participant engagement and staffing the WEDP helpline were additional measures to minimize disruption and adapt project delivery. ACKNOWLEDGMENTS This work has been supported through generous contributions from Global Affairs Canada, in partnership with the Umbrella Facility for Gender Equality (UFGE). UFGE is a multidonor trust fund administered by the World Bank to advance gender equality and women’s empowerment through experimentation and knowledge creation that help governments and the private sector focus policy and programs on scalable solutions with sustainable outcomes. UFGE is supported with generous contributions from Australia, Canada, Denmark, Finland, Germany, Iceland, Ireland, Latvia, Netherlands, Norway, Spain, Sweden, Switzerland, United Kingdom, the United States, the Bill & Melinda Gates Foundation, and the Wellspring Philanthropic Fund. For their contributions to and participation in this work we would also like to acknowledge Yohannes Solomon at the Ethiopian Entrepreneurship Development Institute; Mengistu Bessir and Marlon Rawlins at WEDP; Rachel Coleman, Niklas Buehren, and Yemsrach Kinfe at GIL; Lois Aryee, Manasee Desai, and Pranav Trewn at ideas42; and Brook Ashinne, Kellen Eilerts, and Priscilla Obeng at Viamo. FOR MORE INFORMATION, PLEASE CONTACT 1 David McKenzie, “Reassessing the Evidence for ‘Business Training Doesn’t Work,’” World Bank (blog), September 21, 2020, https://blogs.worldbank.org/impactevaluations/reassessing-evidence-business-training-doesnt-work. Girum Abebe Tefera 2 For further information, see https://www.ideas42.org/project/financial-management-training-mobile-phones. gtefera@worldbank.org 3 Naira Kalra et al., Breaking Barriers: Female Entrepreneurs Who Cross Over to Male-Dominated Sectors (Washington, DC: World Bank, 2021), https://openknowledge.worldbank.org/server/api/core/ Tsedey Asheber bitstreams/7031b50b-f682-52f5-9a3a-24dc46c77942/content. tasheber@worldbank.org 4 David McKenzie and Christopher Woodruff, eds., “Training Entrepreneurs: Issue 2,” VoxDevLit (2021), https://voxdev.org/sites/default/files/Training_Entrepreneurs_Issue_2.pdf. Adiam Hagos Hailemicheal 5 Daniel Fernandes et al., “Financial Literacy, Financial Education, and Downstream Financial Behaviors,” ahailemicheal@worldbank.org Management Science 60, no. 8 (2014), https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2013.1849. 6 Salman Alibhai et al., “Full Esteem Ahead? Mindset-Oriented Business Training in Ethiopia,” Policy Research Toni Weis Working Paper 8892, World Bank, Washington, DC, August 4, 2019, https://papers.ssrn.com/sol3/papers. tweis@worldbank.org cfm?abstract_id=3430501. 7 The five cities participating in the pilot were Adama, Addis Ababa, Bahir Dar, Dire Dawa, and Hawassa. 1818 H St NW 8 A follow-up survey was conducted in 2022; results are forthcoming. Washington, DC 20433 USA www.worldbank.org/africa/gil Photo credit: All photos credited to World Bank.