Report No: AUS8941 . Republic of Yemen BDS Grants Impact Evaluation The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen . February 5, 2016 . GTC05 MIDDLE EAST AND NORTH AFRICA . . . Standard Disclaimer: . This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. . Copyright Statement: . The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978-750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. 2 The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen# David McKenzie, World Bank Nabila Assaf, World Bank Ana Paula Cusolito, World Bank Abstract Matching grants are one of the most common types of private sector development programs used in developing countries. But government subsidies to private firms can be controversial. A key question is that of additionality: do these programs get firms to undertake innovative activities that they would not otherwise do, or merely subsidize activities that would take place anyway? Randomized controlled trials can provide the counterfactual needed to answer this question, but efforts to experiment with matching grant programs have often failed. This paper uses a randomized controlled trial of a matching grant program for firms in the Republic of Yemen to demonstrate the feasibility of conducting experiments with well-designed programs, and to measure the additionality impact. In the first year, the matching grant is found to have led to more product innovation, firms upgrading their accounting systems, marketing more, making more capital investments, and being more likely to report their sales grew. Keywords: matching grant; additionality; firm subsidy; fragile states; spurring innovation. JEL codes: O14, H25, D22, C93 1. Introduction Matching grants consist of partial subsidies (typically 50 percent) provided by government programs to try to spur firms to undertake innovative activities such as creating new products, technological upgrading, exporting, using business development services, and more broadly, firm expansion. They are one of the most common policy tools used in private sector development projects in developing countries, and have been included in more than 60 World Bank projects totaling over US$1.2 billion, funding over 100,000 micro, small and medium enterprises.1 Yet credible evidence on whether these grants induce firms to undertake innovative activities that they would not otherwise do (additionality) is limited: there are several case studies and non-experimental evaluations2 that are subject to the concern that firms that self-select into such programs may differ in unobservable dimensions from those not selected for funding, while a number of recent attempts to conduct randomized experiments to evaluate these programs in Sub-Saharan Africa have failed for reasons such as lack of applications or governments # We thank the Umbrella Facility for Gender Equality (UFGE) Trust Fund for supporting this work, Sami Sofan for his work supporting this project, and Mohamed Raouda for research assistance. 1 Data from a World Bank Latin American and the Caribbean overview available at http://go.worldbank.org/OVDGTHSWY0. 2 See Biggs, 1999; Phillips, 2001, 2002; Castillo et al., 2011; Crespi et al., 2011; Gourdon et al., 2011, and Lopez-Acevedo and Tan, 2011. 3 changing their minds on random selection (Campos et al, 2014). The one exception is work by Bruhn et al. (2012) on a matching grant program in Puebla, Mexico, that had a higher than usual match rate (70 to 90% subsidy) for consulting services, and found firms receiving the subsidy did not significantly innovate more, but did do more marketing, improved their accounting, and increased sales and profits. A potential rationale for these subsidies is that owners of small firms may under-invest in learning and innovation, either because they under-estimate the gains from using business development services, because these gains spill over to other firms, or because of credit market imperfections. In particular, two characteristics of innovation make financing more difficult: (i) innovation produces an intangible asset, and (ii) the returns to innovation investment are uncertain (e.g. Hall and Lerner (2009) and Kerr and Nanda (2014)). Reluctance to make such investments may be particularly high in fragile and conflict-afflicted states, in which market failures may be more pervasive and the underlying uncertainty increases the option value of waiting to make investments (Bernanke, 1983; Dixit and Pindyck, 1994). Matching grants may help firms overcome this reluctance and spur new activity, or conversely, these other factors may limit the demand for such programs only to the few firms planning on undertaking these activities anyway, in which case the subsidy would not have any additionality. To date there is no evidence from fragile states as to which channel dominates. This paper reports on a randomized experiment testing the impact of matching grants for business development services in the Republic of Yemen. The program provided firms with a matching grant of up to $10,000 as a 50 percent subsidy towards the cost of business services like finance and accounting systems, website creation, training, marketing, participation in exhibitions, and some associated goods such as office and IT equipment. The program was intended to run in two annual calls, but the second year was canceled due to the outbreak of civil conflict in 2015. We therefore report on the randomized experiment run on the first round. The results show that such programs can be designed in a way that results in many firms applying, randomization is feasible, and that the program does appear to result in additional innovation and firm growth. Building on lessons from past matching grant programs, the program was designed to make it easy for firms to apply, and approximately four times as many eligible firms ended up applying as there was funding available. A public randomization ceremony proved an effective and transparent way of choosing among eligible applicants. A follow-up survey was conducted in March 2015 just as civil conflict was beginning to break out. Although this survey had relatively high attrition (45%), the sample interviewed remain balanced between treatment and control on observed characteristics, and the results show that the matching grant led to firms being 37 percentage points more likely to engage in innovation, introducing more new products, carrying out more marketing, being more likely to introduce a new accounting system, training more workers, making more capital investments, and being 48 percentage points more likely to say their sales grew over the past year. These results suggest that additionality was a result of this matching grant program. The remainder of the paper is structured as follows: Section 2 discusses the context and details of the matching 4 grant program; Section 3 describes the randomization process and take-up of the program; Section 4 provides the results of the impact evaluation, and Section 5 concludes. 2. Context and Details of the Program The Republic of Yemen is the poorest country in the Arab world, with a per-capita GDP of US$1,473 (2013). As in most developing countries, micro and small enterprises constitute the majority of private firms, with 88 percent of firms estimated to have less than five workers, 4.7 percent between 5 and 9, and only 3.5 percent having more than 10 workers. Smaller firms tend to be less productive, and may face more constraints to taking up technology and innovation. In the two years preceding the launch of the program studied here, the Republic of Yemen had undergone a revolution as part of the Arab Spring, marked by protests, armed clashes, and a contraction of the economy. Real GDP shrank 12.7 percent in 2011, and only recovered 2.4 percent in 2012. In a survey of 200 firms in June, 2012, one year before the matching grant program was launched, 72% of firms surveyed reported declining sales, and there was a 12% average decline in employment across firms surveyed. Most firms (about 4 out of 5), reported availability of electricity, macroeconomic instability, and political instability as major or severe constraints to business (Stone et al, 2012). Coupled with an already existing difficult business environment, and productivity levels which are among the lowest in the region,3 these economic conditions made it particularly difficult for businesses to develop and grow and motivated the project here. 2.1 The Matching Grant Program The Enterprise Revitalization and Employment Pilot (EREP) was designed as a two year pilot project aimed at improving firm capabilities and the employability of recent graduates. The matching grant component of this program aimed to provide grants for business development services (BDS) to 400 enterprises, 200 in each of two years.4 Firms could apply for a grant of up to $10,000, as a 50 percent subsidy towards the cost of procurement of business development services, training, and goods (as a minority component) to improve business practices, technology, or products, or to reach new markets, as defined in a Business Development Plan submitted by the firm. The program implementation was designed with the lessons of other matching grant programs in mind, many of which had struggled to get sufficient applicants due to issues such as too strict eligibility criteria, complicated application procedures, political capture, and insufficient advertising (see Campos et al, 2014). In particular, the focus on small and medium enterprises was achieved through the setting of a modest grant ceiling; the project did not limit eligibility by firm size. The eligibility criteria were simple, requiring that the firms must be operating in Yemen, have offices in either Sana’a or Aden, be in operation for at least 6 months, not be in 3 Compared with Lebanon, Egypt, Saudi Arabia and Syria – see Yemen Investment Climate Assessment Update, World Bank, 2011. 5 a prohibited activity (such as weapons manufacture), and have submitted a complete application form. In order to make it easy for firms with limited access to the Internet to apply, applications were accepted either online or on paper. The application form was reasonably short, and collected basic information on the firm such as employment level, sales, whether they export, types of innovative activities conducted in the past three years, and then basic information on what business development services they would like and an agreement to pay half the costs of these. The program was launched in August 2013 (see timeline, appendix 1), and widely advertised through a 45 day campaign that included a project website, advertisements on the Yemeni official television channel and on radio, in many local newspapers, on Facebook and Google Ads, and through workshops and events held at local universities, hospitals, and business organizations. BDS Advisors hired as consultants to the program also visited firms in person to promote the program and attract applications. Firms could apply between October 1, 2013 and December 26, 2013. They were then subjected to a simple screening process to ensure they met the eligibility criteria. 2.2 Context: Conflict Resumes The first year of the program took part in 2014, with the matching grants implemented as described below. However, in August 2014, part-way through this process, the rebel Houthis began demonstrations in Sana’a against increased fuel prices, and in September they took control of the city of Sana ’a. A U.N.-brokered peace agreement was made in which they agreed to withdraw once a national unity government was formed. The situation worsened at the start of 2015, with the Houthis seizing control of the state television, President Hadi resigning and fleeing to Aden, and civil conflict breaking out in late March and early April 2015. As a result of these events, the second year of the program was canceled, and as a result, this analysis focuses on the first year of the program. 3. Randomization, Take-up, and Data Collection A key issue in a number of other matching grant programs has been a lack of take-up for the program, with the small number of applications received dooming plans for experimental analysis based on oversubscription designs (Campos et al, 2014). The combination of an easy application process and the need among firms for the program meant that this was not an issue in Yemen. In total 820 applications were received (53% from Sana’a, 47% from Aden). This was slightly more than four times the number of grants available for the first round (200). Nineteen applications were rejected due to firms not being eligible for the program or not agreeing to cover their share of the costs. 3.1 Random Selection Firms were selected for the program from among the eligible applicants in public randomization events held in 4 Firms could also apply for subsidies to hire interns, with some firms applying only for these internship subsidies, some applying for the BDS grants, and some applying for both. 20% of the firms in our treatment group also applied for interns. We discuss the impact of the internship program on youth employment in Assaf et al. (2015). 6 Sana’a on January 9, 2014 and Aden on January 12, 2014. The only variable used in stratification was city, with separate drawings occurring in the two cities. The events were open to the public, and attendees included some of the applicants, members of the project’s Advisory Committee, and media including television. In each city 100 firms were randomly drawn, along with a reserve list in case any of the originally selected firms withdrew from the program. In the end this resulted in 216 firms being randomly assigned to the treatment group. We choose a random sample of 100 firms from the eligible applicants in each city to be the control group for follow-up. Table 1 compares the baseline (application form) characteristics of the firms in the treatment and control groups. The firms are quite heterogeneous, ranging in size from the smallest firm having no paid employees to one firm with 950 employees. The median firm has 5 employees, with a mean of 14.8 and standard deviation of 53. One-third of firms report no sales at baseline, which may be missing data in many cases. Conditional on positive sales, the median firm has annual sales of 20,000 rials ($93) and the mean $18.9 million rials ($88,000), with a standard deviation of 143 million rials. This heterogeneity (along with item non-response) severely limits the statistical power of this experiment to measure impacts on financial outcomes and employment, and as a result we will focus largely on binary outcomes. Table 2 provides a tabulation of the types of firms in terms of industry sector. We again see substantial heterogeneity, with the most common firm types being retail sales, selling and producing food, clothing, education and training, and computer services. The firms are almost entirely domestically focused (only 2 percent export), are mostly owned by men, and the majority have not been undertaking many innovative activities in the three years prior to application: only 16 percent had improved a product line, only 2 percent had any type of quality certification like ISO, 24 percent had trained workers, 6 percent had attended an international exposition or fair, and 13 percent had expanded their market. Random selection ensures that the treatment and control samples look similar on average: a joint test of orthogonality cannot reject the null hypothesis of balance (p = 0.318). 3.2 Take-up Firms which were selected in the random draw were invited to workshops in February in which they were told about the procedures for procurement, and the process for filling out a business development plan (BDP). The plan set out what the firm planned to do with the matching grant, and was reviewed by a project management committee for approval. These BDPs were reviewed as received between March and June 2014, and once approved, agreements were signed. Firms were then reimbursed for the agreed services upon completion of the consulting services and providing evidence of payment.5 Figure 1 shows the main services requested by firms. The most popular requests were for help in marketing, accounting, and worker training. According to administrative records, 170 firms had signed agreements by October 30, 2014. However, a number of these firms dropped out as a result of concerns about the unstable security situation, and a fuel crisis meaning 7 that they did not want to take the risk of paying 50 percent of the cost of the planned activity. According to the follow-up survey, 57.8 percent of the treatment group used the matching grant in the end. Table 3 compares the baseline attributes of those firms which did take up the matching grant to those in the treatment group which did not, for the sample answering the follow-up survey. We see that the two groups are not statistically different in most characteristics. This in part reflects the relatively small sample size, but it is not the case that those who take the grant are large firms while those who drop out are small firms for example. Those who take-up the grant are more likely to have expanded their business in the three years prior to their application, but no more likely to have increased employment, done new marketing, or improved their product during this time. A probit of take-up on these observable characteristics drops having attended an international expo since none of the drop-outs had done this, and having exported, but then cannot reject that the other variables jointly cannot predict take-up. This is consistent with take-up decisions being driven by idiosyncratic factors arising from the general economic environment, rather than sorting the firms according to their inherent productivity or innovativeness. 3.3 Data Collection A call for the second year of the program took place in November 2014, with random selection in December 2014. However, with the worsening security situation in Yemen this second year of the program was abandoned. We therefore quickly launched a follow-up survey in March 2015. This timing is approximately 4 to 10 months after firms had used their matching grants, and right before widespread civil conflict broke out. The survey was conducted as a phone survey given that safety issues, gas shortages, and time constraints prohibited an in-person survey. The survey collected basic company information such as operating status and employment, the firm’s innovation activities and use of consultants, financing from banks and the grant program, and on the firms’ exports and sales. Like most surveys in developing countries of firms this size we must rely on self-reporting of these activities, lacking any alternative sources of data with which to cross-check these responses. The survey was conducted by Apex consulting, and succeeded in interviewing 226 of the target 416 firms (a response rate of 54.3%). Attrition was higher in the control group (51%) than the treatment group (40.7%), with this difference statistically significant (p=0.036). Attrition rates did not vary significantly by city. Of those firms interviewed, 97.8 percent had survived (4 treated and 1 control had closed). It seems likely that those who had closed because of fuel shortages or safety concerns would be more likely to have attrited. This response rate and its difference by treatment status is a concern, although given the circumstances with suicide bombings and airstrikes beginning at the end of March (appendix 1), no further actions could be taken to chase down non-responders. Given the lack of evidence on the performance of matching grants in general, and especially in fragile states, we believe it is still worth reporting on what we can learn from these data. The last 5 In some cases, particularly for very small firms or start-ups, direct payments were made to vendors for the balance of the costs once the firm provided evidence of payment of 50%. 8 three columns of Table 1 compare the baseline application characteristics for the treatment and control samples that did respond to the survey. We see that these characteristics look similar for most variables to the full sample in the first three columns, and more importantly, are similar for treatment and control. We cannot reject the null hypothesis of balance on these observables with a joint orthogonality test (p=0.863). Typically we might expect grants and training to enhance the survival prospects of firms that would otherwise close down (e.g. de Mel et al, 2014; McKenzie, 2015). If this is the case, this would suggest that any bias from differential response would be in the direction of the additional respondents in the treatment group being less innovative firms that would have closed (and therefore attrited) if they had been in the control group – in which case our estimates could be lower bounds on the program effects. As a final check on robustness, in appendix 2 we construct Lee (2009) lower bounds and find that even in this case, the program had a significant impact. 4. Results We have seen from Figure 1 that the main uses firms requested the matching grants for were for implementation of accounting systems, marketing plans, and training. When asked directly in the follow-up survey whether they would have undertaken these activities without the grant (i.e. about additionality), 43 percent of those using the grant say they would not have done so, 41 percent say they would have done so at a smaller scale, and 16 percent say they would have done the same activity anyway. We use our experiment to provide more rigorous measurement of the additionality of the program. We estimate the impact of being selected for a matching grant (the intention to treat, ITT) effect by estimating the following equation for outcome Y and firm i: = + + (1) Where Ti is an indicator for being randomly selected for a grant, and Aden is a dummy variable for the city of Aden to account for the stratified randomization. To show robustness to concerns that attrition may have resulted in imbalance between the treatment and control groups, we also re-estimate equation (1) controlling for the set of baseline variables in Table 1. Since we have seen that the two groups in fact seem balanced on these characteristics, we should expect that adding these characteristics will not change b substantively. In addition, we estimate the impact of actually receiving and using the matching grant (the local average treatment effect, LATE) by instrumenting the receipt of treatment with assignment to treatment. Since none of the control group received a grant, the LATE is also the treatment-effect-on-the-treated (TOT). Given the logistical and time constraints on fielding the follow-up survey, we were limited in the number and complexity of questions we could ask. In addition, as noted above, the heterogeneity of firms limits our power for examining impacts on financial outcomes. We did try asking about sales at the end of the survey, but had 51 percent item-non-response amongst the firms that did answer the survey. As a result, we largely examine binary outcomes that help address the issue of additionality: did the grants get firms to undertake more innovative activities than they otherwise would have done, or did the grants just subsidize firms to do what they were going 9 to do anyway? 4.1 Impacts on Innovative Activities Table 4 examines the impact of the matching grant on the likelihood the firm undertook different innovative activities. Panel A presents the ITT impacts with only a control for city (the randomization strata) and Panel B the same impacts after controlling further for the range of observable baseline characteristics in Table 1. We see the two rows yield very similar results in terms of magnitudes and statistical significance, which is in line with the evidence presented above that the follow-up sample appears balanced on baseline characteristics. Panel C then presents the LATE estimates. Column 1 shows that the matching grants resulted in a sizeable and statistically significant increase in product innovation: firms receiving the grants were 30.3 percentage points (p.p.) more likely to introduce a new product, which is more than a doubling of the 26.5 percent rate in the control group. Column 2 shows the impact on process innovation is positive, but not statistically significant. Columns 3, 4 and 5 show that in accordance with what firms reported wanting the grants for, they do more marketing (19.8 p.p. increase), introduce new accounting systems (48.5 p.p. increase), and do more worker training (33.2 p.p. increase) with the grants – in each case approximately doubling the control group rates. We saw at baseline that it was incredibly rare for firms to get quality certification, and Column 6 shows that this continues to be the case, with only 4 percent of the control group getting a new quality certification and the matching grants having no impact on this rate. Column 7 shows those with a matching grant are 27.4 p.p. more likely to introduce a new human resources (HR) system. Column 8 shows a positive, but not statistically significant, increase in the likelihood of attending a trade show abroad. We consider two aggregate measures of innovative activities as a way of summarizing this information and dealing with multiple hypothesis testing. Column 9 aggregates to form the number of activities, while Column 10 examines whether any innovative activity was undertaken. We see strong and statistically significant impacts on both measures. Firms receiving grants are 37.1 percentage points more likely to have undertaken an innovative activity, and take 1.8 innovative activities more than the control group on average. Taken together these results indicate that the matching grant program did have additionality in terms of getting firms to innovate more. 4.2 Impacts on Firm Inputs and Growth We then turn in Table 5 to examining the extent to which these activities increased firm inputs and firm outcomes. A key additional caveat here is that some of the innovative activities pursued by the firm with the grant may take some time to ultimately affect firm growth. For example, in their study of matching grants for consulting in Mexico, Bruhn et al. (2012) find no impact on employment in the first year, but that administrative data appear to show employment increases three years later. A second caveat here is that the instability in Yemen may have caused firms to hesitate in hiring additional workers, as well as limited the scope for firms to grow. Columns 1 and 2 of Table 5 show that there is a negative, but not statistically significant, impact on permanent and temporary workers. Recall that the sample is rather heterogeneous in terms of initial employment size, and as a result, our power is low to detect changes in employment: a 95% confidence interval for the LATE impact on 10 permanent employees is (-22, +14) relative to a control group mean of 11.6. So the data are largely uninformative about the employment impact. Column 3 shows that firms receiving the matching grant were substantially more likely to make a new capital investment that cost more than US$1000.6 Just over one-third of the control group (34.7 percent) had made such an investment, and the point estimate suggests 97 percent of those receiving grants made such an investment. This is consistent with some use of the matching grants being for capital investments to accompany the services provided. Column 4 shows that firms receiving the grants are also considerably more likely to have used a consultant. Columns 5, 6 and 7 show no significant impacts on the likelihood the firm has a website, receives a new loan, or exports. The heterogeneity in sales and high item non-response for a question on the volume of sales limits our ability to measure the return to the matching grant in terms of amount of sales. However, our questionnaire did ask firms how their sales in 2014 compared to their sales in 2013. 31.6 percent of the control group said their sales had grown over the year. The last column of Table 5 shows that recipients of the matching grant are 47.7 p.p. more likely to say their sales grew during 2014. This provides some evidence to suggest that the new products introduced and other innovative activities did translate into an improvement in firm outcomes in the first year. 5. Conclusions Matching grants have been widely used by governments to attempt to spur private sector innovation and growth. However, it has proved difficult to assess the impact of these programs, and a key concern has been whether they create additionality, resulting in benefits beyond those which the private sector would have otherwise produced even in the absence of the program (Phillips, 2001). This paper demonstrates the feasibility of a randomized evaluation of a matching grant program, and finds strong evidence that it has generated additional innovative activities. It provides the first such evidence from a fragile state environment, showing that even under these difficult circumstances, a well-designed program can succeed in getting firms to innovate. There are several caveats that must be taken into account when interpreting the results of this paper. The first is that the eruption of civil conflict in Yemen led to higher than usual attrition in our follow-up survey. The data available suggest that this is not greatly biasing our results. Second, the conflict also means that we cannot say anything about the long-term impacts of these grants. Third, the fact that the second year of the grants program was cancelled halved our sample size from what was originally planned, which together with the heterogeneity in the sample of applicants, means we have no statistical power to examine impacts on employment or sales levels, nor to examine heterogeneity of impacts with firm characteristics. Finally, while our analysis addresses the issue of additionality, we are not able to measure whether these grants result in externalities for other firms in the economy. Nevertheless, given the popularity of these programs and the dearth of evidence on their effects, we believe the results are a useful first step in assessing the effectiveness of matching grants. 11 References Assaf, Nabila, Ana Paula Cusolito and David McKenzie (2015) “The Demand for, and Impact of, Youth Internships: Evidence from a Randomized Experiment in Yemen”, Mimeo. World Bank. Bernanke, Ben (1983) “Irreversibility, Uncertainty, and Cyclical Investment,” Quarterly Journal of Economics, 98(1): 85–106. Biggs, Tyler (1999) “A microeconomic evaluation of the Mauritius Technology Diffusion Scheme (TDS)”, Regional Program on Enterprise Development Discussion Paper no. 108. Bruhn, Miriam, Dean Karlan and Antoinette Schoar (2012) “The impact of consulting services on small and medium enterprises: Evidence from a randomized trial in Mexico”, Yale Economics Department Working Paper no. 100. Campos, Francisco, Aidan Coville, Ana Fernandes, Markus Goldstein and David McKenzie (2014) “Learning from the experiments that never happened: Lessons from trying to conduct randomized evaluations of matching grant programs in Africa”, Journal of the Japanese and International Economies, 33: 4-24. Castillo, Victoria, Alessandro Maffioli, Sofá Rojo and Rodolfo Stucchi (2011) “Innovation policy and employment: Evidence from an impact evaluation in Argentina”, IDB Working Paper no. IDB-TN-341 Crespi, Gustavo, Alessandro Maffioli, and Marcela Melendez (2011) “Public support to innovation: The Colciencas experience”, IDB Working Paper no. IDB-TN-264 De Mel, Suresh, David McKenzie and Christopher Woodruff (2014) “Business Training and Female Enterprise Start-up, Growth, and Dynamics: Experimental evidence from Sri Lanka”, Journal of Development Economics, 106: 199-210. Dixit, Avinash and Robert Pindyck, (1994) Investment under Uncertainty, Princeton, New Jersey: Princeton University Press). Gourdon, Julien, Jean Michel Marchat, Siddharth Sharma and Tara Vishwanath (2011) “Can matching grants promote exports? Evidence from Tunisia’s FAMEX II program”, pp. 81-106 in Olivier Cadot, Ana Fernandes, Julien Gourdon and Aaditya Mattoo (eds.) Where to spend the next million? Applying impact evaluation to trade assistance. World Bank: Washington, D.C. Hall, Bob, and Josh Lerner (2009). “The Financing of R&D and Innovation.” NBER Working Paper #15325. Kerr, William, and Ramada Nanda (2014). “Financing Innovation.” Harvard Business School, Working Paper 15-034. Lee, David (2009) “Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects", Review of Economic Studies 76(3): 1071-1102 Lopez-Acevedo, Gladys and Hong Tan (2011). Impact evaluation of small and medium enterprise programs in Latin America and the Caribbean. The World Bank. McKenzie, David (2015) “Identifying and Spurring High-Growth Entrepreneurship: Experimental Evidence from a Business Plan Competition”, World Bank Policy Research Working Paper no. 7391 Phillips, David (2001) “Implementing the market-based approach to enterprise support: An evaluation of ten matching grant schemes”, World Bank Policy Research Working Paper no. 2589. 6 This was asked as a binary question, about whether they had made such an investment or not, so we do not know the amount of investment made. 12 Phillips, David (2002) “The market-based approach to enterprise assistance—an evaluation of the World Bank's market development grant funds”, Small Enterprise Development 13(1): 26-37. Stone, Andrew, Lina Badawy, and Nabila Assaf, (2012) “The Plight of Yemeni Private Enterprises since the 2011 Crisis: A Rapid Assessment” The World Bank MENA Quick Note Series No. 72, September 2012. Figure 1: Main Business Development Services Requested by Firms 90 80 70 60 Number of Firms 50 40 30 20 10 0 Marketing Accounting Training Exhibition Website Research Campaign System Sana'a Aden Source: Administrative data from matching grant applicants. 13 Table 1: Baseline Characteristics for Experimental Sample and for Follow-up Sample Full Experimental Sample Sample Interviewed at Follow-up Treatment Control P-value Treatment Control P-value Owner is Female 0.09 0.16 0.027 0.11 0.16 0.202 Firm is in Aden 0.50 0.50 s 0.50 0.55 s Number of Male Employees 13.56 9.41 0.320 8.46 6.51 0.306 Number of Female Employees 3.59 2.87 0.499 2.11 2.15 0.832 Expanded business in last 3 years 0.25 0.24 0.713 0.26 0.20 0.372 Increased employment in last 3 years 0.12 0.13 0.773 0.11 0.11 0.929 Marketing campaign in last 3 years 0.30 0.31 0.821 0.27 0.31 0.595 Expanded to new market in last 3 years 0.13 0.15 0.556 0.11 0.14 0.452 Improved product line in last 3 years 0.19 0.14 0.210 0.16 0.15 0.821 Has an ISO qualification 0.02 0.01 0.286 0.02 0.00 0.159 Trained workers in last 3 years 0.27 0.22 0.201 0.27 0.23 0.636 Went to international expo in last 3 years 0.06 0.06 0.845 0.05 0.05 0.966 Exported in 2012 0.02 0.03 0.656 0.02 0.01 0.749 Sales in 2012 6757172 18833812 0.309 2502169 5695261 0.274 Joint orthogonality test p-value 0.318 0.863 Sample Size 216 200 128 98 Notes: s denotes variable was used for stratification. P-values are from regression controlling for location strata. Table 2: Types of Firms Firm Type Number Retail Sales 27 Foodstuffs 25 Clothing 23 Education and Training 17 Miscellaneous services 17 Computer training, internet café, computer services 15 Medical diagnosis, lab services, equipment provision, etc. 13 Telecom and selling scratch cards 13 Advertising services 12 Perfumes and cosmetics, beauty 8 Restaurant 8 Construction materials, consulting services and distributing 7 Manufacturing 7 Miscellaneous consulting services 7 Travel and car rental 7 Accounting services 4 Dental treatment 3 Administrative sciences and projects 1 Other 12 Notes: classification for firms responding to follow-up survey 14 Table 3: Differences in Baseline Attributes by Take-up Status Take-up Don't Take-up P-value Owner is Female 0.122 0.093 0.522 Firm is in Aden 0.527 0.463 s Number of Male Employees 8.041 9.037 0.664 Number of Female Employees 2.338 1.796 0.400 Expanded business in last 3 years 0.338 0.148 0.010 Increased employment in last 3 years 0.108 0.111 0.930 Marketing campaign in last 3 years 0.284 0.241 0.657 Expanded to new market in last 3 years 0.108 0.111 0.959 Improved product line in last 3 years 0.189 0.130 0.387 Has an ISO qualification 0.014 0.019 0.817 Trained workers in last 3 years 0.297 0.222 0.302 Went to international expo in last 3 years 0.081 0.000 0.013 Exported in 2012 0.014 0.019 0.817 Sales in 2012 3954254 512275 0.127 Joint orthogonality test p-value 0.424# Sample Size 74 54 Notes: s denotes variable was used for stratification. P-values are from regression controlling for location strata. # note this drops exported in 2012 and went to international expo from probit 15 Table 4: Impacts of the Matching Grant Program on Firm Innovation New New New New Attended Number of Any Introduced Introduced Marketing Accounting training quality HR trade show Innovative Innovative new product new process Campaign System for workers certification system abroad Activities Activity Panel A: ITT Impact with only Stratification Controls Selected for Grant 0.154** 0.070 0.101* 0.248*** 0.169** -0.001 0.140*** 0.051 0.932*** 0.189*** (0.063) (0.055) (0.056) (0.065) (0.066) (0.027) (0.043) (0.059) (0.249) (0.060) Panel B: ITT Impact with Baseline Controls Selected for Grant 0.170** 0.046 0.097* 0.246*** 0.178*** -0.005 0.109*** 0.060 0.901*** 0.181*** (0.067) (0.055) (0.058) (0.066) (0.067) (0.026) (0.040) (0.058) (0.248) (0.061) Panel C: LATE Impact Received Grant 0.303** 0.137 0.198* 0.485*** 0.332*** -0.003 0.274*** 0.100 1.825*** 0.371*** (0.124) (0.107) (0.108) (0.126) (0.126) (0.053) (0.085) (0.114) (0.471) (0.113) Sample Size 226 226 226 226 226 226 226 226 226 226 Control Mean 0.265 0.173 0.173 0.306 0.418 0.041 0.061 0.224 1.663 0.622 Notes: Robust standard errors in parentheses, *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Panels A and C control for city (used for stratifying the randomization), Panel B for the full set of baseline controls in Table 1. 16 Table 5: Impacts of the Matching Grant Program on Firm Inputs and Outcomes New Used Has Received Full-time Temporary capital a a a new Sales workers workers investment consultant website loan Exported grew Panel A: ITT Impact with only Stratification Controls Selected for Grant -2.013 -2.712 0.321*** 0.214*** 0.011 -0.013 -0.009 0.243*** (4.717) (3.299) (0.064) (0.057) (0.055) (0.058) (0.033) (0.065) Panel B: ITT Impact with Baseline Controls Selected for Grant -3.417 -3.082 0.310*** 0.221*** 0.017 -0.024 0.008 0.236*** (4.369) (3.576) (0.064) (0.059) (0.057) (0.059) (0.035) (0.067) Panel C: LATE Impact Received Grant -3.913 -5.271 0.630*** 0.419*** 0.022 -0.026 -0.017 0.477*** (9.176) (6.433) (0.132) (0.110) (0.108) (0.114) (0.066) (0.132) Sample Size 225 225 226 226 226 226 226 226 Control Mean 11.571 6.724 0.347 0.163 0.214 0.245 0.071 0.316 Notes: Robust standard errors in parentheses, *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Panels A and C control for city (used for stratifying the randomization), Panel B for the full set of baseline controls in Table 1. 17 Appendix 1: Timeline August 2013: Program launched October 1, 2013 – December 26, 2013: Applications accepted December 29, 2013 – January 5, 2014: Screening for eligibility January 2014: Random selection of firms for matching grants February 2014: workshop to inform firms of procurement process, and firms fill out templates for approval March-October 2014: firms implement business development plans, receive grants September 2014: Houthi rebels take control of capital of Sana’a, UN peace deal announced in which they agree to withdraw once national unity government is formed. January-February 2015: Houthis seize state TV; President Hadi resigns and flees to Aden; Houthis take charge of government March 2015: Follow-up survey conducted March 2015: Islamic state carries out suicide bombings (March 20), Saudi-led coalition launches air strikes and imposes naval blockade (March 26), clashes in Aden and other cities, by April widespread conflict. Appendix 2: Lee Lower Bounds We have shown in the text that the treatment and control groups are balanced in terms of observable baseline characteristics, despite having differential attrition. As a result, our results were robust to the inclusion of these characteristics in the regression. Moreover, we argued that any bias from differential attrition may well be a downward one, if the matching grants keep firms open that would have otherwise closed. Nevertheless, as a worst case scenario, we also consider here Lee (2009) bounds. We do this for the outcomes found to have the largest and most significant treatment impacts in Tables 4 and 5. If the treatment group had had the same attrition rate as the control group, it would have had 22 fewer observations in it at follow-up. To construct the Lee lower bounds, we assume that all of these observations were successes. For example, we randomly set as missing 22 of the 54 treatment group observations that reported introducing a new product. The table below shows the resulting estimates of the ITT. As we would expect, these lower bounds reduce the estimated impacts. Nevertheless, all of the lower bounds are still positive, and three out of six still statistically significant. Therefore, even in the unlikely case that all the additional control group firms who were not interviewed were innovating and growing their sales, the matching grant program is still found to have additionality. Appendix 2: Lee Lower Bounds on Key Impacts New New Any New Used Introduced Accounting training Innovative capital a Sales new product System for workers Activity investment consultant Grew ITT Impact 0.034 0.154** 0.084 0.149** 0.253*** 0.088 0.151** (0.064) (0.068) (0.070) (0.064) (0.068) (0.056) (0.068) Sample Size 204 204 204 204 204 204 204 Notes: Robust standard errors in parentheses, *, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively. Regressions also control for city. Document of the World Bank