Publication: Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
Loading...
Published
2012-12
ISSN
Date
2013-01-29
Author(s)
Editor(s)
Abstract
Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. The authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with "interesting" results get published. The hope is to mitigate this bias by learning from the experiments that never happened. This paper describes the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delves into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in "last mile" issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. Lessons are drawn from these experiences for both the implementation and the possible evaluation of future projects.
Link to Data Set
Citation
“Campos, Francisco; Coville, Aidan; Fernandes, Ana M.; Goldstein, Markus; McKenzie, David. 2012. Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa. Policy Research Working Paper; No. 6296. © World Bank. http://hdl.handle.net/10986/12200 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication The Evolution of Local Participatory Democracy in Nepal(Washington, DC: World Bank, 2025-11-05)Nepal is, according to its constitution, among the world’s most decentralized countries, with a long and complex tradition of local-level public participation. This paper traces the evolution of Nepal’s modern participatory institutions, examining the extent to which they are “induced” by external interventions versus being “organically” rooted in indigenous practices. The paper identifies three broad phases: an initial focus on participation in project implementation; a subsequent phase that expanded citizen engagement; and a third phase of citizen empowerment, culminating in the 2015 federal constitution, which granted unprecedented local autonomy. The analysis yields five key findings. First, over the past 50 years, successive reforms have progressively expanded opportunities for citizens to influence local decision-making. Second, these reforms have integrated traditional participatory mechanisms into formal institutions of local government. Third, although central-level initiatives exist, most participatory platforms continue to operate at the local level. Fourth, the federal constitution has created a new landscape of local democracy, embedding autonomy and accountability. Fifth, although they are still valued in many ethnic and territorial communities, traditional participatory practices are gradually disappearing. The paper concludes by offering policy recommendations to help donor agencies and governments strengthen Nepal’s democratic trajectory. It argues that effective interventions should build on Nepal’s deep participatory traditions while recognizing the constitutional reality of far-reaching local autonomy.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication Closing the Gender Gap in Entrepreneurship: Overcoming Challenges in Law and Practice for Female Entrepreneurs(Washington, DC: World Bank, 2026-01-07)Despite significant strides toward gender equality, women around the world continue to encounter systemic obstacles that hinder their entrepreneurial success. This paper systematically reviews the literature on the barriers female entrepreneurs face and the solutions proposed to overcome these challenges. It discusses institutional factors, financial factors, human capital factors, and social and cultural factors. The literature overview is complemented by a series of stylized facts that illustrate how overcoming some of these existing barriers is correlated with improved women’s entrepreneurship and female labor force participation, drawing on the World Bank’s Women, Business and the Law database as well as the World Bank’s Enterprise Surveys. The findings underscore the need for creating an enabling environment where women can thrive as entrepreneurs.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Enhancing Access to Finance for Technology Entrepreneurs : Analysis of Highly Innovative, High Growth Start-Ups in Vietnam, Cambodia, and Nepal(Washington, DC, 2014)The first part of the study provides contextual background to the financing gaps and associated barriers, which restrict access to finance for HI start-ups. These barriers are driven by both supply and demand sides of the financing equation. Supply side barriers include: high transaction costs associated with financing; high levels of credit risk associated with HI start-ups; high collateral required by financial institutions; non-conducive legal and regulatory environments for investment in HI start-ups; lack of start-up expertise and dedicated resources by financiers; and finance products that are not tailored to HI start-ups needs and circumstances. Demand side barriers include: reliance by HI start-ups on informal financing sources; lack of awareness on the process to apply for funding from formal financing sources; low levels of financial literacy by HI start-ups; and the fear of losing control by involving external investors. This section also contains a broad overview of the country frameworks governing the start-up sector, together with some of the initiatives relating to access to finance. The second part of the study covers the fieldwork undertaken in Vietnam, Cambodia, and Nepal. The fieldwork gathered views from investee firms (irrespective of whether they were successful in raising finance or not), investors, and other stakeholders. The fieldwork was aimed at understanding the severity of the financing gap for HI start-ups, the stage(s) of financing impacted by lack of access to finance, and the sources of financing for HI start-ups. Lastly, financing catalyst recommendations address non-financing impediments, which if overcome will have a positive impact on access to finance.Publication Growing Women-led Enterprises in the Mekong : Testing a Methodology for Accelerating Growth(Washington, DC, 2014)According to a recent global entrepreneurship monitor, 48 percent of entrepreneurs globally are women. Growth-oriented women entrepreneurs face challenges such as lack of access to support services and relevant knowledge as well as regulatory and legal barriers. Recognizing the barriers that often prevent women from growing their businesses, infoDev embarked on a small pilot project designed to test a new methodology for increasing the capacity, confidence, and networks of growth-oriented women leading small businesses. The objective was to test whether this methodology will indeed lead to business expansion. infoDevapos;s pilot - the Mekong womenapos;s entrepreneurship challenge (MWEC) - set out to test a methodology based primarily on structured facilitation of peer-to-peer learning among growth-oriented women entrepreneurs. This report discusses the design, outcomes, and lessons of MWEC in further detail and provides a few preliminary recommendations for the design of future programs targeting the growth of women-owned enterprises.Publication Garanti Bank SA : Combining SME Banking Excellence with a Proposition for Women Entrepreneurs in Turkey(World Bank Group, Washington, DC, 2014-03)Garanti Bankasi (Garanti) is a regional pioneer in offering products and services specifically targeted towards women entrepreneurs as a specific market segment. In 2006, Garanti created a women's entrepreneur support package specifically designed to help Turkey s women entrepreneurs to establish and grow their businesses. This was followed up with the creation of Women Entrepreneur Gatherings, in 2007, which provided additional training and educational tools and new networking opportunities. And in 2013, Garanti began collaborating with Bogazici University, a top university in Turkey, to offer an intensive mini-MBA training program for women entrepreneurs. Garanti also launched Turkey s first Woman Entrepreneur of the Year award, an initiative that showcases women who have successfully grown their business, encouraging more women entrepreneurship in the country. This case study (the study ) explores how Garanti leaders decided to focus on women entrepreneurs. It also documents and analyzes the bank s efforts to build a profitable and sustainable women in business franchise, through its three-pillar approach, including financial support, client education, and the encouragement of entrepreneurship. The case study also provides insights into how Garanti tapped into, and integrated with its highly innovative, market leading Small and Medium Enterprise, or SME banking franchise to more effectively target women in business. The study concludes with an objective assessment of the results of the program to date, offering suggestions from IFC specialists on how Garanti may further scale up its women in business initiative in the next phase of development.Publication Bhutan Investment Climate Assessment Report : Vitalizing the Private Sector, Creating Jobs, Volume 1. Summary Report(Washington, DC, 2010-09)The objective of the Bhutan Investment Climate Assessment (ICA) is to evaluate the investment climate in Bhutan in all its operational dimensions and promote policies to strengthen the private sector. This ICA consists of two volumes. Volume 1 summarizes the main results. Volume 2 presents a more detailed analysis of each of the three main themes of the report: labor productivity and skills, access to finance, and business government relations, and is supplemented by a chapter on the gender dimensions of the investment climate. Volume 2 also includes the supporting econometric analysis of the enterprise survey data. The purpose of Volume 2 is to provide additional detail on the underlying data and analysis which supports the main results summarized in Volume 1. In Volume 2, Chapter 1 sets the Bhutanese context and background, explains what's new in the second Bhutan Investment Climate Assessment, and presents the overall findings of the 2010 report. Chapter 2 reviews the characteristics of the investment climate in Bhutan, including the macroeconomic perspective. Chapter 3 discusses labor productivity and skills and the importance of focusing on high value added niche industries. Chapter 4 reviews access to finance as a primary constraint to firms in Bhutan and discusses an apparent access to finance paradox. Chapter 5 reviews the business government enabling environment in Bhutan and the importance of licensing reform. Chapter 6 discusses the gender dimension of the investment climate in Bhutan.Publication Enhancing Access to Finance for Technology Entrepreneurs in Southern Africa(Washington, DC, 2014-04)This report examines the financing gaps for early stage and growth that high-growth technology entrepreneurs are facing in the information and communication technology (ICT), climate technology, and innovative agribusiness sectors in Zambia and Mozambique, with a more regional review of Namibia and Botswana as possible. It analyzes the unmet needs of these entrepreneurs - perceived demand and latent demand - as well as the existing sources of supply of private capital for seed and early-stage investment, and defines the adequacy to meet these needs. The report also explores the challenges that angel investors have experienced in the past when attempting to finance early- and growth-stage start-ups in the region and, if and where appropriate, make recommendations relating to infoDev's possible interventions.
Users also downloaded
Showing related downloaded files
Publication The Additionality Impact of a Matching Grant Program for Small Firms(World Bank, Washington, DC, 2016-02-05)Matching grants are one of the most common types of private sector development programs used in developing countries. But government subsidies to private firms can be controversial. A key question is that of additionality: do these programs get firms to undertake innovative activities that they would not otherwise do, or merely subsidize activities that will take place anyway? Randomized controlled trials can provide the counterfactual needed to answer this question, but efforts to experiment with matching grant programs have often failed. This paper uses a randomized controlled trial of a matching grant program for firms in the Republic of Yemen to demonstrate the feasibility of conducting experiments with well-designed programs, and to measure the additionality impact. In the first year, the matching grant is found to have led to more product innovation, firms upgrading their accounting systems, marketing more, making more capital investments, and being more likely to report their sales grew.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Missing Information(World Bank, Washington, DC, 2022-10)Seeking out business advice, for example through classroom training or consulting, can be a profitable investment. Previous research has found that obtaining business advice through classroom training or one-on-one consulting can increase profits and sales, and firms can recover the cost of training in one to three years. Yet, many firms do not take up business training, even when it is heavily subsidized. Anecdotal evidence suggests that one reason for this low take-up is lack of information on business practices and on the benefits of improving these practices. The authors collaborated with the Parana office of the Brazilian Micro and Small Business Support Service (SEBRAE-PR), to test empirically whether providing more information on business practices can lead firms to seek out advice and improve their performance.Publication Long-Run Effects of Temporary Incentives on Medical Care Productivity(World Bank, Washington, DC, 2015-06)The adoption of new clinical practice patterns by medical care providers is often challenging, even when the patterns are believed to be efficacious and profitable. This paper uses a randomized field experiment to examine the effects of temporary financial incentives paid to medical care clinics for the initiation of prenatal care in the first trimester of pregnancy. The rate of early initiation of prenatal care was 34 percent higher in the treatment group than in the control group while the incentives were being paid, and this effect persisted at least 15 months and likely 24 months or more after the incentives ended. These results are consistent with a model where the incentives enable providers to address the fixed costs of overcoming organizational inertia in innovation, and suggest that temporary incentives may be effective at motivating improvements in long-run provider performance at a substantially lower cost than permanent incentives.Publication World Development Report 1984(New York: Oxford University Press, 1984)Long-term needs and sustained effort are underlying themes in this year's report. As with most of its predecessors, it is divided into two parts. The first looks at economic performance, past and prospective. The second part is this year devoted to population - the causes and consequences of rapid population growth, its link to development, why it has slowed down in some developing countries. The two parts mirror each other: economic policy and performance in the next decade will matter for population growth in the developing countries for several decades beyond. Population policy and change in the rest of this century will set the terms for the whole of development strategy in the next. In both cases, policy changes will not yield immediate benefits, but delay will reduce the room for maneuver that policy makers will have in years to come.