Publication: Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
Loading...
Published
2012-12
ISSN
Date
2013-01-29
Author(s)
Editor(s)
Abstract
Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. The authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with "interesting" results get published. The hope is to mitigate this bias by learning from the experiments that never happened. This paper describes the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delves into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in "last mile" issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. Lessons are drawn from these experiences for both the implementation and the possible evaluation of future projects.
Link to Data Set
Citation
“Campos, Francisco; Coville, Aidan; Fernandes, Ana M.; Goldstein, Markus; McKenzie, David. 2012. Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa. Policy Research Working Paper; No. 6296. © World Bank. http://hdl.handle.net/10986/12200 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication Intergenerational Income Mobility around the World(Washington, DC: World Bank, 2025-07-09)This paper introduces a new global database with estimates of intergenerational income mobility for 87 countries, covering 84 percent of the world’s population. This marks a notable expansion of the cross-country evidence base on income mobility, particularly among low- and middle-income countries. The estimates indicate that the negative association between income mobility and inequality (known as the Great Gatsby Curve) continues to hold across this wider range of countries. The database also reveals a positive association between income mobility and national income per capita, suggesting that countries achieve higher levels of intergenerational mobility as they grow richer.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Higher Education Expansion and Firm Organization(Washington, DC: World Bank, 2025-09-17)This paper investigates the impact of higher education expansion on firm performance in developing countries. It focuses on the significant expansion of higher education in Brazil between 2000 and 2012, which substantially increased higher education enrollment and graduation rates, thereby reducing the costs of hiring college-educated workers. Building on the theory of knowledge-based hierarchies and using a difference-in-differences approach and matching techniques, the paper finds that the surge in skilled labor supply led to a rise in the proportion of college-educated workers within firms in the treated microregions. This increase was accompanied by an increased prob-ability of firms adding knowledge hierarchies, followed by a rise in productivity and an increased likelihood of export. The findings suggest that policies affecting the cost and accessibility of hiring professionals and managers can significantly influence firms’ organizational structures, with implications for firm performance and productivity.Publication The Exposure of Workers to Artificial Intelligence in Low- and Middle-Income Countries(Washington, DC: World Bank, 2025-02-05)Research on the labor market implications of artificial intelligence has focused principally on high-income countries. This paper analyzes this issue using microdata from a large set of low- and middle-income countries, applying a measure of potential artificial intelligence occupational exposure to a harmonized set of labor force surveys for 25 countries, covering a population of 3.5 billion people. The approach advances work by using harmonized microdata at the level of individual workers, which allows for a multivariate analysis of factors associated with exposure. Additionally, unlike earlier papers, the paper uses highly detailed (4 digit) occupation codes, which provide a more reliable mapping of artificial intelligence exposure to occupation. Results within countries, show that artificial intelligence exposure is higher for women, urban workers, and those with higher education. Exposure decreases by country income level, with high exposure for just 12 percent of workers in low-income countries and 15 percent of workers in lower-middle-income countries. Furthermore, lack of access to electricity limits effective exposure in low-income countries. These results suggest that for developing countries, and in particular low-income countries, the labor market impacts of artificial intelligence will be more limited than in high-income countries. While greater exposure to artificial intelligence indicates larger potential for future changes in certain occupations, it does not equate to job loss, as it could result in augmentation of worker productivity, automation of some tasks, or both.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Growing Women-led Enterprises in the Mekong : Testing a Methodology for Accelerating Growth(Washington, DC, 2014)According to a recent global entrepreneurship monitor, 48 percent of entrepreneurs globally are women. Growth-oriented women entrepreneurs face challenges such as lack of access to support services and relevant knowledge as well as regulatory and legal barriers. Recognizing the barriers that often prevent women from growing their businesses, infoDev embarked on a small pilot project designed to test a new methodology for increasing the capacity, confidence, and networks of growth-oriented women leading small businesses. The objective was to test whether this methodology will indeed lead to business expansion. infoDevapos;s pilot - the Mekong womenapos;s entrepreneurship challenge (MWEC) - set out to test a methodology based primarily on structured facilitation of peer-to-peer learning among growth-oriented women entrepreneurs. This report discusses the design, outcomes, and lessons of MWEC in further detail and provides a few preliminary recommendations for the design of future programs targeting the growth of women-owned enterprises.Publication Enhancing Access to Finance for Technology Entrepreneurs : Analysis of Highly Innovative, High Growth Start-Ups in Vietnam, Cambodia, and Nepal(Washington, DC, 2014)The first part of the study provides contextual background to the financing gaps and associated barriers, which restrict access to finance for HI start-ups. These barriers are driven by both supply and demand sides of the financing equation. Supply side barriers include: high transaction costs associated with financing; high levels of credit risk associated with HI start-ups; high collateral required by financial institutions; non-conducive legal and regulatory environments for investment in HI start-ups; lack of start-up expertise and dedicated resources by financiers; and finance products that are not tailored to HI start-ups needs and circumstances. Demand side barriers include: reliance by HI start-ups on informal financing sources; lack of awareness on the process to apply for funding from formal financing sources; low levels of financial literacy by HI start-ups; and the fear of losing control by involving external investors. This section also contains a broad overview of the country frameworks governing the start-up sector, together with some of the initiatives relating to access to finance. The second part of the study covers the fieldwork undertaken in Vietnam, Cambodia, and Nepal. The fieldwork gathered views from investee firms (irrespective of whether they were successful in raising finance or not), investors, and other stakeholders. The fieldwork was aimed at understanding the severity of the financing gap for HI start-ups, the stage(s) of financing impacted by lack of access to finance, and the sources of financing for HI start-ups. Lastly, financing catalyst recommendations address non-financing impediments, which if overcome will have a positive impact on access to finance.Publication Bhutan Investment Climate Assessment Report : Vitalizing the Private Sector, Creating Jobs, Volume 1. Summary Report(Washington, DC, 2010-09)The objective of the Bhutan Investment Climate Assessment (ICA) is to evaluate the investment climate in Bhutan in all its operational dimensions and promote policies to strengthen the private sector. This ICA consists of two volumes. Volume 1 summarizes the main results. Volume 2 presents a more detailed analysis of each of the three main themes of the report: labor productivity and skills, access to finance, and business government relations, and is supplemented by a chapter on the gender dimensions of the investment climate. Volume 2 also includes the supporting econometric analysis of the enterprise survey data. The purpose of Volume 2 is to provide additional detail on the underlying data and analysis which supports the main results summarized in Volume 1. In Volume 2, Chapter 1 sets the Bhutanese context and background, explains what's new in the second Bhutan Investment Climate Assessment, and presents the overall findings of the 2010 report. Chapter 2 reviews the characteristics of the investment climate in Bhutan, including the macroeconomic perspective. Chapter 3 discusses labor productivity and skills and the importance of focusing on high value added niche industries. Chapter 4 reviews access to finance as a primary constraint to firms in Bhutan and discusses an apparent access to finance paradox. Chapter 5 reviews the business government enabling environment in Bhutan and the importance of licensing reform. Chapter 6 discusses the gender dimension of the investment climate in Bhutan.Publication Customer Management in SME Banking(Washington, DC, 2012)The objective of the Customer Management in Small and Medium Enterprise (SME) Banking Guide is to share and disseminate critical information for managing the SME client relationship, allowing banks that already serve the SME sector to move beyond lending to better capture the SME Banking opportunity. This Guide leverages IFC s SME Banking Diagnostic framework used to assess SME banking operations, as well as its SME Banking Benchmarking exercise used to analyze good practice business models. In addition, the Guide provides practical examples of customer management focused on SME banking from a number of featured financial institutions. Such examples may serve to highlight a good practice, or may simply serve to illustrate a learning experience. Financial institutions featured in this publication include Bankinter, DBS Bank, Diamond Bank, Garanti Bank, ICICI Bank, Banco Santander, and Türk Ekonomi Bankasi (TEB). Additional SME banking experiences are drawn from a variety of other banks and are cited throughout the text as appropriate. Profitably serving the SME segment requires a tailored customer management approach that will allow banks to answer these four questions: How can banks better understand SME customer needs? How can they match diverse needs with the right offer, service level, and delivery channel? How can customer management be used to maximize the revenue opportunity when servicing this market segment? How can banks effectively manage SME customers across their life-cycle? This guide will address these questions as well as take the reader through the customer life-cycle to target and acquire new SME clients.Publication Fostering Entrepreneurship in Georgia(Washington, DC: World Bank, 2013)Job creation and productivity growth are at the forefront of today's global development agenda. The 2013 world development report on jobs identified entrepreneurship as an important tool in addressing these dual goals. This study uses the entrepreneurship model put forth in the organization for Economic Co-operation and Development (OECD)-Eurostat Entrepreneurship Indicator Program (2009), with minor modifications. The model comprises various determinants that policy can affect and that in turn influence entrepreneurial performance, or the amount and type of entrepreneurship that take place. The model then refers to the impact of entrepreneurship on higher-level goals such as economic growth, job creation, and poverty reduction. This study focuses on determining the level of entrepreneurship in Georgia and analyzes the role of each determinant in both fostering and constraining entrepreneurial activity. Entrepreneurs view opportunities in the economy by measuring their profit-making potential; that is their first motivation. Entrepreneurs will not pursue a societal need unless they can successfully make a profit. Entrepreneurship also depends on various social and individual characteristics of gender and culture. It involves taking risks, and potential entrepreneurs cannot be risk-averse. A fundamental question stemming from analyzing these determinants of entrepreneurship is how to design effective public policy that promotes innovative firm creation and enables existing firms to catch up, improve productivity, and grow. Experiences from Asian countries and developed economies have shown that innovative Small Medium Enterprises (SMEs) and knowledge-based firm creation have played a major role in the development of new national economic advantages. In this context public policy is pivotal in creating an enabling environment that helps alleviate the market failures that inhibit firm growth. Emerging markets have other environmental conditions that are not present (or are less prevalent) in developed markets, and investors considering investing in emerging markets will face added risk as a result.
Users also downloaded
Showing related downloaded files
Publication Argentina Country Climate and Development Report(World Bank, Washington, DC, 2022-11)The Argentina Country Climate and Development Report (CCDR) explores opportunities and identifies trade-offs for aligning Argentina’s growth and poverty reduction policies with its commitments on, and its ability to withstand, climate change. It assesses how the country can: reduce its vulnerability to climate shocks through targeted public and private investments and adequation of social protection. The report also shows how Argentina can seize the benefits of a global decarbonization path to sustain a more robust economic growth through further development of Argentina’s potential for renewable energy, energy efficiency actions, the lithium value chain, as well as climate-smart agriculture (and land use) options. Given Argentina’s context, this CCDR focuses on win-win policies and investments, which have large co-benefits or can contribute to raising the country’s growth while helping to adapt the economy, also considering how human capital actions can accompany a just transition.Publication World Development Report 2006(Washington, DC, 2005)This year’s Word Development Report (WDR), the twenty-eighth, looks at the role of equity in the development process. It defines equity in terms of two basic principles. The first is equal opportunities: that a person’s chances in life should be determined by his or her talents and efforts, rather than by pre-determined circumstances such as race, gender, social or family background. The second principle is the avoidance of extreme deprivation in outcomes, particularly in health, education and consumption levels. This principle thus includes the objective of poverty reduction. The report’s main message is that, in the long run, the pursuit of equity and the pursuit of economic prosperity are complementary. In addition to detailed chapters exploring these and related issues, the Report contains selected data from the World Development Indicators 2005‹an appendix of economic and social data for over 200 countries. This Report offers practical insights for policymakers, executives, scholars, and all those with an interest in economic development.Publication Lebanon Economic Monitor, Fall 2022(Washington, DC, 2022-11)The economy continues to contract, albeit at a somewhat slower pace. Public finances improved in 2021, but only because spending collapsed faster than revenue generation. Testament to the continued atrophy of Lebanon’s economy, the Lebanese Pound continues to depreciate sharply. The sharp deterioration in the currency continues to drive surging inflation, in triple digits since July 2020, impacting the poor and vulnerable the most. An unprecedented institutional vacuum will likely further delay any agreement on crisis resolution and much needed reforms; this includes prior actions as part of the April 2022 International Monetary Fund (IMF) staff-level agreement (SLA). Divergent views among key stakeholders on how to distribute the financial losses remains the main bottleneck for reaching an agreement on a comprehensive reform agenda. Lebanon needs to urgently adopt a domestic, equitable, and comprehensive solution that is predicated on: (i) addressing upfront the balance sheet impairments, (ii) restoring liquidity, and (iii) adhering to sound global practices of bail-in solutions based on a hierarchy of creditors (starting with banks’ shareholders) that protects small depositors.Publication The Journey Ahead(Washington, DC: World Bank, 2024-10-31)The Journey Ahead: Supporting Successful Migration in Europe and Central Asia provides an in-depth analysis of international migration in Europe and Central Asia (ECA) and the implications for policy making. By identifying challenges and opportunities associated with migration in the region, it aims to inform a more nuanced, evidencebased debate on the costs and benefits of cross-border mobility. Using data-driven insights and new analysis, the report shows that migration has been an engine of prosperity and has helped address some of ECA’s demographic and socioeconomic disparities. Yet, migration’s full economic potential remains untapped. The report identifies multiple barriers keeping migration from achieving its full potential. Crucially, it argues that policies in both origin and destination countries can help maximize the development impacts of migration and effectively manage the economic, social, and political costs. Drawing from a wide range of literature, country experiences, and novel analysis, The Journey Ahead presents actionable policy options to enhance the benefits of migration for destination and origin countries and migrants themselves. Some measures can be taken unilaterally by countries, whereas others require close bilateral or regional coordination. The recommendations are tailored to different types of migration— forced displacement as well as high-skilled and low-skilled economic migration—and from the perspectives of both sending and receiving countries. This report serves as a comprehensive resource for governments, development partners, and other stakeholders throughout Europe and Central Asia, where the richness and diversity of migration experiences provide valuable insights for policy makers in other regions of the world.Publication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.