Publication: Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
Loading...
Date
2012-12
ISSN
Published
2012-12
Author(s)
Editor(s)
Abstract
Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. The authors attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case were unable to complete an experimental evaluation. One critique of randomized experiments is publication bias, whereby only those experiments with "interesting" results get published. The hope is to mitigate this bias by learning from the experiments that never happened. This paper describes the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delves into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in "last mile" issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. Lessons are drawn from these experiences for both the implementation and the possible evaluation of future projects.
Link to Data Set
Citation
“Campos, Francisco; Coville, Aidan; Fernandes, Ana M.; Goldstein, Markus; McKenzie, David. 2012. Learning from the Experiments That Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa. Policy Research Working Paper; No. 6296. © World Bank. http://hdl.handle.net/10986/12200 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Asymmetric Bank Distress Amplifier of Recessions(Washington, DC: World Bank, 2025-07-11)One defining feature of financial crises, evident in U.S. and international data, is asymmetric bank distress—concentrated losses on a subset of banks. This paper proposes a model in which shocks to borrowers’ productivity dispersion lead to asymmetric bank losses. The framework exhibits a “bank distress amplifier,” exacerbating economic downturns by causing costly bank failures and raising uncertainty about the solvency of banks, thereby pushing banks to deleverage. Quantitative analysis shows that the bank distress amplifier doubles investment decline and increases the spread by 2.5 times during the Great Recession compared to a standard financial accelerator model. The mechanism helps explain how a seemingly small shock can sometimes trigger a large crisis.Publication From Tailwinds to Headwinds(Washington, DC: World Bank, 2025-07-10)The first quarter of the twenty-first century has been transformative for emerging market and developing economies (EMDEs). These economies now account for about 45 percent of global GDP, up from about 25 percent in 2000, a trend driven by robust collective growth in the three largest EMDEs—China, India, and Brazil (the EM3). Collectively, EMDEs have contributed about 60 percent of annual global growth since 2000, on average, double the share during the 1990s. Their ascendance was powered by swift global trade and financial integration, especially during the first decade of the century. Interdependence among these economies has also increased markedly. Today, nearly half of goods exports from EMDEs go to other EMDEs, compared to one-quarter in 2000. As cross-border linkages have strengthened, business cycles among EMDEs and between EMDEs and advanced economies have become more synchronized, and a distinct EMDE business cycle has emerged. Cross-border business cycle spillovers from the EM3 to other EMDEs are sizable, at about half of the magnitude of spillovers from the largest advanced economies (the United States, the euro area, and Japan). Yet EMDEs confront a host of headwinds at the turn of the second quarter of the century. Progress implementing structural reforms in many of these economies has stalled. Globally, protectionist measures and geopolitical fragmentation have risen sharply. High debt burdens, demographic shifts, and the rising costs of climate change weigh on economic prospects. A successful policy approach to accelerate growth and development should focus on boosting investment and productivity, navigating a difficult external environment, and enhancing macroeconomic stability.Publication Intergenerational Income Mobility around the World(Washington, DC: World Bank, 2025-07-09)This paper introduces a new global database with estimates of intergenerational income mobility for 87 countries, covering 84 percent of the world’s population. This marks a notable expansion of the cross-country evidence base on income mobility, particularly among low- and middle-income countries. The estimates indicate that the negative association between income mobility and inequality (known as the Great Gatsby Curve) continues to hold across this wider range of countries. The database also reveals a positive association between income mobility and national income per capita, suggesting that countries achieve higher levels of intergenerational mobility as they grow richer.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Growing Women-led Enterprises in the Mekong : Testing a Methodology for Accelerating Growth(Washington, DC, 2014)According to a recent global entrepreneurship monitor, 48 percent of entrepreneurs globally are women. Growth-oriented women entrepreneurs face challenges such as lack of access to support services and relevant knowledge as well as regulatory and legal barriers. Recognizing the barriers that often prevent women from growing their businesses, infoDev embarked on a small pilot project designed to test a new methodology for increasing the capacity, confidence, and networks of growth-oriented women leading small businesses. The objective was to test whether this methodology will indeed lead to business expansion. infoDevapos;s pilot - the Mekong womenapos;s entrepreneurship challenge (MWEC) - set out to test a methodology based primarily on structured facilitation of peer-to-peer learning among growth-oriented women entrepreneurs. This report discusses the design, outcomes, and lessons of MWEC in further detail and provides a few preliminary recommendations for the design of future programs targeting the growth of women-owned enterprises.Publication Enhancing Access to Finance for Technology Entrepreneurs : Analysis of Highly Innovative, High Growth Start-Ups in Vietnam, Cambodia, and Nepal(Washington, DC, 2014)The first part of the study provides contextual background to the financing gaps and associated barriers, which restrict access to finance for HI start-ups. These barriers are driven by both supply and demand sides of the financing equation. Supply side barriers include: high transaction costs associated with financing; high levels of credit risk associated with HI start-ups; high collateral required by financial institutions; non-conducive legal and regulatory environments for investment in HI start-ups; lack of start-up expertise and dedicated resources by financiers; and finance products that are not tailored to HI start-ups needs and circumstances. Demand side barriers include: reliance by HI start-ups on informal financing sources; lack of awareness on the process to apply for funding from formal financing sources; low levels of financial literacy by HI start-ups; and the fear of losing control by involving external investors. This section also contains a broad overview of the country frameworks governing the start-up sector, together with some of the initiatives relating to access to finance. The second part of the study covers the fieldwork undertaken in Vietnam, Cambodia, and Nepal. The fieldwork gathered views from investee firms (irrespective of whether they were successful in raising finance or not), investors, and other stakeholders. The fieldwork was aimed at understanding the severity of the financing gap for HI start-ups, the stage(s) of financing impacted by lack of access to finance, and the sources of financing for HI start-ups. Lastly, financing catalyst recommendations address non-financing impediments, which if overcome will have a positive impact on access to finance.Publication Bhutan Investment Climate Assessment Report : Vitalizing the Private Sector, Creating Jobs, Volume 1. Summary Report(Washington, DC, 2010-09)The objective of the Bhutan Investment Climate Assessment (ICA) is to evaluate the investment climate in Bhutan in all its operational dimensions and promote policies to strengthen the private sector. This ICA consists of two volumes. Volume 1 summarizes the main results. Volume 2 presents a more detailed analysis of each of the three main themes of the report: labor productivity and skills, access to finance, and business government relations, and is supplemented by a chapter on the gender dimensions of the investment climate. Volume 2 also includes the supporting econometric analysis of the enterprise survey data. The purpose of Volume 2 is to provide additional detail on the underlying data and analysis which supports the main results summarized in Volume 1. In Volume 2, Chapter 1 sets the Bhutanese context and background, explains what's new in the second Bhutan Investment Climate Assessment, and presents the overall findings of the 2010 report. Chapter 2 reviews the characteristics of the investment climate in Bhutan, including the macroeconomic perspective. Chapter 3 discusses labor productivity and skills and the importance of focusing on high value added niche industries. Chapter 4 reviews access to finance as a primary constraint to firms in Bhutan and discusses an apparent access to finance paradox. Chapter 5 reviews the business government enabling environment in Bhutan and the importance of licensing reform. Chapter 6 discusses the gender dimension of the investment climate in Bhutan.Publication Customer Management in SME Banking(Washington, DC, 2012)The objective of the Customer Management in Small and Medium Enterprise (SME) Banking Guide is to share and disseminate critical information for managing the SME client relationship, allowing banks that already serve the SME sector to move beyond lending to better capture the SME Banking opportunity. This Guide leverages IFC s SME Banking Diagnostic framework used to assess SME banking operations, as well as its SME Banking Benchmarking exercise used to analyze good practice business models. In addition, the Guide provides practical examples of customer management focused on SME banking from a number of featured financial institutions. Such examples may serve to highlight a good practice, or may simply serve to illustrate a learning experience. Financial institutions featured in this publication include Bankinter, DBS Bank, Diamond Bank, Garanti Bank, ICICI Bank, Banco Santander, and Türk Ekonomi Bankasi (TEB). Additional SME banking experiences are drawn from a variety of other banks and are cited throughout the text as appropriate. Profitably serving the SME segment requires a tailored customer management approach that will allow banks to answer these four questions: How can banks better understand SME customer needs? How can they match diverse needs with the right offer, service level, and delivery channel? How can customer management be used to maximize the revenue opportunity when servicing this market segment? How can banks effectively manage SME customers across their life-cycle? This guide will address these questions as well as take the reader through the customer life-cycle to target and acquire new SME clients.Publication Fostering Entrepreneurship in Georgia(Washington, DC: World Bank, 2013)Job creation and productivity growth are at the forefront of today's global development agenda. The 2013 world development report on jobs identified entrepreneurship as an important tool in addressing these dual goals. This study uses the entrepreneurship model put forth in the organization for Economic Co-operation and Development (OECD)-Eurostat Entrepreneurship Indicator Program (2009), with minor modifications. The model comprises various determinants that policy can affect and that in turn influence entrepreneurial performance, or the amount and type of entrepreneurship that take place. The model then refers to the impact of entrepreneurship on higher-level goals such as economic growth, job creation, and poverty reduction. This study focuses on determining the level of entrepreneurship in Georgia and analyzes the role of each determinant in both fostering and constraining entrepreneurial activity. Entrepreneurs view opportunities in the economy by measuring their profit-making potential; that is their first motivation. Entrepreneurs will not pursue a societal need unless they can successfully make a profit. Entrepreneurship also depends on various social and individual characteristics of gender and culture. It involves taking risks, and potential entrepreneurs cannot be risk-averse. A fundamental question stemming from analyzing these determinants of entrepreneurship is how to design effective public policy that promotes innovative firm creation and enables existing firms to catch up, improve productivity, and grow. Experiences from Asian countries and developed economies have shown that innovative Small Medium Enterprises (SMEs) and knowledge-based firm creation have played a major role in the development of new national economic advantages. In this context public policy is pivotal in creating an enabling environment that helps alleviate the market failures that inhibit firm growth. Emerging markets have other environmental conditions that are not present (or are less prevalent) in developed markets, and investors considering investing in emerging markets will face added risk as a result.
Users also downloaded
Showing related downloaded files
Publication Argentina Country Climate and Development Report(World Bank, Washington, DC, 2022-11)The Argentina Country Climate and Development Report (CCDR) explores opportunities and identifies trade-offs for aligning Argentina’s growth and poverty reduction policies with its commitments on, and its ability to withstand, climate change. It assesses how the country can: reduce its vulnerability to climate shocks through targeted public and private investments and adequation of social protection. The report also shows how Argentina can seize the benefits of a global decarbonization path to sustain a more robust economic growth through further development of Argentina’s potential for renewable energy, energy efficiency actions, the lithium value chain, as well as climate-smart agriculture (and land use) options. Given Argentina’s context, this CCDR focuses on win-win policies and investments, which have large co-benefits or can contribute to raising the country’s growth while helping to adapt the economy, also considering how human capital actions can accompany a just transition.Publication Guide to the Debt Management Performance Assessment Tool(Washington, DC, 2008-02-05)The purpose of this document is to provide guidance and supplemental information to assist with country assessments of debt management performance, using the Debt Management Performance Assessment (DeMPA) tool. The DeMPA is a methodology used for assessing public debt management performance through a comprehensive set of 15 performance indicators spanning the full range of government Debt Management (DeM) functions. It is based on the principles set out in the International Monetary Fund (IMF) and World Bank guidelines for public debt management, initially published in 2001 and updated in 2003. It is modeled after the Public Expenditure and Financial Accountability (PEFA) framework for performance measurement of public financial management. The DeMPA has been designed to be a user-friendly tool to undertake an assessment of the strengths and weaknesses in government DeM practices. This guide provides additional background and supporting information so that a no specialist in the area of debt management may undertake a country assessment effectively. The guide can be used by assessors in preparing for and undertaking an assessment. It is particularly useful for understanding the rationale for the inclusion of the indicators, the scoring methodology, and the list of supporting documents or evidence required, and the questions that could be asked for the assessment.Publication Crime and Violence in Central America : A Development Challenge - Main Report(World Bank, 2011-01-01)Crime and violence are now a key development issue for Central American countries. In three nations El Salvador, Guatemala, and Honduras crime rates are among the top five in Latin America. This report argues that successful strategies require actions along multiple fronts, combining prevention and criminal justice reform, together with regional approaches in the areas of drug trafficking and firearms. It also argues that interventions should be evidence based, starting with a clear understanding of the risk factors involved and ending with a careful evaluation of how any planned action might affect future options. In addition, the design of national crime reduction plans and the establishment of national cross-sectoral crime commissions are important steps to coordinate the actions of different government branches, ease cross-sectoral collaboration and prioritize resource allocation. Of equal importance is the fact that national plans offer a vehicle for the involvement of civil society organizations, in which much of the expertise in violence prevention and rehabilitation resides. Prevention efforts need to be complemented by effective law enforcement. The required reforms are no longer primarily legislative in nature because all six countries have advanced toward more transparent adversarial criminal procedures. The second-generation reforms should instead help deliver on the promises of previous reforms by: (i) strengthening key institutions and improving the quality and timeliness of the services they provide to citizens; (ii) improving efficiency and effectiveness while respecting due process and human rights; (iii) ensuring accountability and addressing corruption; (iv) increasing inter-agency collaboration; and (v) improving access to justice, especially for poor and disenfranchised groups. Specific interventions reviewed in the report include: information systems and performance indicators as a prerequisite to improve inter-institutional coordination and information sharing mechanisms; an internal overhaul of court administration and case management to create rapid reaction, one-stop shops; the strengthening of entities that provide legal counseling to the poor and to women; and the promotion of alternative dispute-resolution mechanisms and the implementation of community policing programs.Publication The Mexican Social Protection System in Health(World Bank, Washington DC, 2013-01)With a population of 113 million and a per-capita Gross Domestic Product, or GDP of US$10,064 (current U.S. dollars), Mexico is one of the largest and highest-income countries in Latin America and the Caribbean (LAC). The country has benefited from sustained economic growth during the last decade, which was temporarily interrupted by the financial and economic crisis. Real GDP is projected to grow 3.8 percent and 3.6 percent in 2012 and 2013, respectively (International Monetary Fund, or IMF 2012). Despite this growth, poverty in the country remains high; with half of the population living below the national poverty line. The country is also highly heterogeneous, with large socioeconomic differences across states and across urban and rural areas. In 2010, while the extreme poverty ratio in the Federal District and the states of Colima and Nuevo Leon was below 3 percent, in Chiapas, Guerrero, and Oaxaca it was 25 percent or higher. These large regional differences are also found in other indicators of well-being, such as years of schooling, housing conditions, and access to social services. This case study assesses key features and achievements of the Social Protection System in Health (Sistema de Proteccion Social en Salud) in Mexico, and particularly of its main pillar, Popular Health Insurance (Seguro Popular, PHI). It analyzes the contribution of this policy to the establishment and implementation of universal health coverage in Mexico. In 2003, with the reform of the General Health Law, the PHI was institutionalized as a subsidized health insurance scheme open to the population not covered by the social security schemes. Today, the PHI covers all of its intended affiliates, about 52 million peoplePublication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.