Publication: How Do Ex Ante Simulations Compare with Ex Post Evaluations?
Evidence from the Impact of Conditional Cash Transfer Programs
Loading...
Published
2011-06-01
ISSN
Date
2012-03-19
Author(s)
Editor(s)
Abstract
This paper compares the ex ante simulation of the impacts of conditional cash transfer programs against the ex post estimates of impacts obtained from experimental evaluations. Using data on program-eligible households in treatment areas from the same baseline surveys that are used for experimental evaluations of conditional cash transfer programs in Mexico and Ecuador, the authors use a micro-simulation model to derive ex ante estimates of the impact of the programs on enrollment rates and poverty. The estimates reveal that ex ante predictions of certain impacts of conditional cash transfer programs match up well against the benchmark estimates of ex post experimental studies. The findings seem to support the use of this model to assess the potential impact and cost efficiency of a conditional cash transfer program ex ante, in order to inform decisions about how the program would be designed.
Link to Data Set
Citation
“Leite, Phillippe; Narayan, Ambar; Skoufias, Emmanuel. 2011. How Do Ex Ante Simulations Compare with Ex Post Evaluations?
Evidence from the Impact of Conditional Cash Transfer Programs. Policy Research working paper ; no. WPS 5705. © World Bank. http://hdl.handle.net/10986/3469 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication The Evolution of Local Participatory Democracy in Nepal(Washington, DC: World Bank, 2025-11-05)Nepal is, according to its constitution, among the world’s most decentralized countries, with a long and complex tradition of local-level public participation. This paper traces the evolution of Nepal’s modern participatory institutions, examining the extent to which they are “induced” by external interventions versus being “organically” rooted in indigenous practices. The paper identifies three broad phases: an initial focus on participation in project implementation; a subsequent phase that expanded citizen engagement; and a third phase of citizen empowerment, culminating in the 2015 federal constitution, which granted unprecedented local autonomy. The analysis yields five key findings. First, over the past 50 years, successive reforms have progressively expanded opportunities for citizens to influence local decision-making. Second, these reforms have integrated traditional participatory mechanisms into formal institutions of local government. Third, although central-level initiatives exist, most participatory platforms continue to operate at the local level. Fourth, the federal constitution has created a new landscape of local democracy, embedding autonomy and accountability. Fifth, although they are still valued in many ethnic and territorial communities, traditional participatory practices are gradually disappearing. The paper concludes by offering policy recommendations to help donor agencies and governments strengthen Nepal’s democratic trajectory. It argues that effective interventions should build on Nepal’s deep participatory traditions while recognizing the constitutional reality of far-reaching local autonomy.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication Closing the Gender Gap in Entrepreneurship: Overcoming Challenges in Law and Practice for Female Entrepreneurs(Washington, DC: World Bank, 2026-01-07)Despite significant strides toward gender equality, women around the world continue to encounter systemic obstacles that hinder their entrepreneurial success. This paper systematically reviews the literature on the barriers female entrepreneurs face and the solutions proposed to overcome these challenges. It discusses institutional factors, financial factors, human capital factors, and social and cultural factors. The literature overview is complemented by a series of stylized facts that illustrate how overcoming some of these existing barriers is correlated with improved women’s entrepreneurship and female labor force participation, drawing on the World Bank’s Women, Business and the Law database as well as the World Bank’s Enterprise Surveys. The findings underscore the need for creating an enabling environment where women can thrive as entrepreneurs.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Conditional Cash Transfers, Schooling, and Child Labor : Micro-Simulating Brazil's Bolsa Escola Program(Washington, DC: World Bank, 2003-05)A growing number of developing economies are providing cash transfers to poor people that require certain behaviors on their part, such as attending school or regularly visiting health care facilities. A simple ex ante methodology is proposed for evaluating such programs and used to assess the bolsa escola program in Brazil. The results suggest that about 60 percent of poor 10- to 15-year-olds not in school enroll in response to the program. The program reduces the incidence of poverty by only a little more than one percentage point, however, and the Gini coefficient falls just half a point. Results are better for measures more sensitive to the bottom of the distribution, but the effect is never large.Publication Planning, Monitoring, and Evaluation : Methods and Tools for Poverty and Inequality Reduction Programs(World Bank, Washington, DC, 2013-01)As the author enter the second decade of the twenty-first century, governments, international organizations, nongovernmental organizations (NGOs), philanthropic organizations, and civil society groups worldwide are actively focusing on evidence-based policy and increased accountability to stakeholders (results agenda).The widespread implementation of the Results Agenda has generated a plethora of books, guides, academic papers, trainings, and case studies, which has enabled an ongoing maturation process in the field. Consequently, specialists are now better equipped to understand what works under which circumstances. Broadly speaking there are two interrelated questions which must be answered when assessing the sustainability of a government results agenda. First, is the institutional design and practice of government conducive to evidence-based policy making? Second, are the overarching monitoring and evaluation (M&E) methods and specific tools used appropriate for garnering the evidence demanded by government? These series of notes aim to make a small contribution to the latter question by summarizing and highlighting a selection of PM&E methods and the tools that governments and international organizations around the world have developed to put these into practice in their own contexts. The central goal of this initiative is to prompt a process of learning, reflection and action by providing practical information to those whose leadership role requires them to understand PM&E methods and their potential for enhancing evidence-based policy making.Publication Handbook on Impact Evaluation : Quantitative Methods and Practices(World Bank, 2010)This book reviews quantitative methods and models of impact evaluation. The formal literature on impact evaluation methods and practices is large, with a few useful overviews. Yet there is a need to put the theory into practice in a hands-on fashion for practitioners. This book also details challenges and goals in other realms of evaluation, including monitoring and evaluation (M&E), operational evaluation, and mixed-methods approaches combining quantitative and qualitative analyses. This book is organized as follows. Chapter two reviews the basic issues pertaining to an evaluation of an intervention to reach certain targets and goals. It distinguishes impact evaluation from related concepts such as M&E, operational evaluation, qualitative versus quantitative evaluation, and ex-ante versus ex post impact evaluation. Chapter three focuses on the experimental design of an impact evaluation, discussing its strengths and shortcomings. Various non-experimental methods exist as well, each of which are discussed in turn through chapters four to seven. Chapter four examines matching methods, including the propensity score matching technique. Chapter five deal with double-difference methods in the context of panel data, which relax some of the assumptions on the potential sources of selection bias. Chapter six reviews the instrumental variable method, which further relaxes assumptions on self-selection. Chapter seven examines regression discontinuity and pipeline methods, which exploit the design of the program itself as potential sources of identification of program impacts. Specifically, chapter eight presents a discussion of how distributional impacts of programs can be measured, including new techniques related to quantile regression. Chapter nine discusses structural approaches to program evaluation, including economic models that can lay the groundwork for estimating direct and indirect effects of a program. Finally, chapter ten discusses the strengths and weaknesses of experimental and non-experimental methods and also highlights the usefulness of impact evaluation tools in policy making.Publication Evaluating the Impact of Mexico's Quality Schools Program : The Pitfalls of Using Nonexperimental Data(World Bank, Washington, DC, 2006-10)The authors evaluate whether increasing school resources and decentralizing management decisions at the school level improves learning in a developing country. Mexico's Quality Schools Program (PEC), following many other countries and U.S. states, offers US$15,000 grants for public schools to implement five-year improvement plans that the school's staff and community design. Using a three-year panel of 74,700 schools, the authors estimate the impact of the PEC on dropout, repetition, and failure using two common nonexperimental methods-regression analysis and propensity score matching. The methods provide similar but nonidentical results. The preferred estimator, difference-in-differences with matching, reveals that participation in the PEC decreases dropout by 0.24 percentage points, failure by 0.24 percentage points, and repetition by 0.31 percentage points-an economically small but statistically significant impact. The PEC lacks measurable impact on outcomes in indigenous schools. The results suggest that a combination of increased resources and local management can produce small improvements in school outcomes, though perhaps not in the most troubled school systems.Publication Policy Options for Meeting the Millennium Development Goals in Brazil : Can Micro-Simulations Help?(World Bank, Washington, DC, 2003-02)The authors investigate whether micro-simulation techniques can shed light on the types of policies that should be adopted by countries wishing to meet their Millennium Development Goals. They compare two families of micro-simulations. The first family of micro-simulations decomposes required poverty changes into a change in the mean and a reduction in inequality. Although it highlights the importance of inequality reduction, it appears to be too general to be of much use for policymaking. The second family of micro-simulations is based on a richer model of behavior in the labor markets. It points to the importance of combining different policy options, such as educational expansion and targeted conditional redistribution schemes, to ensure that the poorest people in society are successfully reached. But the absence of market equilibria in these statistical models, as well as the strong stability assumptions which are implicit in their use, argue for extreme caution in their interpretation.
Users also downloaded
Showing related downloaded files
Publication MIGA Annual Report 2013 : Insuring Investments, Ensuring Opportunities(Washington, DC: World Bank Group, 2013-10-11)In fiscal year 2013, Multilateral Investment Guarantee Agency (MIGA) issued 2.8 billion dollars in investment guarantees for projects in our developing member countries. At 1.5 billion dollars, representing more than half of new business, the bulk of MIGA's guarantees issued support investments in Sub-Saharan Africa. Sixty-nine percent of new business volume this year was in complex projects in infrastructure and extractive industries, a strategic priority for the Agency. This year, 82 percent of MIGA's new volume fell into one or more of strategic priority areas: investments in the world's poorest countries, "South-South" investments, investments in conflict-affected countries, and investments in complex projects. MIGA also established the conflict-affected and fragile economies facility to further deepen support to this priority area.Publication The Role of Social Ties in Factor Allocation(Published by Oxford University Press on behalf of the World Bank, 2019-10)We investigate whether social structure helps or hinders factor allocation using unusually rich data from the Gambia. Evidence indicates that land available for cultivation is allocated unequally across households; and that factor transfers are more common between neighbors, co-ethnics, and kinship-related households. Does this lead to the conclusion that land inequality is due to flows of land between households being impeded by social divisions? To answer this question, a novel methodology that approaches exhaustive data on dyadic flows from an aggregate point of view is introduced. Land transfers lead to a more equal distribution of land and to more comparable factor ratios across households in general. But equalizing transfers of land are not more likely within ethnic or kinship groups. In conclusion, ethnic and kinship divisions do not hinder land and labor transfers in a way that contributes to aggregate factor inequality. Labor transfers do not equilibrate factor ratios across households. But it cannot be ruled out that they serve a beneficial role, for example, to deal with unanticipated health shocks.Publication The Firm-Level Impact of the Covid–19 Pandemic(World Bank, Washington, DC, 2020-09-02)The World Bank commissioned a firm-level survey to provide quantitative evidence of the impact of the Coronavirus (COVID-19) pandemic. Two rounds of data have now been collected for the months of March and May using a nationally representative World Bank survey providing information on the impact of the Coronavirus (COVID-19) pandemic. The survey includes five hundred firms spanning a wide range of industries and firm sizes, as well as the formal and informal sector. This note provides a snapshot of how the firms’ outcomes and response to the pandemic have changed between the months of March and May 2020.Publication Impact of Migration on Economic and Social Development : A Review of Evidence and Emerging Issues(2011-02-01)This paper provides a review of the literature on the development impact of migration and remittances on origin countries and on destination countries in the South. International migration is an ever-growing phenomenon that has important development implications for both sending and receiving countries. For a sending country, migration and the resulting remittances lead to increased incomes and poverty reduction, and improved health and educational outcomes, and promote economic development. Yet these gains might come at substantial social costs to the migrants and their families. Since many developing countries are also large recipients of international migrants, they face challenges of integration of immigrants, job competition between migrant and native workers, and fiscal costs associated with provision of social services to the migrants. This paper also summarizes incipient discussions on the impacts of migration on climate change, democratic values, demographics, national identity, and security. In conclusion, the paper highlights a few policy recommendations calling for better integration of migration in development policies in the South and the North, improving data collection on migration and remittance flows, leveraging remittances for improving access to finance of recipient households and countries, improving recruitment mechanisms, and facilitating international labor mobility through safe and legal channels.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.