Publication: Timing and Duration of Exposure in Evaluations of Social Programs
Loading...
Published
2008-08
ISSN
Date
2012-05-31
Author(s)
Editor(s)
Abstract
Impact evaluations aim to measure the outcomes that can be attributed to a specific policy or intervention. Although there have been excellent reviews of the different methods that an evaluator can choose in order to estimate impact, there has not been sufficient attention given to questions related to timing: How long after a program has begun should one wait before evaluating it? How long should treatment groups be exposed to a program before they can be expected to benefit from it? Are there important time patterns in a program's impact? Many impact evaluations assume that interventions occur at specified launch dates and produce equal and constant changes in conditions among eligible beneficiary groups; but there are many reasons why this generally is not the case. This paper examines the evaluation issues related to timing and discusses the sources of variation in the duration of exposure within programs and their implications for impact estimates. It reviews the evidence from careful evaluations of programs (with a focus on developing countries) on the ways that duration affects impacts.
Link to Data Set
Citation
“King , Elizabeth M.; Behrman, Jere R.. 2008. Timing and Duration of Exposure in Evaluations of Social Programs. Policy Research Working Paper No. 4686. © World Bank. http://hdl.handle.net/10986/6812 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication Direct and Indirect Impacts of Transport Mobility on Access to Jobs: Evidence from South Africa(Washington, DC: World Bank, 2025-11-12)Access to jobs is essential for economic growth. In Africa, unemployment rates are notably high. This paper reexamines the relationship between transport mobility and labor market outcomes, with a particular focus on the direct and indirect effects of transport connectivity. As predicted by theory, wages are influenced by the level of commuting deterrence. Generally, higher earnings are associated with longer commute times and/or higher commuting costs. Local accessibility is also important, especially for individuals with time constraints. Both direct and indirect impacts are found to be significant in South Africa, where job accessibility has been challenging since the end of apartheid. For the direct impact, the wage elasticity associated with commuting costs is significant. Returns on commute are particularly high for women. Local accessibility to socioeconomic facilities, such as shops and health services, is also found to have a significant impact, consistent with the concept of mobility of care. To enhance employment, therefore, it is crucial to connect people not only to job locations but also to various socioeconomic points of interest, such as markets and hospitals, in an integrated manner. This integration will enable individuals to spend more time working and commuting longer distances.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication From Policy to Practice: Lessons from the Implementation of the Refugee Work Rights Policy in Ethiopia(Washington, DC: World Bank, 2025-11-10)This paper examines the early implementation of Ethiopia’s refugee work rights policy, with a focus on the issuance of permits that enable refugees to engage in economic activities. Building on significant legal and institutional advances under the 2019 Refugee Proclamation and subsequent directives, the analysis explores how these reforms are being operationalized in practice. Using a mixed-methods approach, combining document review, administrative data analysis, and semi-structured interviews, the paper identifies both progress and remaining challenges. Permit issuance has increased since the adoption of detailed operational guidance in 2024, reflecting the Government of Ethiopia’s commitment to operationalizing its progressive legal framework and ensuring that refugees can exercise their right to work. However, take-up remains modest, with about 5.2 percent of the working-age population holding a permit. Preliminary evidence suggests that coordination gaps, limited subnational capacity, low awareness among refugees and employers, and disincentives to formalize in a largely informal labor market are contributing to the low take-up. The paper offers policy suggestions, grounded in the Ethiopian context and emerging evidence, to help translate legal commitments into improved labor market outcomes for refugees.Publication Monitoring Global Aid Flows: A Novel Approach Using Large Language Models(Washington, DC: World Bank, 2025-11-04)Effective monitoring of development aid is the foundation for assessing the alignment of flows with their intended development objectives. Existing reporting systems, such as the Organisation for Economic Co-operation and Development’s Creditor Reporting System, provide standardized classification of aid activities but have limitations when it comes to capturing new areas like climate change, digitalization, and other cross-cutting themes. This paper proposes a bottom-up, unsupervised machine learning framework that leverages textual descriptions of aid projects to generate highly granular activity clusters. Using the 2021 Creditor Reporting System data set of nearly 400,000 records, the model produces 841 clusters, which are then grouped into 80 subsectors. These clusters reveal 36 emerging aid areas not tracked in the current Creditor Reporting System taxonomy, allow unpacking of “multi-sectoral” and “sector not specified” classifications, and enable estimation of flows to new themes, including World Bank Global Challenge Programs, International Development Association–20 Special Themes, and Cross-Cutting Issues. Validation against both Creditor Reporting System benchmarks and International Development Association commitment data demonstrates robustness. This approach illustrates how machine learning and the new advances in large language models can enhance the monitoring of global aid flows and inform future improvements in aid classification and reporting. It offers a useful tool that can support more responsive and evidence-based decision-making, helping to better align resources with evolving development priorities.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication China Early Child Development : Early Childhood Education in Yunnan(Washington, DC, 2013-11-18)Yunnan is a medium-sized and relatively poor Chinese province on the southwestern border of China. In 2012, the Yunnan department of education formally requested Bank support in conducting a review of early childhood education policies and programs in order to gain an in-depth and evidence-based understanding of the challenges the province faces in expanding early childhood education-in particular to rural and mountainous regions. The Bank's China education team embarked on raising funds, designing and implementing a rather elaborate research agenda around early childhood education. The goal was to investigate key challenges, and to propose policy interventions for expanding the Early Child Development (ECD) coverage in rural Yunnan. This report presents the findings from the background studies, and draws potential policy implications for improving the access to and quality of preschool education in Yunnan province. China has now almost achieved universal 9-year basic education. Over the last decade, the country has devoted increasing attention to policy development in early childhood education. Even though China does not yet have a specific early childhood education law, it has established a rather elaborate set of guidelines and regulations pertaining to early childhood education. Early childhood education has expanded significantly within the last few years. There are two main types of preschool programs for 3-6 year olds including: a regular 3-year program which is called kindergarten, and a one-year program attached usually to primary schools. The rapid growth of preschool teacher supply has contributed to the drop in pupil-teacher ratios across the nation. In Yunnan in particular, the ratio has decreased from approximately 30 to 20 in recent years. However, urban areas still enjoy a more favorable pupil-teacher ratio, as well as a higher proportion of qualified teachers compared to rural areas. Rural areas account for 50 percent of total preschool enrollment, but only 22 percent of all trained teachers serve in rural areas.Publication Early Childhood Care and Development Project : Implementation Completion Memorandum(Washington, DC, 2008-04)The objective of the Project is to assist the Socialist Republic of Vietnam, through Save the Children UK on behalf of International Save the Children Alliance acting under the coordination of the Early Childhood Education Department of the Ministry of Education and Training (MOET) in establishing an enabling environment in the poorest and most vulnerable communities, and providing young children the opportunities to develop to their full potential. All communes in three selected provinces of project locations (Quang Tri, Yen Bai and Dien Bien) face food insecurity, limited access to markets, limited provision of basic infrastructures, poor health, inadequate education, high illiteracy, and environmental degradation. Despite the challenging contextual environment in which this grant has taken place, the grant objectives were largely highly satisfactorily met and fulfilled though four key intermediate project results: increased access and availability of early childhood care and development services and supplies; increased quality of early childhood care and development services (Health and Education); increased family and community awareness and capacity to promote utilization of early childhood care and development services; and improved implementation of policies in support of ethnic minority children.Publication Quality of education in Colombia - Achievements and challenges ahead : analysis of the results of TIMSS 1995-2007(World Bank, 2010-08-06)The main objective of this report is to analyze the effect of time changes and factors associated with student achievement in Colombia in order to foster policies to improve education quality that are grounded in research and the Colombian context. In 2007, Colombia participated for the second time in the Trends in International Mathematics and Science Study (TIMSS), providing a first-time opportunity to analyze the effects of time changes on student achievement over a period (1995-2007) during which a number of education reforms were made. Using the TIMSS 2007 results offers a chance to deepen the study on the factors associated with learning in Colombia and to benchmark Colombia's education system against that of other countries. This effort began during the first phase of the Colombia Programmatic Quality and Relevance of Education Analytic and Advisory Activities (AAA), in which an analysis of Colombia's debut in the 2006 Program for International Student Assessment (PISA), resulted in publication of, 'the quality of education in Colombia: an analysis and options for a policy agenda' report (hereafter, 'PISA report'). The present report builds on this work through an analysis of Colombia's participation in TIMSS 2007 in relation to its performance on TIMSS 1995, and reaffirms the urgent need for improved student learning outcomes in Colombia, further confirming a number of the policy options put forward in the PISA report to inform a future agenda for system design and reform.Publication Liberia - Public Expenditure Review : Human Development(Washington, DC, 2012-11)The human development sector (essentially, education, health, and social protection) plays an important role in promoting economic development and social equity, and improvement in these areas can have a salutary impact on peace and stability. This report is a compilation of three separate sector studies, covering education, health, and social protection. Notwithstanding that each study focuses on progress achieved in distinct areas, a number of cross cutting themes permeate the different chapters. The main cross-cutting threads can be summarized as follows: (i) overall public spending on human development is low given the needs of the population and is highly dependent on donor financing for a large number of activities, a situation that entails risks to sustainability and future development; (ii) intra sectoral allocation is not sufficiently pro-poor; (iii) resource allocation is inequitable; (iv) inefficiency in public expenditures, with large shares of funds going to overhead; leakage of wage funds; and low value for money; and (v) severe lack of data and capacity to monitor and analyze developments.Publication Snakes and Ladders : Factors Influencing Successful Primary School Completion for Children in Poverty Contexts(Washington, DC, 2004-09)The World Bank (New Delhi) commissioned this qualitative study in June 2002 with the objective of exploring various factors that facilitate or impede successful primary school completion, with special reference to children living in diverse poverty situations. The study looks at children from conception to 11 years - tracking key milestones of children in diverse poverty situations in one district each in the states of Andhra Pradesh, Karnataka and Uttar Pradesh. It explores the synergy between health, education and nutritional inputs in the overall development of children in different poverty situations and reflects on whether the proximal and distal environment promotes a holistic development of the child. This study includes the following headings: executive summary; background; through the life cycle of children; emerging issues; and policy implications.
Users also downloaded
Showing related downloaded files
Publication World Development Report 2006(Washington, DC, 2005)This year’s Word Development Report (WDR), the twenty-eighth, looks at the role of equity in the development process. It defines equity in terms of two basic principles. The first is equal opportunities: that a person’s chances in life should be determined by his or her talents and efforts, rather than by pre-determined circumstances such as race, gender, social or family background. The second principle is the avoidance of extreme deprivation in outcomes, particularly in health, education and consumption levels. This principle thus includes the objective of poverty reduction. The report’s main message is that, in the long run, the pursuit of equity and the pursuit of economic prosperity are complementary. In addition to detailed chapters exploring these and related issues, the Report contains selected data from the World Development Indicators 2005‹an appendix of economic and social data for over 200 countries. This Report offers practical insights for policymakers, executives, scholars, and all those with an interest in economic development.Publication The Journey Ahead(Washington, DC: World Bank, 2024-10-31)The Journey Ahead: Supporting Successful Migration in Europe and Central Asia provides an in-depth analysis of international migration in Europe and Central Asia (ECA) and the implications for policy making. By identifying challenges and opportunities associated with migration in the region, it aims to inform a more nuanced, evidencebased debate on the costs and benefits of cross-border mobility. Using data-driven insights and new analysis, the report shows that migration has been an engine of prosperity and has helped address some of ECA’s demographic and socioeconomic disparities. Yet, migration’s full economic potential remains untapped. The report identifies multiple barriers keeping migration from achieving its full potential. Crucially, it argues that policies in both origin and destination countries can help maximize the development impacts of migration and effectively manage the economic, social, and political costs. Drawing from a wide range of literature, country experiences, and novel analysis, The Journey Ahead presents actionable policy options to enhance the benefits of migration for destination and origin countries and migrants themselves. Some measures can be taken unilaterally by countries, whereas others require close bilateral or regional coordination. The recommendations are tailored to different types of migration— forced displacement as well as high-skilled and low-skilled economic migration—and from the perspectives of both sending and receiving countries. This report serves as a comprehensive resource for governments, development partners, and other stakeholders throughout Europe and Central Asia, where the richness and diversity of migration experiences provide valuable insights for policy makers in other regions of the world.Publication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Europe and Central Asia Economic Update, Fall 2024: Better Education for Stronger Growth(Washington, DC: World Bank, 2024-10-17)Economic growth in Europe and Central Asia (ECA) is likely to moderate from 3.5 percent in 2023 to 3.3 percent this year. This is significantly weaker than the 4.1 percent average growth in 2000-19. Growth this year is driven by expansionary fiscal policies and strong private consumption. External demand is less favorable because of weak economic expansion in major trading partners, like the European Union. Growth is likely to slow further in 2025, mostly because of the easing of expansion in the Russian Federation and Turkiye. This Europe and Central Asia Economic Update calls for a major overhaul of education systems across the region, particularly higher education, to unleash the talent needed to reinvigorate growth and boost convergence with high-income countries. Universities in the region suffer from poor management, outdated curricula, and inadequate funding and infrastructure. A mismatch between graduates' skills and the skills employers are seeking leads to wasted potential and contributes to the region's brain drain. Reversing the decline in the quality of education will require prioritizing improvements in teacher training, updated curricula, and investment in educational infrastructure. In higher education, reforms are needed to consolidate university systems, integrate them with research centers, and provide reskilling opportunities for adult workers.