Publication: Using Post-Double Selection Lasso in Field Experiments
Loading...
Date
2024-09-27
ISSN
Published
2024-09-27
Author(s)
Editor(s)
Abstract
The post-double selection Lasso estimator has become a popular way of selecting control variables when analyzing randomized experiments. This is done to try to improve precision, and reduce bias from attrition or chance imbalances. This paper re-estimates 780 treatment effects from published papers to examine how much difference this approach makes in practice. PDS Lasso is found to reduce standard errors by less than one percent compared to standard Ancova on average and does not select variables to model treatment in over half the cases. The authors discuss and provide evidence on the key practical decisions researchers face in using this method.
Link to Data Set
Citation
“Cilliers, Jacobus; Elashmawy, Nour; McKenzie, David. 2024. Using Post-Double Selection Lasso in Field Experiments. Policy Research Working Paper; 10931. © World Bank. http://hdl.handle.net/10986/42206 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Geopolitics and the World Trading System(Washington, DC: World Bank, 2024-12-23)Until the beginning of this century, the GATT/WTO system worked. Economic research provided a compelling explanation. It showed that if governments maximize the well-being of their own countries broadly defined, GATT/WTO principles would facilitate mutually beneficial cooperation over their trade policy choices. Now heightened geopolitical rivalry seems to have undermined the WTO. A simple transposition of the previous rationalization suggests that geopolitics and trade cooperation are not compatible. The paper shows that this is only true if rivalry eclipses any consideration of own-country well-being. In all other circumstances, there are gains from trade cooperation even with geopolitics. Furthermore, the WTO’s relevance is in question only if it adheres too rigidly to its existing rules and norms. Through measured adaptation to the geopolitical imperative, the WTO can continue to thrive as a forum for multilateral trade cooperation in the age of geopolitics.Publication Chinese Imports and Industrialization in Africa(Washington, DC: World Bank, 2025-05-12)The rise of China in the global economy has been linked with negative impacts on employment across many high- and middle-income countries. However, evidence for African countries is limited. This paper investigates the causal relationship between Chinese imports and manufacturing employment in Ethiopia. Imports may harm domestic firms through a revenue effect (lower market shares) or benefit them, indirectly if competition spurs innovation or directly through access to better quality or cheaper inputs. The analysis shows that a one unit increase in import penetration leads to a 15.2 percent increase in industry employment. The inputs effect is disentangled from the other two effects by decomposing total Chinese imports by their end-use category using input-output tables. The evidence shows that imported intermediate inputs are driving the employment gains. The findings are consistent with the idea that employment gains are a result of productivity gains and increases in capacity utilization. These employment gains appear to benefit large firms and labor-intensive industries disproportionately.Publication VAT Exemptions, Embedded Tax, and Unintended Consequences(Washington, DC: World Bank, 2025-05-15)The value-added tax (VAT) has proved to be a highly effective tool at raising revenue in developed and developing countries alike. However, the effective operation of the VAT breaks down in the presence of exemptions. Unlike zero rates, exemptions deny input tax credits, thereby increasing production costs and resulting in VAT being embedded within the prices of goods and services. This paper develops a VAT model based on input-output table and household budget survey data for 29 European countries to examine the effects of VAT exemptions on final prices and to assess the merits of their use. Simulation results show that exemptions suffer from the same targeting problems as reduced VAT rates, but, in addition, they are non-transparent and have unpredictable and counterproductive indirect effects. These effects are in addition to the well-known distortionary impact of exemptions on production decisions, and their creation of incentives to self-supply. The paper concludes that the use of exemptions should be limited to addressing pragmatic concerns, such as the disproportionate compliance costs of small businesses and the practical difficulty in taxing margin-based financial services.Publication Disentangling the Key Economic Channels through Which Infrastructure Affects Jobs(Washington, DC: World Bank, 2025-04-03)This paper takes stock of the literature on infrastructure and jobs published since the early 2000s, using a conceptual framework to identify the key channels through which different types of infrastructure impact jobs. Where relevant, it highlights the different approaches and findings in the cases of energy, digital, and transport infrastructure. Overall, the literature review provides strong evidence of infrastructure’s positive impact on employment, particularly for women. In the case of electricity, this impact arises from freeing time that would otherwise be spent on household tasks. Similarly, digital infrastructure, particularly mobile phone coverage, has demonstrated positive labor market effects, often driven by private sector investments rather than large public expenditures, which are typically required for other large-scale infrastructure projects. The evidence on structural transformation is also positive, with some notable exceptions, such as studies that find no significant impact on structural transformation in rural India in the cases of electricity and roads. Even with better market connections, remote areas may continue to lack economic opportunities, due to the absence of agglomeration economies and complementary inputs such as human capital. Accordingly, reducing transport costs alone may not be sufficient to drive economic transformation in rural areas. The spatial dimension of transformation is particularly relevant for transport, both internationally—by enhancing trade integration—and within countries, where economic development tends to drive firms and jobs toward urban centers, benefitting from economies scale and network effects. Turning to organizational transformation, evidence on skill bias in developing countries is more mixed than in developed countries and may vary considerably by context. Further research, especially on the possible reasons explaining the differences between developed and developing economies, is needed.Publication Economic Consequences of Trade and Global Value Chain Integration(World Bank, Washington, DC, 2025-04-04)This paper introduces a new approach to measuring Global Value Chains (GVC), crucial for informed policy-making. It features a tripartite classification (backward, forward, and two-sided) covering trade and production data. The findings indicate that traditional trade-based GVC metrics significantly underestimate global GVC activity, especially in sectors like services and upstream manufacturing, and overstate risks in early trade liberalization stages. Additionally, conventional backward-forward classifications over-estimate backward linkages. The paper further applies these measures empirically to assess how GVC participation mediates the impact of demand shocks on domestic output, highlighting both the exposure and stabilizing potential of GVC integration. These new measures are comprehensively available on the World Bank’s WITS Platform, providing a key resource for GVC analysis.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Testing the Importance of Search Frictions, Matching, and Reservation Prestige through Randomized Experiments in Jordan(World Bank Group, Washington, DC, 2014-09-01)Unemployment rates for tertiary-educated youth in Jordan are high, as is the duration of unemployment. Two randomized experiments in Jordan were used to test different theories that may explain this phenomenon. The first experiment tested the role of search and matching frictions by providing firms and job candidates with an intensive screening and matching service based on educational backgrounds and psychometric assessments. Although more than 1,000 matches were made, youth rejected the opportunity to even have an interview in 28 percent of cases, and when a job offer was received, they rejected this offer or quickly quit the job 83 percent of the time. A second experiment built on the first by examining the willingness of educated, unemployed youth to apply for jobs of varying levels of prestige. Youth applied to only a small proportion of the job openings they were told about, with application rates higher for higher prestige jobs than lower prestige jobs. Youth failed to show up for the majority of interviews scheduled for low prestige jobs. The results suggest that reservation prestige is an important factor underlying the unemployment of educated Jordanian youth.Publication In Pursuit of Balance : Randomization in Practice in Development Field Experiments(World Bank, Washington, DC, 2008-10)Randomized experiments are increasingly used in development economics, with researchers now facing the question of not just whether to randomize, but how to do so. Pure random assignment guarantees that the treatment and control groups will have identical characteristics on average, but in any particular random allocation, the two groups will differ along some dimensions. Methods used to pursue greater balance include stratification, pair-wise matching, and re-randomization. This paper presents new evidence on the randomization methods used in existing randomized experiments, and carries out simulations in order to provide guidance for researchers. Three main results emerge. First, many researchers are not controlling for the method of randomization in their analysis. The authors show this leads to tests with incorrect size, and can result in lower power than if a pure random draw was used. Second, they find that in samples of 300 or more, the different randomization methods perform similarly in terms of achieving balance on many future outcomes of interest. However, for very persistent outcome variables and in smaller sample sizes, pair-wise matching and stratification perform best. Third, the analysis suggests that on balance the re-randomization methods common in practice are less desirable than other methods, such as matching.Publication Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan(World Bank, Washington, DC, 2012-07)Throughout the Middle East, unemployment rates of educated youth have been persistently high and female labor force participation, low. This paper studies the impact of a randomized experiment in Jordan designed to assist female community college graduates find employment. One randomly chosen group of graduates was given a voucher that would pay an employer a subsidy equivalent to the minimum wage for up to 6 months if they hired the graduate; a second group was invited to attend 45 hours of employability skills training designed to provide them with the soft skills employers say graduates often lack; a third group was offered both interventions; and the fourth group forms the control group. The analysis finds that the job voucher led to a 40 percentage point increase in employment in the short-run, but that most of this employment is not formal, and that the average effect is much smaller and no longer statistically significant 4 months after the voucher period has ended. The voucher does appear to have persistent impacts outside the capital, where it almost doubles the employment rate of graduates, but this appears likely to largely reflect displacement effects. Soft-skills training has no average impact on employment, although again there is a weakly significant impact outside the capital. The authors elicit the expectations of academics and development professionals to demonstrate that these findings are novel and unexpected. The results suggest that wage subsidies can help increase employment in the short term, but are not a panacea for the problems of high urban female youth unemployment.Publication Beyond Baseline and Follow-up : The Case for More T in Experiments(2011-04-01)The vast majority of randomized experiments in economics rely on a single baseline and single follow-up survey. If multiple follow-ups are conducted, the reason is typically to examine the trajectory of impact effects, so that in effect only one follow-up round is being used to estimate each treatment effect of interest. While such a design is suitable for study of highly autocorrelated and relatively precisely measured outcomes in the health and education domains, this paper makes the case that it is unlikely to be optimal for measuring noisy and relatively less autocorrelated outcomes such as business profits, household incomes and expenditures, and episodic health outcomes. Taking multiple measurements of such outcomes at relatively short intervals allows the researcher to average out noise, increasing power. When the outcomes have low autocorrelation, it can make sense to do no baseline at all. Moreover, the author shows how for such outcomes, more power can be achieved with multiple follow-ups than allocating the same total sample size over a single follow-up and baseline. The analysis highlights the large gains in power from ANCOVA rather than difference-in-differences when autocorrelations are low and a baseline is taken. The paper discusses the issues involved in multiple measurements, and makes recommendations for the design of experiments and related non-experimental impact evaluations.Publication Propensity Score Matching and Policy Impact Analysis : A Demonstration in EViews(World Bank, Washington, DC, 2006-04)Effective development policymaking creates a need for reliable methods of assessing effectiveness. There should be, therefore, an intimate relationship between effective policymaking and impact analysis. The goal of a development intervention defines the metric by which to assess its impact, while impact evaluation can produce reliable information on which policymakers may base decisions to modify or cancel ineffective programs and thus make the most of limited resources. This paper reviews the logic of propensity score matching (PSM) and, using data on the National Support Work Demonstration, compares that approach with other evaluation methods such as double difference, instrumental variable, and Heckman's method of selection bias correction. In addition, it demonstrates how to implement nearest-neighbor and kernel-based methods, and plot program incidence curves in E-Views. In the end, the plausibility of an evaluation method hinges critically on the correctness of the socioeconomic model underlying program design and implementation, and on the quality and quantity of available data. In any case, PSM can act as an effective adjuvant to other methods.
Users also downloaded
Showing related downloaded files
Publication The Experience of Public Works Programs in Myanmar(World Bank, Washington, DC, 2015-07-07)Public Work Programs (PWPs) can be a useful tool for poverty reduction and social protection provision in Myanmar by presenting a coherent framework to achieve several objectives (seasonal income and food security, disaster recovery, community resilience, social cohesion). For PWPsto be effective and sustainable, design should respond to local conditions and implementation should engage government structures that provide a scalable platform.Publication Working Without Borders(Washington, DC: World Bank, 2023-07-24)Online gig work poses both opportunities and challenges for governments and workers. On the upside, it offers prospects for income generation, especially in developing countries, where most people work in low-productivity, low-quality, often informal jobs. The virtual and often temporary nature of gig work also provides flexibility for often neglected groups such as women, youth, migrants, and people with disabilities. These jobs could be a stepping-stone to bet¬ter-quality jobs for low-skilled workers by helping them learn critical digital skills and close the digital divide. But most gig jobs offer little to no protection for workers, with uncertain income streams and no clear career pathways. Depending on local labor regulations, many gig workers are not protected against unfair practices, abuse or injuries while working. Gig work also raises challenges for managing data security and privacy. The report examines how countries can navigate the promise and perils of online gig work. It reveals that the online gig workforce is much larger than previously assumed with an estimated 154 million to 435 million Online gig workers around the globe. For the first time ever, the report mapped and tracked regional platforms and gig workers who work in languages other than English. Key messages are: • Online gig work is expanding, accounting for up to 12% of the global labor force and is a growing source of income for millions. • Demand for online gig workers is rising faster in developing countries than in industrialized countries. • Local gig platforms play a vital role in the local labor market, but they face challenges in establishing a viable business model, and opportunities for long-term growth. • Online gig work can support inclusion by providing work opportunities for youth, women, and low-skilled workers. • Gig workers, like most other informal sector workers in developing countries, are often outside the purview of labor regulations. • The gig economy can offer opportunities locally to build digital skills, increase income-earning opportunities, and facilitate social protection coverage of informal workers.Publication Measuring Teaching Practices at Scale(World Bank, Washington, DC, 2018-11)What goes on inside the classroom is central to student learning. Despite its importance, low- and middle-income countries rarely measure teaching practices, in part due to a lack of access to adequate classroom observation tools and the high transaction costs associated with administering them. Teach, a new, open-source classroom observation tool for primary classrooms, was developed to capture the quantity and quality of teaching practices in these settings with a simple, easy-to-administer tool. This paper validates the use of Teach scores for system diagnostics by providing four types of evidence. First, it provides evidence that the practices included in the tool have a clear conceptual underpinning. Second, almost 90 percent of local observers in Mozambique, Pakistan, the Philippines, and Uruguay were highly accurate using Teach after a four-day training. Third, using data from 845 classrooms in Pakistan, the paper shows that Teach scores are internally consistent, present moderate to high inter-rater reliability in the field (.75 intraclass correlation coefficient), and provide substantial information that allows to differentiate teachers, even those with similar but not equal scores. Finally, teachers who display effective practices, as measured by Teach, are associated with students who achieve higher learning outcomes.Publication Equivalent Years of Schooling(World Bank, Washington, DC, 2019-02)In the past decade, hundreds of impact evaluation studies have measured the learning outcomes of education interventions in developing countries. The impact magnitudes are often reported in terms of "standard deviations," making them difficult to communicate to policy makers beyond education specialists. This paper proposes two approaches to demonstrate the effectiveness of learning interventions, one in "equivalent years of schooling" and another in the net present value of potential increased lifetime earnings. The results show that in a sample of low- and middle-income countries, one standard deviation gain in literacy skill is associated with between 4.7 and 6.8 additional years of schooling, depending on the estimation method. In other words, over the course of a business-as-usual school year, students learn between 0.15 and 0.21 standard deviation of literacy ability. Using that metric to translate the impact of interventions, a median structured pedagogy intervention increases learning by the equivalent of between 0.6 and 0.9 year of business-as-usual schooling. The results further show that even modest gains in standard deviations of learning -- if sustained over time -- may have sizeable impacts on individual earnings and poverty reduction, and that conversion into a non-education metric should help policy makers and non-specialists better understand the potential benefits of increased learning.Publication The Global Fund to Fight AIDS, Tuberculosis, and Malaria, and the World Bank's Engagement with the Global Fund(Washington, DC: World Bank Group, 2011-02-08)The principal purpose of this Global Program Review (GPR) is to learn lessons from the experience of the Global Fund and its interaction with the Bank in three areas: (a) the design and operation of large global partnership programs like the Global Fund that are financing country-level investments, (b) the engagement of the World Bank with these partnership programs, and (c) the evaluation of these programs. The Review has an intensive focus on the Bank's engagement with the Global Fund at the country level because of the potential for competition or collaboration between Global Fund-supported activities and the Bank's lending operations at the country level. Therefore, it also focuses on the design and operation of the Global Fund-supported activities at the country level. This review was initiated before the high-level independent review panel on fiduciary controls and oversight mechanisms of the Global Fund was commissioned in February 2011, and it was drafted before their final report, turning the page from emergency to sustainability, was issued on September 19, 2011. While the two studies are complementary and overlap to some extent, they were conducted independently of each other, for different audiences, and for different purposes.