Publication: On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics
Loading...
Files in English
249 downloads
Date
2017-10-01
ISSN
Published
2017-10-01
Author(s)
Editor(s)
Abstract
Estimation of empirical relationships is prone to bias. Economists have carefully studied sources of bias in structural and quasi-experimental approaches, but the randomized control trial (RCT) has only begun to receive such scrutiny. In this paper, we argue that several lessons from medicine, derived from analysis of thousands of RCTs establishing a clear link between certain practices and biased estimates, can be used to reduce the risk of bias in economics RCTs. We identify the subset of these lessons applicable to economics and use them to assess risk of bias in estimates from economics RCTs published between 2001 and 2011. In comparison to medical studies, we find most economics studies do not report important details on study design necessary to assess risk of bias. Many report practices that suggest risk of bias, though this does not necessarily mean bias resulted. We conclude with suggestions on how to remedy these issues.
Link to Data Set
Citation
“Eble, Alex; Boone, Peter; Elbourne, Diana. 2017. On Minimizing the Risk of Bias in Randomized Controlled Trials in Economics. World Bank Economic Review. © World Bank. http://hdl.handle.net/10986/31473 License: CC BY-NC-ND 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
World Bank Economic Review
1564-698X
Journal Volume
Journal Issue
Citations
- Cited 5 times in Scopus (view citations)
Collections
Related items
Showing items related by metadata.
No results found.
Users also downloaded
Showing related downloaded files
Publication South Asia Development Update, April 2024: Jobs for Resilience(Washington, DC: World Bank, 2024-04-02)South Asia is expected to continue to be the fastest-growing emerging market and developing economy (EMDE) region over the next two years. This is largely thanks to robust growth in India, but growth is also expected to pick up in most other South Asian economies. However, growth in the near-term is more reliant on the public sector than elsewhere, whereas private investment, in particular, continues to be weak. Efforts to rein in elevated debt, borrowing costs, and fiscal deficits may eventually weigh on growth and limit governments' ability to respond to increasingly frequent climate shocks. Yet, the provision of public goods is among the most effective strategies for climate adaptation. This is especially the case for households and farms, which tend to rely on shifting their efforts to non-agricultural jobs. These strategies are less effective forms of climate adaptation, in part because opportunities to move out of agriculture are limited by the region’s below-average employment ratios in the non-agricultural sector and for women. Because employment growth is falling short of working-age population growth, the region fails to fully capitalize on its demographic dividend. Vibrant, competitive firms are key to unlocking the demographic dividend, robust private investment, and workers’ ability to move out of agriculture. A range of policies could spur firm growth, including improved business climates and institutions, the removal of financial sector restrictions, and greater openness to trade and capital flows.Publication Media and Messages for Nutrition and Health(World Bank, Washington, DC, 2020-06)The Lao People’s Democratic Republic (Lao PDR) has experienced rapid and significant economic growth over the past decade. However, poor nutritional outcomes remain a concern. Rates of childhood undernutrition are particularly high in remote, rural, and upland areas. Media have the potential to play an important role in shaping health and nutrition–related behaviors and practices as well as in promoting sociocultural and economic development that might contribute to improved nutritional outcomes. This report presents the results of a media audit (MA) that was conducted to inform the development and production of mass media advocacy and communication strategies and materials with a focus on maternal and child health and nutrition that would reach the most people from the poorest communities in northern Lao PDR. Making more people aware of useful information, essential services and products and influencing them to use these effectively is the ultimate goal of mass media campaigns, and the MA measures the potential effectiveness of media efforts to reach this goal. The effectiveness of communication channels to deliver health and nutrition messages to target beneficiaries to ensure maximum reach and uptake can be viewed in terms of preferences, satisfaction, and trust. Overall, the four most accessed media channels for receiving information among communities in the study areas were village announcements, mobile phones, television, and out-of-home (OOH) media. Of the accessed media channels, the top three most preferred channels were village announcements (40 percent), television (26 percent), and mobile phones (19 percent). In terms of trust, village announcements were the most trusted source of information (64 percent), followed by mobile phones (14 percent) and television (11 percent). Hence of all the media channels, village announcements are the most preferred, have the most satisfied users, and are the most trusted source of information in study communities from four provinces in Lao PDR with some of the highest burden of childhood undernutrition.Publication Economic Recovery(World Bank, Washington, DC, 2021-04-06)World Bank Group President David Malpass spoke about the world facing major challenges, including COVID, climate change, rising poverty and inequality and growing fragility and violence in many countries. He highlighted vaccines, working closely with Gavi, WHO, and UNICEF, the World Bank has conducted over one hundred capacity assessments, many even more before vaccines were available. The World Bank Group worked to achieve a debt service suspension initiative and increased transparency in debt contracts at developing countries. The World Bank Group is finalizing a new climate change action plan, which includes a big step up in financing, building on their record climate financing over the past two years. He noted big challenges to bring all together to achieve GRID: green, resilient, and inclusive development. Janet Yellen, U.S. Secretary of the Treasury, mentioned focusing on vulnerable people during the pandemic. Kristalina Georgieva, Managing Director of the International Monetary Fund, focused on giving everyone a fair shot during a sustainable recovery. All three commented on the importance of tackling climate change.Publication Remarks at the United Nations Biodiversity Conference(World Bank, Washington, DC, 2021-10-12)World Bank Group President David Malpass discussed biodiversity and climate change being closely interlinked, with terrestrial and marine ecosystems serving as critically important carbon sinks. At the same time climate change acts as a direct driver of biodiversity and ecosystem services loss. The World Bank has financed biodiversity conservation around the world, including over 116 million hectares of Marine and Coastal Protected Areas, 10 million hectares of Terrestrial Protected Areas, and over 300 protected habitats, biological buffer zones and reserves. The COVID pandemic, biodiversity loss, climate change are all reminders of how connected we are. The recovery from this pandemic is an opportunity to put in place more effective policies, institutions, and resources to address biodiversity loss.Publication World Bank East Asia and the Pacific Economic Update, October 2024: Jobs and Technology(Washington, DC: World Bank, 2024-10-07)East Asia and the Pacific, seen in the context of the world economy, stands out as a paragon of development. Despite the recent ravages of the pandemic and the persistent tensions of geopolitics, the region is growing at stably high rates and the benefits are widely shared. But compared to its own past and potential, the region’s economic performance is less impressive. Growth is still below pre-pandemic rates, except in Indonesia, and output has not yet recovered to pre-pandemic levels in several countries, especially in the Pacific. This Economic Update highlights three key developments: shifting regional growth dynamics as China’s growth slows, changing trade patterns due to global tensions, and the impact of technologies such as robots, artificial intelligence, and digital platforms on jobs. The report calls for productivity-enhancing structural reforms to strengthen domestic growth drivers through; deeper international trade agreements to foster more open and stable trade regimes; deeper technical, digital, and soft skills while addressing impediments to labor mobility, factor price distortions and expanding social protection for workers in the digital informal economy to boost productivity and employment.