Publication: External Validity and Policy Adaptation: From Impact Evaluation to Policy Design
Loading...
Files in English
408 downloads
Date
2020-07
ISSN
1564-6971
Published
2020-07
Author(s)
Editor(s)
Abstract
With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context.
Link to Data Set
Citation
“Williams, Martin J.. 2020. External Validity and Policy Adaptation: From Impact Evaluation to Policy Design. World Bank Research Observer. © Published by Oxford University Press on behalf of the World Bank. http://hdl.handle.net/10986/36809 License: CC BY-NC-ND 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Citations
- Cited 25 times in Scopus (view citations)