Can We Trust Shoestring Evaluations?

Ravallion, Martin

doi:10.1093/wber/lht016

Publication: Can We Trust Shoestring Evaluations?

cbn

Files in English

English PDF (135.8 KB)

227 downloads

Date

2014-10-02

ISSN

1564-698X

Published

2014-10-02

Author(s)

Ravallion, Martin

Abstract

Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, asked post-intervention. But would the results be reliable? This paper tests a rapid-appraisal, “shoestring” method using subjective recall for welfare changes. The recall data were collected at the end of a full-scale evaluation of a large World Bank supported poor-area development program in China. Qualitative recalls on how living standards have changed are found to provide only weak and biased signals of the changes in consumption as measured from contemporaneous surveys. Importantly, the shoestring method was unable to correct for the selective placement of the program favoring poor villages. The results of this case study are not encouraging for future applications of the shoestring method, although similar tests are needed in other settings.

Citation

“Ravallion, Martin. 2014. Can We Trust Shoestring Evaluations?. World Bank Economic Review. © Oxford University Press on behalf of the World Bank. http://hdl.handle.net/10986/24191 License: CC BY-NC-ND 3.0 IGO.”