Publication: Can We Trust Shoestring Evaluations?
Loading...
Files in English
289 downloads
Published
2014-10-02
ISSN
1564-698X
Date
2016-05-03
Author(s)
Editor(s)
Abstract
Many more impact evaluations could be done, and at lower unit cost, if evaluators could avoid the need for baseline data using objective socio-economic surveys and rely instead on retrospective subjective questions on how outcomes have changed, asked post-intervention. But would the results be reliable? This paper tests a rapid-appraisal, “shoestring” method using subjective recall for welfare changes. The recall data were collected at the end of a full-scale evaluation of a large World Bank supported poor-area development program in China. Qualitative recalls on how living standards have changed are found to provide only weak and biased signals of the changes in consumption as measured from contemporaneous surveys. Importantly, the shoestring method was unable to correct for the selective placement of the program favoring poor villages. The results of this case study are not encouraging for future applications of the shoestring method, although similar tests are needed in other settings.
Link to Data Set
Citation
“Ravallion, Martin. 2014. Can We Trust Shoestring Evaluations?. World Bank Economic Review. © World Bank. http://hdl.handle.net/10986/24191 License: CC BY-NC-ND 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
World Bank Economic Review
1564-698X
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
No results found.
Users also downloaded
Showing related downloaded files
Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Thailand Monthly Economic Monitor, October 2025(Washington, DC: World Bank, 2025-10-22)Fiscal conditions remained stable, with a modest widening of the deficit to 3.1 percent of GDP. New stimulus measures are expected to support short-term demand without breaching the public debt ceiling. Inflation stayed negative, reflecting lower energy and food prices amid subdued domestic demand. The central bank kept the policy rate unchanged, citing limited policy space. Thailand’s growth momentum has slowed further as manufacturing activity and services weakened as projected. Tourism remained subdued, largely due to fewer Chinese visitors. Goods exports also slowed as earlier front-loaded orders faded, particularly in agriculture and industrial goods. The Thai baht depreciated in early October as the US dollar appreciated and the current account turned negative.Publication Regional Poverty and Inequality Update: Latin America and the Caribbean, October 2025(Washington, DC: World Bank, 2025-10-23)This brief summarizes recent facts related to poverty and inequality in Latin America and the Caribbean (LAC) using the latest wave of harmonized household surveys from the Socio-Economic Database for LAC (SEDLAC). This brief was produced by the Poverty Global Practice in the LAC Region of the World Bank.Publication Housing Subsidies for Refugees(Washington, DC: World Bank, 2025-01-22)Refugees require assistance for basic needs like housing but local host communities may feel excluded from that assistance, potentially affecting community relations. This study experimentally evaluates the effect of a housing assistance program for Syrian refugees in Jordan on both the recipients and their neighbors. The program offered full rental subsidies and landlord incentives for housing improvements, but saw only moderate uptake, in part due to landlord reluctance. The program improved short-run housing quality and lowered housing expenditures, but did not yield sustained economic benefits, partly due to redistribution of aid. The program unexpectedly led to a deterioration in child socio-emotional well-being, and also strained relations between Jordanian neighbors and refugees. In all, housing subsidies had limited measurable benefits for refugee well-being while worsening social cohesion, highlighting the possible need for alternative forms of aid.Publication Ukraine Country Environmental Analysis(World Bank, Washington, DC, 2016-01)The objective of the Country Environmental Analysis (CEA) is to assess the adequacy and performance of the policy, legal, and institutional framework for environmental management in Ukraine, in light of the decentralization process of environmental governance and wider reform objectives, and to provide recommendations to government to address the key gaps identified. Ukraine is the second largest country in Europe and has a population of 43 million, the majority of whom live in urban areas. It is a lower middle income country, with the services, industry and agriculture sectors being main contributors to the country’s Gross Domestic Product (GDP). Ukraine faces a number of environmental challenges, as identified in its National Environmental Strategy 2020 (NES). Key among these are: air pollution; quality of water resources and land degradation; solid waste management; biodiversity loss; human health issues associated with environmental risk factors; in addition to climate change. The scope of Ukrainian environmental legislation is quite broad and comprehensive (more than 300 legal acts) and covers most areas of environmental protection and natural resources management. However, the environmental legislation faces a number of weaknesses:The environmental legislation is largely declaratory in nature and does not have all the essential enforcement mechanisms for the implementation of legal acts and international agreements; Many of the acts are not coordinated with each other; and Legislation undergoes limited analysis of its impact—for example, no in-depth analysis such as Regulatory Impact Analysis is conducted for proposed pieces of legislation.