Publication: Evaluation of Large-Scale Combination HIV Prevention Programs : Essential Issues
No Thumbnail Available
Published
2011
ISSN
1944-7884 (Electronic)
1525-4135 (Linking)
Date
2012-03-30
Editor(s)
Abstract
HIV prevention research has shifted to the evaluation of combination prevention programs whereby biomedical, behavioral, and structural interventions are implemented concurrently to maximize synergies among interventions. However, these kinds of combination prevention packages, particularly when implemented at scale, pose significant evaluation challenges, including how best to determine impact and how and whether to measure the effectiveness of component strategies. In addition, methodological challenges unique to HIV infection such as the absence of a reliable incidence assay, the lack of naive control groups, and no suitable surrogates further complicate rigorous evaluation. In this commentary, we discuss the key considerations for planning impact evaluations of combination HIV prevention programs in light of these challenges, including defining the evaluable package, determining which component programs require independent assessment of impact, choosing study designs with valid counterfactuals, selecting appropriate outcomes of interest, and the importance of mid-course program corrections.
Link to Data Set
Digital Object Identifier
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Almost Random : Evaluating a Large-scale Randomized Nutrition Program in the Presence of Crossover(Washington, DC: World Bank, 2008-11)Large-scale randomized interventions have the potential to uncover the causal effect of programs applying to a large population, thereby improving on the insights gained from currently dominant smaller randomized studies. However, the external validity gained through larger interventions typically implies less supervision and often comes at the cost of some deviation from the randomization plan. This paper investigates the impact of the Nutrition Enhancement Program, which aims to improve child nutrition in Senegal based on a large-scale randomized community intervention. The analysis explicitly deals with deviation from the planned treatment and suggests approaches for combining ex-post adjustments such as propensity score matching with the randomized treatment plan. The authors do not detect a strong overall program impact on the outcome measure of weight-for-age based on planned treatment status, but do find an impact on the youngest children. Moreover, the project impact is clearer when the analysis considers treatment crossover using alternative estimators of two-stage least-squares and propensity score matching. The findings underscore the importance of addressing the shortcomings of large-scale randomization interventions in a systematic manner in order to understand the selection process that can guide further implementation of such projects, as well as to expose the true, causal effect of such programs.Publication Almost Random: Evaluating a Large-Scale Randomized Nutrition Program in the Presence of Crossover(2011)Large-scale randomized interventions have the potential to uncover the causal effect of programs applying to a large population, thereby improving on the insights gained from currently dominant smaller randomized studies. However, the external validity gained through larger interventions typically risks deviation from the randomization protocol. This paper investigates the impact of the Nutrition Enhancement Program, which aims to improve child nutrition in Senegal. The analysis deals with deviation from the planned treatment and suggests approaches for combining ex-post adjustments such as propensity score matching with the randomized treatment plan. The authors do not detect a strong overall program impact on the outcome measure of weight-for-age based on planned treatment status, but do find an impact on the youngest children. Moreover, the project impact is clearer when the analysis considers treatment crossover using alternative estimators of two-stage least-squares and propensity score matching. The findings underscore the importance of addressing the shortcomings of large-scale randomization interventions in a systematic manner to guide further implementation of such projects, as well as to expose the true causal effect of such programs.Publication Cash for Coolers : Evaluating a Large-Scale Appliance Replacement Program in Mexico(American Economic Association, 2014-11)This paper evaluates a large-scale appliance replacement program in Mexico that from 2009 to 2012 helped 1.9 million households replace their old refrigerators and air conditioners with energy-efficient models. Using household-level billing records from the universe of Mexican residential customers, we find that refrigerator replacement reduces electricity consumption by 8 percent, about one-quarter of what was predicted by ex ante analyses. Moreover, we find that air conditioning replacement actually increases electricity consumption. Overall, we find that the program is an expensive way to reduce externalities from energy use, reducing carbon dioxide emissions at a program cost of over $500 per ton.Publication Decentralized Beneficiary Targeting in Large-Scale Development Programs : Insights from the Malawi Farm Input Subsidy Program(World Bank, Washington, DC, 2013-11)This paper contributes to the long-standing debate on the merits of decentralized beneficiary targeting in the administration of development programs, focusing on the large-scale Malawi Farm Input Subsidy Program. Nationally-representative household survey data are used to systematically analyze the decentralized targeting performance of the program during the 2009-2010 agricultural season. The analysis begins with a standard targeting assessment based on the rates of program participation and the benefit amounts among the eligible and non-eligible populations, and provides decompositions of the national targeting performance into the inter-district, intra-district inter-community, and intra-district intra-community components. This approach identifies the relative contributions of targeting at each level. The results show that the Farm Input Subsidy Program is not poverty targeted and that the national government, districts, and communities are nearly uniform in their failure to target the poor, with any minimal targeting (or mis-targeting) overwhelmingly materializing at the community level. The findings are robust to the choice of the eligibility indicator and the decomposition method. The multivariate analysis of household program participation reinforces these results and reveals that the relatively well-off, rather than the poor or the wealthiest, and the locally well-connected have a higher likelihood of program participation and, on average, receive a greater number of input coupons. Since a key program objective is to increase food security and income among resource-poor farmers, the lack of targeting is a concern and should underlie considerations of alternative targeting approaches that, in part or completely, rely on proxy means tests at the local level.Publication Impact Evaluation of a Large-scale Rural Sanitation Project in Indonesia(World Bank, Washington, DC, 2013-02)Lack of sanitation and poor hygiene behavior cause a tremendous disease burden among the poor. This paper evaluates the impact of the Total Sanitation and Sanitation Marketing project in Indonesia, where about 11 percent of children have diarrhea in any two-week period and more than 33,000 children die each year from diarrhea. The evaluation utilizes a randomized controlled trial but is unusual in that the program was evaluated when implemented at scale across the province of rural East Java in a way that was designed to strengthen the enabling environment and so be sustainable. One hundred and sixty communities across eight rural districts participated, and approximately 2,100 households were interviewed before and after the intervention. The authors found that the project increased toilet construction by approximately 3 percentage points (a 31 percent increase in the rate of toilet construction). The changes were primarily among non-poor households that did not have access to sanitation at baseline. Open defecation among these households decreased by 6 percentage points (or 17 percent). Diarrhea prevalence was 30 percent lower in treatment communities than in control communities at endline (3.3 versus 4.6 percent). The analysis cannot rule out that the differences in drinking water and handwashing behavior drove the decline in diarrhea. Reductions in parasitic infestations and improvements in height and weight were found for the non-poor sample with no sanitation at baseline.
Users also downloaded
Showing related downloaded files
No results found.