WPS6207 Policy Research Working Paper 6207 Handwashing Behavior Change at Scale Evidence from a Randomized Evaluation in Vietnam Claire Chase Quy-Toan Do The World Bank Sustainable Development Network Water and Sanitation Program & Development Research Group Poverty and Inequality Team September 2012 Policy Research Working Paper 6207 Abstract Handwashing with soap, which has been shown to reduce the household, and caregivers in the treatment group diarrhea in young children by as much as 48 percent, is were more likely to report washing hands at some of the frequently mentioned as one of the most effective and times emphasized by the campaign. However, observed inexpensive ways to save children’s lives. Yet rates of handwashing with soap at these times is low, and there handwashing remain very low throughout the world. isn’t any difference between the treatment and control Handwashing with soap campaigns are de rigueur in groups. As a result, no impact on health or productivity developing countries, but little is known about their is found. These results suggest that even under seemingly effectiveness. Few have been rigorously evaluated, and optimal conditions, where knowledge and access to soap none on a large-scale. This paper evaluates a large-scale and water are not main constraints, behavior change handwashing campaign in three provinces of Vietnam campaigns that take place on a large scale face tradeoffs in in 2010. Exposure to the campaign resulted in a slight terms of intensity and effectiveness. increase in the availability of handwashing materials in This paper is a joint product of the Water and Sanitation Program, Sustainable Development Network; and Poverty and Inequality Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at cchase@worldbank.org and qdo@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Handwashing behavior change at scale: Evidence from a randomized evaluation in Vietnam Claire Chase1 with Quy-Toan Do2 JEL codes: I, Q2 Key words: hygiene, handwashing, behavior change, randomized evaluation 1 World Bank, WSP, corresponding author. 2 World Bank Development Economics Research Group. Claire Chase is Evaluation Specialist at the World Bank Water and Sanitation Program. Quy- Toan Do is Senior Economist in the Development Research Group at the World Bank. The Global Scaling Up Handwashing project impact evaluation task team is led by Bertha Briceño (in its early stages was led by Jack Molyneaux), together with Alexandra Orsola-Vidal and Claire Chase. Professor Paul Gertler has provided guidance and advice throughout the impact evaluation. Advisors also include Sebastian Galiani, Jack Colford, Ben Arnold, Pavani Ram, Lia Fernald, and Patricia Kariger. The authors are grateful to Nga Nguyen and Minh Hien Thi Nguyen, the country task team for project implementation in Vietnam, Almud Weitz, Regional Team Leader for WSP-East Asia and Pacific region, and to Eduardo Perez, overall task team leader for the Scaling Up project. Generous financial support was provided by the Bill and Melinda Gates Foundation. 1. INTRODUCTION Preventable diseases resulting from poor hygiene behavior are responsible for a tremendous disease burden among the world‟s poor, especially infants and children under five years old. Globally, diarrheal disease is said to contribute to more child deaths than HIV/AIDS, Tuberculosis and Malaria combined. A large body of evidence suggests that improvements in hygiene behavior and handwashing with soap in particular, can reduce diarrheal disease substantially. For example, a recent systematic review of observational and experimental studies cites reductions in diarrhea of 48 percent for handwashing with soap (Cairncross, et al., 2010), and a synthetic review carried out by the International Initiative for Impact Evaluation (3ie) of impact evaluations in water, sanitation and hygiene found that handwashing at critical times including before eating or preparing food and after using the toilet can reduce diarrhea rates by almost 40 percent (Waddington, et al., 2009). Handwashing works by interrupting the transmission of harmful pathogens obtained through contact with human feces in the environment. When ingested these pathogens cause diarrhea and other gastro-enteric infections and lead to longer term adverse outcomes for young children who are infected, including growth faltering, malnutrition, and cognitive and learning impairments (see (The World Bank, 2008) for a complete review). It has been called the „do-it-yourself‟ vaccine, yet despite its low cost and proven benefits, rates of handwashing with soap are very low throughout the developing world (The World Bank, 2005). Campaigns employing a range of methods are de rigueur in developing countries, however little is known about the effectiveness of these campaigns in getting people to wash their hands with soap. Few have been rigorously evaluated, and none on a large-scale. Where evaluations have been done, they are often under trial conditions, with provision of soap and close follow-up of trial participants (see for example Haggerty, et al., 1994; Luby, et al., 2005; Ejemot, et al., 2009). While the interventions studied have proven effective in reducing diarrhea morbidity, they are not feasible on a large scale due to the vast amount of resources they require. Thus, rigorous evidence on the effectiveness of handwashing behavior change promotion in real-world settings is lacking. In December 2006, the Water and Sanitation Program (WSP) began implementation of a large-scale hygiene project, called Global Scaling Up Handwashing (HWWS), with funding from the Bill & Melinda Gates Foundation. The HWWS project set out to learn how to apply a combination of commercial marketing and public health promotional approaches to behavior change to generate large scale and sustainable improvements in handwashing with soap. The project also spearheaded efforts to strengthen the enabling environment of local and national governments, NGOs and local implementing agencies to carry out handwashing promotion 2 beyond the lifetime of the project. The overarching goal of the project was to stimulate and sustain handwashing with soap behavior at critical times in 5.4 million people in Peru, Senegal, Tanzania, and Vietnam over the four years of the project. This in turn was hypothesized to lead to improvements in child health and development outcomes and increases in household productivity. As part of the HWWS monitoring and evaluation plan, the project incorporated a randomized controlled trial impact evaluation (IE) in each of the four countries to rigorously test the effectiveness of these approaches to handwashing promotion in caregivers of children under five. The results of the impact evaluation in Vietnam suggest that handwashing with soap behavior in the target population has not changed substantially as a result of the intervention, and thus no health or productivity impacts are found. Knowledge about the correct way to wash hands was found to be high at baseline, and while the intervention led to an increase in knowledge about some of the key times for handwashing, it had little differential effect on already high access to soap and water in households and only modest effects on the self-reported handwashing behavior of mothers of children under five. Structured observations of handwashing show that rates of handwashing with soap at key junctures, especially after defecation and contact with child‟s feces, are very low among the target group, and no differences are found between treatment and control groups. More often, caregivers are foregoing soap to rinse their hands with water only. These results suggest that even under seemingly optimal conditions where knowledge and access to soap and water for handwashing are not main constraints, behavior change campaigns that intend to reach a mass audience face tradeoffs in terms of effectiveness. The remainder of this paper proceeds as follows. In the next section, we describe the setting for the study, the design of the campaign and the intervention components that were evaluated. Section 3 describes the theory of change, the evaluation design, and discusses threats to identification of the counterfactual such as baseline balance and sample attrition. In section 4 we describe the estimation strategy and Section 5 presents the main results of the impact evaluation along the causal chain. Section 6 discusses potential reasons for the limited behavior change impacts found and Section 7 concludes. 2. BACKGROUND AND DESCRIPTION OF THE PROGRAM Vietnam is a lower middle income country in Southeast Asia bordered by China to the north, Lao PDR to the northwest and Cambodia to the southwest, and with a GNI per capita in 2010 of $3,070 (PPP, current international $). Seventy-two percent of the population of 86 million live in rural areas. Access to water and sanitation infrastructure in Vietnam is high, even in rural areas, with 92% of the rural population having access 3 to an improved water source and 75% of the total population having access to an improved sanitation facility (World Development Indicators, 2008). Alongside rapid economic growth, Vietnam has witnessed remarkable improvements in child health over the past several decades and successfully halved infant and child mortality rates well ahead of the 2015 Millennium Development Goals (MDG) deadline.3 In spite of this, diarrheal diseases and acute respiratory infections remain two of the most common causes of child illness and deaths. The most recent national level surveys report 2 week prevalence of these diseases at 6.8 and 6.3% respectively (GSO, 2006) but these figures appear to mask the burden of child malnutrition that still exists in Vietnam. In particular, child stunting, or low height for age, still affects more than one-third of children in Vietnam. (NIN, 2010). The setting for this research differs in important ways when compared with national level indicators. To begin with, poverty in Vietnam is largely concentrated in ethnic minority and mountainous communities. Two of the three provinces selected for the handwashing campaign, Hung Yen in the North and Tien Giang in the South have poverty rates that are well below the national poverty rate in 2006 of 20% at 11.9% and 6.2% respectively. Thanh Hoa province on the North Central Coast, however, has a higher poverty rate of 36.1% (Nguyen, et al., 2010). Secondly, 95% of the study sample identifies with the Kinh majority ethnic group, the largest of 54 officially recognized ethnic groups in Vietnam, and represented by 85.7% of the population nationally.4 Finally, on key water, sanitation, and health indicators the study sample meets or exceeds the national level statistics. Ninety-six percent of households have access to an improved water source, 68% improved sanitation, 81% a place for handwashing with soap and water available, and just 14% of children under two are stunted at baseline. Vietnam was selected for the HWWS project due to its engagement in handwashing promotion under the Public Private Partnership for Handwashing (PPPHW), a global handwashing initiative established in 2001. Handwashing promotion activities began in 2006 under the name of the Handwashing Initiative (HWI) with various partners including the Vietnam Ministry of Health, Women‟s Union and WSP. The HWWS project sought to improve and scale-up these efforts through capacity building of local organizations, such as the Vietnam Women‟s Union, training them in behavior change approaches, and providing technical support for development of behavior change communication campaign materials. WSP-supported HWWS activities targeted at caregivers and schoolchildren have been underway since 2007 in 7 provinces. These activities had a target of reaching over 2 million mothers and children with the goal of improving the handwashing 3 MDG 4 seeks to reduce infant and child mortality by two-thirds 4 Vietnam Population and Housing Census, 2009 4 behavior of 750,000, or approximately 37.5% of those reached (Water and Sanitation Program, 2011). The impact evaluation study covers a period of the caregiver communications campaign that was implemented between January and October 2010 in 3 of the 7 provinces, selected to be geographically representative of the north, central and southern regions of Vietnam. We refer to the intervention evaluated under this impact evaluation as the handwashing interpersonal communication campaign (HWIPC) in order to differentiate it from the larger and more comprehensive Vietnam Handwashing Initiative. Hygiene behavior change, and handwashing campaigns in particular, have been regular components of disease prevention, water, and sanitation projects in Vietnam. Since the 1940s handwashing has been an integral part of national social development efforts by the government of Vietnam, including one major campaign effort known as the Three Cleans Movement which sought to educate the population on clean food, water and living conditions. Past campaigns have emphasized threat of disease as the main driver to get people to change their behavior, and have primarily focused on increasing knowledge about handwashing and transmission of disease in the target population. However, recent evidence from handwashing behavior change research suggests this may not be the most effective way to change behavior, but rather promotional messages should appeal to the emotion, habits and motivations of the target audience (Curtis, et al., 2009). The HWIPC campaign was designed using a conceptual behavior change framework developed by WSP known as FOAM (Focus on Opportunity, Ability and Motivation)5 (Coombes & Devine, 2010) . The framework draws on a range of well-known behavior change theories and models in health, psychology and the social sciences which hypothesize that a particular set of internal and external factors determine individual behavior, and that interventions which target these determinants will lead to behavior change. The HWIPC campaign identified mothers of children under five as the primary target audience, but included other caregivers of young children, such as grandparents, in the target group given their involvement in childrearing in rural Vietnam. The available evidence suggests that effective handwashing among caregivers of young children; that is, handwashing with the right materials (soap and water) and at the right times (after contact with feces and before touching food) will reduce diarrheal disease burden in infants and children under five. As such, the caregiver campaign emphasized handwashing at critical times, after defecation and cleaning a child‟s bottom, before food preparation and before feeding children, rather than frequent or regular handwashing or handwashing by other family members. 5 The FOAM framework is based on the PERForM model developed by Population Services International (PSI) 5 In order to identify the handwashing behavior change determinants specific to this target audience and behavior, the HWIPC carried out audience research in 2007 with mothers of children under 5 in rural and peri-urban areas from 8 provinces geographically representative of Vietnam. The research used a range of methods to elicit determinants of caregiver handwashing with soap, including product (soap) trials, in- depth interviews, focus group discussions, structured observations of handwashing and face-to-face structured surveys. The research found that the majority of mothers who report washing their hands do not find soap necessary and the tendency is to wash hands only when they smell or are visibly soiled. In addition, researchers found that soap was widely available in households but that cleansing agents were generally found in toilet and bathing areas, and far from cooking facilities (Indochina Research (Vietnam) Ltd., 2007). The findings from the formative research led to the design of a communication campaign that focused on changing beliefs and addressing other motivational barriers to handwashing with soap. More specifically, messages highlighted the importance of using soap to wash hands and that even clean-looking and clean- smelling hands can harbor germs (belief determinants). The campaign sought to promote handwashing as something practiced by „good mothers‟ to ensure the health and development of their children (locus of control determinant). In addition, communications activities emphasized the need to make soap and water available for handwashing (access and availability determinant). A series of materials were developed for the campaign including a television ad with the messages “Hands are not clean if you wash only with water, soap is needed� and “Wash your hands with soap for the health and development of children�; posters showing the four key junctures for HWWS with the tagline: “Remember to wash your hands with soap for children‟s health and development�; a paper handout of the four key junctures poster for people to stick on their walls to remind them to wash hands with soap; and various promotional items such as stickers, hand clappers, and washcloths printed with the campaign logo and tagline. The study evaluates the joint effect of mass media and interpersonal communication (IPC) activities at the community level compared with mass media alone6. Given the lack of control group for the mass media arm we are unable to evaluate the effect of this component separately. The components are described in detail below: 6 The HWWS Project initially included a third component of direct consumer contact (DCC) activities. These were intended to be 1-day events in each treatment community. Due to difficulty of finding qualified firms and the limited scale of the DCC interventions the DCC activities were never carried out. This component was thus dropped from the study. 6 Component 1—Mass Media Campaign: The mass media campaign features 30 second and 15 second television spots carried out across ten popular national and regional channels. The frequency and timing of the spots varied over time in an effort to reach the target audience as often as possible. The television ad makes use of the popular Vietnamese tradition of proverbs and songs to teach children and incorporated the song „Five Clean Fingers‟, the lyrics of which are: “One plus one is two. Two plus two is four. Four plus one is five. All five fingers clean.� The campaign comprised a total of 363 national and 165 local television spots that ran from March 2010 to January 2011. Component 2—Interpersonal Communication (IPC) Activities: The Vietnam Women‟s Union (VWU), a mass organization with over 13 million members, carried out an extensive training program for over 14,000 (20-26 per commune) village health workers, teachers, and Women‟s Union members on how to promote group and household level IPC activities that reinforce handwashing with soap behavior in the target population. These trained handwashing motivators were responsible for carrying out IPC activities in their communities, including group meetings with mothers of children under five, grandparents, and women between the ages of 18-49, household visits, market meetings, loudspeaker announcements, Women‟s Union club meetings, handwashing with soap festivals, cooking contests, and distribution of HWWS informational and promotional materials at key locations in the village. These activities took place over a period of approximately 9 months. 3. EVALUATION DESIGN The study uses a cluster-randomized controlled trial impact evaluation to establish the causal linkages between the HWIPC campaign and the outcomes of interest. The study‟s theory of change is illustrated in Figure 1. As discussed in the previous section, the project carried out formative research, leading to the design of a behavior change communication campaign comprising mass media and interpersonal communication activities with the target audience. These activities were hypothesized to change behavioral determinants of handwashing, resulting in increased access and availability of handwashing materials in the household, individual behavior change, and reduced household and environmental contamination. Finally, improved handwashing behavior among caretakers was hypothesized to result in reductions in diarrhea and acute-respiratory infections in young children, relieving caregivers of the burden of caring for their sick children and resulting in increased time for more productive activities. 7 Figure 1: HWIPC campaign theory of change 3.1. Treatment assignment and sample selection Since the intervention was carried out at the commune administrative level, and was designed to be confined to the geographic borders of the commune, treatment assignment was made at the commune level. 7 Starting with a list of 401 communes across 18 districts in the three intervention provinces, a total of 15 rural districts (huyện) were selected by the VWU to participate in the experimental phase of the HWIPC. These districts were selected because of dense population and willingness, commitment, and capacity of VWU staff to carry out the planned activities. Five districts were chosen in Hung Yen, 4 in Thanh Hoa, and 6 in Tien Giang. Within the 15 selected districts a total of 315 rural and urban communes (xã and Thị trấn) were used as the sampling frame. The sample was first stratified by province to account for regional variation between the provinces. Then, within each province communes were matched into groups of three using the Mahalanobis matching metric, to minimize the statistical distance between the units based on population size, number of households, and geographic location (coastal, flat, or mountainous area). A total of 70 groups of three were then randomly selected into the study (24 in Hung Yen, 20 in Thanh Hoa and 26 in Tien Giang).8 As a final step, the communes in each group of three were randomly assigned to one of three arms to account for the original design of the evaluation that comprised 7 A Vietnamese commune is an administrative sub-division of the district. The average population of the communes in this study is 7,577 people (1,807 households) with a population range of 409 to 27,898 (172 to 5,531 households) 8 The VWU imposed a restriction that no more than 40 communes could be selected for implementation of the HWIPC project in Thanh Hoa province, thus only 20 matched triplets were selected. 8 of two separate treatment arms.9 A total of 140 communes were assigned to treatment and 70 to control. 10 1112 The study was designed to detect a 20% relative reduction in the primary outcome indicator of diarrhea in children under five. This called for a sample size of 3,150 households; 15 households in each commune with at least one child under the age of two at baseline. The study focused on households with children under two in order to capture changes in outcomes for the age range during which children are most sensitive to changes in hygiene in their environment. To assess impact, outcomes in the treatment group (D) are compared against outcomes in the control group (C). Both the treatment and control group comprise a representative sample of the population of households in intervention communes with at least one child under the age of two at baseline, however the sample is not representative of the Vietnam, nor should it be taken to represent the communes and districts where the HWIPC campaign took place. 13 Approximately one month prior to the baseline survey a list of all children under the age of two was obtained from the health post in each commune. From this listing a random selection of 15 households was made with an additional 10 replacement households selected to accommodate households that refused to participate in the survey, or to replace households that did not meet eligibility criteria at the time of the survey. Households in which specially trained community motivators lived were excluded from the sample, since these volunteers would later play a role in delivering handwashing messages to the community. For structured observations carried out at the endline survey a sample of 600 households (2 – 3 per commune) were randomly selected from among the 3,150 households. 3.2. Data collection Data were collected for this study in three rounds. For the first round, a baseline survey was carried out from September to December 2009 before the intervention began in 3,149 households. A midline monitoring survey was carried out in the same households in July 2010. Finally, 1 – 4 months after 9 A direct consumer contact (DCC) component of the intervention was dropped during the implementation stage but these communes were maintained in the study 10 The remaining 191 communes were not part of the evaluation sample and did not receive the IPC project interventions, but were exposed to the national and regional level TV ad campaign. 11 For map of intervention communes see (Chase & Do, 2010) 12 Random assignment of treatment, whereby a statistically random selection of communities receives the treatment and the remainder serve as controls, gives us a robust counterfactual to measure the causal effect of the HWIPC intervention. The randomization process ensures that on average the treatment and comparison groups are equal in both observed and unobserved characteristics, (Hernan, et al., 2004) and that an appropriate counterfactual can be measured. 13 See (Chase & Do, 2010) for discussion of representativeness of the sample 9 intervention activities had ended in October 2010,14 an endline household survey was carried out in 3,147 households (December 2010 – March 2011). Approximately 5.7% of households could not be reinterviewed at the endline survey mostly due to temporary relocation. These households were replaced by the next household on the list of replacement households that had been pre-selected during the baseline. A community survey was also carried out in the 210 communes with village and commune officials during the baseline and endline rounds of data collection. See Table 1 below for an overview of data collected throughout the study. Table 1: Data collection tools Instrument Baseline Midline Endline Description (Sep-Dec (July ‘10) (Dec ’10 – ‘09) Mar ‘11) Household Conducted in all households. Includes: questionnaire roster, demographics, labor, income, assets, spot-check observation of �  � handwashing facilities, handwashing behavior, handwashing determinants, dwelling characteristics, water sources, drinking water, and sanitation Child health Conducted in all households. Includes: questionnaire � � � caregiver reported health symptoms for 7 day recall Exposure module Conducted in all households. Includes:  � � caregiver exposure to HWIPC campaign Community Conducted with group of key questionnaire informants in all communes. Includes: socio-demographics of community, �  � accessibility, connectivity, education and health facilities, water and sanitation facilities, and government or other development projects Anemia & Conducted on all children < 2 years. Anthropometrics Includes: height, weight, head and arm �   circumference, hemoglobin measurement Structured Conducted in 600 households. Includes: observations of 3 hour observation of household handwashing   � activities to record handwashing behavior of primary caregiver of oldest child under two years 14 IPC activities were most intensive from January to July 2010 10 The survey instrument was developed by a group of experts in economics, epidemiology, child development, nutrition, and behavior change. All instruments were translated into Vietnamese, back- translated into English, and pre-tested prior to use in the field. Data collection instruments were administered to respondents in Vietnamese by native speakers. The National Institute of Hygiene and Epidemiology (NIHE) was contracted to conduct the field work for the baseline and midline survey and Mekong Economics (MKE) was contracted for the endline survey. With support from the principal investigator and the global IE team, the survey firm trained field supervisors and enumerators on all data collection protocols and instruments during a 3-day training in each province, separately for baseline and endline rounds of data collection. Field teams for the baseline survey consisted of one supervisor who oversaw quality control of the interviews, one health technician responsible for interviewing the household, and one laboratory technician responsible for child anthropometrics and hemoglobin measurement, in addition to two members of commune level health cadres responsible for backstopping child growth measures and administering the child health calendar for the baseline and midline survey. Activities in each province were supervised by a field manager from NIHE. Survey team structure for the endline survey was largely similar. A provincial field manager oversaw two teams, each comprising one supervisor and six enumerators. Since anemia and anthropometrics were not part of the endline survey, specially trained health technicians were not necessary. Therefore, each enumerator was responsible for administering the household questionnaire and structured observations. The community questionnaire was administered by a senior enumerator. The survey teams were supported by a commune liaison officer who was responsible for setting up household interviews and ensuring adequate replacement households in the case of households no longer willing to participate, respondent unavailability and / or vacant households. 3.3 Baseline balance and sample attrition The baseline report for this study presented a series of balance tests comparing each treatment group (D1 and D2) to the control group. Due to changes in implementation during the study that removed differences between the two treatment groups, and the loss of three communes due to changes in administrative borders, here we recalculate baseline balance between the merged treatment groups with 11 the control group.15 Tables 2.A through 2.J present mean comparison tests16 between treatment and control for key variables collected during the baseline survey. The null hypothesis of equality of means was rejected at the 10% level in 8.4% of the tests on key characteristics (12 out of 143 tests). Random chance would predict we reject the null 10% of the time, so we are confident that the randomization was carried out successfully. Along key handwashing behavior, child health and development outcomes there is balance between the treatment and control arm at baseline, except for length/height for age z-score, which is -0.756 standard deviations below the median in the treatment arm as opposed to -0.604 in the control arm (p=0.069). However, the raw length measures are balanced so the difference is more likely due to age variation. The study was able to successfully follow up over 94% of households across the three rounds of data collection, with no differential attrition found between treatment and control arms (Table 3A). The households that were successfully followed up were compared across key baseline demographic and socioeconomic characteristics to determine if they differed in important ways that could be correlated with the treatment (Table 3B). We find no significant differences between treatment and control panel households. Additionally, we conduct mean comparison tests for those households that were included as replacement households during the follow-up survey (Table 3C). Several characteristics, such as number of livestock owned, availability of soap and water at or near the toilet and soap available anywhere in the home are higher in the control arm, suggesting that these household are somehow better off, at least along these dimensions. When we compare the entire endline sample (panel plus replacement) on characteristics presumably independent of treatment, we find there are still differences in livestock ownership and that control households are more likely to have access to an improved water source. While access to an improved water source is an important characteristic, it is over 95% for both groups and thus is not likely to help explain much of the variation in outcomes across households. Therefore, we maintain the full sample of panel plus replacement households in all models for estimation of impact. In addition to household attrition we analyze attrition of caregivers as they are the primary target of the HWIPC campaign and the main respondent for the household survey (Table 3A). Importantly, the study defines the primary caregiver as the person who has spent the most time with the child over the past 6 months, usually the mother. Between baseline and endline approximately 26% of primary caregivers 15 Balance tests using original treatment assignment are available in the Scaling Up Handwashing Behavior: Findings from the Impact Evaluation Baseline Survey report 16 The standard errors used in the comparison of means tests were clustered at the commune level, allowing for the possibility of intra-commune correlation. 12 changed. New caregivers are on average older, less educated and more likely to be male.17 In rural Vietnam when a child reaches the age at which he or she is no longer breastfeeding it is common for mothers to leave him with the grandparents or other family members during the daytime while she works in the fields.18 This could explain the shift in demographics of the primary caregiver, since both grandmothers and grandfathers tend to take on the role of caregiver in these cases. Since the HWIPC intervention was targeted at caregivers, including grandparents, we leave these new caregivers in the sample for estimation of program impact on outcomes at the caregiver level but include a dummy in the adjusted models to indicate there has been a change in caregiver. 4. EVALUATION METHODOLOGY Since the evaluation of the HWIPC campaign was prospectively designed as a randomized experiment the evaluation is relatively straightforward. Given a robust counterfactual generated through random assignment we are able to assess the causal impact of the HWIPC campaign by simply comparing average outcomes between those communes assigned to treatment to those communes assigned to control. This is what is known as the intention-to-treat parameter (ITT). Randomized assignment of Treati in the HWIPC campaign ensures that E (�i | Treati )  0 such that the unadjusted OLS estimates of � will be unbiased. Additionally, we may choose to control for baseline characteristics known to be strongly correlated with the outcome. Since observed characteristics were balanced across the treatment arms at baseline, inclusion of these covariates does not change the results, but results in more precision of the estimates.19 Where possible, we control for the outcome at baseline. Both unadjusted and adjusted estimates are reported. To examine the overall impact of the HWIPC campaign, for each outcome of interest, we estimate the ITT parameter using the following regression for 207 of the original 210 communes (unadjusted model)20: (1) Yi  a  �Treati  � i 17 Throughout the paper we sometimes refer to the primary as „she‟ even though 5.7 percent of primary care givers are male in the study sample 18 Based on personal communication with project staff 19 While inclusion of time-invariant Xi can increase the efficiency of the estimates, it may also lead to bias since randomization does not ensure that E (�i | Xi, Treati )  0 . See (Freedman, 2006) and (Lin, 2011). 20 Three of the original 210 communes are dropped from the analysis since administrative borders were reassigned during the study, Thoi Son and Phuoc Thanh communes did not receive the handwashing intervention although at the time of treatment assignment they were assigned to treatment. Four villages of Thanh Phu, originally a control commune, were moved to Phuoc Thanh commune, assigned to treatment. 13 Where Yi is the outcome of interest for individual or household i, Treati is a dummy equal to 1 if the household has been randomized to treatment and 0 otherwise. Where outcomes are highly correlated between baseline and endline we add the outcome observed at baseline to the right hand side of the equation to increase the precision of our estimates21, as shown below in Model 2 (lag dependent variable model): (2) Yi  a  �Treati  �Yi1  � i Where Yi1 is the lagged dependent variable for individual or household i at baseline. In Model 3 (full model) we add caregiver age, education and sex for caregiver level outcomes, and caregiver education, child age and sex for child level outcomes. In addition we add an indicator for the province and month of interview to account for seasonality of some health outcomes and the systematic difference between when the endline survey was carried out in treatment group 2 (D2) and treatment group 1 (D1)/control communities, Xi : (3) Yi  a  �Treati  �Yi1  �X i  � i Finally, Model 4 (DID) is the double difference estimate where the parameter of interest is � : (4) Yi  a  �Treati  �ti  � (Treati  ti )  � i All regression estimates include triplet dummies for each treatment and control matched triplet, and cluster-randomized standard errors since the experiment was clustered at the commune level. 5. ESTIMATING PROGRAM EFFECTS In this section we present the main results of the HWIPC campaign. Results are presented along the hypothesized causal chain of handwashing behavior change (Figure 1) leading from exposure to the campaign and its messages, the effect of campaign exposure on knowledge of handwashing, the effect of the intervention on changes in behavioral determinants of handwashing, handwashing behavior, child health and, finally, caregiver productivity. The results tables are organized by outcome grouping following (Figure 1) with each row representing a different outcome. The main results are shown in Tables 4 – 10 for the entire sample. Reading the tables 21 This is highly unlikely for acute child illness outcomes (see Schmidt 2011 for example) 14 from left to right, Column 1 is the mean and standard deviation of the variable at baseline in 2009, if available22. Otherwise, the first column is the mean and standard deviation of the outcome variable for the control group at follow-up (midline for exposure and endline for all other outcomes). The next column is the coefficient and standard error on the outcome variable from the intention to treat estimation equation (1) presented in section 4.1 above. Since we estimate a linear probability model the coefficient can be interpreted as the marginal effect of treatment such that the average outcome in the treatment arm is the sum of the control mean and the coefficient. The remaining columns show the conditional mean effect of treatment on the outcome variable from the intention to treat estimation equation for the respective models presented in section 4.1. Impact estimates reported in the text are absolute percentage point differences estimated from the full model except for household exposure to the campaign (Table 4) which reports on the unadjusted model. 5.1. Exposure to the HWIPC campaign As the HWIPC was designed as a behavior change communication campaign, expected impact depends on sufficient exposure to the campaign. Program designers theorized that the most effective way to change behavior would be to reach the target audience with a consistent message through multiple channels. Thus, the campaign used 4 distinct channels: mass media TV ads, print materials, interpersonal communication (IPC) through household visits or face-to-face group meetings, and community events. Exposure to these channels and messages was measured midway through the campaign by asking primary caregivers whether they remember seeing or hearing anything about handwashing in the past month. 23 The respondent was asked to spontaneously recall what messages she heard, from whom or where she heard them, and the frequency of the exposure. Caregivers in the treatment group were approximately 10% more likely to report high exposure to the campaign, defined as exposure through more than 3 channels. They were likewise more likely to have talked to someone from the Women‟s Union about handwashing in the past month (56.4% in treatment vs. 34.7% in control). However, on average respondents in the control arm reported exposure to handwashing messages via 2.8 channels and 46% reported exposure to more than 3 channels. At least one of these 22 Given underreporting of child health symptoms at baseline difference-in-difference estimates are not presented for child health outcomes 23 The exposure module asked about activities that took place over the previous month. A one month recall period was used in order to balance recall bias with targeting to the intervention. All activities were underway during the month prior to the midterm survey. Exposure was measured at the endline survey as well, however since the question asked about exposure in the past month and the intervention had ended more than three months before endline, these results are not analysed here. 15 channels, the mass media TV ad, was aired in both treatment and control communities, and thus we might expect control households to report hearing or seeing something about handwashing in the past month from 1 channel on average. However, it is not clear why nearly 50% of control arm respondents reported exposure to handwashing promotion through 3 or more channels. It could be that messages about handwashing are now more common in rural Vietnam due to recent concerns surrounding H1N1 and Avian influenza infection. Likewise, private soap companies rigorously promote their products in rural Vietnam and routinely carry out large marketing campaigns, sometimes in partnership with researchers and government agencies24, so exposure to handwashing messages could come from sources other than the HWIPC. 5.2. Effect of the HWIPC campaign on caregiver knowledge, beliefs and attitudes Prior phases of the handwashing initiative, carried out by the MOH, had spent their efforts on educating the target audience on how, when and why to wash hands with soap, and formative research had shown that knowledge was already high in the target audience. Baseline figures confirmed this, with 79.4% of caregivers reporting that the best way to wash hands is with soap and water. Thus, educating the target audience was not an explicit objective of the campaign (Nguyen, et al., 2011). Instead, the campaign sought to influence the motivating factors and barriers that were identified as determinants of handwashing to effectively translate that knowledge into action. Knowledge is measured here as a necessary, but not sufficient, requirement for behavior change. Overall, knowledge about the best way to wash hands increased from a baseline of 79.4% to 97.3% at endline in the control arm, with a similar increase found in the treatment arm. Additionally, knowledge about the importance of handwashing and soap use in prevention of diarrhea was found to be higher in the treatment group (84.9% in control vs. 87.8% in treatment). The HWIPC campaign emphasized four critical junctures for handwashing: After defecation and touching a child‟s feces, before food preparation and before feeding / breastfeeding a child. While a majority of respondents from both arms know that handwashing after using the toilet is important (77%), nearly11% more respondents in the treatment arm stated „before preparing food‟ as an important time for handwashing with soap (29.6% in control vs. 40.5% in treatment). 24 Soap company Lifebuoy claims that nearly half of the brand‟s soap consumption occurs in rural areas of Asia where the majority of the population lives on less than $1 per day 16 As described above, the HWIPC campaign sought to influence motivating factors and address barriers to effective handwashing using the FOAM framework. Behavioral determinants such as locus of control, beliefs and access/availability were measured in the target population using a series of Likert scale25 statements, and the behavioral constructs tracked over time to see how they respond to the program.26 These findings are presented in Tables 6A and 6B. In particular, we find that some of the statements measuring beliefs were responsive to the program (e.g. You only need to wash hands with soap, if they look dirty or smell bad; You only need to wash hands with soap if you touch unhygienic objects). In the treatment group 3.7% (41.2% in control vs. 44.9% in treatment) and 6.2% (43.2% in control vs. 49.4% in treatment) more caregivers disagreed with these statements respectively, i.e. they gave the correct answer. In addition, some of the statements measuring automaticity or habit were also found to be responsive to the program (e.g. You start washing your hands before you realize you are doing it; You wash your hands with soap automatically; You have to think about it each time you wash your hands with soap), although the effects are small. As we discuss below, despite some movement along these measures in response to the campaign, we do not observe changes in handwashing behavior as hypothesized. 5.3. Effect of the HWIPC campaign on handwashing behavior Asking respondents to self-report their behavior is a simple, low-cost metric commonly used to obtain population estimates of the prevalence of handwashing behavior and to gauge the effectiveness of handwashing campaigns. However, self-reported measures alone cannot be relied upon to give accurate point estimates of prevalence of handwashing behavior. Due to the perceived social desirability of handwashing these measures are subject to substantial bias (Stanton, et al., 1987; Manun'Ebo, et al., 1997; Biran, et al., 2008). Respondents may over-report handwashing because they believe it is the right thing to do or because they fear being judged by the interviewer and others around them. Several methods have been developed and tested in an effort to get more valid rates of handwashing. These methods have had differing degrees of success, validity, reliability and cost (Ram, 2010). In this study, several different methods were used to triangulate the results as well as to gather evidence on the reliability and bias of particular measures (Ram, et al., 2012). The results of the campaign in 25 A Likert scale evaluates a series of statements in which the respondent is asked to provide their level of agreement / disagreement with the statement. In this study the following levels were used: strongly disagree, disagree, agree, strongly agree. The statements administered did not provide an option for „neither agree nor disagree‟, however respondents could provide the response „Don‟t Know‟ 26 Factor analysis of the Likert scale was unable to reveal a robust underlying construct for any of the behavioral determinants, so only individual statements are analysed here. See (Hernandez, 2012) for discussion of the analysis of behavioral determinants in Vietnam. 17 regards to handwashing behavior are presented as three separate panels in Tables 7 – 9: availability of handwashing facilities and cleansing agents, self-reported handwashing with soap and observed hand cleanliness, and structured observations of handwashing behavior. Facilities for handwashing were observed in each household at baseline and endline. Respondents were asked whether or not family members wash hands with soap after using the toilet and before food preparation. If they responded yes, enumerators asked to observe where handwashing took place. At each handwashing place in the household enumerators recorded the distance in meters to either the toilet or food preparation area, the type of handwashing device (basin, bucket, etc.), and whether water and soap were available at the place for handwashing. At baseline 80.7% of households already had access to a place for handwashing with soap and water at or near the place of defecation and 76.4% had a place at or near the food preparation area of the household. Moreover, some type of cleansing agent (liquid, bar or powder soap) was available in nearly all households surveyed (96.2% at baseline). Even with this high baseline, a general trend is observed over time in both groups whereby soap and water in both places (defecation and food preparation) increased by approximately 10 percentage points (see Figure 2). A statistically higher increase in soap and water availability at the place of defecation is observed in the treatment arm (89.0% vs 91.6%), but the magnitude of the difference between treatment and control at endline is small. Figure 2: Trend in availability of handwashing facilities and cleansing agents Note: Figures show the trend from baseline (2009) to endline (2011) on availability of soap and water for handwashing in the household for treatment group receiving mass media (MM) and interpersonal communication activities (IPC) versus control group receiving mass media only 18 To measure self-reported handwashing behavior, caregivers were asked to recall over a period of 24 hours prior to the survey the circumstances under which they last washed their hands with soap. They were then asked to report all other times they washed their hands with soap during this time. At baseline 59.9% of caregivers on average reported to wash their hands with soap after fecal contact (after using toilet and after contact with child‟s feces), 30% before food preparation, and 31.6% before feeding or breastfeeding a child 27. At follow-up self-reported handwashing increased at all critical junctures measured (see Figure 3), with statistically significant differences between treatment and control in self-reported handwashing after fecal contact (68.1% vs. 73%) and before feeding/breastfeeding a child (36.3% vs. 41.4%). Figure 3: Trend in self-reported handwashing behavior at critical times Note: Figures show the trend from baseline (2009) to endline (2011) on self-reported handwashing measures for treatment group, receiving mass media (MM) and interpersonal communication activities (IPC) versus control group receiving mass media only Given the known biases of self-reported behavior, the study emphasized a series of objective measures of handwashing, including visual inspection of the hands of caregivers to assess cleanliness, and observations of handwashing in a sub-sample of households selected for the endline survey.28 There were no evident differences in cleanliness of caregivers‟ hands between treatment and control when controlling 27 For comparison of self-reported measures see WASH Child Nutrition report (after defecation 36.2%; before eating 22.8%, before preparing food 19%; after helping child stool 14.9%) (MOH 2010) 28 Structured observations were not carried out during the baseline survey 19 for covariates. However, overall hand cleanliness appears to have decreased at follow-up, but this could be due to the difficulty of standardizing such observations. Direct observation of handwashing is considered the gold standard for measuring handwashing, although even these are subject to bias if the person being observed changes his or her behavior in the presence of an observer (Ram, et al., 2010). Furthermore, the reliability and repeatability of this measure has been questioned (Cousens, et al., 1996). Nevertheless, structured observations are the most objective measure of handwashing available in this study. Observations were carried out in a subset of 600 households in treatment and control arms. They took place prior to the main endline survey over a 3 hour period in the morning, typically from 6:00AM to 9:00 AM29, and focused on the primary caregiver of the oldest child under 2 in the household, and the child under his or her care. Enumerators received extensive training on how to conduct structured observations and were instructed to be discreet and unobtrusive and to focus their observation on exposure events, i.e. food preparation, feeding children, eating, after using the toilet, not on when handwashing took place. Households were informed that the enumerator would be observing daily household activities. The analysis of structured observations focuses on the primary caregiver since she is the main target audience for the intervention. The results are presented separately for two handwashing behaviors: rinsing with water only and using soap. Unadjusted estimates indicate rinsing of hands before food preparation is more common in treatment households, and that overall the percentage of exposure events that were accompanied by handwashing is higher in the treatment group, however we find no evidence of a difference between treatment and control on observed handwashing rates when controlling for covariates. Furthermore, the rates observed suggest that handwashing with soap is still only practiced by a minority of the target population. As demonstrated in previous studies, observed handwashing in this study is considerably lower than self- reported behavior. Figure 4 presents self-reported and observed handwashing behavior side by side to illustrate this variation. Whereas 69.2% of caregivers in the treatment group report to wash hands with soap after fecal contact, only 25.5% of fecal related exposure events were observed to be accompanied by handwashing with soap.30 The differences hold for all critical junctures measured. Keeping in mind that proxy measures that use availability of a place for handwashing with soap and water present suggest handwashing prevalence after defecation of around 91% in treatment and 89% in control it is clear that 29 During pretesting of the survey this time period was recommended by households and community members as the most opportune time to observe the caretaker activities in the home 30 On average caregivers were observed during 1.54 fecal contact events, 1.29 child feeding events and 1.87 food preparation events per household in the structured observation sample 20 self-report and proxy may serve as indicators of the direction of impact, but cannot be relied on to provide anywhere near accurate point estimates. Figure 4: % Caregivers reporting handwashing with soap vs. % exposure events accompanied by soap use 100.0% 90.0% 80.0% 69.2% 68.1% 70.0% % Caregivers 60.0% 50.0% 39.5% 40.0% 34.3% 36.3% 31.1% 25.5% 30.0% 24.0% 20.0% 7.1% 7.1% 10.0% 4.6% 6.3% 0.0% HW with soap after fecal HW with soap before food HW with soap before contact preparation feeding a child Self-report Control Self-report Treatment Observe Control Observe Treatment Note: Figures are unadjusted means in treatment and control arms Given the inherent bias in self-reports and low reliability of proxy indicators, can we trust the observed measures to provide accurate point estimates? To begin with, comparisons between self-report and observed measures demonstrate that the structured observations in this study were likely not subject to the high levels of reactivity that have been found in other studies (Ram, et al., 2010). However, we cannot rule out reactivity and the levels observed (25.5% after fecal contact, 7.1% before food preparation, and 21 6.3% before feeding a child) should be viewed as upper bounds of actual behavior. Second, the prevalence of handwashing with soap suggested by these findings is in line with other recent studies that have observed handwashing behavior. One recent 11 country review found the prevalence of handwashing after using the toilet to be 17% on average (Curtis, et al., 2009). A baseline study in rural Bangladesh (Halder, et al., 2010) observed 33% of mothers washing their hands with soap after defecation and fewer than 1% before eating and feeding a child. Washing hands with water only was more common: 23% were observed to do so after defecation and 5% before eating. These findings are in line with results for the treatment group in Vietnam (35 and 6% respectively). 5.4. Effect of HWIPC on child health and caregiver productivity Given the lack of substantial changes in handwashing behavior resulting from the HWIPC campaign we do not anticipate impacts on child health outcomes or caregiver productivity. Indeed, we find no impact on caregiver reported diarrhea symptoms or acute respiratory infection31. However, symptoms indicative of acute lower respiratory infection, short breath with cough or difficulty breathing, are reportedly lower in the treatment group and the differences are large: approximately 34% relative reductions (prevalence is 4.4% in control vs. 2.9% in treatment). Handwashing with soap has previously been shown to be associated with reductions in lower respiratory infections (Rabie & Curtis, 2006), but the evidence is far from conclusive. Without concurrent compliance data showing handwashing behavior has increased in the treatment arm these results should be viewed with caution. As we might expect given that no impacts were found on child health, there was no impact on caregiver productivity, measured as instances of time lost to care for a sick child. 6. DISCUSSION In this section we present some hypotheses that have emerged as potential explanations for the limited impacts of the HWIPC campaign on handwashing behavior and describe some limitations of the study. While none of these hypotheses have been properly tested, they should nevertheless help put the findings into context and potentially guide future research. 31 The analysis excludes health symptoms measured at baseline. Extremely low prevalence of diarrhea (1.2 percent 7-day recall period prevalence) and other health symptoms led the research to conclude that health symptoms were likely underreported. 22 Given the large scale of the HWIPC campaign and the use of a mass organization in partnership with national government agencies to implement the intervention, there was limited control by the project implementing team over intervention activities at the commune and village level, with the result that actual implementation of the intervention reflects more closely „real-world‟ conditions as opposed to trial conditions. Efforts were made to standardize the behavior change messages and delivery of those messages by village motivators, but there is no way to confirm that motivators „stayed on message‟. In fact, field supervision visits by the country implementation team suggested that some motivators tended to overemphasize health messages and germ theory and reverted to delivering the messages in a didactic manner, despite training that emphasized participatory methods. In terms of whether the activities actually took place, the Women‟s Union reported on the number and type of activities that were carried out in each commune and district, and the number of participants at each event as part of its contractual obligation with WSP. These monitoring data show that all activities (and in some cases more) took place as planned in the time frame expected. While systematic third party monitoring data on these activities do not exist, the program implementation team expressed confidence that the activities were carried out. The HWIPC campaign relied on a common formula for behavior change in Vietnam: commune meetings, face-to-face communication, loudspeaker announcements and mass media print and television, with grassroots behavior change led by mass organizations like the Women‟s Union32. The methodology, in the case of the HWIPC, necessarily relied on the active participation and engagement of mothers and caregivers during meetings and home visits in order to be fully effective. However, engaging all mothers and caregivers in a commune can be a challenge given the competing demands that rural households face. Qualitative research on gender norms in Vietnam has shown that rural Vietnamese women are often overwhelmed by the demands of private and public commitments and find it difficult to meet the expectations of their family and community to simultaneously care for children, work on the farm and tend to the household chores, in addition to attending obligatory commune and club meetings (Schuler, et al, 2006). Most meetings with mothers were scheduled in the evenings to maximize participation, but it‟s possible that women did not actively attend or engage in meetings given the constraints on their time. Village handwashing motivators were instructed to visit the homes of caregivers who failed to attend, but even a few absentee mothers at each meeting would mean an additional burden on the motivator. Since it was these meetings and participation in one-on-one conversations that differentiated the treatment arm from the control arm, full participation seems a necessary (although perhaps not sufficient) criterion for behavior change. 32 The Farmer‟s Union, Communist Youth Union, and Vietnam Veteran‟s Association are other mass organizations found in nearly every rural commune that are active in poverty reduction programs in rural Vietnam 23 Large-scale effectiveness trials are subject to contamination of control areas by contemporaneous projects, or alternatively, ongoing health and development projects that could have an impact on the outcomes studied across both treatment and control arms, leading to no differential impacts in the treatment arm. To investigate this possibility, data were collected in January 2012 from provincial authorities on other hygiene or health related activities and projects that were underway from 2009 – 2010. We found several concurrent provincial wide projects underway during the study period. In particular, the education and communication component of the National Target Program on Rural Water Supply and Sanitation (NTP II) in Tien Giang reportedly emphasized HWWS through training of village health workers and worked with VWU as a partner. All three provinces reported Avian Influenza (AI) projects over the period as well as yearly diarrhea and general disease prevention campaigns. Many of these projects make use of similar channels (mass media, IPC, group meetings) and methodology (cascade training) as the HWIPC. Handwashing specific campaigns over the study period include the Unilever supported „Share Love Not Germs‟ campaign alongside the Ministry of Health Department of Preventive Health in 10 provinces (2006 – present), the UNICEF Water, Sanitation and Health Program and various NGO programs (Plan International, Church World Services, East Meets West, etc). The timing of the endline survey itself in relation to the intervention implementation schedule determines what effects ultimately get measured. Some communes in the study were surveyed up to four months after IPC activities had completed in October 2010,33 but the most intensive IPC activities happened between January and July 2010, followed by less intensive household visits and an additional group meeting in October 2010. Therefore, a full eight months had passed in some communes between when the „intensive‟ implementation ended and follow up measures were taken. If changes in behavior were stimulated by the intervention and over time these behaviors diminished we would expect the pooling of D1 and D2 treatment groups to have a downward bias on the behavior change estimates. However, analysis of impact for only those communities in the D1 treatment group does not reveal any differences when compared with the combined D1 and D2 results (results available upon request). Since no midline measures of behavior change are available we cannot say with certainty whether or not substantial changes in behavior were ever realized, however the lack of impact on child health symptoms provides further confirmation that handwashing behavior was not meaningfully impacted. An alternative explanation might be that the implementation period was too condensed and the time between baseline and endline measures too short to capture changes in behavior. 33 Mass media television ads ran through January 2011 24 7. CONCLUSION Washing hands with soap has been shown to reduce diarrhea in young children by as much as 48%, and may also contribute to reductions in acute respiratory infection, the two main killers of infants and young children worldwide. Despite the effectiveness of handwashing and the ubiquity of handwashing messages in hygiene, water, sanitation and health campaigns, very few people practice this behavior the right way and at the right time. This paper presents the results of an impact evaluation of a large-scale handwashing with soap behavior change campaign that used mass media television advertisements, door-to-door visits and group meetings to communicate handwashing messages to caregivers of children under five. To our knowledge it is the first randomized evaluation of a large-scale behavior change campaign that focused specifically on handwashing with soap. In contrast to small scale and highly controlled studies, this study does not find large health impacts, which suggests that previous estimates are likely overstated for handwashing campaigns that take place in real-world contexts. A forthcoming cost-benefit analysis conducted alongside the impact evaluation will demonstrate whether given these limited impacts, large scale handwashing campaigns can provide good value for money. Knowledge of the importance of handwashing with soap and the correct way to wash hands with soap is nearly universal among caregivers in this study. While the HWIPC campaign resulted in increased knowledge about some of the key times for handwashing, it had little differential effect on already high access to soap and water in households and only modest effects on the self-reported handwashing behavior of mothers of children under five. Structured observations reveal that more handwashing is not taking place in the treatment arm when compared with the control arm and that rates of handwashing with soap at key junctures, especially after contact with feces, are very low among the target group. As a result, we do not observe improvements in child health or caregiver productivity gains. The results in Vietnam are consistent with other countries that were part of the study. In the three countries that have completed an impact analysis the findings show the intervention to be successful in reaching the target audience and improving knowledge along some dimensions measured. However translating this knowledge into changes in handwashing behavior has been uneven. Moreover, the results in Vietnam suggest that even under seemingly optimal conditions, where knowledge and access to soap and water are not main constraints, changing behavior is difficult. This study targeted caregivers from a relatively advantaged population. Diarrhea prevalence is low in the households studied and child growth measures taken at baseline indicate just a small proportion of children are clinically malnourished. Furthermore, based on indicators of access to improved sanitation, 25 safe water sources and safe drinking water treatment practices, fecal contamination of the environment that these children live in might be expected to be small. Caregivers may be more likely to change their behavior when they feel an immediate and personal threat that their children will fall ill, but when children are observed to be relatively healthy caregivers may lack appropriate incentives to change their behavior. Additionally, although we do not observe large impacts on handwashing behavior in this study, we cannot rule out the possibility that had the study been done on a more vulnerable population, that the rates of handwashing observed would have resulted in health impacts. Indeed, handwashing with soap may be a more effective preventative measure when these other environmental health improvements are not in place. This is the subject of ongoing research.34 Handwashing with soap remains a key preventive measure, especially in areas with a high burden of diarrheal disease and malnutrition in children, and the results of this study are not likely to change the priority placed on hygiene by the water and health communities. Other research carried out by WSP has shown the handwashing initiative to be successful in strengthening local capacity to implement large- scale hygiene campaigns and encouraging policies that integrate handwashing with soap into national guidelines on water, sanitation and health. The impact evaluation was not designed to measure the effectiveness of these efforts, nor is it able to measure the long-term impact of this improved enabling environment for handwashing with soap in Vietnam. 34 See for example the WASH Benefits project, also funded by the Bill and Melinda Gates Foundation, that is testing the individual and combined effects of different water, sanitation and hygiene interventions (http://www.gatesfoundation.org/watersanitationhygiene) 26 REFERENCES Biran, A., Rabie, T., Schmidt, W., Juvekar, S., Hirve, S., & Curtis, V. (2008). Comparing the performance of indicators of hand-washing practices in rural India households. Tropical Medicine and International Health , 13, 278-285. Cairncross, S., Hunt, C., Boisson, S., Bostoen, K., Curtis, V., Fung, I., et al. (2010). Water, sanitation and hygiene for the prevention of diarrhea. International Journal of Epidemiology , i193-205. Chase, C., & Do, Q.-T. (2010). Scaling Up Handwashing Behavior: Findings from the Impact Evaluation Baseline Survey in Vietnam. Washington DC: Water and Sanitation Program. Coombes, Y., & Devine, J. (2010). Introducing FOAM: A Framework to Analyze Handwashing Behaviors to Design Effective Handwashing Programs. Washington DC: Water and Sanitation Program. Cousens, S., Kanki, B., Toure, S., Diallo, I., & Curtis, V. (1996). Reactivity and repeatability of hygiene behavior: structured observations from Burkina Faso. Social Science and Medicine , 43 (9), 1299-308. Curtis, V. A., Danquah, L. O., & Aunger, R. V. (2009). Planned, motivated and habitual hygiene behaviour: an eleven country review. Health Education Research , 1-19. Ejemot, R., Ehiri, J., Meremikwu, M., & Critchley, J. (2009). Cochrane review: Hand washing for preventing diarrhoea. Evidence-Based Child Health: A Cochrane Review Journal , 4 (2), 893–939. Freedman, D. A. (2006). Statistical Models for Causation What Inferential Leverage Do They Provide? Evaluation Review , 30 (6), 691-713. GSO. (2003). Vietnam Demographic and Health Survey 2002. Hanoi: General Statistics Office. GSO. (2006). Vietnam Mulitiple Indicator Cluster Survey 2006, Final Report. Hanoi: General Statistics Office. Haggerty, P. A., Muladi, K., Kirkwood, B. R., Ashworth, A., & Manunebo, M. (1994). Community-based Hygiene Education to Reduce Diarrhoeal Disease in Rural Zaire: Impact of the Intervention on Diarrhoeal Morbidity. International Journal of Epidemiology , 23 (5), 1050-1059. 27 Halder, A. K., Tronchet, C., Akhter, S., Bhuiya, A., Johnston, R., & Luby, S. P. (2010). Observed hand cleanliness and other measures of handwashing behavior in rural Bangladesh. BMC Public Health , 10 (545). Hernan, M. A., Hernandez-Diaz, J., & Robins, J. M. (2004). A Structural Approach to Selection Bias. Epidemiology (15), 615-625. Hernandez, O., Devine, J., Karver, J., Chase, C., & Coombes, Y. (2012). Measuring the Behavioral Determinants of Handwashing with Soap. The World Bank. Washington DC: Water and Sanitation Program. Indochina Research (Vietnam) Ltd. (2007). Vietnam National Handwashing Initiative Consumer Research Baseline Survey: Final Report. Hanoi. Lin, W. (2011). Agnostic notes on regression adjustments to experimental data: reexamining freedman‟s. Unpublished manuscript. Luby, S. P., Halder, A. K., Huda, T., Unicomb, L., & Johnston, R. B. (2011). The Effect of Handwashing at Recommended Times with Water Alone and With Soap on Child Diarrhea in Rural Bangladesh. Plos Medicine , 8 (6). Luby, S. P., Mubina, A., Feikin, D. R., Painter, J., Billhimer, W., Altaf, A., et al. (2005). Effect of handwashing on child health: a randomised controlled trial. The Lancet (366), 225–33. Manun'Ebo, M., Cousens, S., Haggerty, P., Kalengaie, M., Ashworth, A., & B, K. (1997). Measuring hygiene practices: a comparison of questionnaires with direct observations in rural Zaire. Tropical Medicine and International Health , 2 (11), 1015-21. MOH. (2010). Study on the correlation between household environmental sanitation, household water supply, mothers' hygiene behavior for children under 5 and the status of child nutrition in Vietnam. Hanoi: Ministry of Health. Nguyen, N. (2010). Designing Evidence-based Communications Programs to Promote Handwashing with Soap in Vietnam. Dhaka: South Asia Hygiene Practitioners. Nguyen, N., Paynter, N., & Nguyen, M. H. (2011). Vietnam: A Handwashing Behavior Change Journey. Washington, DC: Water and Sanitation Program, The World Bank. 28 Nguyen, V. C., Truong, T. N., & van der Weid, R. (2010). Poverty and Inequality Maps for Rural Vietnam: An Application of Small Area Estimation. Policy Research Working Paper Series . NIN. (2010). Annual Nutrition Surveillance System 2009. Hanoi: National Institute of Nutrition. Rabie, T., & Curtis, V. (2006). Handwashing and risk of respiratory infections: a quantitative systematic review. Tropical Medicine and International Health , 11, 258-267. Ram, P. (2010). Guidance for Measuring Handwashing Behavior. The Water and Sanitation Program. Ram, P., Briceno, B., Chase, C., Arnold, B., & Colford, J. (2012). Analysis of Handwashing Behaviors Measured in Baseline Impact Evaluation Surveys: Findings from Peru, Senegal, and Vietnam. Unpublished manuscript. Ram, P., Halder, A., Granger, S., Jones, T., Hall, P., Hitchcock, D., et al. (2010). Is structured observation a valid technique to measure handwashing behavior? Use of acceleration sensors embedded in soap to assess reactivity to structured observations. American Journal of Tropical Medicine and Hygiene , 83 (5), 1070-6. Schuler, S. R., Anh, T. H., Vu, H. S., Tran, M. H., Bui, M. T., & Pham, T. V. (2006). Constructions of gender in Vietnam: In pursuit of the „Three Criteria‟. Culture, Health & Sexuality , 383–394. Stanton, B., Clemens, J., Aziz, K., & Rahman, M. (1987). Twenty-four-hour recall, knowledge-attitude- practice questionnaires, and direct observations of sanitary practices: a comparative study. Bulletin of the World Health Organization , 65 (2), 217-22. The World Bank. (2008). Environmental Health and Child Survival: Epidemology, Economics, Experiences. Washington DC: The World Bank. The World Bank. (2005). The handwashing handbook: a guide for developing a hygiene promotion program to increase handwashing with soap. Washington DC: The World Bank. The World Bank. (2008). World Development Indicators. Washington DC, USA. Waddington, H., Snilstveit, B., White, H., & Fewtrell, L. (2009). Water, sanitation and hygiene interventions to combat childhood diarrhea in developing countries. New Delhi: 3ie. Water and Sanitation Program. (2011). Progress Report . Washington DC. 29 Water and Sanitation Program. (2011). WSP Handwashing with Soap (HWWS) Project Semi-Annual Progress Report. Washington DC. 30 TABLE 2A: COMPARISON OF MEANS TEST FOR HOUSEHOLD DEMOGRAPHICS Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value HH size 2070 4.625 0.042 1034 4.596 0.058 0.683 2070 4.983 0.039 1032 4.947 0.060 0.618 Number children under 5 years age (per HH) 2070 1.191 0.011 1034 1.200 0.017 0.646 2070 1.200 0.010 1032 1.198 0.014 0.916 Age of HH head 2070 41.936 0.507 1034 42.462 0.744 0.559 2070 43.048 0.414 1031 43.164 0.596 0.873 Age of other HH members 7504 18.905 0.245 3718 18.616 0.312 0.466 7685 20.325 0.197 3790 20.126 0.304 0.583 HH head is male 2070 0.872 0.011 1034 0.862 0.014 0.561 2070 0.835 0.011 1031 0.834 0.013 0.949 Other HH members are male 7504 0.375 0.006 3718 0.372 0.008 0.749 8244 0.353 0.005 4074 0.348 0.006 0.475 HH head ever attended school 2045 0.981 0.004 1018 0.985 0.004 0.440 2055 0.972 0.005 1028 0.967 0.006 0.529 Other HH members ever attended school 4880 0.989 0.002 2431 0.984 0.003 0.171 5105 0.977 0.003 2514 0.974 0.004 0.543 Educational attainment of HH head Incomplete primary 1983 0.151 0.012 979 0.151 0.016 0.984 1982 0.209 0.011 982 0.238 0.016 0.141 Complete primary 1983 0.417 0.014 979 0.402 0.019 0.532 1982 0.445 0.013 982 0.420 0.016 0.218 Incomplete secondary 1983 0.241 0.012 979 0.279 0.020 0.099 1982 0.225 0.011 982 0.244 0.015 0.277 Complete secondary 1983 0.130 0.011 979 0.129 0.013 0.932 1982 0.121 0.008 982 0.098 0.012 0.106 Higher 1983 0.062 0.008 979 0.039 0.007 0.028 1982 0.001 0.001 982 0.000 0.000 0.317 Educational attainment of other HH members Incomplete primary 4734 0.208 0.007 2331 0.194 0.010 0.267 4966 0.331 0.008 2434 0.316 0.012 0.282 Complete primary 4734 0.354 0.009 2331 0.357 0.015 0.869 4966 0.354 0.008 2434 0.359 0.012 0.717 Incomplete secondary 4734 0.243 0.008 2331 0.256 0.013 0.386 4966 0.213 0.007 2434 0.220 0.011 0.588 Complete secondary 4734 0.138 0.007 2331 0.137 0.011 0.907 4966 0.101 0.005 2434 0.104 0.008 0.707 Higher 4734 0.056 0.005 2331 0.055 0.007 0.906 4966 0.001 0.000 2434 0.000 0.000 0.489 31 TABLE 2B: COMPARISON OF MEANS TEST FOR HOUSEHOLD PRIMARY WORK, LABOR INCOME, AND NON-LABOR INCOME Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value HH head is employed 2056 0.857 0.010 1031 0.855 0.013 0.905 2058 0.830 0.010 1029 0.815 0.014 0.400 Others in HH are employed 3926 0.750 0.012 1946 0.750 0.019 0.995 3923 0.722 0.009 1941 0.743 0.011 0.135 Females in HH are employed 3163 0.698 0.015 1578 0.699 0.023 0.983 3238 0.677 0.010 1618 0.692 0.015 0.399 Last week activity of unemployed HH head Studying 293 0.010 0.006 149 0.007 0.007 0.695 350 0.006 0.004 190 0.005 0.005 0.946 Taking care of home 293 0.372 0.041 149 0.383 0.045 0.863 350 0.600 0.027 190 0.611 0.036 0.815 Rent earner 293 0.031 0.014 149 0.054 0.034 0.531 350 0.020 0.008 190 0.005 0.005 0.129 Permanently unable to work 293 0.109 0.030 149 0.034 0.022 0.041 350 0.051 0.013 190 0.026 0.011 0.142 Retired 293 0.208 0.028 149 0.228 0.042 0.695 350 0.157 0.023 190 0.158 0.025 0.982 Not working 293 0.263 0.039 149 0.289 0.054 0.700 350 0.166 0.024 190 0.195 0.033 0.477 Last week activity of unemployed other HH members Looking for work 981 0.014 0.004 486 0.019 0.007 0.615 1092 0.007 0.002 499 0.010 0.004 0.582 Studying 981 0.167 0.017 486 0.177 0.024 0.739 1092 0.217 0.013 499 0.232 0.025 0.589 Taking care of home 981 0.610 0.028 486 0.576 0.042 0.511 1092 0.603 0.016 499 0.571 0.024 0.277 Rent earner 981 0.011 0.005 486 0.031 0.016 0.253 1092 0.016 0.004 499 0.006 0.003 0.073 Permanently unable to work 981 0.019 0.005 486 0.019 0.008 0.930 1092 0.013 0.004 499 0.006 0.003 0.165 Retired 981 0.033 0.008 486 0.047 0.012 0.303 1092 0.036 0.007 499 0.050 0.011 0.276 Not working 981 0.114 0.020 486 0.109 0.024 0.871 1092 0.091 0.010 499 0.120 0.016 0.119 32 Primary employment status (over all employed individuals) Self-employed 4901 0.068 0.008 2455 0.072 0.014 0.801 4929 0.123 0.007 2479 0.112 0.009 0.331 Employee 4901 0.241 0.012 2455 0.237 0.017 0.821 4929 0.321 0.010 2479 0.311 0.013 0.545 Employer or boss 4901 0.004 0.001 2455 0.004 0.003 0.949 4929 0.020 0.003 2479 0.017 0.003 0.447 Worker with no remuneration 4901 0.000 0.000 2455 0.000 0.000 0.316 4929 0.001 0.000 2479 0.000 0.000 0.472 Day laborer 4901 0.050 0.007 2455 0.051 0.010 0.897 4929 0.078 0.006 2479 0.077 0.007 0.979 Working in household activities or production 4901 0.632 0.017 2455 0.630 0.027 0.966 4929 0.456 0.012 2479 0.480 0.016 0.226 Other 4901 0.005 0.001 2455 0.005 0.002 0.997 4929 0.001 0.001 2479 0.002 0.001 0.399 Monthly salary (in VND millions) 1781 2.127 0.108 892 2.184 0.246 0.831 2659 2.439 0.064 1271 2.279 0.073 0.100 Months worked per year 4916 9.145 0.127 2448 9.268 0.180 0.576 … … … … … … … Days worked per month 4915 20.192 0.284 2447 20.592 0.397 0.412 4923 21.786 0.195 2477 21.979 0.269 0.561 Hours worked per day 4909 7.524 0.076 2440 7.578 0.102 0.667 4920 7.202 0.054 2475 7.398 0.063 0.018 HH has non-labor income 2070 0.771 0.020 1034 0.730 0.034 0.310 2070 0.857 0.012 1032 0.866 0.017 0.650 Total HH non-labor income (in VND millions) 2070 2.074 0.325 1034 2.585 0.885 0.588 2070 3.854 0.503 1032 3.238 0.373 0.326 33 TABLE 2C: COMPARISON OF MEANS TEST FOR HOUSEHOLD ASSETS Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Radio, CD, cassette 2069 0.090 0.010 1033 0.108 0.014 0.300 2070 0.130 0.009 1032 0.146 0.013 0.286 Television 2070 0.924 0.008 1034 0.910 0.011 0.297 2070 0.953 0.005 1031 0.956 0.007 0.705 Videocassette, VCR, DVD player 2070 0.609 0.019 1034 0.605 0.025 0.904 2070 0.727 0.012 1032 0.694 0.019 0.147 Computer 2070 0.062 0.007 1034 0.065 0.009 0.806 2070 0.094 0.008 1032 0.091 0.011 0.840 Bicycle 2070 0.752 0.021 1034 0.771 0.029 0.589 2070 0.787 0.014 1032 0.803 0.018 0.476 Motorcycle 2070 0.796 0.012 1034 0.763 0.017 0.111 2070 0.838 0.010 1032 0.834 0.014 0.847 Automobile or truck 2070 0.026 0.004 1033 0.027 0.005 0.816 2070 0.029 0.004 1032 0.030 0.006 0.889 Refrigerator 2070 0.262 0.015 1034 0.246 0.017 0.468 2070 0.368 0.017 1032 0.333 0.018 0.162 Gas stove 2070 0.454 0.020 1034 0.423 0.024 0.318 2070 0.663 0.017 1032 0.652 0.021 0.689 Blender 2070 0.297 0.018 1034 0.290 0.025 0.833 2070 0.024 0.004 1032 0.016 0.005 0.164 Microwave 2070 0.016 0.004 1034 0.007 0.002 0.030 2070 0.316 0.017 1032 0.305 0.022 0.706 Washing machine 2070 0.081 0.010 1034 0.068 0.010 0.363 2070 0.038 0.005 1032 0.042 0.007 0.679 Water boiler, hot water heater 2070 0.235 0.023 1034 0.137 0.023 0.003 2070 0.115 0.008 1032 0.118 0.013 0.827 Machinery, equipment for household business 2070 0.023 0.005 1034 0.025 0.006 0.767 2070 0.948 0.006 1032 0.951 0.006 0.795 Boat 2070 0.029 0.008 1034 0.019 0.006 0.305 2070 0.021 0.004 1032 0.016 0.005 0.457 Telephone (including mobile) 2070 0.771 0.017 1034 0.743 0.027 0.377 2070 0.961 0.006 1032 0.967 0.008 0.545 Air conditioner 2070 0.011 0.003 1034 0.010 0.003 0.817 … … … … … … … Electric fan 2070 0.958 0.007 1034 0.949 0.009 0.389 … … … … … … … HH owns other piece of land 2070 0.190 0.024 1034 0.170 0.031 0.608 2070 0.812 0.016 1032 0.808 0.023 0.902 HH owns farm equipment 2070 0.186 0.019 1034 0.223 0.031 0.300 2070 0.536 0.018 1032 0.571 0.025 0.262 HH has animals 2070 0.622 0.026 1034 0.668 0.039 0.319 2070 0.713 0.015 1032 0.766 0.021 0.039 Number of different kinds of livestock owned per HH 2070 1.044 0.055 1034 1.140 0.077 0.310 2070 1.188 0.042 1032 1.301 0.052 0.089 Asset base wealth index 2058 1.807 0.035 1027 1.768 0.050 0.529 2070 4.203 0.027 1031 4.189 0.038 0.760 34 TABLE 2D: COMPARISON OF MEANS TEST FOR CLEANLINESS OF CAREGIVER HANDS Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Nails, palms, fingerpads 1961 8.097 0.077 982 8.143 0.125 0.758 2107 7.612 0.060 1046 7.355 0.090 0.017 Nails 2078 0.587 0.026 1035 0.628 0.036 0.348 2109 0.523 0.016 1047 0.453 0.020 0.006 Palms 2078 0.739 0.024 1035 0.754 0.031 0.712 2109 0.694 0.015 1047 0.644 0.021 0.054 Fingerpads 2078 0.732 0.023 1035 0.753 0.031 0.601 2109 0.701 0.016 1047 0.631 0.022 0.010 TABLE 2E: COMPARISON OF MEANS TEST FOR HANDWASHING SELF-REPORTED BEHAVIOR Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Washed hands with soap during the last 24 hours 1963 0.934 0.014 987 0.965 0.009 0.073 2109 0.984 0.003 1047 0.978 0.006 0.325 Washed hands with soap during the last 24 hours in the following instances: After fecal contact 2080 0.601 0.024 1040 0.594 0.036 0.869 2111 0.692 0.011 1048 0.681 0.018 0.628 Before food preparation 2080 0.295 0.026 1040 0.310 0.038 0.746 2111 0.346 0.013 1048 0.311 0.018 0.117 Before feeding / breastfeeding child 2080 0.307 0.023 1040 0.336 0.037 0.516 2111 0.392 0.016 1048 0.363 0.019 0.224 Before eating 2080 0.182 0.022 1040 0.222 0.038 0.360 2111 0.156 0.009 1048 0.162 0.016 0.728 Because hands look or feel dirty 2080 0.451 0.031 1040 0.467 0.043 0.756 2111 0.214 0.013 1048 0.194 0.017 0.327 After or while doing laundry 2080 0.415 0.033 1040 0.484 0.046 0.230 2111 0.352 0.013 1048 0.351 0.021 0.973 35 TABLE 2F: COMPARISON OF MEANS TEST FOR AVAILABILITY OF HANDWASHING FACILITIES Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Soap present anywhere in the home 2070 0.964 0.006 1034 0.957 0.010 0.591 2070 0.986 0.003 1032 0.982 0.005 0.499 Soap & water present, HW place used post- defecation 2038 0.802 0.021 1011 0.817 0.027 0.654 2067 0.910 0.007 1031 0.890 0.012 0.161 Soap & water present, HW place used before food prep in different place 2056 0.296 0.022 1031 0.292 0.029 0.907 2070 0.340 0.015 1032 0.333 0.022 0.811 Soap & water present, HW place used before food prep in same place 2038 0.466 0.029 1011 0.503 0.038 0.433 2067 0.564 0.014 1031 0.555 0.020 0.718 Soap & water present, HW place used before food prep. 2067 0.754 0.023 1033 0.784 0.028 0.404 2070 0.902 0.007 1032 0.888 0.011 0.245 Soap & water present, anywhere in HH 2070 0.806 0.020 1034 0.815 0.027 0.789 2070 0.937 0.006 1032 0.930 0.010 0.567 36 TABLE 2G: COMPARISON OF MEANS TEST FOR HANDWASHING FACILITIES (TOILET) Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Location of handwashing device, toilet Inside toilet facility 2027 0.239 0.020 1008 0.222 0.026 0.611 2060 0.214 0.015 1026 0.213 0.023 0.982 Inside food preparation area 2027 0.052 0.012 1008 0.029 0.008 0.103 2060 0.018 0.003 1026 0.023 0.006 0.415 In yard, less than 1 meter from toilet facility (pond/stream or wash basin) 2027 0.168 0.026 1008 0.173 0.035 0.911 2060 0.073 0.006 1026 0.071 0.008 0.873 In yard, between 1 and 3 meters from toilet facility (pond/stream or wash basin) 2027 0.108 0.015 1008 0.126 0.022 0.491 2060 0.139 0.009 1026 0.132 0.014 0.658 In yard, more than 3 meters from toilet facility (pond/stream or wash basin) 2027 0.338 0.029 1008 0.352 0.039 0.776 2060 0.554 0.019 1026 0.558 0.026 0.923 Type of handwashing device, toilet Tap, faucet 1831 0.252 0.028 908 0.309 0.045 0.281 2056 0.632 0.023 1022 0.606 0.029 0.474 Homemade water tap 1831 0.438 0.033 908 0.437 0.048 0.989 . . . . . . . Basin, bucket 1831 0.208 0.023 908 0.163 0.026 0.196 2056 0.344 0.021 1022 0.361 0.027 0.626 Other container from which water is poured 1831 0.092 0.018 908 0.085 0.024 0.802 2056 0.017 0.004 1022 0.013 0.003 0.444 Other 1831 0.010 0.003 908 0.006 0.004 0.379 2056 0.007 0.002 1022 0.021 0.005 0.015 Water is available at the place for handwashing, toilet 1810 0.980 0.005 891 0.980 0.006 0.969 2055 0.979 0.004 1023 0.987 0.004 0.158 Soaps available at the place for handwashing, toilet Multipurpose bar soap 1835 0.498 0.022 909 0.536 0.025 0.248 2056 0.529 0.015 1023 0.482 0.020 0.063 Powder soap, detergent 1835 0.673 0.028 909 0.726 0.035 0.237 2056 0.613 0.015 1023 0.600 0.020 0.609 Liquid soap 1835 0.183 0.022 909 0.221 0.035 0.365 2056 0.526 0.014 1023 0.473 0.019 0.022 No soap observed 1835 0.076 0.015 909 0.046 0.014 0.153 2056 0.062 0.006 1023 0.089 0.011 0.030 37 TABLE 2H: COMPARISON OF MEANS TEST FOR HANDWASHING FACILITIES (FOOD PREPARATION) Baseline Endline Treatment Control Treatment Control p- N Avg. SE N Avg. SE p-value N Avg. SE N Avg. SE value Location of handwashing device, food preparation Inside toilet facility 1988 0.032 0.006 999 0.037 0.008 0.593 2057 0.016 0.003 1021 0.017 0.004 0.901 Inside food preparation area 1988 0.172 0.019 999 0.126 0.021 0.114 2057 0.138 0.010 1021 0.162 0.016 0.202 Type of handwashing device, food preparation Tap, faucet 669 0.450 0.042 326 0.482 0.063 0.674 778 0.724 0.024 390 0.618 0.038 0.019 Tippy Tap 669 0.302 0.038 326 0.368 0.056 0.329 . . . . . . . Basin, bucket 669 0.152 0.028 326 0.117 0.037 0.440 778 0.252 0.023 390 0.351 0.036 0.019 Other container from which water is poured 669 0.085 0.019 326 0.034 0.011 0.017 778 0.021 0.006 390 0.018 0.007 0.778 Other 669 0.010 0.004 326 0.000 0.000 0.015 778 0.004 0.002 390 0.013 0.007 0.197 Water is available at the place for handwashing, food preparation 670 0.979 0.008 327 0.982 0.007 0.811 778 0.990 0.004 390 0.990 0.005 0.997 Soaps available at the place for handwashing, food preparation Multipurpose bar soap 670 0.551 0.036 327 0.538 0.053 0.845 778 0.201 0.017 390 0.208 0.025 0.815 Powder soap, detergent 670 0.610 0.038 327 0.636 0.062 0.725 778 0.338 0.018 390 0.367 0.032 0.435 Liquid soap 670 0.279 0.043 327 0.388 0.067 0.170 778 0.771 0.017 390 0.718 0.027 0.099 No soap observed 670 0.054 0.013 327 0.052 0.022 0.946 778 0.087 0.011 390 0.118 0.017 0.134 38 TABLE 2I: COMPARISON OF MEANS TEST FOR CHILD HEALTH SYMPTOMS 7-DAY PERIOD PREVALENCE (% OF CHILDREN <5) Baseline Endline Treatment Control p- Treatment Control p- N Avg. SE N Avg. SE value N Avg. SE N Avg. SE value Diarrhea 2594 0.010 0.002 1303 0.016 0.005 0.203 2483 0.029 0.004 1236 0.047 0.007 0.038 Acute respiratory infection 2594 0.152 0.010 1303 0.160 0.015 0.686 2483 0.331 0.012 1236 0.375 0.019 0.052 Fever 2594 0.170 0.010 1303 0.177 0.017 0.721 2483 0.193 0.010 1236 0.227 0.015 0.062 Cough in previous week 2594 0.150 0.010 1303 0.156 0.015 0.733 2483 0.288 0.011 1236 0.308 0.016 0.308 Congestion/coryza 2594 0.152 0.010 1303 0.154 0.017 0.922 2483 0.343 0.014 1236 0.434 0.018 0.000 Difficulty breathing 2594 0.012 0.003 1303 0.015 0.003 0.492 2483 0.119 0.008 1236 0.164 0.015 0.008 Cramps 2594 0.002 0.001 1303 0.002 0.001 0.996 2483 0.039 0.004 1236 0.069 0.007 0.001 Nausea 2594 0.001 0.001 1303 0.002 0.001 0.276 2483 0.026 0.003 1236 0.055 0.007 0.000 Vomiting 2594 0.007 0.002 1303 0.008 0.003 0.532 2483 0.043 0.005 1236 0.066 0.008 0.020 Three or more bowel movements 2594 0.018 0.003 1303 0.024 0.005 0.371 2483 0.033 0.004 1236 0.052 0.007 0.024 Watery or soft stool 2594 0.013 0.002 1303 0.021 0.005 0.131 2483 0.031 0.004 1236 0.053 0.007 0.011 Mucus or blood in stool 2594 0.004 0.001 1303 0.004 0.002 0.993 2483 0.010 0.002 1236 0.018 0.004 0.084 Refusal to eat 2594 0.023 0.005 1303 0.029 0.008 0.510 2483 0.166 0.010 1236 0.203 0.015 0.033 Bruising 2594 0.004 0.001 1303 0.002 0.001 0.419 2483 0.066 0.006 1236 0.104 0.012 0.003 Itching 2594 0.009 0.003 1303 0.008 0.003 0.773 2483 0.061 0.005 1236 0.078 0.009 0.083 Anemic: Hb level < 11 g/dl 1961 0.322 0.015 965 0.307 0.022 0.571 . . . . . . . 39 TABLE 2J: COMPARISON OF MEANS TEST FOR CHILD GROWTH MEASURES (Z-SCORES) Baseline Endline Treatment Control p- Treatment Control N Avg. SE N Avg. SE value N Avg. SE N Avg. SE p-value Child weight (to 0.1 kg) 2131 9.580 0.053 1057 9.668 0.074 0.334 . . . . . . . Child height (to 0.1 cm) 2129 77.180 0.199 1054 77.680 0.275 0.141 . . . . . . . Child arm circumference (to 0.1 cm) 2131 14.935 0.057 1057 14.903 0.063 0.705 . . . . . . . Child head circumference (to 0.1 cm) 2131 45.281 0.067 1057 45.327 0.098 0.699 . . . . . . . Weight-for-age z-score 2093 -0.647 0.030 1039 -0.578 0.046 0.205 . . . . . . . Length/height-for-age z-score 2086 -0.760 0.045 1035 -0.609 0.072 0.075 . . . . . . . BMI-for-age z-score 2074 -0.236 0.036 1025 -0.298 0.054 0.342 . . . . . . . Weight-for-length/height z-score 2086 -0.324 0.033 1034 -0.368 0.049 0.455 . . . . . . . Arm circumference-for-age z-score 2094 0.111 0.036 1043 0.157 0.049 0.452 . . . . . . . Head circumference-for-age z-score 2104 -0.651 0.035 1044 -0.599 0.051 0.395 . . . . . . . 40 TABLE 3A: HOUSEHOLD AND CAREGIVER ATTRITION (n=3102) Treatment Control N Avg. SE N Avg. SE p-value Household followed up at endline 2069 0.943 0.006 1033 0.942 0.009 0.884 Primary caregiver followed up at endline 2085 0.747 0.012 1043 0.730 0.017 0.399 TABLE 3B: COMPARISON OF MEANS TEST FOR PANEL HOUSEHOLDS (n=2925) Baseline Treatment Control N Avg. SE N Avg. SE p-value HH size 1952 4.639 0.043 973 4.599 0.058 0.583 Employment status of HH head 1939 0.858 0.010 969 0.852 0.014 0.740 Years of school of HH head 1871 7.688 0.119 921 7.657 0.153 0.873 HH head is male 1952 0.870 0.011 973 0.858 0.014 0.499 Age of HH head 1952 41.964 0.505 973 42.569 0.742 0.500 Total HH income 1952 2.104 0.341 973 2.674 0.934 0.567 HH wealth score 1941 1.810 0.036 966 1.777 0.051 0.595 HH has non-labor income 1952 0.766 0.021 973 0.733 0.035 0.410 Number livestock owned by HH 1952 1.049 0.055 973 1.156 0.078 0.264 Soap and water at/near toilet 1920 0.802 0.021 953 0.817 0.028 0.657 Soap and water at/near food preparation facility 1949 0.755 0.023 972 0.785 0.028 0.414 Soap observed somewhere in HH 1952 0.966 0.006 973 0.962 0.009 0.731 41 TABLE 3C: COMPARISON OF MEANS TEST FOR REPLACEMENT HOUSEHOLDS (n=177) Endline Treatment Control N Avg. SE N Avg. SE p-value HH size 118 4.864 0.157 59 4.915 0.230 0.855 Employment status of HH head 117 0.855 0.032 59 0.831 0.062 0.730 Years of school of HH head 115 6.835 0.228 56 6.571 0.453 0.604 HH head is male 118 0.831 0.034 59 0.847 0.043 0.756 Age of HH head 118 42.246 1.219 59 46.102 2.151 0.119 Total HH income 118 2.543 0.387 59 2.707 0.588 0.816 HH wealth score 118 4.178 0.063 59 4.322 0.093 0.201 HH has non-labor income 118 0.873 0.035 59 0.864 0.047 0.885 Number livestock owned by HH 118 1.169 0.116 59 1.475 0.104 0.050 Soap and water at/near toilet 118 0.881 0.027 59 0.949 0.028 0.082 Soap and water at/near food preparation facility 118 0.915 0.025 59 0.915 0.038 1.000 Soap observed somewhere in HH 118 0.966 0.017 59 1.000 0.000 0.040 TABLE 3D: COMPARISON OF MEANS TEST FOR PANEL + REPLACEMENT HOUSEHOLDS (n=3102) HH size 2070 4.983 0.039 1032 4.947 0.060 0.618 Employment status of HH head 2058 0.830 0.010 1029 0.815 0.014 0.400 Years of school of HH head 1982 7.048 0.079 982 6.910 0.120 0.335 HH head is male 2070 0.835 0.011 1031 0.834 0.013 0.949 Age of HH head 2070 43.048 0.414 1031 43.164 0.596 0.873 Total HH income (VND millions) 2070 3.854 0.503 1032 3.238 0.373 0.326 HH wealth score 2070 4.203 0.027 1031 4.189 0.038 0.760 HH has non-labor income 2070 0.857 0.012 1032 0.866 0.017 0.650 Number livestock owned by HH 2070 1.188 0.042 1032 1.301 0.052 0.089 HH has improved sanitation facility (JMP) 2070 0.669 0.023 1032 0.690 0.026 0.540 HH has improved water source (JMP) 2070 0.952 0.008 1032 0.971 0.008 0.087 42 Table 4: Household exposure to HWIPC campaign Mean effect of treatment Midterm (July 2010) Control Mean Unadjusted (SD) model Number of channels of exposure 2.822 0.324* 1.356 (0.101) High exposure (More than 3 channels) 0.459 0.096* 0.499 (0.040) Medium exposure (2 - 3 channels) 0.354 -0.032 0.479 (0.040) Low exposure (1 or fewer channels) 0.186 -0.064* 0.390 (0.029) Exposure to IPC by Women's Union 0.347 0.217* 0.476 (0.041) N observations 2919 Control variables No + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis 43 Table 5: Caregiver knowledge Mean effect of treatment Control Mean Unadjusted (SD) model Full model Number of spontaneous mentions of critical times for HWWS 2.236 0.042+ 0.048+ 1.122 (0.025) (0.028) Knowledge of critical time: After using toilet 0.774 0.007 0.001 0.419 (0.029) (0.031) Knowledge of critical time: After washing baby's bottom/changing diaper 0.385 -0.034 0.010 0.488 (0.040) (0.041) Knowledge of critical time: Before preparing food 0.296 0.118* 0.109* 0.458 (0.037) (0.040) Knowledge of critical time: Before feeding/breastfeeding baby 0.389 0.038 0.022 0.489 (0.038) (0.041) What is best way to wash hands? With soap 0.973 0.004 0.005 0.161 (0.005) (0.006) Does not HWWS cause diarrhea? Yes 0.849 0.013 0.029* 0.358 (0.012) (0.014) Does not HW cause diarrhea? Yes 0.834 0.020 0.035* 0.372 (0.014) (0.015) Do changes in weather cause diarrhea? Yes 0.427 -0.028 -0.019 0.495 (0.018) (0.020) N observations 3159 3078 Control variables No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age, sex, month of interview and province dummies 44 Table 6A: Caregiver attitudes, practices and beliefs about handwashing Mean effect of treatment Control Mean Unadjusted (SD) model Full model % Agree or Strongly Agree You wash your hands with soap without needing to remind your self 0.900 -0.002 -0.014 0.301 (0.010) (0.012) When washing your hands with soap, you know you are protecting your children‟s health 0.980 0.005 -0.001 0.139 (0.005) (0.005) Handwashing with soap is something you can do to prevent your children from getting sick 0.966 0.005 0.005 0.180 (0.006) (0.007) Handwashing with soap is something you can do for your child's development 0.975 0.000 -0.000 0.155 (0.005) (0.006) You start washing your hands before you realize you are doing it 0.435 0.032+ 0.043* 0.496 (0.018) (0.018) You feel strange when you do not wash hands with soap 0.700 0.025 0.004 0.459 (0.015) (0.017) You wash your hands with soap automatically 0.959 0.013* 0.015* 0.199 (0.006) (0.006) You have been washing your hands with soap for a long time 0.920 0.015 0.008 0.271 (0.009) (0.010) You often wash your hands with soap 0.917 0.019* 0.009 0.275 (0.009) (0.011) Washing your hands with soap is typically "you" 0.906 -0.001 -0.006 0.292 (0.010) (0.012) You would feel uncomfortable if you didn't wash your hands 0.907 0.011 -0.006 0.290 (0.009) (0.010) N observations 3052 2971 Control variables No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age and sex of caregiver, month of interview and province dummies 45 Table 6B: Caregiver attitudes, practices and beliefs about handwashing Intention to treat (ITT) Unadjusted Control Mean (SD) model Full model % Disagree or Strongly Disagree Washing your hands with soap requires effort 0.139 -0.003 -0.001 0.346 (0.013) (0.014) You only need to wash your hands with soap, if they look dirty or smell bad 0.412 0.016 0.037+ 0.492 (0.019) (0.022) It is not necessary to wash hands with soap before cooking as germs on hands are killed when the food is cooked 0.801 0.019 0.030+ 0.399 (0.015) (0.016) It is easy to forget to wash hands with soap after going to the toilet if you do not see soap 0.737 -0.015 -0.008 0.440 (0.017) (0.019) Washing hands with soap before feeding children is only important if you use your hands to feed them 0.624 -0.014 0.016 0.485 (0.021) (0.021) Diarrhea in children is unpreventable 0.595 -0.015 -0.029 0.491 (0.019) (0.021) Cough and cold are normal symptoms during the growth process of children and do not affect their long term health 0.568 0.046* 0.020 0.496 (0.018) (0.020) You forget to wash your hands with soap when you are busy 0.644 -0.006 -0.011 0.479 (0.018) (0.020) Diarrhea is a normal symptom during the growth process of children and does not affect their long term health 0.754 0.058* 0.045* 0.431 (0.017) (0.018) Washing hands with soap is not part of your daily routine 0.791 0.010 0.019 0.407 (0.016) (0.019) You only need to wash your hands with soap if you touch unhygienic objects 0.432 0.046* 0.062* 0.496 (0.019) (0.022) You have to think about it, each time you wash your hands with soap 0.739 0.007 0.026+ 0.439 (0.015) (0.015) It is not necessary to wash hands with soap after cleaning children‟s bottom as their feces is not dangerous 0.925 0.006 0.011 0.263 (0.011) (0.012) You always have to remind yourself to wash your hands with soap 0.083 -0.004 0.005 0.275 (0.009) (0.010) N observations 3052 2971 Control variables No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age and sex of caregiver, month of interview and province dummies 46 Table 7: Handwashing facilities and cleansing agents in household Mean effect of treatment Baseline Control Mean Mean Unadjusted Lag Full (SD) (SD) model dependent Model DID Soap and Water available at HW place used after defecation 0.807 0.890 0.020+ 0.026* 0.026* 0.036 0.395 0.313 (0.011) (0.011) (0.011) (0.034) Soap and Water available at HW place used before food preparation 0.764 0.888 0.015 0.017 0.016 0.045 0.425 0.316 (0.010) (0.010) (0.010) (0.038) Soap present anywhere in the home 0.962 0.982 0.004 0.006 0.006 -0.002 0.192 0.134 (0.005) (0.005) (0.005) (0.013) N observations 3102 2927 2927 6206 Lag dependent variable No Yes Yes No Province dummies No No Yes No Double difference No No No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis 47 Table 8: Self-reported handwashing with soap and Observed hand cleanliness Mean effect of treatment Baseline Control Mean Mean Unadjusted Lag Full (SD) (SD) model dependent model DID Self-reported HWWS after fecal contact in past 24 hours 0.599 0.681 0.011 0.013 0.049* 0.004 0.490 0.466 (0.018) (0.019) (0.021) (0.046) Self-reported HWWS before food preparation in past 24 hours 0.300 0.311 0.032+ 0.029 0.030 0.051 0.458 0.463 (0.019) (0.019) (0.021) (0.050) Self-reported HWWS before feeding/breastfeeding child in past 24 hours 0.317 0.363 0.032 0.022 0.051* 0.060 0.465 0.481 (0.020) (0.020) (0.023) (0.049) Clean hands index (observed) 8.111 7.355 0.264* 0.259* 0.044 0.309+ 1.397 1.876 (0.066) (0.070) (0.077) (0.168) Clean fingernails (observed) 0.600 0.453 0.073* 0.069* 0.024 0.113* 0.490 0.498 (0.018) (0.018) (0.020) (0.048) Clean palms (observed) 0.744 0.644 0.051* 0.049* 0.008 0.065 0.437 0.479 (0.017) (0.018) (0.020) (0.044) Clean fingerpads (observed) 0.739 0.631 0.070* 0.067* 0.026 0.090* 0.439 0.483 (0.017) (0.018) (0.020) (0.043) N observations 3159 2912 2838 6281 Lag dependent variable No Yes Yes No Control variables No No Yes No Double difference No No No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age and sex of caregiver, month of interview and province dummies 48 Table 9: Structured observations of handwashing (Caregivers) Mean effect of treatment Control Mean Unadjusted (SD) model N Full model N Rinse after fecal contact 0.359 0.011 911 0.051 873 0.481 (0.032) (0.039) Rinse before food preparation 0.194 0.052+ 1103 0.019 1071 0.396 (0.028) (0.032) Rinse before feeding a child 0.119 0.030 765 0.029 743 0.324 (0.022) (0.027) HW with soap after fecal contact 0.240 0.015 911 -0.030 873 0.428 (0.030) (0.035) HW with soap before food preparation 0.071 0.000 1103 0.003 1071 0.258 (0.018) (0.021) HW with soap before feeding a child 0.046 0.017 765 0.019 743 0.210 (0.017) (0.019) Percent of exposure events accompanied by handwashing and/or soap use 0.293 0.041* 4295 0.024 4159 0.455 (0.016) (0.018) Control variables No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age, sex, month of interview and province dummies 49 Table 10: Child illness symptoms (7-day recall) Mean effect of treatment Baseline Control Mean Mean Unadjusted (SD) (SD) model Full model Diarrhea (Caregiver defined) . 0.054 -0.020* -0.009 . 0.227 (0.006) (0.007) Diarrhea (Symptom defined) 0.012 0.047 -0.017* -0.004 0.108 0.212 (0.006) (0.007) ARI 0.155 0.375 -0.044* 0.001 0.362 0.484 (0.017) (0.018) Symptoms of ALRI (short breath with cough or difficulty breathing) . 0.044 -0.021* -0.015+ . 0.206 (0.007) (0.008) Abrasions, scrapes, bruising 0.003 0.104 -0.037* -0.029* 0.058 0.306 (0.009) (0.010) Itch 0.008 0.078 -0.017* -0.005 0.092 0.269 (0.008) (0.009) Treatment seeking behavior Sought medical care in past 7-days 0.239 0.469 -0.052* -0.023 0.427 0.499 (0.018) (0.020) Treated for ARI symptoms 0.934 0.873 0.016 -0.001 0.248 0.334 (0.013) (0.015) Treated for gastrointestinal symptoms 0.611 0.672 0.005 0.023 0.489 0.471 (0.042) (0.044) Caregiver productivity Cases of lost hours for child care due to illness 0.147 0.110 -0.015 -0.004 0.354 0.314 (0.011) (0.013) N observations 3719 3618 Control variables No Yes Date of interview dummies No Yes + p<0.10, * p<0.05 [1] All models include matched triplet dummies. Robust standard errors clustered at commune level in parenthesis [2] Control variables include caregiver education, age and sex of child and province dummies 50