A Multiple-Arm, Cluster-Randomized Impact Evaluation of the Clean India (Swachh Bharat) Mission Program in Rural Punjab, India

This study reports the findings of a large-scale, multiple-arm, cluster-randomized control study carried out in rural Punjab, India, to assess the impact of a flagship sanitation program of the Government of India. The program, the Clean India Mission for Villages, was implemented between October 2014 and October 2019 and aimed to encourage the construction of toilets, eliminate the practice of open defecation, and improve the awareness and practice of good hygiene across rural India. It utilized a combination of behavioral change campaigns, centered on the community-led total sanitation approach, and financial incentives for eligible households. The study also evaluates the incremental effects of intensive hygiene awareness campaigns in selected schools and follow-up initiatives in selected communities. The study finds that the coverage of ?safely managed? toilets among households without toilets increased by 6.8?10.4 percentage points across various intervention arms, compared with a control group. Open defecation was reduced by 7.3?7.8 percentage points. The program also had significant positive impacts on hygiene awareness among adults and children, although the interventions of school campaigns and intensive follow-up were of limited additional impact.


Introduction
Poor sanitation has been identified as one of the major causes of malnutrition (Briend 1990;Prüss-Üstün and Corvalan 2006;Humphrey 2009) Punjab, with a population of 27.7 million living in 22 districts, is one of the richest states of India. Even before the initiation of the SBMG, it had made great strides in implementing water supply, sanitation, and hygiene (WASH) reforms in rural areas, while delivering services appropriate to the specific needs of local communities. Punjab was perhaps the first state in India to provide safe water supply to all its 12,258 villages. According to the 2011 Census, rural sanitation coverage was impressive, at about 71.9 percent (the average across India was 32.7 percent), mainly because of state government efforts to prioritize sanitation.
Yet, despite this high coverage rate, before the initiation of the SBMG, only 1 percent of all Gram Panchayats (GPs; "village councils") in the state had received the Government of India's Nirmal Grama Puraskar ("Clean Village Award") for the achievement of a 100 percent open defecation free (ODF) status. Though Punjab ranked 7th out of 28 states for toilet coverage, it ranked 19th for the number of GPs winning the award, and 25th for the percentage of total awards granted. Of 47 GPs in Punjab that won the "Clean Village Award" in the years 2009-11, only 17 percent were found to still be ODF in 2014 (World Bank 2016). So, by the time the SBMG was launched, it was not just toilet construction but also the promotion of toilet use, and the achievement and maintenance of ODF status, that were critical priorities in Punjab.
The World Bank-assisted Punjab Rural Water and Sanitation Sector Improvement Project (PRWSSIP) supported Punjab's Department of Water Supply and Sanitation (DWSS) in implementing the SBMG across all villages in Punjab, with the objectives of 100 percent toilet coverage and 100 percent toilet use. Its main goal was to eradicate open defecation and facilitate the achievement and sustenance of ODF communities. An ODF community is one in which all human excreta are safely disposed and all persons use an "improved" toilet 1 at all times. A community-led total sanitation (CLTS) approach was adopted by the DWSS to motivate communities to attain ODF status. The central pillar of this approach is a behavior change communication (BCC) campaign that focuses on raising awareness and demand for health, hygiene, and sanitation; mobilizing collective action toward behavioral change; and generating peer pressure through the creation of community-based committees that monitor neighborhoods and encourage sanitary practices. The project also supported the construction of toilets in households across the state. As an incentive, Rs. 15,000 was offered to eligible beneficiaries to cover the full cost of constructing twin leach pit latrines.
It is expected that project interventions will have positive effects on people's awareness and practice of household hygiene and sanitation, resulting in improved health and nutritional outcomes. The project promotes handwashing with soap and safe disposal of child feces, and encourages households to construct and use toilets. The use of toilets will lead to a reduction in the fecal contamination of soil and groundwater, thus improving water quality at the point of extraction and reducing intestinal worm infections. While project impacts on hygienic and sanitary practices, including toilet construction and use, are expected to be observable in the short run, positive health and nutritional outcomes will take longer to materialize.
A multiple-arm, cluster-randomized impact evaluation was designed to measure the impact of the SBMG in Punjab. This reports the results from a midline survey, focusing on the project's short-term impacts on hygiene awareness and related practices and toilet construction and use. An end-line survey, planned for October 2020, will capture the project's impacts on children's health and nutritional outcomes.
This study provides evidence of the project's success in triggering behavior change and promoting toilet construction and use. Specifically, the impact evaluation sought answers to the following research questions: i) Did the various project interventions influence households' decisions to construct toilets?
ii) Did the project interventions lead to a reduction in open defecation rates in beneficiary communities?
iii) How did the BCC campaign affect hygiene awareness and related practices? iv) Did school programs focused on raising students' awareness of hygienic practices effectively boost such practices, toilet construction, and toilet use? 1 An improved toilet is designed to hygienically separate excreta from human contact.

Behavior Change Communication (BCC)
BCC is core to the project's interventions geared toward generating demand for toilets and creating awareness of the linkages between sanitation, hygiene, and health. The ultimate goal of the BCC campaign is to mobilize communities to collectively strive for ODF status. BCC activities in Punjab have been focusing on the following critical actions: (i) construction and usage of toilets, (ii) handwashing with soap, and (iii) safe disposal of child feces.
BCC activities have been led by the motivators and master motivators, who in turn identify, sensitize, and support sanitation "champions" at the GP level from local government institutions, cooperatives, schools, health care facilities, women's groups, community-based organizations, self-help groups, etc., during a phase called pre-triggering. These sanitation champions spearhead the campaign at the community level. Following pre-triggering, the motivator, along with local sanitation champions, makes door-to-door visits identifying and preparing a list of households that do not have toilets and are eligible to receive incentive money for toilet construction under the SBMG program. The list is vetted and approved by officials at the GP, block, and district levels. Households' first exposure to BCC occurs at that initial visit, when a motivator and sanitation champion discuss the ill effects of open defecation and the importance of using toilets. At this point, the time and venue of a so-called triggering meeting is communicated to the household.
The triggering is held in a public place and led by the motivator. Villagers are asked to draw a map of their community, identifying their homes and the places commonly used for open defecation. Following this mapping exercise, villagers are taken for a walk to one of the open defecation sites. They are asked to not cover their noses. The objective is for the participants to experience the way open defecation is polluting the village environment. An important part of the triggering process is the "dirty fly" demonstration. The motivator offers participants a glass of clean water to drink, which most people gladly accept. Then the motivator pulls out a hair, touches feces with that hair, and dips the hair back into the water. Nobody is willing to drink the water now. This is a very effective way of communicating how flies transmit contamination from exposed feces through food to the human body, and how open defecation implies that people are consuming one another's feces. Triggering activities are meant to convey: (i) the link between exposed feces and fecal contamination of food; (ii) the importance of washing hands with soap; and (iii) the need for constructing and using toilets that can effectively separate human excreta from the human environment.
For the day-to-day monitoring of open defecation, Nigrani Samitis (Vigilance Committees) have been set up in each GP. These committees comprise volunteers from among the local sanitation champions, who carry out morning visits to places commonly used for open defecation. The purpose is to identify people who defecate in the open and convince them to use a toilet-or to construct a toilet if they do not have one. People without access to a toilet are asked, in the meantime, to cover their feces with dirt.
Also, under this program, Swachhata ("cleanliness") Clubs have been reactivated in primary schools.
These were formed in all primary schools in 2014 under an Education Department initiative to promote good hygiene among school students, but many such clubs stopped functioning over time. Under the auspices of the SBMG, motivators restarted these clubs and discussed good practices related to hygiene and sanitation with students. These clubs, headed by students in the 5th standard, oversee and ensure fellow students' personal hygiene, including washing hands with soap before meals, and the cleanliness of classrooms and surroundings.

Construction of Toilets
It was hoped that BCC activities would spur local demand for toilets. Once this had been accomplished, motivators, along with DWSS engineers, would provide technical support in choosing the location and technology of new toilets. Also, beneficiary communities shared information via social media on material and labor costs, helping households make cost-effective choices. The SBMG laid out specific eligibility criteria for the incentive money offered under the program. However, the DWSS decided that all households without toilets were eligible, and had been actively promoting the construction of twin leach pit latrines. According to the SBMG guidelines, the incentive money was supposed to be transferred to households only once a toilet had been completed and its use verified. However, many households did not have the financial resources to pay the up-front costs of construction. Therefore, the DWSS decided to release the money in three tranches of Rs. 5,000 each. The first tranche was released after the twin pits were dug-which could be done by beneficiary household members themselves. The second tranche was released after completion of the superstructure, and the final installment was released after the toilet (including a door) was completed and its use had been verified by the motivator and by district and block officials. For households opting for more expensive toilets with septic tanks, the DWSS required that soak pits be constructed as well. Once all toilets have been completed in a village, motivators ensure that morning vigils continue for a month after construction.

Additional Activities
In addition to the project interventions discussed above, the DWSS has been undertaking intensive awareness campaigns in schools in one set of GPs, and an intensive follow-up campaign in another set.
The awareness campaigns have been carried out in both primary and middle schools, and involved the following activities: (i) School triggering. "Catch them young" is the main aim. Students were told about the ill effects of open defecation and the importance of using toilets and washing hands with soap before eating meals and after defecation. All students were encouraged to ensure that they had a toilet in their house, used by all their family members. All teachers as well as school support staff were involved in this effort.
(ii) WASH training sessions. These focused on handwashing and also provided guidance on personal hygiene. Adolescent boys and girls were separated into two groups, and guided in maintaining good health and hygiene. Menstrual hygiene is an important issue that was discussed with the girls.
(iii) School sanitation program. On three consecutive days, for two hours each, motivators and master motivators led middle school students in interactive games and the preparation of sketches focused on water and sanitation issues. After a communal discussion of key issues, students were asked to fill a questionnaire relating to water and sanitation issues. A motivator also filled out a survey based on his/her personal observation of the physical conditions of the water and sanitation infrastructure in the school. Finally, the report of the school was discussed with all school members-teachers, students, and staff. A special training manual for this three-day program was prepared by the DWSS.
The intensive follow-up campaigns in select villages have involved the following activities: (i) Frequent morning vigils. All the motivators as well as master motivators participated in morning vigils in select GPs three times a week (instead of twice a week, as under the regular project intervention).
(ii) Female mohalla (community) committees. The formation of these committees, comprising 15-20 women volunteers, was meant to make morning vigil campaigns more impactful.
Each committee included representatives from all segments of society, including the marginalized and vulnerable, who carried out the following activities: (a) conducting morning vigils in the GP; (b) encouraging open defecators to construct toilets and use them; (c) facilitating the toilet construction process; (d) promoting the construction of bath-cumtoilets; and (e) encouraging women's involvement in the decision-making process, including in the selection of a toilet's location, superstructure, and accessories, and in the consideration of issues relevant to users' safety. Though CLTS has been widely adopted in developing countries worldwide, empirical evidence of its effectiveness is limited. While the available literature indicates that CLTS has been successful in promoting toilet ownership and reducing open defecation (Pattanayak et al. 2009;Patil et al. 2014;Borja-Vega 2014;Godfrey et al. 2014;Guiteras, Levinsohn, and Mobarak 2015;Dickinson et al. 2015;Pickering et al. 2015;Gertler et al. 2015;Crocker et al. 2016), the size of the effect varies widely within and across countries depending on preexisting local conditions and the nature and intensity of the interventions.

Theory of Change
One multicountry, cluster-randomized study that compared the effectiveness of CLTS interventions in India, Indonesia, Mali, and Tanzania (Gertler et al. 2015) (Gertler et al. 2015). The results of this study highlight the importance of community mobilization and repeated follow-ups in CLTS interventions. The results also indicate that while subsidies work to expand toilet access, they do not lead to a proportionate decline in open defecation, and that behavior change is mostly limited to those households without toilets at the start. Thus, the results support an argument put forth by the proponents of CLTS that any subsidy is going to be counterproductive in "spurring genuine household demand for private toilets" and eradicating open defecation (Kar and Chambers 2008).
The success of CLTS interventions in promoting toilet construction and use have been further documented by impact evaluations in Mali (Pickering et al. 2015) and Ghana (Crocker et al. 2016). A cluster-randomized trial of CLTS interventions in the Koulikoro region of rural Mali reported a 32 percentage point increase in latrine ownership among households in the treatment villages, and a decrease in open defecation by 24 and 23 percentage points among men and women, respectively, and by 43 percentage points among children 10 years or younger (Pickering et al. 2015). In Ghana, a study of the effectiveness of stand-alone CLTS interventions relative to those that included the training of "natural leaders" in beneficiary communities found that the training of these leaders resulted in a decline in open defecation 19.9 percentage points greater than in those villages that received only CLTS interventions (Crocker et al. 2016 (Guiteras, Levinsohn, and Mobarak 2015).
Similar results have been reported by cluster-randomized studies of CLTS interventions with subsidies in Orissa (Pattanayak et al. 2009;Dickinson et al. 2015) and Madhya Pradesh (Patil et al. 2014) in India. Pattanayak et al. (2009) (Crocker et al. 2016). With the current focus on improved and safely managed toilets (as guided by SDG Target 6.2 3 and, specifically, Indicator 6.2.1 4 ), subsidies and supply-side interventions are effective complements to CLTS interventions to promote toilet construction and use, especially among the poorer segments of a community.
Poor sanitation is associated with a wide range of diseases, including diarrhea (Waddington et al. 2009) and soil-transmitted helminth (STH) infections (Ziegelbauer et al. 2012 Additionally, the STH infections can have a negative effect on cognitive development in school-age children, creating learning gaps (Hadidjaja et al. 1998).
In spite of the link between poor sanitation and hygiene practices and diarrhea, many studies of the impact of CLTS interventions on health outcomes find no significant reduction in diarrheal disease (Patil et al. 2014;Dickinson et al. 2015;Pickering et al. 2015). Studies that do find any significant relationship either focus on CLTS programs accompanied by water-supply interventions (Godfrey et al. 2014) or find the reduction in only a specific subgroup of project beneficiaries (Borja-Vega 2014).
The empirical literature, however, indicates that CLTS interventions have a positive impact on nutritional outcomes in children under five. In their study of rural Orissa, India, Dickinson et al. (2015) find that mid-upper-arm circumference increased by two standard deviations in treatment villages relative to control villages. In Mali, children under five in treatment villages were taller and less likely to be severely underweight than their counterparts from control villages. The effect was more pronounced among children under two years (Pickering et al. 2015). A similar result is reported for Mali by Gertler et al. (2015), where CLTS interventions led to a 0.17 standard deviation increase in Moreover, the project in Punjab involves a holistic look at hygiene and sanitation, and is not focused on toilets alone. It promotes handwashing with soap and educates community members on how to safely dispose of child feces. The incentive money offered to eligible households for the construction of new toilets is electronically transferred to the bank accounts of beneficiaries, reducing the chances of corruption.
The project interventions are likely to expand toilet coverage and reduce open defecation in Punjab.
This will reduce the fecal contamination of soil and water, which in turn will reduce the possibility of diarrheal and STH infections and have a positive effect on children's nutritional outcomes. It is important to note, however, that these expected health and nutritional benefits take time to materialize and capture.
So, the expected benefits of the project can be categorized into two groups based on their time horizons.
In the short run, we expect an improvement in people's awareness and practice of hygienic behaviors, an increase in the number of households with toilets, and a reduction in the number of people defecating in the open. In the medium to long run, we expect a decrease in cases of diarrhea and an improvement in nutritional outcomes (that is, a decrease in stunting, wasting, and being underweight) in children younger than five years. In this study, we will explore the short-term impacts of the project.

Evaluation Design and Methodology
The evaluation used a multiple-arm cluster-randomization design with three treatment arms and one control group to establish causal impacts of the project interventions. The project is delivered at the GP level and the project interventions are aimed at influencing hygiene and sanitation behavior as well as health and nutritional outcomes of the entire community. So, the cluster randomization was done at the GP level. Considering the District Sanitation Committee's pivotal role in project implementation, the randomization has been stratified at the district level. Of the three treatment arms, treatment arm 1 included GPs that received regular project interventions. In GPs in treatment arm 2, intensive hygiene and sanitation awareness campaigns were carried out in schools (alongside regular project interventions). Treatment arm 3 included GPs where there was an intensive follow-up campaign (in addition to the regular project interventions). Project interventions were withheld in the control GPs.
The evaluation used surveys at the household, community, and school levels to collect data on both the project's beneficiaries and its implementation. Baseline data are from before the start of project's intervention, and these are compared with midline data, collected after the completion of project interventions, to assess the short-term outcomes of the project. A follow-up survey is expected to be carried out in 2021, toward the end of the project, to assess its medium-term impacts.

Sample Size and Sampling Strategy
As per the original design, there were 50 GPs per arm or group, and 24 households per GP, leading to a total sample size of 200 GPs and 4,800 households. However, there is a distinct possibility that some of the control GPs could have, in fact, received interventions (most likely for political reasons), while some of the treatment GPs could have seen them withdrawn due to a shortage of funds. The attrition of control GPs was thought to be more likely than the attrition of treatment GPs. Accordingly, it was decided to sample 60 GPs from each of the treatment arms and 80 GPs from the control arm. So, the final sample had 260 GPs of 24 households each, for a total number of 6,240.

Sampling Strategy for GPs
Baseline SBMG data on GP-level toilet coverage were used to select the GPs in the sample. Of the 12,786 GPs from Punjab listed in the SBMG data set, those excluded (i) had already received interventions under the SBMG (Tranche 1 and 2); (ii) were going to benefit from improvements in water service delivery; (iii) had water quality problems that were going to be addressed in the near future under the project or the National Rural Water Development Program; (iv) were GPs where political interests might make it difficult to withhold interventions, as identified by the officials of the DWSS; and (v) lacked data on toilet coverage. Once we excluded these GPs, the total number of GPs in our target population was reduced to 7,764.
Further, to ensure that a full-fledged BCC campaign would be undertaken in the sample GPs, a construction requirement of at least 25 toilets was set as a minimum threshold for being selected.
Similarly, to ensure that the project interventions would be completed within six months, GPs where the number of households without individual latrines exceeded 300 were excluded from the sample.
After imposing these conditions, the remaining 4,868 GPs were identified as our baseline population from which the samples were drawn. A computer program was used to draw random samples stratified at the district level and maintain proportional representation of GPs in districts in each of the treatment and control arms similar to the baseline population.

Sampling Strategy for Households
Up to 250 households were listed in each of the selected GPs. Basic information on these was then collated: the total number of household members, number of children below 5 years of age, number of children aged 6 to 14, and toilet ownership. All the households were listed in GPs of 250 or fewer. For larger GPs, households were listed by ward until the maximum of 250 was reached. For sampling purposes, only households with children were included. The listing data were stratified into two categories-households with children below 5 years and households with children aged 6 to14 years. Each of these was further stratified into households with toilets and without toilets. So, altogether four strata were created, and six households were randomly drawn from each of the four strata.

Data Collection
Data on the selected sample were collected using household, community, and school surveys. The following is a brief description of the various survey instruments designed for the study.
(i) Household surveys. These collected information on the number and demographic profiles of household members, their occupations, the ownership of assets (including toilets), access to water supply and sanitation, usage of toilets by household members, hygiene awareness and hygiene behavior, and incidence of water-related diseases (including diarrhea). Within the survey, a separate module for children aged 6 to 14 collected data on hygiene awareness and behavior. In addition, anthropometric measurements of children under age 5 were also carried out.
(ii) Community survey. The sarpanch (head of the GP) was interviewed. The survey collected information on GP characteristics like the presence of schools, primary health care centers, electricity connections, and implementation of water and sanitation programs. The survey also collected personal information on the sarpanch.
Baseline data were collected from September 25, 2016, to October 28, 2016. A follow-up midline survey of the same households was undertaken from November 2, 2017, to January 15, 2018. A total of 6,067 households from 260 GPs were surveyed at baseline, and 5,275 of these households, from 251 GPs, were followed up with during the midline survey. The project interventions were implemented from November 2016 to the first week of November 2017, when data collection for the midline survey started. So, none of the health-and nutrition-related impacts would have materialized by then. As has been noted, a follow-up survey has been planned for 2021 to measure these impacts.
The same household and community questionnaires were used for both the baseline and midline surveys, though the midline survey included some additional questions to assess the implementation of the project interventions. A school survey was included at midline. The community survey and the school survey were used mostly to assess implementation.

Measurement of Outcomes
The main indicators of interest for this study are hygiene awareness and hygienic behaviors, toilet construction, and toilet use. All these outcomes of interest were measured using data collected from the household survey. A brief description of the indicators used to measure these outcomes follows.
Toilet construction. Toilet construction was measured using households' self-reported ownership of toilets, and toilet technology as observed by the survey team. In line with the Joint Monitoring

Programme (JMP) for Water Supply and Sanitation, coordinated by the World Health Organization and
United Nations Children's Fund, we have defined toilets in three categories. Safely managed toilets are those involving piped sewer systems, septic tanks (with or without soakage pits), and leach pit latrines.
Basic facilities include safely managed toilets as well as pit latrines with a slab. Limited facilities are improved toilets shared among several households. We also included data on toilets under construction. Hygiene awareness and hygienic behavior. Hygiene awareness was measured using respondents' correct identification of the critical times when handwashing is required, the benefits of using soap, and the risks of defecating in the open. Hygienic behavior was measured using the following: (i) percentage of critical times that the respondent washes hands; (ii) percentage of critical times soap is used; (iii) whether the respondent washes hands with soap after defecation and before eating; (iv) the availability of soap in the household; (v) the treatment of water for drinking; and (vi) the disposal of child feces (in a toilet). Similar hygiene awareness and behavior outcomes were measured among children aged 6 to 14 years.

Design Validation
We compared the baseline characteristics of the three treatment arms and the control arm to check for any imbalances in the observed characteristics. Since we had randomized the selection of GPs as well as of households, we expect any imbalances in baseline characteristics to be purely due to chance. We did not test for significance in these baseline differences. As has been argued by various authors (Senn 2013;De Boer et al. 2015), significance tests are not suitable for detecting meaningful differences in baseline characteristics that can lead to confounding. Confounding occurs when certain differences in baseline characteristics can predict the outcomes. For covariates that are highly correlated with the outcomes, even a small difference that is statistically insignificant can be far more important than large significant differences in covariates that are otherwise uncorrelated with the outcomes. So, following Imbens and Wooldridge (2009), we reported normalized differences in means, defined as follows: Here, X is the covariate of interest and N refers to the number of observations. As a rule of thumb, a normalized mean difference of 0.25 or less indicates good balance, whereas a normalized mean difference exceeding unity implies a severe imbalance that cannot be handled using standard regression techniques (Rubin 2001;Imbens and Wooldridge 2009).
The baseline socioeconomic and demographic profiles of households, along with their normalized mean differences across various treatment arms and the control arm, are reported in table 1. We have reported the weighted means with weights based on ownership of toilets to derive population estimates. A more detailed comparison of the baseline characteristics of households as well as their GP-level characteristics for both weighted and unweighted data is presented in tables A.1 and A.2 in the appendix.
Our results indicate that our sample is well balanced. The absolute value of the maximum normalized mean differences of household baseline characteristics is 0.13 in the unweighted data and 0.15 in the weighted data and that of village-level characteristics is 0.2, all indicating a good baseline balance in the sample.
We also looked into the impact of attrition on baseline balance. Attrition in our data comes from two sources-attrition of GPs and attrition of households. The attrition of GPs occurred either due to the census reclassification of GPs into towns (meaning they were no longer eligible for inclusion in the project) or due to the implementation of the project in control GPs. Attrition of households was due to migration, a split in a household, or a lack of availability at the time of the interview. Table A.3 reports attrition for the three treatment arms and control arm. The overall attrition rate for our sample was 13.1 percent. It varied between 10.9 percent for treatment arm 3 and 14.3 percent for the control arm. A 10 percent attrition rate was incorporated during the power calculations at the design stage. Moreover, during sample selection we included 50 additional GPs (10 each in the three treatment arms and 20 in the control arm) to account for possible attrition during project implementation, so the attrition in our sample is unlikely to affect the power of our estimates. A comparison of baseline characteristics for households in various treatment and control arms, post attrition, shows a balance similar to the sample without attrition (table A.4). So, our sample remained well balanced even after attrition. We also compared the characteristics of the households who were present at midline with those who were lost due to attrition (table A.5). We did not find any significant differences between these two groups across a large majority of characteristics. But there were significant differences in characteristics such as asset ownership, occupation of the household head, and toilet access. Households with no land, mobile phone, or access to toilets, and for which the household head's primary occupation was listed as either cultivation or salaried work, were more likely to be absent at midline. To somewhat account for these differences, we analyzed the data separately for households with and without access to toilets at baseline.

Determining Program Impacts: Econometric Model
We used a difference-in-differences (DID) estimation to assess the impact of project interventions. The analysis was done at the household level for all outcome indicators except those related to health and nutrition, for which the analysis was conducted at the individual level. In a DID setup, the first difference (i.e., the difference in outcomes between the periods before and after the intervention) corrects for any remaining fixed effects (i.e., time-invariant differences between treatment and control groups, since we are comparing the same group to itself). The second difference within groups corrects for any time-varying external factors, unrelated to the treatment, that affect the target population during the interval of analysis. Assuming outcomes display equal trends in the absence of treatment (i.e., outcomes in the treatment group would have moved in tandem with the outcomes in the control group in the absence of the intervention), the second difference isolates the true causal effect of the intervention. The following DID specification has been used for the purposes of estimation: = + 1 1 + 2 2 + 3 3 + 4 + 5 * 1 + 6 * 2 + 7 * 3 + + + Where is the outcome of household i in the GP g and district d at time t, before and after the intervention; 1 , 2 , 3 are the intervention indicators for the three treatment arms, respectively; are the household-and GP-level characteristics at the baseline; is the district indicator capturing district fixed effects, since randomization was done at the district level; and is the error term. Coefficients 5 , 6 , 7 are the DID estimates of the impact of the program interventions for treatment arms 1, 2, and 3, respectively. Accordingly, ( 6 − 5 ) estimates the incremental impact of the school interventions and ( 7 − 5 ) estimates the incremental impact of the intensive follow-up campaigns.
We have estimated both the unadjusted and adjusted model. In the unadjusted model, the householdand GP-level characteristics, , were omitted from the specification. In the adjusted model, for household specifications, we have included the following covariates: GPs where the majority of residents are members of a scheduled caste (SC) or scheduled tribe (ST), households where at least one member has a college degree, SC/ST households, and BPL households. The regression specifications have been estimated using ordinary least squares (OLS) with Huber-White robust standard errors clustered at the GP level. Since we oversampled households without toilets, population weights have been used in all specifications involving the full sample.
In addition to the DID specification, we have also used the analysis of covariance (ANCOVA) estimation method. It has been argued that ANCOVA is more efficient than DID (Frison and Pocock 1992), and can lead to improvement in power so long as the autocorrelation is not equal to 1 (McKenzie 2012). We have used the following specification for ANCOVA: = + 1 1 + 2 2 + 3 3 + + + Where and are the outcomes of household i in GP g and district d after and before the intervention. Here, coefficients 1 , 2 , 3 are the ANCOVA estimates of the impact of the program interventions for treatment arms 1, 2, and 3, respectively; and ( 2 − 1 ) estimates the incremental impact of the school interventions, and ( 3 − 1 ) estimates the incremental impact of the intensive follow-up campaigns. The model has been estimated using OLS with Huber-White robust standard errors clustered at the GP level. Though the World Bank evaluation team had an understanding with the government that Punjab could be declared "open defecation free" minus the control GPs, the evaluation team feared the possibility of contamination if the control GPs were withheld for too long. In a meeting with the DWSS, it was decided that all remaining work in the treatment GPs would be wrapped up by October 30, 2017, and all project personnel would be withdrawn and no further work undertaken until the completion of the midline surveys. However, even by the end of October, the project activities in the treatment GPs were not over. So, it was decided to first survey those districts where efforts had been completed. Thus, the midline surveys were conducted in five batches.

Project Implementation
It should be noted that motivators and master motivators were engaged in the treatment GPs right after the completion of the baseline surveys, and the BCC activities were being carried out from December 2016 onward, even though there was no incentive money available for the construction of toilets.
However, there were some prominent departures from the plan. Intensive awareness campaigns in schools could not get in full swing until July 2017 because of the long summer vacation. Moreover, because of delays in implementation, the morning vigils planned to follow the completion of toilets could not be carried out in some GPs. During an evaluation readiness mission in October 2017, it was observed that some of the GPs receiving regular interventions (treatment arm 1) were also included in activities specific to raising awareness in schools (treatment arm 2) and intensive follow-up (treatment arm 3). In some cases, it was found that the intensity of the school awareness campaign was greater in regular-intervention GPs compared with some of the treatment arm 2 GPs. Though the evidence is anecdotal, it does indicate some fundamental inconsistencies between the project's design and implementation.

Summary Statistics
Summary statistics of our outcomes of interest are presented in tables A.6 and A.7. Since a stratified sampling of households was carried out based on toilet ownership, households without toilets were overrepresented in our sample. So, we used weights for all toilet-related variables as well as for hygiene awareness and hygienic behaviors among adult survey respondents to derive population estimates of these outcomes. No weights were used for child-related outcomes or for those variables that were defined for a subpopulation, such as toilets under construction (i.e., applicable only to households without toilets), and safe disposal of feces (i.e., applicable only to households with toilets). Table A.6 reports the summary statistics of outcomes without weights and table A.7 reports the weighted summary statistics.
Discrepancy between unweighted and weighted means in the summary statistics tables reveals the overrepresentation of households without toilets in our sample in general, and in the control group in particular. In the unweighted data, 56 percent of households in the control arm and around 58 percent of households in the three treatment arms had access to toilets during the baseline survey. After the project's implementation, access to toilets increased to 68 percent for the control group and between 72 and 73 percent for the treatment GPs. In the weighted data, access to toilets in the control GPs increased from 80 percent of households at the baseline to 84 percent after the completion of project interventions.
A relatively larger improvement was noticed in the treatment arms, where access to toilets improved from around 74 percent of households at the baseline to around 82-83 percent of households after implementation. Improvement in access in treatment GPs was mostly due to the expanded coverage of safely managed facilities, which increased from around 59-61 percent at the baseline to 66-68 percent after project completion. There was also an increase in "limited" sanitation facilities-that is, toilets shared among households-in the treatment GPs after completion.
For the control GPs, most of the improvement in access seen between the baseline and midline is due to limited sanitation facilities, whose coverage increased from 6 to 11 percent. Coverage of basic and safely managed sanitation facilities in the control GPs mostly remained unaltered (table A. The households in our sample displayed a high degree of awareness of hygiene and sanitation issues.
Baseline survey respondents, both adults and school-going children, were asked whether they had soap at home, and whether they thought that handwashing with soap was a better option than without. Almost all answered yes. A majority of these respondents correctly identified the need for using soap. percentage points and 7 percentage points, respectively. The unweighted summary statistics relating to hygiene awareness and hygienic behaviors among adults were very similar to the weighted ones.
In the baseline survey, around 18 to 20 percent of households from the treatment and the control arms reported the treatment of drinking water, which increased by 2 to 5 percentage points post implementation. So, the impact of the project on households' decision to treat drinking water was limited. Moreover, safe-disposal practices of child feces in households with toilets seem to have deteriorated post completion. In the control arm, 68 percent of households at baseline reported using safe practices to dispose of child feces, which declined to around 65 percent at the midline. A similar pattern was observed in the treatment arms as well, though the magnitudes of the decline were much larger-around 6 percentage points for arm 1, 15 percentage points for arm 2, and 17 percentage points for arm 3.
Around 81 percent of school-going children in the treatment arms and 83 percent in the control arm were aware of the importance of handwashing before eating at the baseline. In the control arm, this awareness increased to 87 percent at midline. Larger improvements were noticed in the treatment arms, where between 95 and 96 percent of school-going children reported being aware of the importance of handwashing before eating, post completion. Awareness of the importance of handwashing after defecation also followed almost identical patterns before and after the project in both control and treatment groups. Similarly, larger proportions of children reported using soap before eating and after defecation in the treatment arms compared with the control arm after project completion. Use of soap before eating among school-going children increased from 61 to 69 percent in the control arm, whereas in the treatment arms it increased from 51-56 percent at the baseline to 73-78 percent at the midline.
Around 83 percent of children from various treatment arms and 88 percent of children from the control group reported using soap after defecation at baseline. After the project's completion, the use of soap after defecation increased by 3 percentage points in the control group and by 7 to 10 percentage points in the three treatment arms.

Results from DID Estimations
Toilet Construction Table 2   Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences.
The project interventions had a significant impact on the ownership of safely managed facilities in intervention GPs in arms 1 and 3, where overall coverage increased between 5 and 6 percentage points compared with control GPs. Coverage of "safely managed toilets in arm 2 increased by 3.5 percentage points compared with control GPs, though the improvement was not statistically significant. Increases in the coverage of basic and safely managed facilities resulted in an overall increase in access to toilets in the intervention GPs. Households with access to toilets increased significantly, by around 3 percentage points in arm 1 and around 4 percentage points in arms 2 and 3 compared with control GPs.
Since the project's financial incentives for toilet construction were meant only for households without toilets, we reported the results separately for households without access to toilets at the baseline (table   3) and for households with toilets (table A.9). In the subsample of households without toilets at baseline, improvement in the coverage of basic and safely managed toilet facilities was higher in arms 2 and 3 than in arm 1. Households with basic toilets increased by around 9 percentage points in arms 2 and 3,  Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences.
We also considered the project's effects on toilets under construction in households without previous access. Toilets under construction increased by 8.2 percentage points in arm 1, 11.9 percentage points in arm 2, and 13.4 percentage points in arm 3 compared with control GPs. Though both arms 2 and 3 had a substantially larger impact on toilets under construction compared with arm 1, the differences were not statistically significant.
When we used the adjusted DID model, the above results remained mostly unchanged (table A.8). We also compared the increase in toilet coverage between households with and without access to toilets at the baseline (tables 3 and A.9). As expected, the increase in coverage in the intervention GPs was driven by households lacking access to toilets at the baseline; these were the major beneficiaries of the program. However, even among the households with access to toilets at the baseline, we would expect a decline in households that had access to limited facilities (i.e., toilets shared among households) and that were to be provided with their own toilets under the project. In the subsample of households with access to toilets at the baseline, we observed almost no impact on those with limited facilities in arm 2 and, consequently, the coverage of basic and safely managed facilities remained mostly unaltered. In arm 3, the coverage of limited facilities declined by around 1 percentage point and the coverage of basic and safely managed facilities expanded by around 3 percentage points compared with the control group.
However, none of these coefficients was statistically significant. In arm 1, basic coverage expanded by 5.3 percentage points compared with the control group and was significant at the 10 percent level. We also observed a 3.5 percentage point decline in the coverage of limited facilities in arm 1 compared with the control group, though it was not statistically significant.

Open Defecation
Survey We also reported the results from the adjusted model (see table A.8 in the appendix). The results are very similar to the unadjusted model with similar levels of significance. We also compared changes in the defecation practices of households with and without access to toilets at the baseline (table 3 and   table A points, respectively, relative to the control group. None of these coefficients was statistically significant. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Table 4 reports the project's impact on the hygiene awareness and hygienic behaviors of adult survey respondents. We mostly focused on responses that were not unanimously correct at the baseline. The survey respondents were asked an open-ended question regarding the critical times for handwashing.

Adults
The proportion of respondents who thought that handwashing is important before eating went up by 9.4 percentage points relative to the control group in arm 3, and the estimate was significant at the 10 percent level. We also observed an increase in awareness related to handwashing before eating in arms 1 and 2 by around 6.5 percentage points relative to the control group. But these results were not statistically significant. However, the project had a statistically significant impact on adult awareness of the importance of handwashing after defecation across different treatment arms. The adult respondents who thought that handwashing after defecation is important increased relative to the control group by 6.9 percentage points in arm 1, 9.1 percentage points in arm 2, and 10.2 percentage points in arm 3.
The project's impact on improving hygienic behaviors among adults was limited. Use of soap among adults before eating went up between 7 and 8 percentage points across various treatment groups compared with the control group. But none of these coefficients was significantly different from zero.
Nonsignificant impacts were also observed on the use of soap after defecation in arms 2 and 3. However, soap use after defecation rose significantly by 7.1 percentage points relative to the control group in GPs receiving regular project interventions. There was no significant impact on the treatment of drinking water or the safe disposal of child feces in the intervention households. Safe disposal of child feces has a negative coefficient across all intervention arms (though not a significant one), implying that a smaller percentage of treatment households were practicing the safe disposal of child feces relative to the control group. The result is puzzling.
The adjusted DID model has results similar to those of the unadjusted model (table A.10). A comparison of outcomes between households with and without access to toilets at the baseline (tables A.12 and A.11) showed that the impact of the program was stronger among households that had access to toilets to start with. For example, an increase in adults' awareness of the importance of handwashing before eating among households with access to toilets at the baseline ranged from 6 to 12 percentage points over the control group across various treatment arms (though it was statistically significant only in arm 3). The impact among households without access to toilets at the baseline ranged from 1 to 7 percentage points (and none was statistically significant). Awareness of the importance of handwashing after defecation among households with access to toilets at the baseline increased relative to the control group by 8.6 percentage points in arm 2, and 11.3 percentage points in arm 3-both significant at the 5 percent level. For households without access to toilets at the baseline, the impact on handwashing after defecation is only significant at the 10 percent level for arm 2, where awareness increased by 10.5 percentage points relative to the control group. We also did not observe any significant impact on hygiene practices in these two subgroups except for the use of soap after defecation in arm 1. The use of soap in GPs receiving regular project interventions increased by 5.9 percentage points and 10.7 percentage points, respectively, among households with and without access to toilets at the baseline.

Children Aged 6-14 Years
The hygiene awareness and sanitary practices of school-going children between ages 6 and 14 are reported in table 5. Awareness of the importance of handwashing before eating increased relative to the control group by 10.4 percentage points in arm 1, 10.7 percentage points in arm 2, and 10.6 percentage points in arm 3. All the coefficients were significant at the 5 percent level. Awareness of handwashing after defecation among school-going children also improved in the treatment arms compared with the control group by 5 to 7 percentage points-from 4.7 percentage points in arm 1 to 7.3 percentage points in arm 3. However, none of these positive impacts was statistically significant.
The use of soap before eating among school-going children increased significantly relative to the control group by 13.8 percentage points in arm 1, 13.3 percentage points in arm 2, and 13.7 percentage points in arm 3. The coefficients were significant at the 5 percent level. The practice of handwashing with soap after defecation among children, however, showed no statistically significant improvement. That said, the impacts observed in all treatment arms were positive, varying between 4.1 percentage points in arm 3 to 6.4 percentage points in arm 2. We also did not find any significant differential impact of intensive school awareness campaigns and intensive follow-up campaigns across households on the handwashing knowledge or practices of school-going children. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences.
The results of the adjusted DID specification reported in table A.13 are similar to those of the unadjusted model. When we compared results for subgroups of children from households with and without access to toilets at the baseline (tables A.15 and A.14), we observed a large positive impact on awareness of the importance of handwashing before eating in both subgroups, though not statistically significant across all arms. Similarly, the project's impact on awareness of the importance of handwashing after defecation was positive in both subgroups, but statistically significant only for arm 3 in the subgroup of children from households with toilets at baseline. The use of soap before eating among children from these households also increased significantly, by 15 to 18 percentage points, across treatment arms relative to the control arm. However, the impact on use of soap after defecation was relatively small and statistically insignificant in arms 2 and 3, and significant at the 10 percent level in arm 1. For children belonging to households without access to toilets at the baseline, the use of soap before eating increased between 9 and 12 percentage points relative to the control group across treatment arms, but the coefficients were not statistically significant. The project's impact on the use of soap after defecation in this sub-group was relatively small and insignificant in arms 1 and 3, but was found to be significant at the 5 percent level in arm 2, which witnessed a 13 percentage points' improvement relative to control post-completion of the project.

Results from the ANCOVA Estimations
Estimates from the ANCOVA specifications are reported in table 6. We also estimated the model for the two subsamples: households with and without access to toilets at the baseline. The results are reported in tables A.16 and A.17, respectively. Since ANCOVA specifications used baseline values of the outcome variables as regressors, access to toilets as well as outcome variables that were conditioned on access to toilets could not be estimated for the subsamples.

Toilet Construction and Use
ANCOVA estimates for outcomes related to the construction and use of toilets were generally smaller in magnitude than the DID estimates. For the full sample, unlike DID estimates, no significant impact was observed on the coverage of basic facilities across treatment arms or access to toilets in arm 1. In arms 2 and 3, access improved significantly relative to the control group, by around 2 percentage points, compared with 4 percentage points estimated under DID. Estimated effects on the coverage of safely managed facilities, toilets under construction, and open defecation also had significance levels similar to DID estimates but smaller in magnitude. As per ANCOVA estimates, open defecation declined in the treatment arms relative to the control group by around 7 to 8 percentage points, and households without initial access but reporting construction of toilets increased by 5.9 percentage points in arm 1, 7 percentage points in arm 2, and 12 percentage points in arm 3.
For the subsamples of households without access to toilets at baseline, we observed a significant impact of the project on the coverage of basic and safely managed facilities. The estimated coefficients were very similar to those of DID estimates. Impacts on open defecation across treatment arms were also similar. Open defecation declined by 1.2 percentage points in arm 1, 3.8 percentage points in arm 2, and 6.5 percentage points in arm 3. However, unlike DID estimates, in which all the coefficients were non-significant, the impact on arm 3 was found to be significant at the 10 percent level. For the subsample of households with access to toilets at the baseline, we found no significant impact on basic coverage. The estimated impacts on safely managed facilities were also similar to DID, except that the impact on arm 1 was significant at the 10 percent level. We also found large reductions in open defecation across treatment arms for this subgroup, which is in line with the findings from the DID analysis.

Adults
Based on the ANCOVA estimates, awareness of the importance of handwashing, both before eating and after defecation, improved significantly among adults across all treatment arms compared with the control group. All the coefficients were found to be significant at least at the 5 percent level. Awareness of the importance of handwashing after defecation increased by 4 to 5 percentage points, which is less than that shown by the DID estimates. Moreover, unlike the DID estimates, which found a statistically significant impact on the awareness of handwashing before eating only in arm 3, the ANCOVA results suggest an improvement of 5 and 7 percentage points across all treatment arms. Similar to the findings of the DID analysis, the ANCOVA results suggest that the program had no statistically significant impact on motivating more people to use soap for handwashing or to treat their drinking water. Coefficients from the ANCOVA estimates of soap use after defecation were negative, though close to zero, and nonsignificant for arms 1 and 2. However, this estimate was negative and significant at the 10 percent level for arm 3. In contrast, the DID estimates suggested positive impacts across treatment arms, but these are statistically significant only for arm 1.
A comparison of results for households with and without access to toilets at the baseline showed statistically significant improvements in awareness among those with access, across the three treatment arms. For households without access to toilets, the impacts on the awareness of handwashing before eating across treatment arms, though positive, were smaller in magnitude and not statistically significant. However, significant positive impacts on the awareness of handwashing after defecation were observed in arms 1 and 2 for the subsample of households without toilets at baseline. Finally, no significant effect on handwashing practices was observed in any treatment arm in either of these subsamples.

Children Aged 6-14 Years
Awareness of handwashing among school-going children showed significant improvement across all treatment arms. All the coefficients were significant at the 1 percent level. Awareness of handwashing before eating in the treatment arms improved relative to the control group by 7 to 8 percentage points, which was slightly smaller than the 9 to 10 percentage points derived from the DID estimates.
Awareness of handwashing after defecation improved across treatment arms by around 6 percentage points. In contrast, DID analysis found significant improvement only in arm 3.
However, the program had little impact on the use of soap after defecation. The coefficients were close to zero and not significant. Moreover, though we found a positive impact on the use of soap before eating, the impact was significant only in arm 2 GPs, where intensive awareness campaigns were conducted in schools. The results of the DID analysis were mostly similar but larger in magnitude.
Along with arm 2, DID analysis also showed a large, positive impact (significant at the 10 percent level) on the use of soap before eating in arm 3.
When we compared the results for the two subsamples-households with and without access to toilets at the baseline-significant improvements in awareness of handwashing before eating were observed in both the subsamples. Awareness of handwashing after defecation also improved by 6 to 8 percentage points across treatment arms in households with access to toilets at baseline. In contrast, for the subgroup without access to toilets, a statistically significant impact was observed only in arm 2, where awareness improved by 7 percentage points relative to control GPs. The project's impact on the use of soap in either of the subgroups (i.e., with and without access to toilets) was limited. Use of soap before eating increased by around 8 percentage points in arm 2 in both subgroups, and use of soap after defecation increased by 3.4 percentage points in arm 1 for households with access at baseline. However, these coefficients were significant only at the 10 percent level.

Discussion of Results
One of the main objectives of the project was to create demand for toilets. It was expected that BCC campaigns along with incentive money for toilet construction would lead to an increase in toilet coverage across the intervention GPs. Our results indicate that the project was successful in increasing ownership of safely managed facilities in intervention GPs, especially among households that did not have access to toilets earlier. This is in line with the project's design, which financed the construction of only safely managed facilities. Universal coverage of individual household latrines was also made implicit in the project design by defining all households without individual toilets as beneficiaries. So, we would expect a decline in limited facilities and an increase in safely managed facilities even in those households with access to toilets at the baseline. We would also expect a proportional increase in the coverage of basic facilities in the treatment GPs since "safely managed" facilities constitute a subset of "basic" facilities. However, our results indicate that for the subsample of households with access to toilets at the baseline, the ownership of toilets both individual and shared was not impacted in arm 2. In the other two treatment arms, while the coverage of safely managed and limited facilities moved in the expected direction, the magnitudes of the impact were small and statistically insignificant. There was also an expansion of limited facilities in intervention GPs in the subsample of households without toilets at baseline. Moreover, summary statistics indicate that basic facilities increased less than what would be proportionate, compared with safely managed facilities, and the proportion of households with limited facilities also increased in the treatment GPs after the completion of project interventions.
During project implementation, the government decided that covered pit latrines with slabs (basic facilities, according to the JMP's definition), were unacceptable and would be destroyed and replaced by safely managed facilities. Subsequently, during the latter part of implementation, when the project ran into financial constraints, the focus shifted from providing all households with safely managed facilities to concentrating on those households that did not have access and also encouraging households to share toilets with their extended families. The combined effect of these decisions by the project management could explain some of our findings.
Our results also suggest that the project significantly increased the number of toilets under construction.
A large proportion of households without access at the baseline, and across treatment arms, reported having toilets under construction at the time of the midline survey. So, the project's long-term benefits, in terms of providing access to toilets as well as expanding coverage of safely managed facilities, might be higher than was observed in the short term.
Our results indicate that the BCC campaign undertaken throughout the implementation of the project Our results indicate that the BCC campaign had a positive and significant impact in raising awareness of the importance of handwashing before eating and after defecation among adults across all treatment arms. The project's impact appeared to be strongest on households that had access to toilets at the baseline. For households without access to toilets at the baseline, the project's impact on adults' awareness of handwashing before eating seemed mostly insignificant, whereas significant improvement in awareness of handwashing after defecation among adults was restricted to GPs receiving regular project interventions and the intensive school awareness campaigns.
The BCC campaign was also very successful in raising awareness of handwashing among school-going children in all treatment arms. Significant improvements in awareness of handwashing before eating were also observed among subgroups of children from households with and without access to toilets at the baseline. Awareness of handwashing after defecation also improved significantly across all treatment arms among households with access to toilets at the baseline. For households without such access, however, a significant improvement was seen only in GPs where intensive school awareness campaigns were conducted.
While the project was mostly successful in raising awareness of the importance handwashing among adults, it had no significant effect on mobilizing more people to use soap, treat their drinking water, or use safe methods for the disposal of child feces. Similarly, the project's impact in promoting improved hygiene and sanitation practices among subgroups with and without access to toilets was mostly insignificant.
The project also had limited impact on improving handwashing practices among school-going children.
We found no systematic impact of the BCC campaign on promoting the use of soap after defecation.
GPs where intensive school campaigns were conducted showed positive significant improvements in use of soaps before eating in the full sample as well as in both sub-samples of households with and without access to toilets. However, the BCC campaign's impact on using soap before eating on other treatment arms was mostly insignificant. That said, DID estimates indicate that the project, overall, had a positive significant impact on the use of soap before eating, especially among households with access to toilets at the baseline.
Our results also indicate a possible incremental impact of intensive school awareness campaigns and intensive follow-up campaigns, though we did not find any statistically significant differential impact of either of these interventions. Awareness of the importance of handwashing before eating and after defecation was relatively higher among school-going children in GPs where intensive school awareness campaigns had been conducted. A significantly larger proportion of children from these GPs also reported using soap before eating. GPs that received the intensive follow-up campaign also witnessed the largest increase in safely managed facilities, and larger proportion of households without initial access to toilets in these GPs reported having toilets under construction. While a reduction in open defecation was mostly observed among households who had access to toilets at the baseline, GPs receiving intensive follow-up campaigns also saw a reduction in open defecation among households without access to toilets at the baseline.

Conclusion
The present study is one of the first rigorous impact evaluations of India's flagship sanitation program, the Swachh Bharat Mission-Gramin. The study used a multiple-arm, cluster randomized control design to assess the short-run effects of the program as implanted in rural Punjab, India on the construction of toilets, the practice of open defecation, and beneficiaries' hygiene-related awareness and practices. The study found that the program was successful in increasing the coverage of safely managed toilets and led to a decline in open defecation among rural households in project intervention areas. A significant reduction in open defecation was observed among households that already had toilets. The effect was less pronounced among households without toilets, which were also the primary targets of the program.
Since the midline survey was conducted immediately after project completion, many households whose toilets had just been completed might not yet have had time to get habituated to using them. Moreover, many other households were still in the process of seeing their toilets constructed. So, the potential impact of the program on toilet construction and open defecation is likely to increase over time.
Interestingly, GPs with intensive follow-up campaigns achieved a significant reduction in open defecation even among households without access to toilets at baseline, and also had a relatively larger impact on expanding the coverage of safely managed facilities as well as access.
The program had a positive impact on raising awareness of the importance of handwashing among adults and school-going children. The program's impact on raising awareness of handwashing among adults appears to be strongest among households with access to toilets at the baseline. GPs with intensive awareness campaigns in schools also had a relatively larger impact on raising awareness of handwashing among school-going children and promoting the use of soap before eating. Otherwise, the program's impact in promoting the use of soap among children and adults was mostly limited. We also found no significant impact on motivating households to treat drinking water, or use safe practices for the disposal of child feces.
At the time of the midline survey, the DWSS was planning to continue the BCC campaigns in the treatment GPs until October 2019, the target end date of the SBMG. Since the program has already shown promise in raising hygiene awareness and discouraging open defecation, more intensive followups are likely to result in better outcomes. The positive impacts found at the midline evaluation of the program promise that positive health and nutritional outcomes associated with better sanitation practices are very likely to materialize over the long term. A follow-up survey and evaluation planned for 2021 will be able to capture these impacts.          Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. # F-values and P-values (reported in parentheses). DID = difference-in-differences. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. Standard errors reported in parentheses. ANCOVA = analysis of covariance. Note: * significance at the .1 level, ** significance at the .05 level, *** significance at the .01 level. Standard errors reported in parentheses. ANCOVA = analysis of covariance.