Policy Research Working Paper 9957 Socioemotional Skills Development in Highly Violent Contexts Measurements and Impacts Lelys Dinarte-Diaz Pablo Egana-delSol Claudia Martinez A. Development Economics Development Research Group March 2022 Policy Research Working Paper 9957 Abstract Non-cognitive skills can determine socioeconomic success curriculum that aims to strengthen participants’ character and the transmission of economic status across genera- (Virtue), or a mindfulness and relaxation technique pro- tions. Yet, evidence of cost-effective interventions that aim gram (Mindful). To estimate the protection component, 8 to develop these skills for at-risk youth living in highly schools were selected as pure controls with a propensity violent contexts is still scarce. This paper experimentally score approach. Results show that the net learning com- studies the social-emotional learning and protection com- ponent improved behavior at school by 0.46 standard ponents of an After School Program (ASP) for teenagers in deviations and reduced a proxy for stress by 0.45 standard the most violent neighborhoods of El Salvador, Honduras, deviations relative to the Clubs only ASP. These results were and Guatemala. By combining administrative records and driven by the Virtue curriculum. Although the protection data gathered on-site via computer from task-based games component negatively impacts social-emotional skills, it is, and AI-powered emotion detection algorithms, this paper on average, more effective for students with worse behavior measures the ASP’s impacts on behavior, academic perfor- at baseline, indicating that the ASP curriculum and the mance, and non-cognitive skills. To measure the learning characteristics of the population served are key in designing component, 21 public schools were randomly assigned policies aimed at improving students’ behavior. to extracurricular activities (Clubs), a psychology-based This paper is a product of the Development Research Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at ldinartediaz@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Socioemotional Skills Development in Highly Violent Contexts: Measurements and Impacts* Lelys Dinarte-Diaz† Pablo Egana-delSol‡ Claudia Martinez A.§ Keywords: Psychology-based Interventions, Non-cognitive Skills, School-based Violence, Socioe- motional Skills Measurement. JEL Classification: I29, K42, Z13, I25, D87. * We appreciate the feedback from Sofia Amaral, Christina Brown, Gemma Dipoppa, Rafael Di Tella, Valentina Duque, Claudio Ferraz, Fran- cisco Gallego, Felipe Gonzalez, Jeanne Lafortune, David McKenzie, Daniel Mejia, Ernesto Schargrodsky, and to participants at seminars in 2021 NEUDC, 2021 RIDGE Forum, 2021 SECHI Annual Meeting, 10th ALCAPONE-LACEA Meeting, ifo-University of Munich, UC Denver, PUC Chile, U. Adolfo Ibanez, II Economics PUC Alumni Workshop, and 2021 Psychology and Economics of Poverty (PEP) Convening-CEGA, UC Berkeley. Egana acknowledges the support and generosity of Rosalind Picard and the Affective Computing group at MIT and Affectiva for help- ing in the measurements of emotional responses. We are truly grateful for the incredible support of Glasswing International as an implementer partner, especially to Stephanie Martinez and Betsabe Vasquez and the Monitoring and Evaluation team. Thanks to Carlos Espinosa, Tatiana Mojica, David Quiroz, and Steffanny Romero for their support as research assistants. We are also grateful to principals, teachers, and students of the public schools in El Salvador, Honduras and Guatemala that participated in this project. This work was supported by the Templeton World Charity Foundation, Inc. Grant Proposal TWCF0362 titled, “Building the local case for Character Development in high-conflict settings of the Northern Triangle in Central America” and by the Research Support Budget (RSB) from the World Bank. Egana-delSol acknowledges the sup- port provided by the Centre for Social Conflict and Cohesion Studies (ANID/FONDAP/15130009) and the ANID/Millenium Science Initiative Program/NCS2021-033. Mart´ ınez A. acknowledges the support from Fondecyt 1201305 and ANID/Millennium Nucleus on Intergenerational Mobility: From Modelling to Policy (MMOVI) [NCS2021072]. The research project’s protocol was reviewed and approved by the Institutional Review Board at the Pontificia Universidad Catolica de Chile under ID Protocol 190129001. The contents of this study are the responsibility of the authors and do not necessarily reflect the views of their employers, funding agencies, or governments. This study was registered in the AEA RCT Registry with unique identifying number: AEARCTR-0003976 https://www.socialscienceregistry.org/trials/3976. † Development Research Group. The World Bank. Email: ldinartediaz@worldbank.org ‡ Adolfo Ibanez University School of Business, EGAP, and Millenium Center of The Evolution of Work (MNEW) affiliate. Email: pablo.egana@uai.cl. § Corresponding author. Pontificia Universidad Catolica ´ de Chile, J-PAL affiliate, and Millennium Nucleus on Intergenerational Mobility: From Modelling to Policy (MMOVI) affiliate. Email: clmartineza@uc.cl 1 Introduction The relevance of non-cognitive or socioemotional skills (SES) in explaining socioeconomic success over the life cycle of an individual and the transmission of economic status across generations is now firmly established. From the seminal works of Bowles et al. (2001); Heckman (2000); Heckman and Rubinstein (2001),1 a growing body of empirical research shows that SES can determine im- provements in academic success and educational transitions,2 cognitive development,3 participation in risky behaviors, or engagement in healthy decisions,4 and labor market outcomes.5 Exposure to high stress or violence can have a substantial effect on an individual’s well-being and economic opportunities, especially during adolescence.6 Existing evidence indicates that expo- sure to stressful, violent or unstable environments may produce unwanted results in terms of SES and other welfare outcomes (Peterson and Seligman, 2003; Baysan et al., 2018; Loewenstein, 2000). More specifically, high stress and emotional instability may lead to short-sighted and risk-averse de- cision making; instead of adopting behaviors that result in greater returns, individuals favor habitual low-yield actions, thereby perpetuating a vicious cycle of remaining stuck in psychological poverty traps (Haushofer and Fehr, 2014; DellaVigna, 2009; Loewenstein, 2000). Moreover, for at-risk peo- ple, the inability to regulate emotions can increase the likelihood that they will respond violently to some stimuli (Peterson and Seligman, 2003). Considering the documented positive effects of SES on socioeconomic outcomes and the undesirable effects of exposure to violent contexts, promoting SES in a cost-effective manner for at-risk individuals living in highly violent environments is extremely relevant to policy. How can we promote SES in at-risk adolescents and children? After-school programs (ASP) are an attractive approach, since they provide at least three services. First, ASPs can protect children by keeping them occupied and off the streets at a time when they might otherwise be left unsuper- vised and exposed to external risks, thereby preventing their victimization and delinquent behavior (Gottfredson et al., 2004; Jacob and Lefgren, 2003). Second, when these programs include a specific 1 For a review of recent literature on measuring and boosting cognitive and non-cognitive skills, see the work of Kautz et al. (2014). Also, for a discussion of the impact of early non cognitive skills on later outcomes, see Carneiro et al. (2007). 2 See Jackson et al. (2020); Dasgupta et al. (2017); Acosta et al. (2015b). For a review of recent studies of SES in education, see (Lechner et al., 2019). 3 See Cunha and Heckman (2008); Cunha et al. (2010). 4 See Mitchell et al. (2020); Chiteji (2010); Jackson et al. (2020). 5 See Heckman et al. (2006); Acosta et al. (2015a,b). 6 Violence can substantially increase the economic costs of health and justice services, constrain human capital acquisi- tion—particularly for adolescents and youth, and significantly hinder economic growth (Krug et al., 2002; Soares and Narit- omi, 2010; Caudillo and Torche, 2014; Monteiro and Rocha, 2017). 1 curriculum designed to foster SES and help students control of impulsive responses, they also offer an alternative source of social-emotional learning and development (Taheri and Welsh, 2016; Durlak et al., 2010; Eccles and Templeton, 2002).7 Third, the adults who are involved in such programs can serve as role models for the participants (Falk et al., 2020). Existing rigorous impact evaluations of ASPs show mixed effects on students’ academic and noncognitive outcomes (Dinarte and Egana-delSol, 2019; ınez and Perticar´ Mart´ a, 2020). Although, the mechanism by which these programs and their impacts operate is unclear, it is plausible that they affect student outcomes through a protection mechanism (supervision or incapacitation), through the activities and skills that the program fosters, or through the program instructors’ role modeling. In this paper, we experimentally disentangle the social-emotional learning and protection components8 of an ASP to determine its impact on measures of academic performance, behavior at school, SES, as well as emotion regulation outcomes. We also provide evidence of the effectiveness of the social-emotional learning components of two psychology-based curricula, and we test these two ASPs differential impacts components by student’s vulnerability and by maternal protection at home. Finally, unlike other studies of this kind, we use innovative tools (i.e., task-based games and artificial intelligence (AI) applications) to estimate objective measures of the participants’ SES and emotion regulation. We partnered with the NGO Glasswing International, which implemented the ASP (and its corresponding treatment arms) in public schools located in the most violent neighborhoods of El Salvador, Honduras, and Guatemala. Glasswing’s ASP, (Clubs), includes extracurricular activities such as dance, sports, and art, among others. These clubs gather in school facilities after school two days per week during the academic year. Each meeting lasts approximately 1.5 hours. Before the intervention, the study sample included 1,975 enrolled students ages 12 to 16 years. Within the Central American context, this age range is significant because gangs are most likely to recruit adolescents of this age (International Crisis Group, 2017). Offering these students protection and fostering their SES can promote traits that will enable them to thrive outside of criminal groups. Participating schools were randomly assigned to one of three treatment arms. The ”active control” group receives Glasswing’s ASP with activities Clubs. The other two groups receive psycho- logical interventions in addition to Glasswing’s ASP, while maintaining the same session duration. 7 Recent evidence of psychological interventions that efficiently improve human behavior and reduce criminal outcomes can be found in Blattman et al. (2017) and Heller et al. (2017). 8 Despite the potential importance of the role model effect, we focus only on the other two components and aim to keep the role model effect similar across treatment arms that test for the other two components, as we describe below. 2 The first psychological curriculum, Character Strengths Development Program (Virtue), which aims to strengthen the participants’ character and increase their psychological well-being, is inspired by the Peterson et al. (2004) model. Peterson hypothesized that by working on one’s strengths and practicing them consistently,individuals can reach their maximum potential and achieve well-being. The Virtue curriculum was specially designed with the Central American context in mind through a asquez and Dinarte, 2021). The second curriculum we study is Calm Class- consultation process (V´ room®, (Mindful), a mindfulness-based and relaxation technique program. Mindfulness in schools has been shown to have a positive impact on students’ socioemotional and academic outcomes (Meiklejohn et al., 2012). This curriculum is a pared-down and less intensive than Virtue. These two different curricula are included to enable us to test two programs that differ in intensity and cost. To experimentally measure and compare the impacts of the social-emotional learning com- ponent to the protection component, we compare the outcomes of students from schools randomly assigned to either Clubs, Mindful, or Virtue. Since all participants are part of an ASP, they automat- ically benefit from the protection and role model channels. The cross-group comparisons identify the net learning channel impacts of the ASP. To estimate the pure-protection component of the ASP, we selected schools similar to those in the interventions before implementation through a propensity score matching.9 These schools are used as a pure control–that is, students enrolled in these schools return to their homes after school. We argue that the effects we can find from Clubs with respect to these pure controls stem from the protection they offer children, while those from the other two inter- ventions with respect to the Clubs are driven by the additional curricular contents to which students are exposed in each of the programs.10 This study includes four stages. The first stage, the enrollment phase, took place before the intervention. During this phase, we collected demographic information from all enrolled children and asked for their and their parents’ consent to access their educational administrative data and to conduct task- and AI-based surveys during the data-collection stages. During the second stage, we collected baseline data, which we completed by March 2019. Specifically, we used task-based games available in the SoftGames App (Danon, 2018) to gather the measures of SES that can be af- fected through these interventions: perseverance, risk-taking behaviors, and self-control (Glasswing, 9 Becauseof implementing partner constraints, it was impossible to randomize a set of pure-control schools. 10 It is important to highlight that we do not rule out that children can learn SES and behaviors through their interactions with other children, as the literature from education has shown. We do argue, however, that we keep this element constant across treatment arms. The social-emotional learning we are measuring arises from the psychology-based curriculum in particular. 3 2019, 2017). Using an affective computing platform based on computer vision and AI (Reactiva), we also collected markers for emotion regulation and stress outcomes such as arousal and valence. In addition, we assembled data on students’ academic performance (i.e., math grades) and behavior at school as reported by teachers. The third stage of this study involved follow-up data collection and adopted a procedure similar to baseline data collection. The follow-up took place after the ASP curricula were completed. Moreover, since evidence shows that teachers’ reports are more predic- tive of students’ behavioral outcomes, grade progression, and adult outcomes (Boon-Falleur et al., 2020; Heckman et al., 2013), we conducted focus groups with teachers and parents one year after the intervention ceased in order to learn whether parents and teachers observed our quantitative findings. We document five main results. First, we find that the social-emotional learning component of the psychology-based interventions had a positive impact of 0.46 standard deviations (sd) on behavior at school and a 0.45 sd decrease in arousal. These results correspond perfectly with the teachers and parents’ qualitative reports, which confirmed a reduction in students’ impulsiveness and improvement in school behavior. Parents and teachers also noted an increase in grades and in students’ expectations for the future. Second, our results show that, on average, the Clubs ASP had some unintended effects. Specif- ically, it has no effect on behavior at school or math grades as well as an adverse effect (in a violent context) insofar as it increased arousal by 0.39 sd relative to the pure-control group. The Clubs’ ad- verse effects indicate that the protection mechanism of this ASP did not have the expected positive impact on children’s outcomes. In this sense, adding a psychology-based curriculum while children are protected in an ASP is necessary to generate positive impacts on student outcomes. Just protect- ing children may not be sufficient to improve their behaviors at school, especially if violence takes place at schools located in unsafe neighborhoods.Thus, protection may generate unexpected results when students are kept in ASPs but are not learning additional skills. Our third result indicates that the type of curriculum for the social-emotional learning compo- nent matters. Overall, we find that the Virtue program had positive effects on school behavior and SES. Compared to students in Clubs, those who participated in Virtue have 0.55 sd better behavior at school, 0.20 sd greater levels of perseverance, and 0.52 sd lower arousal. Moreover, the Mindful cur- riculum has no effect on school behavior or on impulsiveness relative to Clubs. Therefore, the most intensive curriculum–Virtue, which was designed specifically for youth living in violent contexts– was effective in changing behaviors. Although the effects of the Virtue and Mindful curricula are, 4 in most cases, not statistically different, the point estimates are consistent with Virtue being more effective than Mindful in improving student outcomes. Fourth, a heterogeneity analysis by students’ bad behavior at baseline shows that, although the psychological interventions seem more successful in improving students’ behavior, they work only for better behaved students at baseline. The protection mechanism, on the other hand, does not have an impact on the average student, but it does improve behavior of the ex ante badly behaved pupil. Moreover, the impacts of the type of curriculum differ by type of student. Mindful (the pared- down intervention) seems to be more effective for relatively better behaved students, whereas Virtue (a more intensive and custom-fit curriculum) has a greater impact on students with worse behavior at baseline. Finally, we document that the protection mechanism varies depending on the quality of care at home. For example, if a child is under the supervision of parents in the absence of the ASP, he might be better off at home instead of ”protected” in Clubs at school. We study if the Clubs’ impacts vary, depending on whether a child would have been cared for by a mother, and we find that the negative effect of Clubs is observed only for those children without parental care during after-school ınez and Perticar´ hours at baseline, which is similar to the findings of Mart´ a (2020). This paper contributes to three strands of the literature. The first strand includes studies con- cerning how programs (including ASPs) that are oriented toward promoting SES in schools can im- prove children’s and adolescents’ behaviors, academic outcomes, and SES. These works include: Dinarte and Egana-delSol (2019); Jackson et al. (2020); de Chaisemartin and Navarrete (2019); Levine and Zimmerman (2010); Fleming et al. (2008); Vandell et al. (2007). All of these studies measure the total effects of programs that teach participants SES while protecting them for a certain period of time. We contribute to this literature by providing experimental evidence of the specific impacts of the social-emotional learning and protection components of an ASP. Disentangling these two ele- ments has important policy implications. First, if the only accomplishment of an ASP is to protect children and adolescents, then policymakers should redirect resources and simply ensure that par- ticipants are formally supervised, thereby reducing substantially the overall cost of ASPs. Second, if there are differences between the protection and learning channels, as well as differences between the types of learning, then schools should invest in the most effective curricula and implement them among at-risk students, especially those with a greater propensity to demonstrate violent behavior. Second, this paper also contributes to the growing body of economics literature that studies 5 the impacts of psychological interventions on outcomes of at-risk individuals (Heller et al., 2017; Dinarte and Egana-delSol, 2019; de Chaisemartin and Navarrete, 2019; Blattman et al., 2017). To our knowledge, this is the first study that compares different types of psychological approaches to decreasing bad behavior in teenagers, particularly within the context of very violent environments. Our results provide critical information on how to foster good behavior and SES through school- based interventions that differ in intensity and suitability. Moreover, we show that these programs can be implemented in a cost-effective manner. On the one hand, these programs are relatively cheap: the average cost per student of the respective ASPs is US$296.5 (US$269.4 for Clubs, US$292.5 for Mindful, and US$327.6 for Virtue), which is only 1/7 the cost of similar programs for at-risk youth in the United States (Heller et al., 2017). Remarkably, our back-to-the-envelope calculation, which follows Ganimian et al. (2021) and Holla et al. (2021) and the framework of Hendren and Sprung- Keyser (2020) on the marginal value of investments using public funds, shows a benefit-cost ratio that ranges from 12.5 to 50.2. Hence, from a public policy perspective, this program is worth investing in because it is likely to pay for itself in the short run and can even generate large additional welfare gains in the long run. Finally, we also contribute to the innovative approaches to measuring SES. Most of the existing evidence on SES has been obtained through self-reported questionnaires to measure non-cognitive skills. However, self-reported responses can be misleading when comparing levels of personality skills across different groups of people (Kautz et al., 2014). Additionally, self-reporting always entails the risk of bias. This issue becomes more salient when self-reports are used to collect the information necessary to evaluate programs oriented to improving SES, since there is no way to assess the direc- tion of the bias.11 Recent papers have been using task-based games to avoid self-reporting bias and obtain more objective measures of outcomes.12 Unlike previous studies, we use innovative tools, in- cluding data collection through task-based games and AI-powered emotional detection algorithms, to measure non-cognitive skills in our field experiment. We prove that these tools can be used to gather information on SES and emotion regulation in school contexts with minimal infrastructure. Unlike in other studies, our interventions take place within a context of foremost importance– namely, the violent environments of the three Central American countries. Since 2009, violence and crime levels in El Salvador, Honduras, and Guatemala have been labeled an “epidemic of violence” 11 On the one hand, participants may underreport their skills because they may have become more aware of their skill level during the intervention and may feel after the intervention that they have not achieved a proficient enough level. On the other hand, they may overreport their skills to prove that they have learned something from the intervention. 12 See (Danon, 2018). Also, see Kautz et al. (2014) for a discussion of a task-based framework for identifying and measuring non-cognitive skills. 6 WHO (2011). In 2015, homicide rates among boys ages 10 to 19 years in these three countries were 66, 65, and 37 murders per 100,000 adolescents, respectively, placing these three nations among the world’s 10 deadliest places for young boys. Adolescent boys in Honduras are 1.9 times more likely to die from homicide than from any other cause. In El Salvador and Guatemala, one-third of the deaths of adolescent boys is due to homicide (UNICEF, 2017). In addition, during the last decade, these countries have experienced a 13% average reduction in the educational enrollment rate, with over 18% of students reporting that they dropped out of school due to delinquency. Moreover, in El Salvador, 66% of detained unaccompanied children cited violence by organized criminal groups as their main motivation for seeking asylum in the United States (UNHCR, 2014). The grave situa- tion in these countries points to the urgent need to find solutions to such violence–in this case, by implementing and studying the impact of ASPs. The remainder of this paper is organized as follows: Section 2 describes the research design, including the intervention, recruitment process, and experimental design. Section 3 presents the data we collected and how we gathered it. In section 4 , we describe our main empirical approach. Section 5 then summarizes the main results obtained from the specifications explained in Section 4 . Finally, in Section 6 we briefly present our conclusions. 2 Research Design 2 .1 Interventions To experimentally disentangle the impacts of the social-emotional learning and protection compo- nents of an After School Program (ASP) on educational and behavioral outcomes, we study three variations of this ASP: A traditional ASP with extracurricular activities (hereafter Clubs) and two psychology-based curricula, Character Strengths Development Program (hereafter Virtue) and Calm Classroom® (hereafter Mindful). The latter two interventions focus on reducing individuals’ bad behavior and fostering socioemotional skills (SES). In 2019, the NGO Glasswing International imple- mented these three ASP treatment arms in public schools in the most violent neighborhoods of El Salvador, Honduras, and Guatemala.13 13 Glasswing International started operations in El Salvador in 2007 and, since then, has implemented projects throughout Central America, Mexico, Colombia, and the Caribbean. More information about Glasswing International and its work can be found here: https://glasswing.org/. 7 The ASP targets students between 11 and 16 years old in the three countries.14 In Central America, targeting programs at adolescents is important because this is when they are more likely to be forcibly recruited or decide voluntarily to enroll in gangs (Cruz et al., 2016). The ASPs were implemented in school facilities after class time, two days per week during the academic year, which lasted seven months in 2019. On average, each ASP had 13–15 participants. The ASPs were run by adult volunteers who had no formal training in social work or psychology,15 which is a relevant feature of the program in terms of scalability. During the sessions, adults supervised the adolescents and protected them from risky contexts for approximately 1.5 hours. Like supervisors in the Chicago- based program Becoming a Man (Heller et al., 2017), the adult volunteers did not necessarily have similar backgrounds to the participants; consequently, they served as examples to the students. The Clubs ASP, which serves as our curriculum of comparison for the social-emotional learn- ing component, consist of extracurricular activities including sports, science, and arts. For example, in the science category, the club offers students opportunities to conduct experiments such as mock volcano eruptions. The art section includes dancing, singing, and other activities meant to develop fine motor skills and creativity. In the sports section, children play soccer or basketball.16 The in- structors of Clubs played a restricted role as a supervisors or facilitators who registered participants’ attendance, distributed materials, led activities, and helped resolve potential conflicts among partic- ipants. Instructors were not allowed to use any activities from the two curricula described below. The structures of the two psychology-based interventions, Virtue and Mindful, were similar to Clubs, but they also contained supplementary activities related to their specific curricular aims. In each of the sessions, students participating in Virtue and Mindful first received the psychology- based curriculum during the first part of each meeting. During the second part of each meeting, then received the Clubs intervention. The number of volunteers, types of facilities, and number of sessions scheduled per week were the same across all ASPs. We argue, therefore, that any difference in the effects of these interventions was driven by variations in curriculum instead of by how the ASP was implemented. 14 In El Salvador, Guatemala, and Honduras, there are three educational levels: Level 1 includes grades 1-3; Level 2 en- compasses grades 4-6; and Level 3 includes grades 7-9. The target age for this study corresponds to education Levels 2 and 3. 15 Three types of volunteers supported this ASP: community volunteers were tutors living in the community who stood out because of their leadership skills; corporate volunteers belonged to a particular firm involved in a social project with Glasswing; and independent volunteers were usually college students who were involved in social work. Unfortunately, we do not have information on which type of volunteers facilitate each program. 16 More details about this ASP can be found here: https://glasswing.org/program/after-school-programs/. Dinarte and Egana-delSol (2019) experimentally evaluated a version of this ASP in El Salvador and present a detailed descrip- tion of the program. Overall, they find that ASP participants improved their academic performance and behavior at school relative to students in the control group (no ASP). 8 The first psychology-based curriculum we study, Virtue, aims to strengthen participants’ char- acter and increase their development and psychological well-being. In coordination with local ex- perts and psychologists, Glasswing International developed the Virtue program, which was inspired by Peterson et al. (2004)’s model of character strengths and virtues.17 These virtues are: creativ- ity, perspective, courage, perseverance, self-control, social intelligence, and hope. The curriculum includes 32 activities distributed across all of the sessions.18 The Virtue program includes both training and self-reflection activities. The training sessions present concepts, while the reflection sessions invite participants to assess their personal history and environment. To achieve the desired level of reflection, volunteers use an active learning methodol- ogy, which places the students at the center of the learning experience and motivates participation through individual and group activities. Participants actively practice their strengths, reflect on how they have applied them in their daily lives, and acquire tools to adopt them easily.19 For example, the Virtue curriculum includes four sessions (one training and three introspec- tive) to develop perseverance. In the training session, participants discussed the definition of perse- verance and how it can be beneficial to them. Then, the self-reflection sessions included three main activities: The Backpack, My Map, and Fighting for the Puzzle. In The Backpack activity, participants were invited to think about two goals and to describe what tools they would “carry” in their back- packs to have ready at hand to achieve these goals. In the My Map activity, participants discussed the path they would take to achieve their goals, the potential obstacles they may face, and how the tools they included in their backpacks will help them overcome obstacles.20 Finally, in Fighting for the Puzzle, participants had to earn the individual pieces of a puzzle and then complete it. To this end, students had to accomplish some physical tasks such as push-ups in order to obtain each piece of the puzzle. Then, they discussed how achieving the goal (completing the puzzle) required them to exert effort. The second curriculum we study, Mindful, is a mindfulness-based and relaxation response program. This curriculum was adapted to the Central American context from the program developed by the Luster Learning Institute after it conducted a pilot program among 500 Salvadoran students in 2017.21 The program includes directed meditation to reduce stress and anxiety and to control 17 V´ asquez and Dinarte (2021) analyze how the Virtue curriculum was developed. This process includes validation of the most relevant character strengths of the group of students from the three countries in our sample. The program was piloted with a group of adolescents in each country before it was implemented in participant schools in this study. 18 For details on the structure of the program and sessions, see appendix table A1. 19 The instructor’s manual and support materials for virtue are available in Spanish in this link. 20 The activity sheets for these two modules can be found in appendix figures A1 and A2, respectively. 21 For more details on the original program, visit www.calmclassroom.com. 9 automatic responses. It also uses thought techniques to help participants develop self-awareness, mental concentration, and inner calm. Through a series of activities, Mindful provides students with tools to manage their stress more effectively and to regulate their emotions. The Mindful program includes 16 different breathing, stretching, relaxation, and focusing ac- tivities, which were implemented throughout all of the sessions. During each activity, the volunteers explained each technique for about three minutes and then demonstrated and led the students in the practices throughout the session. These activities were supplemented with the Clubs activities. For example, in the I am Calm activity, students used breathing and consciousness techniques such as deep breathing as well as sun and butterfly breathing, among others, to try to calm themselves. Dur- ing sun breathing, participants were invited to sit comfortably in their chairs with their feet on the floor. Then they were asked to breathe for 10 seconds while being conscious of their breathing. They were instructed to inhale through their noses, stretch their arms above their heads, and then lower their arms slowly after 5 seconds. They repeated this exercise 10 times, after which they inhaled and exhaled for 20 to 30 seconds, opened their eyes, and discussed how they felt.22 Mindful cultivates a greater consciousness among students, helping them foster attitudes of respect and kindness toward themselves and others within their school environment. In fact, qual- itative reports from teachers in other schools that had implemented the Mindful intervention before the intervention reported that this program had reduced violent behavior and aggression in their students (Glasswing, 2017). Moreover, Mindful’s essential elements are “non-judgmental and accept- ing awareness of one’s experience from moment to moment” (Keng et al., 2011). Considering the qualitative reports and the essential elements of Mindful, we measured a specific subset of these di- mensions, which were determined by the feasibility of implementing them within the context of our study. We present our measurements in our pre-analysis plan (PAP). Mindful is expected to reduce impulsiveness and risky behavior, and increase fluid intelligence and measures of emotion regula- tion. These hypotheses are consistent with evidence of the positive effects of practicing mindfulness on emotion regulation, the capacity to perform tasks, memory, and attention.23 On the other hand, the “Character Strengths and Virtues” model includes many different classes of virtues.24 Following existing empirical evidence,25 we specified that the Virtue ASP would 22 This activity is presented in figure A3 in the Appendix. A manual of the complete intervention, including support materials along with the activities schedule for every academic year are available in Spanish in this link. 23 See the works of Kral et al. (2018); Saltzman and Goldin (2008); Zelazo and Lyons (2011); Huppert and Johnson (2010); Van de Weijer-Bergsma et al. (2012); and Meiklejohn et al. (2012). 24 Peterson et al. (2004)’s model of character strengths and virtues identify six classes of virtues: Wisdom and Knowledge, Courage, Humanity, Justice, Temperance, and Transcendence. 25 See the seminal works of Linley et al. (2007) and Shimai et al. (2006). 10 increase perseverance, reduce risk-taking behaviors and impulsiveness, and increase emotion regu- lation measures. We preregistered these outcomes in the PAP and collected data to measure these dimensions. 2 .2 Experimental design In this project, we aim to provide evidence that addresses four main research questions: First, what are the effects of the social-emotional learning and protection components of an ASP on behaviors at school and academic performance? Second, how do these two components directly affect par- ticipants’ SES? Third, which curriculum, (Virtue or Mindful), most efficiently improves participants’ behavioral, academic, and socioemotional outcomes? And fourth, do the effects of the interventions vary by bad behavior at baseline and maternal care (as in Dinarte and Egana-delSol (2019); Levine ınez and Perticar´ and Zimmerman (2010) and Mart´ a (2020))? To answer these questions, we devel- oped and implemented the study design described below. A. Experimental Variation: Testing the Social-Emotional Learning Component In January 2019, we identified and recruited 21 public schools to test the social-emotional learning component of our ASP. Our selection process took into account four school criteria: (i) located in the most violent municipalities in El Salvador, Honduras and Guatemala, (ii) have not participated in a Glasswing intervention in the past 5 years, (iii) have both the physical and technological infras- tructure to implement the ASP, and (iv) school principals supported the study and intervention. The schools that met this criteria were chosen and randomized across the three different interventions described above. By stratifying at the country and school risk level (proxied by the homicide rate in the mu- nicipality where the school is located), 21 participating schools were randomly assigned and evenly distributed (i.e., 7 schools per program) in Clubs, Virtue, and Mindful.26 The authors used a self-coded randomization procedure in Stata in order to randomly assign schools to the programs. We then in- formed our implementing partner about the treatment allocation. The NGO distributed materials, trained mentors, and supervised implementation accordingly. Treated schools were aware of their treatment status and the research project’s evaluation goals, but the schools were not aware that another treatment group (or the pure-control group that we describe below) existed. 26 The recruitment strategy used to create a pure-control group from the 8 additional schools is described below. 11 As we describe in Figure 1, we test the social-emotional learning component by comparing the outcomes (i.e., behavior, academic performance, and SES) of students enrolled in schools that were randomly assigned to either Virtue or Mindful, to the same outcomes of the students enrolled in schools assigned to Clubs. B. Non-experimental Variation: Measuring the Protection Component To test the protection channel, a set of pure-control schools (with no intervention at all) were neces- sary. Unfortunately, Glasswing International refused to include the pure-control schools in the ran- domization process because the NGO’s aim is to ensure that the participants of their studies directly benefit from enrollment.27 Since the NGO could not commit additional implementation resources to a number of pure-control schools in upcoming years due to a shortage of funding from the US Gov- ernment, the main donor for its programs,28 we created the pure-control group of schools based on data from national educational censuses. We used a propensity score matching approach to identify schools similar to the experimental sample. To select these schools, we use the following covariates: location (urban or rural), size (enrollment), infrastructure (has its own building, connected to water supply, has computer), programs available to students (food and health), and violence level of the surrounding neighborhood. These covariates were selected because they were present in the school censuses of the three countries before program implementation and because they are not affected by the intervention. Using the Nearest Neighbour Matching algorithm, we selected a total of 10 schools that met criteria (i)-(iii) above and obtained the consent of the principals of 8 of these schools. In this context, we measured the protection component impacts by comparing the outcomes of students enrolled in schools randomly assigned to Clubs and those enrolled in the pure-control schools (see Figure 1). As the experimental setting, we considered impacts on main outcomes (be- havior at school and academic achievement) and mechanisms (SES). To arrive at evidence that would answer our third research question, we use experimental variation and compare the average outcomes of the students who were randomly assigned to Virtue with the average outcomes of those assigned to Mindful. Both of these were then compared to the average outcomes of students allocated to Clubs. Finally, to address the last question, we compare differences in outcomes for children assigned to each curricula by their baseline (bad) behavior at 27 For instance, in Dinarte and Egana-delSol (2019), pure-control schools were told that the team wanted to collect data to measure the schools’ demand for the ASP and that the schools would benefit from the program in the upcoming years. 28 For more details, see: https://www.usatoday.com/story/news/politics/2019/04/04/ us-aid-central-america-what-does-and-why-trump-cut/3340142002/. 12 school and by a proxy for protection at home (mother present).29 This allows us to document the heterogeneous effects of the ASP components on all outcomes of interest. 2 .3 Recruitment of participants Between mid-January and February 2019, before collecting baseline data and implementing the ASP, the Glasswing International visited the 29 schools in our sample.30 presents the different stages of the project. The NGO advertised the ASP and provided informational brochures and videos in the 21 schools that received an active treatment.31 Additionally, Glasswing International invited former participants of ASPs that it ran in other schools to share their experiences. In the pure-control schools, the NGO invited students to join the study.32 In March 2019, the NGO and research team returned to the schools to register and enroll chil- dren in the ASP. Any child was allowed to self-enroll as long as the child signed a consent form. During the registration process, students used tablets to complete the form with personal and family information as well as the application to participate in a particular type of activity (sports, dance, or arts) at school. Out of a total of approximately 4,000 students from the 29 schools, we recruited and enrolled 1,975 students between 11 and 16 years of age (556 in the pure control, 542 in Clubs, 426 in Virtue, and 451 in Mindful). 3 Data and Summary Statistics 3 .1 Data collection In this section, we describe all of the stages during which we collect information from schools and participants, the procedures we followed, and the instruments we used. We determined a baseline and conducted a short-term follow-up immediately after the intervention was finished. We also have administrative data on school enrollment for the year after the intervention ceased. Moreover, we 29 As we registered in our PAP, we also use impulsiveness score at baseline as another proxy for participant vulnerability. As we show in the results section, the estimated differences are very similar across the two measures; for this reason, we show only results for bad behavior at baseline. 30 The timeline in Figure A4 of the Appendix section. 31 The NGO advertised the general ASP, excluding references to any specific activity related to Virtue or Mindful. 32 During its visit to any school, the NGO gave consent forms to both students and their parents to confirm their interest in participating in both the intervention and the study. 13 conducted focus groups with parents and teachers to obtain qualitative insights on the interventions. Figure A4 presents the data collection timeline.33 Baseline data collection We have three sources of baseline data: the registration form, socioemotional tests and games, and administrative data.34 The registration form is available in Table A3 in the Appendix section. It includes a total of 21 questions and takes approximately 15 minutes to complete. The staff of the NGO oversaw registration as part of their operations. Once registered, students received a unique identification number, which enabled us to track them through all data sets.35 Two weeks after registration and before program implementation, enumerators went to schools to collect baseline data on relevant skills: SES, character strengths, and emotion regulation. To collect this data, we used a task-based games application (SoftGames) and an artificial intelligence (AI)-based algorithms application (Reactiva). The SoftGames application includes task-based games such as ad- dition, BART, and Go-NoGo, which can be played on a tablet. The application was developed by Danon (2018) and can be used to measure SES. Reactiva is a smartphone application assembled by one of our co-authors and based on an AI-powered algorithm co-developed by Affectiva and the MIT Media Lab’s Affective Computing Group. Affectiva’s technology analyzes videos captured from the front camera of smartphones or tablets to proxy for emotions. It measures emotional reaction to dif- ferent emotion-laden stimuli (negative and positive videos) based on face detection, facial feature extraction, and expression classification. Via the front cameras of tablets, the computer vision algo- rithm identifies key landmarks on the face. The machine learning algorithm then analyzes pixels to classify facial expressions. The combination of these facial expressions are then mapped to identify emotions. The accuracy level of these algorithms (i.e. success rate in predicting emotional state) is around 75–80%. We used tablets for data entry in the field for all instruments (enrollment survey, games, and AI application). To reduce the risk of fatigue among students, we collected these data sets over two days. On the first day, we asked students to respond to the games-based survey in a tablet. On the following day, the students completed the Reactiva application tasks.36 33 We had to cancel a mid-term follow-up in July 2020 because of COVID-19. As of November 2021, we were still unable to conduct fieldwork due to the outbreak. Considering this time gap, it was no longer useful to conduct this follow-up. 34 A description of all the data collected and outcomes measured is presented in Appendix 1. A summary of outcomes, data source, and type of outcome is shown in appendix table A2. 35 We personally trained enumerators in data-collection procedures to ensure the quality of this process. During the train- ing, we discussed the instruments and protocols, modeled different scenarios that enumerators might encounter, offered ways to handle them, and tested the instruments. 36 In each school, we organized several classrooms where students were scheduled to arrive in groups at specific times. 14 Before the intervention commenced, we were given access to students’ administrative records, which included behavior at school and academic performance (i.e, math grades). Teachers submitted this information as printed reports for their students enrolled in the school. We collected, digitized, and cleaned all these paper reports for the students registered in the program and enrolled in the participating schools in El Salvador, Guatemala, and Honduras. One field coordinator per country was responsible for checking that the digitized data was consistent.37 As in Paluck et al. (2016), if teachers’ reporting of behavioral incidents was affected by the treatment, then our use of behavior reports and math grades could threaten our conclusions regarding the intervention’s causal impact. However, we argue that this occurrence is unlikely because teachers prepare these reports every year and because, in this case, the teachers had already prepared their reports before the research team contacted the schools and before treatment was assigned. In addition, since we requested the records of all students, we did not inform teachers that we were tracking reports on the ASP participants’ school behavior and math grades. Short- and mid-term follow-up As presented in the project timeline in figure A4, the implementation of the ASP and intervention curricula was completed by the end of September 2019. We started short-term follow-up data col- lection in October 2019, while the students were still enrolled in school and before they took final exams in order to maximize our ability to locate them. During this process, we gathered the same outcomes and followed identical procedures as at baseline. As we reported in our PAP, we planed a mid-term data collection process in July and August 2020 (6 months after the intervention ceased) to study whether the impacts found remained stable or changed over time. Unfortunately, we were unable to collect this data at that time because all schools were functioning virtually and were not systematically collecting student data apart from school enrollment due to the pandemic in 2020. Instead, we collected school enrollment in February 2021 in the three countries. For some schools, we also had to digitize these administrative records. Focus groups To understand other potential mechanisms that were driving the short-term results and to obtain reports from parents and teachers, we implemented 24 virtual focus groups between February and March 2021.38 In each country, 4 focus groups were conducted with teachers and 4 with parents. Once the groups arrived, we explained the instructions to them and gave each student a tablet. Enumerators followed up with students who had questions. 37 Due to ethical protocol, we were unable to keep the records of students who did not consent to participate in the study. 38 An independent company in each country organized the focus groups. 15 These focus groups were separated by treatment arm so that we could inquire into each treatment effect. Parents and teachers were recruited from the list of participants. For methodological reasons, we recruited 6 to 8 participants per group. During these groups, we gathered information from the following questions: (i) Have parents/teachers observed the same or different behaviors in their child as those identified in the short-term results? (ii) How easy/difficult is it for teachers/parents to teach skills such as perseverance, self-control, etc.? (iii) What are the parents’/teachers’ perceptions of and expectations for students? 3 .2 Data sources and instruments In this section, we describe the instruments we use for all data-collection stages and to construct the outcomes we study.39 Administrative data a. Behavior at school. To proxy for bad behavior at school, we exploited the availability of teachers’ administrative reports. To standardize behavior reports among countries and schools, we asked school principals and teachers to report this outcome following this Likert scale: Excellent; Average; Below Average. Teachers submitted a printed report for each student enrolled in the school, which we subsequently digitized. Since this scale can be translated into a score from 1 to 3 points, we standardized these scores at the comparison group level as the measure of behavior at school, where a higher score indicates better behavior. b. Academic performance. We measured academic performance using students’ math grades as a proxy for academic performance only because the mathematics curricula in these three countries are similar and, therefore, comparable. Teachers grade students on a scale of 1 to 10 points, where 10 is the highest grade. We standardize these grades at the comparison group level. c. Enrollment. We referred to teacher reports to measure whether the participants enrolled in school for the following academic year (2020-2021) and created a dummy indicator that equals 1 if the stu- dent is enrolled in 2020, and equals 0 if the student is not enrolled in any school. We collected enrollment data from 1,167 students: 79.6% were enrolled in any of the 29 schools in our study, 7.6% were enrolled in another school, and 12.8% were not enrolled in any school.40 39 See appendix table A2 for a summary of this subsection, including the list of outcomes, sources of data used to estimate them, and their category (main or mechanism). 40 We assigned a missing value if the student passed away (1 observation). We were unable to obtain enrollment data for 16 Socioemotional skills According to the NGO’s expected impacts from these interventions (Glasswing, 2017) and to exist- ing evidence summarized in Heller et al. (2017), the behavioral outcomes correlate in a statistically significant way with SES such as self-control/self-regulation, persistence, and prudence. For this rea- son, we proxy persistence with a measure of perseverance, prudence with a measures of risk-taking behaviors, and self-regulation with estimations of self-control and emotion regulation. We used the SoftGames and the Reactiva applications to measure proxies for those skills. a. Perseverance: Perseverance is a continued effort to do or achieve something despite difficulties or obstacles. To proxy for this trait, we estimate a measure of short-term persistence using the Additions Game (Alan and Ertact Grit Task). In this game, participants are given a tablet showing a set of additions that can be easy or difficult to solve. After each round, the participants are asked to choose the level of difficulty for the next set. The outcome is measured as a dummy that equals 1 if a participant persists after failing in round 1, a round of high difficulty level that was predetermined by the tablet. b. Self-control: This trait is defined as the tendency to avoid acting suddenly without thinking care- fully about the consequences of an action. We estimate this trait using the Go-NoGo task-based game, which measures the player’s ability to inhibit an ”inappropriate” response determined by a Go-NoGo rule. Specifically, the participant is presented with a square on the screen for a very short period. If the square is not black, the participant must touch the screen as quickly as possible (the “Go” rule). If the square is black, then the respondent must refrain from responding and touching the screen (the “noGo” rule). A total of 72 trials are presented (48 “Go,” 24 “NoGo”). The score is the number of times a participant responds correctly to the “NoGo” stimulus. c. Self-regulation: Self-regulation is the ability to manage one’s behavior and reactions according to the demands of a given situation. We use two proxies to measure this ability. i. Risk-taking behavior: This behavior is as any consciously or unconsciously controlled behav- ior with a perceived uncertainty about its benefits or detriment to the well-being of oneself or others (Trimpop, 1994). To measure this trait, we used the Balloon Analogue Risk Task (BART). Participants were asked to maximize the number of points they can earn from pumping a balloon. They earned points for every pump. But, they could also lose all of their points if the balloon popped. The out- come was measured as the mean number pumps for the balloons that did not pop. The greater the 277 students who graduated from elementary school and from 73 students due to other reasons. 17 score, the less risk-averse is the individual. ii. Emotion regulation: This trait is defined by Gross (2001) as “all of the conscious and non- conscious strategies we use to increase, maintain, or decrease one or more components of an emo- tional response.” For this endeavour we borrow from advances in the scientific field known as af- fective computing (AC). AC is the study and development of systems and devices that recognize, interpret, process, and simulate human affects. AC is an interdisciplinary field that encompasses computer science, psychology, and cognitive science. Based on AC, we proxy emotion regulation using the Reactiva application. We use a similar protocol similar to that of Egana del Sol (2016) to construct arousal (proxy of alertness and stress) and valence (proxy of emotional self-regulation) indices at the onset of positive and negative stimuli using emotional-laden videos from the GAPEP database. Emotion, expression, and emoji metrics scores indicate when individuals manifest a specific emotion or expression (e.g., a smile) along with the degree of confidence.41 The metrics can be thought of as detectors: as the emotion or facial expres- sion occurs and intensifies, the score rises from 0 (no expression) to 100 (expression fully present). In addition, we also utilize a composite emotional metric called valence which provides feedback on overall experience. Valence values from 0 to 100 indicate a neutral to positive experience, while values from -100 to 0 indicate a negative to neutral experience. We present more details about the rationale for using these metrics, our collection procedure for these measures, and how we analyzed them in Appendix 2. 3 .3 Summary statistics and design validity We present the mean of school characteristics by treatment arm in Table 1. These outcomes were obtained using data from the national censuses of the three countries under analysis. Column (1) exhibits statistics for the total sample, column (2) for the pure-control group, and column (3) for any psychology-based intervention. Columns (4)-(6) show the means of these characteristics but are separated according to the three versions of the ASP. More than half of participating schools are located in very violent communities and around 72% are in urban areas. The schools are mid-sized in terms of enrollment (an average of 456 students in grades 1-9). Most schools have their own building (83%), are connected to a water supply (79%), and have computers within the facilities (76%). Finally, around 86% of schools have a food program for students, but less than 50% of schools provide health 41 See Figure 2 in Appendix 2 as a brief reference. 18 services. The underlying assumption for our method is that the only difference between the schools assigned to treatment and control groups is the treatment status. Thus, we test this assumption using school characteristics data by estimating the following specification: ys = α + β1 Clubss + β2 V irtues + β3 M indf uls + Es + s (1) where ys is a characteristic of school s in our sample. The variables Clubss , V irtues , and M indf uls are dummies that equal 1 if school s was assigned to any of the three treatments. We include stratification block fixed effects (Es ).42 s consists of the specific error term. We estimated robust standard errors for the school-level specifications.43 Overall, our balance tests indicate that schools assigned to the different ASPs are statistically similar across their average characteristics. Since we look at the impact of the intervention on student outcomes, we also test balance in their characteristics before the intervention. First, we estimate mean student characteristics from the total sample and separated by treatment status. Results are summarized in Table 2. Of the total sample of 1,975 students, 556 are from comparison schools and 1,419 are enrolled in treated schools. 52% of the total sample are girls, and the average age is 12 years. The students’ average travel time from home to school is 14.5 minutes. On average, 46% of these adolescents are not living with both parents, which is typical for households in these three countries. 57% of students are regularly cared for by their mother after school at baseline. Only 3% of students report having tried to emigrate to the United States, and 14% report currently working. In terms of academic performance, the mean math grade is above the passing threshold (50 points). Moreover, student misbehavior at school is slightly above the median point. 77% of students are reported as exhibiting bad behavior, which is determined by a score lower than 2. We also test the identification assumption by comparing the means of these baseline vari- ables at the student level. We utilize a modified equation similar to 1 to test for these differences and include strata fixed effects. Standard errors are clustered by schools to account for correlated outcomes among students within the same school. We present unadjusted p-values and sharpened two-stage q-values to account for multiple hypothesis testing. These results are presented in table 42 As mentioned before, the strata were defined by country and violence level (high or low) in the community where a school is located. 43 P-values for each hypothesis tested (including pairwise comparisons between treatment arms) are presented in Table A4 in the Appendix. 19 A5 in the Appendix. As shown, most of the individual and household characteristics are balanced across treatment arms, with the exception of student’s age, travel time (i.e, minutes) from home to school, if student is enrolled in the evening shift,44 work status, mother’s education level, and stu- dent’s household composition (if student lives only with mother). Moreover, some outcomes were imbalanced at baseline, including school dropout, fluid intelligence, self-control, risk-taking, arousal, and emotion regulation. We account for these differences by including the characteristics as controls and by using a difference-in-differences approach as the main estimation model, as we describe in the next section. In this study, we also measured attrition, which we define as the share of students who we were unable to contact for follow-up data collection. Our estimations indicate an average attrition rate of 12%. This rate is slightly higher than in previous studies conducted in similar contexts (Di- narte and Egana-delSol, 2019). The reasons for this difference are twofold. First, the main outcomes of interest from previous studies were obtained from administrative data. Second, although data was gathered using applications, data collection took place on days when other school activities re- lated to the end of the academic year were also taking place on campus. This, unfortunately, lowered the participation rate. We test if the attrition rate varies significantly among treatments and control groups. As presented in appendix table A6, estimations indicate that the attrition rate is balanced across ASP interventions and pure-control groups. To reduce the number of observations lost due to missing data at baseline, we impute the value of the mean for each outcome at the school level and include an indicator (dummy) of imputation in all estimations. We do not impute values in follow-up data. Since nearly all outcome variables of interest are indicator (dummy) or categorical variables, outliers are not an issue. In terms of intervention take-up, attendance reports show that, on average, students assigned to Clubs attended 64% of the sessions, Virtue students attended 82%, and Mindful students attended 74%. Attendance balance tests across treatment arms are presented in table A7 in the Appendix. On average, the attendance rate was greater for the Virtue intervention relative to both Clubs and Mindful interventions (column 1). Although the protocol stipulated maintaining the same number of sessions across each of the ASPs treatment arms, as shown in column (3), the implementing partner changed the number of sessions that were conducted for each ASP because the psychology-based curricula required more time per session, and the NGO was committed to completing the program within the 44 Due to limited infrastructure and to serve most of the demand, schools in El Salvador operate in two shifts, one in the morning and another in the afternoon. 20 academic year. These differences are presented in column (2). One concern is that these differences may compromise the interpretation of the impact from the net social-emotional component, since the longer duration may include more protection time relative to Clubs. In this sense, how we inter- pret the results of the net social-emotional learning component will depend on our findings on the impacts of the protection channel using non-experimental variation. If the average effects of the pro- tection component are positive (or negative), then the findings for the net social-emotional learning co mponent should be interpreted as the lower (or upper) bound of the total effect. Another relevant finding presented in column (4) in table A7 is that dropout rates for the two psychology-based curricula were lower (between 9.8 and 6.4 percentage points) relative to the average dropout rate of students in the Clubs group. Considering attendance and dropout results together, we show that the Virtue ASP was more attractive to the students assigned to it, since they attended more sessions relative to both Mindful and Clubs, and they were less likely to abandon the intervention relative to the Clubs. Finally, while the interventions were assigned at the school level, there is a small chance of non-compliance since children can move to schools assigned to treatment. We have records of students who left the schools in which where they were enrolled, but they may have enrolled in another school. Although we lack exact reports of students transferring to differ- ent schools, our estimations indicate that only a total of 177 (8.96%) students left the schools that participated in this study. 4 Empirical strategy Given our randomized experimental design, it is straightforward to measure the Intent-to-Treat (ITT) effects of the interventions on the outcomes of interest using Ordinary Least Squares approach. How- ever, since differences in the means of our outcomes of interest measured at baseline exist, we use a differences-in-differences approach. In this section, we summarize the specifications used to provide evidence for each our research questions. 4 .1 Measuring the ASP’s social-emotional learning component To measure the effects of the social-emotional learning channel on academic outcomes and behav- ior at school, we exploit experimental variation and use the following specification, restricting the sample to schools assigned to any treatment (Clubs, Virtue, or Mindful): 21 yisjt = µ0 + µ1 AnyTis + µ2 AnyTis × P ost + P ostγXisj + πj + εisjt (2) where yist is the outcome for student i at school s and education level j during period t. AnyTis is a dummy indicating that the student is enrolled in a school that was randomly assigned to treatments Virtue or Mindful. P ost is defined as the post-intervention indicator variable. Xisj is a vector of control baseline variables that include an indicator if the baseline outcome was missing and the variables that are not balanced between treatment arms at the student level: age, travel time, enrolled shift, working status, mother’s education level, and student’s household composition (if the student lives only with mother). πj represents the randomization block fixed effects. We use wild cluster bootstrap at the classroom level. ˆ2 captures the net learning channel—that is, the short-term effect on academic In this model, µ outcomes and behavior at school of being assigned to participate in an ASP plus a psychological curriculum (Virtue or Mindful) compared only to being protected (Clubs). To measure the social-emotional learning component of each type of psychology-based inter- vention, we slightly modify specification (2) to capture the effects of each treatment arm relative to Clubs. Specifically, instead of including the indicator AnyTis , we add two dummies (V irtueis and M indf ulis ) that equal 1 if student i is enrolled in a school s randomly assigned to either Virtue or Mindful. 4 .2 Effects of the protection channel For the protection channel, we restrict the sample to students enrolled in schools assigned to Clubs or pure-control groups. Then, we estimate the following equation: yisjt = θ0 + θ1 Clubsis + θ2 Clubsis × P ost + P ost + δXisj + πj + εisjt (3) where Clubsis indicates if student i in school s in stratification block j is assigned to the extracurric- ular activity. The rest of variables are defined as before. In this sense, the net protection channel—the comparison between children protected in (Clubs) and those who returned home after school—will be measured through the estimated coefficient θ2 . 22 4 .3 Mechanisms To study which SES are driving the effects on academic performance and behavior at school, we use specifications (2) and (3) to test for differences between coefficients (as mentioned before) and to restrict outcomes to the SES and emotion regulation sets. 4 .4 Heterogeneity analysis We study the existence of heterogenous effects on student behavior and counterfactual care. First, the effects of these interventions may differ among students by the baseline level of bad behavior at school, as found in Dinarte and Egana-delSol (2019). Testing for such effects can be important in order to target key policy interventions and direct resources for maximum efficiency. As a measure of bad behavior at school at baseline, we generate the dummy variable Misj , which equals 1 if student i has a below average conduct report (behavior score ≤ 2). Then, we interact the treatment dummies with this indicator and estimate a specification similar to (2) and (3), which also include interactions between the treatment and post variables. We can estimate the difference in the intervention by comparing the worst behaved students and best behaved students who received treatment. ınez and Perticar´ Second, following Mart´ a (2020), we study if the effects depend on the type of after school care the student would have received in absence of the ASP. We generate the dummy variable Cisj , which equals 1 if the mother was regularly home when the student arrived home after school, as reported by the student in the enrollment survey. We then compare Clubs effects depending on the counterfactual care. 5 Results In this section we present the main results by contrasting the different social-emotional learning components against the protection component of the intervention. We also show the estimated dif- ferential impacts by type of curriculum (Virtue or Mindful) on student outcomes. In addition, we provide evidence of some noteworthy heterogeneous effects. Social-emotional Learning and Protection Effects on Students’ Main Outcomes 23 Table 3 shows the net impact of the ASP’s social-emotional learning component relative to the protec- tion component estimated using equation 2 (Panel A) and the total protection effect estimated using equation 3. Column (1) presents the effect on behavior at school and columns (2) and (3) show the effects on academic performance. Compared to students in Clubs, participants of the psychology-based interventions (Virtue and Mindful) have 0.46 sd better behavior at the school (Panel A). In addition, although the point estimates for math grades and school enrollment have the expected sign, they are not statistically significant. These estimated impacts align with focus group feedback from parents and teachers, who perceived that program participants improved in their behavior at school and academic perfor- mance. Our results are consistent with an increase in participants’ willingness to reduce not only bad behavior, but also potentially any tendencies toward violence. The results for the protection component are presented in Panel B. As mentioned before, pro- tection is captured by comparing Clubs, which implies that students remain protected at the school af- ter hours, against the pure-control group where students leave campus immediately after the school day has ended. Overall our results show that the protection component alone impacts neither be- havior at school nor academic outcomes significantly.45 Taken together, these results are consistent with the idea that adding psychology-based curric- ula to interventions like traditional ASPs (i.e., Clubs) is essential to generate impact on student out- comes. The positive effects we find from the net social-emotional learning component on improving conduct at school correspond to the evidence of the positive effects of other curricula intended to improve SES that were used in previous high-quality interventions for adolescents (Vandell et al., 2007; Heller et al., 2017; Dinarte and Egana-delSol, 2019). For example, Vandell et al. (2007) find that high-quality ASPs can foster significant improvements in behavior among disadvantaged stu- dents in the USA. Similarly, Heller et al. (2017) finds that the Becoming A Man program for youth in Chicago reduced violent crime arrests. In El Salvador, Dinarte and Egana-delSol (2019) estimates positive effects of an ASP that includes components of cognitive behavioral therapy on participants’ violent behaviors using teachers’ and students’ self reports. Social-emotional Learning and Protection Effects: Mechanisms We study the impact of the social-emotional learning and protection components on SES and emotion 45 A potential concern regarding these results is the fact that that they depend on the comparison group that is used for the analysis. To address this, we estimate equation 2 using schools in the pure-control group as counterfactual. We find that the estimated effects are similar in signs and statistical significance. These results are available upon request. 24 regulation, which are the potential mechanisms driving the impacts on behavior at school and aca- demic performance. Panel A in Table 4 shows the net social-emotional learning effects, and Panel B presents the protection component impacts. We include four outcomes: SES Index, arousal, valence, and fluid intelligence. The SES Index was created in order to address multiple inference due to the number of outcomes within this category. The index was computed as a weighted standardized av- erage using the approach described in Anderson (2008) and included the measures of perseverance, risk-taking behavior, and self-control collected using the SoftGames application. Our results show that psychology-based interventions decrease arousal by 0.45 sd. Given the Yerkes-Dodson inverted u-shaped relationship between arousal and performance (Yerkes and Dodson, 1908), we conclude that a decrease in arousal can be positive in the context of high violence and stress. In other words, decreased arousal indicates that the stressful contexts in which students live have a less of an affect on their capacity to focus on particular tasks and learn, which is consistent with concepts such as cognitive load or mental bandwidth theories (Kaur et al., 2021; Schilbach et al., 2016; Mullainathan, 2013). These results are consistent with previous estimates of the impact of an ASP on arousal using biomarkers in El Salvador (Dinarte and Egana-delSol, 2019). Additional results show that the point estimates for SES Index, valence, and fluid intelligence have the expected sign but are not statistically significant. In this study, the magnitude of the SES Index, in particular, is relatively important and similar to evidence found in previous studies (Dinarte and Egana-delSol, 2019). Moreover, the magnitude of the SES Index is aligned to qualitative results obtained from the teacher and parent focus groups. These groups report observing a positive effect on self control46 On the other hand, panel B in Table 4 shows that the average arousal score for participants in the Clubs group was higher by 0.39 sd than the score for students in the pure-comparison group. In this instance, the point estimates for the SES Index and valence have the expected sign but are not statistically significant. This adverse effect of Clubs is consistent with the finding that the pro- tection component of the ASP did not have the expected positive impact on children’s outcomes. In this sense, just keeping children within school facilities engaged in extracurricular activities and not learning additional SES may not be enough to improve their behavior at school. The existence of such negative or null impacts from the pure-protection component is not surprising to the extent that school facilities in these three countries are located in areas with high number of gangs, and gang violence is one of the main causes of school dropout (Cu´ ´ ellar-Marchelli and Gochez, 2017)47 46 Teachers and parents specifically reported observing a reduction in impulsiveness in students who participated in the psychology-based interventions Virtue and Mindful. 47 Using data from official sources in El Salvador, Cu´ ´ ellar-Marchelli and Gochez (2017) reports that 24% of schools expe- rience issues with gang members within schools, and 65% of schools are affected by community gangs. Moreover, 61% of 25 In sum, two important policy implications regarding the implementation of these interven- tions arise from these results. First, psychology-based curricula should be incorporated into ASPs to ensure the greatest impact, which is consistent with the expected effects of the intervention according to the NGO. Second, if only a protection-based ASP is financially feasible, then it should guarantee the safety of participants both within and outside of schools; otherwise, the program might have not only null but potentially negative unintended effects. Which Curriculum with a Social-emotional Learning Component is More Effective? As demonstrated previously, the social-emotional learning component of this ASP has positive ef- fects on behavior at school and reduces student arousal. To understand if either of the two psychology- based curricula are driving the effects, we measure the impacts of each curriculum separately using a modified version of equation 2. We present our results in Table 5, where panel A summarizes im- pacts on behavior and academic performance and panel B presents the effects on SES and emotion regulation. Since each curriculum may affect different skills included in the SES Index, we estimate the main effects of each curriculum on the skills separately. We find that Virtue is effective in chang- ing behaviors and skills relative to Clubs. As we show in column (1), Virtue curriculum improves behaviors at school by 0.55 sd and reduces arousal by 0.52 sd. In terms of the specific SES skills, Virtue improves short-term perseverance by 0.20 sd relative to Clubs. We find that Mindful reduces students’ arousal by only 0.35 sd relative to Clubs. However, as we demonstrate in column (7), the estimated coefficients for each intervention are statistically different from each other. These results contribute to the currently large body of research that shows the positive effects of mindfulness on an array of behaviors and outcomes for adults (Keng et al., 2011). The main feature of the Mindful intervention is that it is low-cost, which is an attractive feature with regard to potential public policy. However, for the context and target population of our analysis, Mindful may not be the best alternative to improve behavior and self-regulation. To understand the modest impacts of Mindful, we interviewed the staff of Glasswing International, the implementing NGO. Our discussions shed light on several implementation issues with regard to the Mindful intervention that explain the differences between our results and the existing literature. For example, the Mindful activities were more difficult for the students to follow, especially those with less self-control. Also, the school environment was not always suitable for the implementation of the curriculum (e.g., poor acoustics and a noisy environment). Finally, although the Mindful curriculum was adapted from the Calm Classroom® program designed by the Luster Learning Institute, the Virtue curriculum schools are affected by drug-related problems in the community, and 14% of schools deal with drugs on campus. 26 was specifically prepared for youth living in violent neighborhoods in Central America after much asquez and Dinarte (2021). consultation, as described in V´ Based on the impacts estimated from the social-emotional learning and protection components of the ASP, we document two important conclusions. First, only protecting children may not be sufficient to improve their academic achievement or behavior at school. Second, an ASP should also include specific evidence-based curriculum oriented toward improving student outcomes. We find that the resourceful and custom-designed Virtue intervention is the best curriculum to add to Clubs to help vulnerable youth in poor and highly violent countries. Heterogeneous Effects A. Heterogeneity by student vulnerability To better understand the previous results, we argue that the effects might vary depending on the ex ante level of bad behavior or impulsiveness at baseline (i.e., before the program started). Testing for such effects is important to target key policy interventions and direct resources for maximum efficiency. Table 6 shows the heterogeneous effects by bad behavior at baseline. We defined badly be- haved as students with a behavior score ≤ 2 at baseline. Columns (1) and (2) show the total effects on well and badly behaved students, respectively. Column (3) shows the differences in the impacts between badly and well behaved students. Results in Panel A show the that the positive impacts of psychology-based curricula on behavior at school and school enrollment are mitigated for stu- dents categorized as badly behaved at baseline. Well behaved students are driving the effects on improvements in behavior at school and greater school enrollment relative to both other well be- haved students in Clubs and to the badly behaved treated children. Regarding the heterogeneity of SES and emotion regulation, panel B in table 6 shows a positive impact on the SES Index for badly behaved students (0.15 sd), and a moderating effect on arousal (decreasing to 0.39 sd) relative to other badly behaved students in the comparison group. For the well behaved participants, we also find a decrease in arousal by 0.53 sd relative to similar students in the comparison group. For the SES Index and emotion regulation, we do not find differences by behavior at baseline (Panel B, column 3). Taken together, these results show that the ASP’s social- emotional learning component has greater impacts on less vulnerable participants relative to the most vulnerable. 27 From the qualitative interviews that we conducted with the implementing NGO, we learned that it was more challenging for instructors to implement the activities of the Mindful program. The NGO staff also reported that it was more difficult for badly behaved children to follow the instruc- tions of the Mindful activities compared to the Virtue activities. In this sense, the greater effects for the less vulnerable students reported in table 6 could be explained potentially by the fact that the differences were driven by the implementation challenge that the instructors reported. To test this hypothesis, we implement an heterogeneity analysis by bad behavior at baseline according to type of curriculum and present our results in table 7. We document that, in fact, the Mindful curriculum has a greater impact on behavior at school, school enrollment, arousal, and fluid intelligence of students with ex ante good behavior compared to ex ante students with bad behavior who received the same treatment. The improvements in behavior at school, arousal, and fluid intelligence are also greater for the well behaved group when compared to other ex ante well behaved children in the Clubs group. The Virtue intervention, on the other hand, has positive effects for students with bad behavior reports at baseline. We find that the Virtue program, has greater impacts on SES Index of badly behaved students relative to both the Virtue program’s well behaved students and the Clubs intervention’s badly behaved students. Moreover, Virtue reduced the arousal scores of badly behaved children compared categorically to the same students in Clubs. Regarding the pure-protection mechanism, Table 8 uses the non-experimental variation and compares Clubs with the pure-control, differentiating the effects by student behavior at baseline. Columns (1) and (2) of Panel A show that Clubs improves behavior at schools for students with low behavior scores at baseline, compared to both well behaved students treated in Clubs and other badly behaved students in the pure-control group. Our results in panel B also show that badly behaved treated students in Clubs increase their average arousal relative to similar students in the pure-control group. However, we cannot reject the hypothesis that this effect is similar to the increase in arousal from the well behaved students treated in Clubs. In sum, these heterogeneous results shed light on two main elements. First, although the psychological interventions appear more successful in improving better behaved students’ behavior, they do not seem to work for students with lower behavioral scores at baseline. On the other hand, the protection mechanism does not have an impact on the average student, but it does improve behavior for the ex ante badly behaved students. Second, offering different curricula for different types of students seems to be very effective. A pared-down intervention such as Mindful seems more 28 effective for relatively better behaved students, whereas a more intensive and custom-fit curriculum such as Virtue is more suitable for students who exhibit ex ante bad behavior.48 B. Heterogeneity by family vulnerability The protection mechanism might vary depending on the quality of the care that students receive at ınez and Perticar´ home, as recently found by Mart´ a (2020). If children from our study have higher quality protection at home (they are supervised by their parents, for instance), then they might be better-off returning home after school instead of being “protected” in Clubs at school. This is more salient with regard to the public schools that we analyze in this paper, since these environments and ellar-Marchelli and the surrounding communities are plagued by violence, as documented by Cu´ ´ Gochez (2017) and discussed previously. In particular, we study if Clubs impacts vary if children would have been in their mothers’ care. We observed that Clubs had a negative effect only on students cared for by their mothers after school. Table 9, Panel A shows that the marginal effect of being under a mother’s care on behavior at school and math grades is negative and significant for the latter. Furthermore, the increase in arousal is larger for students without maternal care.49 In this way, as previously documented by Mart´ ınez a (2020), the ASP seems to substitute for a mother’s presence. and Perticar´ 6 Conclusions This paper provides the first experimental evaluation that disentangles the social-emotional learning and protection components of an ASP implemented in three developing and highly violent countries on adolescent behavior at school, academic performance, socioemotional skills (SES), and emotion regulation. To our knowledge, this is the first study of its kind to use artificial intelligence (AI) technology and task-based games to estimate proxies for the difficult-to-measure competencies of SES and emotion regulation. 48 We replicated the heterogeneous analysis by impulsiveness at baseline, and we found no significant heterogeneity across curricula. We also conducted heterogeneity analyses by gender. Overall, we find no statistically significant differences be- tween boys and girls. All of these results are available upon request. 49 We also explore two more heterogeneity analyses. First, we study differences by mother’s education. We compare the outcomes of children of mothers with less education (at least elementary education) with the outcomes of children of mothers with more education (college or higher). As we show in table A8 in the Appendix, there are no important differences between these two groups. Second, we consider whether families that are wealthier due to a father who has migrated to another country and sends money home explain these results by comparing students living with both parents (i.e., no migration) and students living with only their mothers (i.e., fathers are absent due to potential migration), but we do not find any statistically significant differences. These results are available upon request. 29 To achieve our end, we created an exogenous variation by randomly assigning participant schools to three ASP interventions: one based only on recreational activities such as sports, art, and dancing (Clubs) and two psychological interventions based, respectively, on a curriculum that aims to strengthen character and virtues (Virtue) and a mindfulness and relaxation program (Mindful). Every Virtue and Mindful meeting was immediately followed by Clubs activities. By comparing the average outcomes of students enrolled in schools assigned to any of the two psychology-based curricula with students enrolled in schools assigned to Clubs, we measure the net social-emotional learning channel. We provide evidence that the psychology-based interventions improve students’ behavior at school and decrease arousal, a proxy for alertness or stress. To measure the effects of the protection component, we use non-experimental variation and compare average outcomes for students assigned to the recreational intervention with the outcomes for students enrolled in a group of selected schools that served as a pure-control (no intervention at all). Our findings indicate that there were no impacts on students’ behavior and academic outcomes, and, unexpectedly, there was an increase in arousal. Considering these results, we conclude that the activities developed in the ASP play a critical role in the program’s effects. A pure-protection ASP can have unintended consequences, which are potentially more salient in contexts where schools are not the safest places for children, relative to their alternative care at home. Moreover, including psychology-based interventions in addition to recreational activities organized within the context of an ASP can have a positive impact on students’ behavior at school and on measures that proxy for participant alertness or stress. We also find policy-relevant heterogeneous effects. First, the type of psychology-based cur- riculum matters for both the average student and for students who are ex ante badly behaved at school. On average, the Virtue curriculum impacts student behavior, short-term perseverance, and arousal. On the other hand, the Mindful curriculum only decreases arousal and has no effect on behavior at school and other academic outcomes. Therefore, we conclude that Virtue, which was specifically designed for youth living in violent environments and involves more intensive activi- ties, is an effective intervention for the average student in our sample. Moreover, a less demanding intervention (i.e., Mindful) seems more effective for relatively better behaved students, whereas a more intensive and custom-fit curriculum (i.e., Virtue) is more suitable for students with ex ante bad behavior. Second, we find that neither of the psychology-based interventions successfully affect the out- comes of students with low behavioral scores at baseline. On the other hand, when compared to the 30 pure control, the Clubs only intervention does improve more vulnerable students’ behavior at school even though it does also increase arousal. In this way, the protection mechanism appears relevant for more vulnerable students. However, more specialized and resourceful interventions available via psychology-based curricula could be necessary to further improve student outcomes. We come to the conclusion that, on average, the learning mechanism is more relevant than the protection mechanism. We also find that, among the two psychological curricula tested, the more intense and tailored curriculum (Virtue) is more effective. For students with worse behavior at baseline, on the other hand, the protection mechanism is more relevant. These results are useful in designing and targeting interventions, and they also open the research question to consider the type of curriculum that can generate greater outcomes for the more vulnerable students who participate in a solely protective ASP. Despite the differences in their impacts, cost estimates provided by Glasswing International show that the costs differences between the different interventions is not significant. The average cost of a 7-month long ASP intervention per student is US$296.5. The cost by specific ASP is: US$269.4 for Clubs, US$292.5 for Mindful, and US$327.6 for Virtue).50 Based on the literature that estimates the relationship between human capital interventions and impacts on adult earnings (Holla et al., 2021; Ganimian et al., 2021; Galasso and Wagstaff, 2019) and following the framework of Hendren and Sprung-Keyser (2020) on the marginal value of investments using public funds, we conducted an approximate calculation of the program’s benefit-cost ratio.51 Our estimates indicate that the present discounted value of earning gains expected to result from this ASP’s impact due to improvements in behavior at school and an indirect potential reduction in school dropout yields a benefit-cost ratio that ranges from 12.5 to 50.2. Consequently, investing in this program should be encouraged as a public policy, as the intervention is likely to pay for itself even in the short run and has the potential to generate large additional welfare gains in the long run. References Acosta, P., Muller, N., Sarzosa, M., 2015a. Adults’ cognitive and socioemotional skills and their labor market outcomes in colombia . 50 Similar interventions involving at-risk youths in the USA (Heller et al., 2017) are 6.8 times more expensive per participant than the average of the interventions involved in this study. 51 See more details of our approach in Appendix 3. 31 Acosta, P., Muller, N., Sarzosa, M.A., 2015b. Beyond qualifications: Returns to cognitive and socio- emotional skills in colombia. World Bank Policy Research Working Paper . Anderson, M.L., 2008. Multiple inference and gender differences in the effects of early intervention: A reevaluation of the abecedarian, perry preschool, and early training projects. Journal of the American statistical Association 103, 1481–1495. alez, F., Hsiang, S., Miguel, E., 2018. Economic and non-economic factors Baysan, C., Burke, M., Gonz´ in violence: Evidence from organized crime, suicides and climate in mexico. Technical Report. National Bureau of Economic Research. Blattman, C., Jamison, J.C., Sheridan, M., 2017. Reducing crime and violence: Experimental evidence from cognitive behavioral therapy in liberia. American Economic Review 107, 1165–1206. Boon-Falleur, M., Bouguen, A., Charpentier, A., Algan, Y., Huillery, E., Chevallier, C., 2020. Measur- ing socio-emotional skills in schools: simple questionnaires outperform behavioral tasks . Bowles, S., Gintis, H., Osborne, M., 2001. The determinants of earnings: A behavioral approach. Journal of economic literature 39, 1137–1176. Carneiro, P., Crawford, C., Goodman, A., 2007. The impact of early cognitive and non-cognitive skills on later outcomes . Caudillo, M.L., Torche, F., 2014. Exposure to local homicides and early educational achievement in mexico. Sociology of education 87, 89–105. de Chaisemartin, C., Navarrete, N., 2019. The direct and spillover effects of a nationwide socio- emotional learning program for disruptive students. Available at SSRN 3316325 . Chiteji, N., 2010. Time preference, noncognitive skills and well being across the life course: Do noncognitive skills encourage healthy behavior? American Economic Review 100, 200–204. URL: https://www.aeaweb.org/articles?id=10.1257/aer.100.2.200, doi:10.1257/aer. 100.2.200. Cruz, J.M., Fonseca, B., Director, J.D., 2016. The new face of street gangs: The gang phenomenon in el salvador. IRB 16, 0322. Cu´ ´ ellar-Marchelli, H., Gochez, G., 2017. La pertinencia de las estrategias para prevenir la violencia ˜ para ´ Salvadorena escolar en El Salvador. Departamento de Estudios Sociales, DES, Fundacion el . . . . 32 Cunha, F., Heckman, J.J., 2008. Formulating, identifying and estimating the technology of cognitive and noncognitive skill formation. Journal of human resources 43, 738–782. Cunha, F., Heckman, J.J., Schennach, S.M., 2010. Estimating the technology of cognitive and noncog- nitive skill formation. Econometrica 78, 883–931. Danon, A., 2018. Do we know how to measure what matters? the measurement and importance of socio-emotional skills in pakistan. Unpublished manuscript . Dasgupta, U., Mani, S., Sharma, S., Singhal, S., 2017. Cognitive, socioemotional and behavioral returns to college quality . DellaVigna, S., 2009. Psychology and economics: Evidence from the field. Journal of Economic literature 47, 315–72. Dinarte, L., Egana-delSol, P., 2019. Preventing violence in the most violent contexts: Behavioral and neurophysiological evidence. Policy Research working paper; no. WPS 8862. Washington, D.C.: The World Bank . Durlak, J.A., Weissberg, R.P., Pachan, M., 2010. A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American journal of community psychology 45, 294–309. Eccles, J.S., Templeton, J., 2002. Chapter 4: Extracurricular and other after-school activities for youth. Review of research in education 26, 113–180. Ekman, P., Friesen, W., 1978. Facial action coding system: Manual. Falk, A., Kosse, F., Pinger, P., 2020. Mentoring and schooling decisions: Causal evidence . Fleming, C.B., Catalano, R.F., Mazza, J.J., Brown, E.C., Haggerty, K.P., Harachi, T.W., 2008. After- school activities, misbehavior in school, and delinquency from the end of elementary school through the beginning of high school: A test of social development model hypotheses. The Journal of Early Adolescence 28, 277–303. Galasso, E., Wagstaff, A., 2019. The aggregate income losses from childhood stunting and the returns to a nutrition intervention aimed at reducing stunting. Economics & Human Biology 34, 225–238. Ganimian, A.J., Muralidharan, K., Walters, C.R., 2021. Augmenting State Capacity for Child De- velopment: Experimental Evidence from India. Technical Report. National Bureau of Economic Research. 33 Glasswing, 2017. Estudio cualitativo de impacto del programa de Mindfulness implementado en Centroamerica. Technical Report. Glasswing, 2019. Desarrollo de Programa de Fortalezas del Caracter. Technical Report. e, D.A., Womer, S.C., Lu, S., 2004. Do after school programs Gottfredson, D.C., Gerstenblith, S.A., Soul´ reduce delinquency? Prevention science 5, 253–266. Gross, J.J., 2001. Emotion regulation in adulthood: Timing is everything. Current directions in psychological science 10, 214–219. Haushofer, J., Fehr, E., 2014. On the psychology of poverty. science 344, 862–867. Heckman, J., Pinto, R., Savelyev, P., 2013. Understanding the mechanisms through which an influen- tial early childhood program boosted adult outcomes. American Economic Review 103, 2052–86. Heckman, J.J., 2000. Policies to foster human capital. Research in economics 54, 3–56. Heckman, J.J., Rubinstein, Y., 2001. The importance of noncognitive skills: Lessons from the ged testing program. American Economic Review 91, 145–149. Heckman, J.J., Stixrud, J., Urzua, S., 2006. The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor economics 24, 411–482. Heller, S.B., Shah, A.K., Guryan, J., Ludwig, J., Mullainathan, S., Pollack, H.A., 2017. Thinking, fast and slow? some field experiments to reduce crime and dropout in chicago. The Quarterly Journal of Economics 132, 1–54. Hendren, N., Sprung-Keyser, B., 2020. A unified welfare analysis of government policies. The Quar- terly Journal of Economics 135, 1209–1318. Holla, A., Bendini, M., Dinarte, L., Trako, I., 2021. Is investment in preprimary education too low? lessons from (quasi) experimental evidence across countries . Huppert, F.A., Johnson, D.M., 2010. A controlled trial of mindfulness training in schools: The importance of practice for an impact on well-being. The Journal of Positive Psychol- ogy 5, 264–274. URL: https://doi.org/10.1080/17439761003794148, doi:10.1080/ 17439761003794148, arXiv:https://doi.org/10.1080/17439761003794148. International Crisis Group, 2017. El Salvador’s Politics of Perpet- ual Violence — Crisis Group , 1–46URL: https://www.crisisgroup. 34 org/latin-america-caribbean/central-america/el-salvador/ 64-el-salvadors-politics-perpetual-violence. Jackson, C.K., Porter, S.C., Easton, J.Q., Blanchard, A., Kiguel, S., 2020. School effects on socioemo- tional development, school-based arrests, and educational attainment. American Economic Re- view: Insights 2, 491–508. URL: https://www.aeaweb.org/articles?id=10.1257/aeri. 20200029, doi:10.1257/aeri.20200029. Jacob, B.A., Lefgren, L., 2003. Are idle hands the devil’s workshop? incapacitation, concentration, and juvenile crime. American economic review 93, 1560–1577. Kaur, S., Mullainathan, S., Oh, S., Schilbach, F., 2021. Do Financial Concerns Make Workers Less Productive? Working Paper 28338. National Bureau of Economic Research. URL: http://www. nber.org/papers/w28338, doi:10.3386/w28338. Kautz, T., Heckman, J.J., Diris, R., Ter Weel, B., Borghans, L., 2014. Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success . Keng, S.L., Smoski, M.J., Robins, C.J., 2011. Effects of mindfulness on psychological health: A review of empirical studies. Clinical psychology review 31, 1041–1056. Kral, T.R., Schuyler, B.S., Mumford, J.A., Rosenkranz, M.A., Lutz, A., Davidson, R.J., 2018. Impact of short- and long-term mindfulness meditation training on amygdala reactivity to emotional stimuli. NeuroImage 181, 301 – 313. URL: http://www.sciencedirect. com/science/article/pii/S1053811918306256, doi:https://doi.org/10.1016/j. neuroimage.2018.07.013. Krug, E.G., Mercy, J.A., Dahlberg, L.L., Zwi, A.B., 2002. The world report on violence and health. The lancet 360, 1083–1088. Lechner, C.M., Anger, S., Rammstedt, B., 2019. Socio-emotional skills in education and beyond: re- cent evidence and future research avenues, in: Research Handbook on the Sociology of Education. Edward Elgar Publishing. Levine, P.B., Zimmerman, D.J., 2010. Targeting investments in children: Fighting poverty when resources are limited. University of Chicago Press. Linley, P.A., Maltby, J., Wood, A.M., Joseph, S., Harrington, S., Peterson, C., Park, N., Seligman, M.E., 2007. Character strengths in the united kingdom: The via inventory of strengths. Personality and individual differences 43, 341–351. 35 Loewenstein, G., 2000. Emotions in economic theory and economic behavior. American economic review 90, 426–432. ınez, C., Perticar´ Mart´ a, M., 2020. Home alone versus after-school programs: The effects of adult supervision on child academic outcomes. International Journal of Educational Research 104. doi:https://doi.org/10.1016/j.ijer.2020.101601. Meiklejohn, J., Phillips, C., Freedman, M.L., Griffin, M.L., Biegel, G., Roach, A., Frank, J., Burke, C., Pinger, L., Soloway, G., et al., 2012. Integrating mindfulness training into k-12 education: Fostering the resilience of teachers and students. Mindfulness 3, 291–307. anchez, A., 2020. Human Capital Development: New Evidence Mitchell, M., Favara, M., Porter, C., S´ on the Production of Socio-Emotional Skills. Technical Report. IZA Discussion Papers. Monteiro, J., Rocha, R., 2017. Drug battles and school achievement: evidence from rio de janeiro’s favelas. Review of Economics and Statistics 99, 213–228. Mullainathan, S., .S.E., 2013. Scarcity: Why Having Too Little Means So Much. Henry Holt Times Books, New York. URL: https://us.macmillan.com/books/9781250056115. Paluck, E.L., Shepherd, H., Aronow, P.M., 2016. Changing climates of conflict: A social network experiment in 56 schools. Proceedings of the National Academy of Sciences 113, 566–571. Peterson, C., Seligman, M.E., 2003. Character strengths before and after september 11. Psychological Science 14, 381–384. Peterson, C., Seligman, M.E., et al., 2004. Character strengths and virtues: A handbook and classifi- cation. volume 1. Oxford University Press. Picard, R., 1995. Affective computing. MIT Technical Report 321 (Abstract) . Picard, R.W., 1997. Affective Computing. MIT Press, Cambridge, MA, USA. Saltzman, A., Goldin, P., 2008. Mindfulness-based stress reduction for school-age children. Accep- tance and mindfulness interventions for children adolescents and families . Schilbach, F., Schofield, H., Mullainathan, S., 2016. The psychological lives of the poor. American Economic Review 106, 435–40. URL: https://www.aeaweb.org/articles?id=10.1257/ aer.p20161101, doi:10.1257/aer.p20161101. 36 Shimai, S., Otake, K., Park, N., Peterson, C., Seligman, M., 2006. Convergence of character strengths in american and japanese young adults. Journal of Happiness Studies 7, 311–322. doi:10.1007/ s10902-005-3647-7. Soares, R.R., Naritomi, J., 2010. Understanding high crime rates in latin america: The role of social and policy factors, in: The economics of crime: Lessons for and from Latin America. University of Chicago Press, pp. 19–55. Egana del Sol, P., 2016. Affective neuroscience meets labor economics: Assessing non-cognitive skills on late stage investment on at-risk youth, in: Skills For Sustainable Development: Essays On How Creativity, Entrepreneurship And Emotions Foster Human Development, Chapter 2. Columbia University Academic Commons, New York. Taheri, S.A., Welsh, B.C., 2016. After-school programs for delinquency prevention: A systematic review and meta-analysis. Youth violence and juvenile justice 14, 272–290. Trimpop, R., 1994. The Psychology of Risk Taking Behavior. ISSN, Elsevier Science. URL: https: //books.google.cl/books?id=rI4c24VTriEC. UNHCR, 2014. Children on the run: Unaccompanied children leaving central america and mexico and the need for international protection . UNICEF, 2017. A familiar face: Violence in the lives of children and adolescents . Vandell, D.L., Reisner, E.R., Pierce, K.M., 2007. Outcomes linked to high-quality afterschool pro- grams: Longitudinal findings from the study of promising afterschool programs. Policy Studies Associates, Inc. . asquez, B., Dinarte, L., 2021. Designing a character strengths development program for youth living V´ in violent contexts. Journal of Youth Development 16, 329–343. Weber, E.U., Johnson, E.J., 2009. Mindful judgment and decision making. Annual review of psychol- ogy 60, 53–85. ¨ Van de Weijer-Bergsma, E., Formsma, A.R., de Bruin, E.I., Bogels, S.M., 2012. The effectiveness of mindfulness training on behavioral problems and attentional functioning in adolescents with adhd. Journal of child and family studies 21, 775–787. WHO, 2011. World health organization statistics . 37 Yerkes, R.M., Dodson, J.D., 1908. The relation of strength of stimulus to ra- pidity of habit-formation. Journal of Comparative Neurology and Psychol- ogy 18, 459–482. URL: https://onlinelibrary.wiley.com/doi/abs/ 10.1002/cne.920180503, doi:https://doi.org/10.1002/cne.920180503, arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/cne.920180503. Zelazo, P.D., Lyons, K.E., 2011. Mindfulness training in childhood. Human Development 54, 61–65. URL: https://www.jstor.org/stable/26764991. 38 7 Tables and Figures Table 1: Mean School Characteristics by Treatment Group (1) (2) (3) (4) (5) (6) Full Pure Virtue or EA Virtue Mindful Sample Control Mindful Clubs School characteristics School is located in a very violent community 0.55 0.63 0.52 0.43 0.71 0.43 School is located in an urban area 0.72 0.75 0.71 0.57 1.00 0.57 Total enrollment (grades 1-6) 341.00 396.25 319.95 294.14 362.86 302.86 Total enrollment ( grades 7-9) 114.79 142.38 104.29 81.43 86.43 145.00 School has its own building 0.83 0.88 0.81 0.71 0.86 0.86 School is connected to water supply 0.79 0.63 0.86 0.86 0.86 0.86 School has computers 0.76 0.75 0.76 0.86 0.71 0.71 School has a health program for students 0.41 0.38 0.43 0.29 0.57 0.43 School has a food program for students 0.86 1.00 0.81 0.71 1.00 0.71 Observations 29 8 21 7 7 7 Notes: This table shows descriptive statistics of the available variables at baseline for the sample of participant schools. All information was obtained at the school level. Description of these school characteristics is presented in Appendix 1. Data was obtained from the Educational Censuses for El Salvador and Guatemala (2018) and self-collected by the authors for Honduras. Column (2) shows mean characteristics for the pure control. Average characteristics for schools assigned to any of the psychology-based interventions are presented in column (3). Column (4) shows mean characteristics for Clubs. Characteristics for schools in each of the interventions (Virtue or Mindful) are presented in columns (5) and (6), respectively. ”Observations” indicates the number of schools within each group. For more information on balance tests, exact p-values, and adjusted sharpened two-stage q-values, see Table A4 in the Appendix. 39 Table 2: Mean Student Characteristics at Baseline by Treatment Group (1) (2) (3) (4) (5) (6) Full Pure EA Virtue or Virtue Mindful Sample Control Clubs Mindful Individual Characteristics Female 0.52 0.55 0.51 0.51 0.53 0.49 Student’s age 12.23 11.88 12.58 12.37 12.39 12.11 Student is enrolled in the evening shift 0.33 0.32 0.53 0.34 0.13 0.31 Student’s course 6.30 6.01 6.46 6.41 6.52 6.25 Travel time (minutes from home to school) 14.50 14.52 15.22 14.5 16.81 11.45 Student has tried to emigrate to USA 0.03 0.03 0.03 0.02 0.02 0.02 Student works 0.14 0.17 0.15 0.13 0.11 0.13 Household Characteristics Student’s household composition Student lives with both parents 0.64 0.63 0.63 0.65 0.68 0.63 Student lives only with mother 0.26 0.25 0.28 0.26 0.23 0.28 Student lives only with father 0.02 0.03 0.02 0.02 0.02 0.02 Student lives with other relatives 0.05 0.06 0.04 0.05 0.04 0.05 Student lives with unrelated adult 0.02 0.03 0.03 0.02 0.03 0.01 Mother is at home after school 0.57 0.57 0.58 0.56 0.58 0.54 Mother’s education level No education 0.17 0.18 0.16 0.17 0.17 0.19 Elementary or high school 0.74 0.73 0.77 0.75 0.72 0.75 University or postgraduate 0.08 0.08 0.07 0.08 0.11 0.06 Main Outcomes Behavior at School 2.05 2.09 2.06 2.04 1.96 2.08 Student has “Bad Behavior” Report 0.76 0.78 0.70 0.80 0.83 0.78 Math Grades 69.58 70.66 67.90 69.16 69.60 70.24 School Enrollment (in 2020) 0.85 0.80 0.82 0.87 0.91 0.89 Mechanisms SES Index 1.19 1.15 1.31 1.20 1.15 1.12 Perseverance 0.28 0.24 0.33 0.29 0.26 0.27 Impulsiveness 0.86 0.82 0.86 0.86 0.85 0.88 Risk-taking Behavior 5.87 6.25 5.47 5.77 6.01 5.86 Arousal 15.26 16.71 12.66 14.87 14.90 17.60 Emotion Regulation (Valence) 59.73 59.77 57.61 59.72 59.47 62.60 Fluid Intelligence 0.61 0.59 0.63 0.61 0.61 0.59 Observations 1,975 556 542 1,419 426 451 Notes: This table shows average student characteristics and outcomes at baseline for the sample of participants. Column (1) presents mean characteristics for the full sample. Columns (2) and (3) show mean characteristics for the pure-control and Clubs groups. Average characteristics for participants assigned to any of the psychology-based interventions are presented in column (4) and for each of the interventions, Virtue or Mindful, in columns (5) and (6), respectively. A description of these characteristics is presented in Appendix 1. All data on individual and family characteristics and outcomes were collected by the authors during enrollment phase and baseline data collection using the different instru- ments as described in table A2 in the Appendix. “Observations” indicates the number of students within each group. For more information on balance tests, exact p-values, and adjusted sharpened two-stage q-values, see Table A5 in the Appendix. 40 Table 3: Effects of ASP Social-Emotional Learning and Protection Components on Behavior and Academic Outcomes (Intention-to-Treat Estimates) Behavior Math School at School Grades Enrollment (1) (2) (3) Panel A: Social-Emotional Learning Component Psychology-based Intervention 0.464** 0.0587 0.0305 (0.220) (0.116) (0.044) [0.040] [0.615] [0.600] Mean Clubs Group -0.007 0.031 0.818 Observations 1334 1332 1141 R-squared 0.043 0.207 0.314 Panel B: Protection Component Clubs 0.162 -0.1345 -0.0147 (0.295) (0.145) (0.050) [0.555] [0.395] [0.900] Mean Pure-Control Group -0.013 0.044 0.794 Observations 1009 1009 823 R-squared 0.031 0.370 0.588 Notes: This table shows the estimated impacts of the ASP’s social-emotional learning and pro- tection components on behavioral and academic outcomes. Panel A presents the net impacts of the social-emotional learning obtained from the experimental variation. “Psychology-based intervention” is a dummy equal to 1 if the student was enrolled in a school that was randomly assigned to the Virtue or Mindful interventions, and 0 if assigned to the Clubs group. “Mean Club Group” is the mean of the outcome for the group that only participates in Club, without any extra curricular activity. Panel B shows the net protection effects obtained from the non- experimental variation. “Clubs” is a dummy equal to 1 if the student was enrolled in a school that was randomly assigned to the extracurricular activities, and 0 if enrolled in a school se- lected for the pure-control group using propensity score matching. “Mean Pure-Control Group” is the mean of the outcome for the pure-control group. We present the estimated coefficients µˆ ˆ 2 and θ2 from specifications 2 and 3 in panels A and B, respectively. The description of de- pendent variables is available in Appendix 1. Behavior at school and math grades are mea- sured in standard deviations relative to Clubs (or Pure-Control) group. ”School enrollment” is a dummy variable that equals 1 if a student was enrolled in any school in our study in the 2020 academic year. All outcomes were obtained from administrative data sources (i.e., teach- ers’ reports). Sample size in each specification varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is im- balance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. Wild bootstrap standard errors are shown in parentheses and adjusted p-values are in brackets. ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1 41 Table 4: Effects of ASP Social-emotional Learning and Protection Components on SES and Emotion Regulation (Intention-to-Treat Estimates) SES Arousal Valence Fluid Index Intelligence (1) (2) (3) (4) Panel A: Social-Emotional Learning Component Psychology-based Intervention 0.113 -0.445*** -0.0126 0.0379 (0.0841) (0.144) (0.125) (0.107) [0.195] [0.015] [0.930] [0.735] Mean Clubs Group -0.006 0.005 -0.008 -0.005 Observations 1114 575 575 575 R-squared 0.278 0.154 0.169 0.103 Panel B: Protection Component Clubs 0.0861 0.397*** -0.191 -0.0826 (0.0971) (0.143) (0.181) (0.137) [0.440] [0.010] [0.325] [0.645] Mean Pure-Control Group -0.002 0.050 0.056 0.013 Observations 860 402 402 402 R-squared 0.376 0.087 0.156 0.118 Data Source SoftGames Reactiva Reactiva Reactiva Notes: This table shows the estimated impacts of the ASP’s social-emotional learning and protection components on SES and emotion regulation outcomes. Panel A presents the net impacts of the social- emotional learning obtained from the experimental variation. “Psychology-based intervention” is a dummy equal to 1 if the student was enrolled in a school that was randomly assigned to the Virtue or Mindful interventions, and 0 if assigned to Clubs. “Mean Club Group” is the mean of the outcome for the group that only participates in Clubs, without any extra curricular activity. Panel B shows the net protection effects obtained from the non-experimental variation. “Clubs” is a dummy equal to 1 if the student was enrolled in a school that was randomly assigned to the extracurricular activities, and 0 if was enrolled in a school selected for the pure-control group using propensity score matching. “Mean Pure-Control group” is the mean of the outcome for the pure-control group. We present the estimated coefficients µ ˆ ˆ 2 and θ2 from specifications 2 and 3 in panels A and B, respectively. The description of dependent variables is available in Appendix 1. All dependent variables are measured in standard deviations relative to the comparison group. All outcomes were obtained from administrative data sources (i.e., teachers’ reports). Sample size in each specification varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is imbalance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. Wild bootstrap standard errors shown in parentheses and adjusted p-values in brackets. ∗∗∗ p < 0.01, ∗∗ p < 0.05 ∗ p < 0.1 42 Table 5: Which Curriculum is the Most Efficient? (Intention-to-Treat Estimates) Estimated Standard P-value P-value Mean Observations P-value Coefficient Error Wild Bootstrap Clubs Group Virtue=Mindful (1) (2) (3) (4) (5) (6) (7) Panel A: Behavior at School and Academic Performance Behavior at School Virtue 0.549** (0.226) [0.017] {0.025} -0.007 1334 0.312 Mindful 0.386 (0.238) [0.108] {0.120} Math Grades Score Virtue 0.00413 (0.129) [0.975] {0.955} 0.031 1332 0.405 Mindful 0.109 (0.132) [0.411] {0.440} School Enrollment Virtue 0.0447 (0.0427) [0.298] {0.430} 0.818 1141 0.553 Mindful 0.0150 (0.0563) [0.790] {0.785} Panel B: SES and Emotion Regulation SES Index Virtue 0.134 (0.105) [0.204] {0.280} -0.006 1114 0.679 Mindful 0.0931 (0.0888) [0.297] {0.295} Short-term Perseverance Virtue 0.201** (0.0789) [0.012] {0.015} 0.001 1117 0.734 43 Mindful 0.117 (0.0850) [0.171] {0.150} Impulsiveness Virtue 0.0582 (0.106) [0.583] {0.635} 0.008 1131 0.734 Mindful 0.0189 (0.112) [0.866] {0.880} Risk-taking Behaviour Virtue -0.118 (0.0961) [0.222] {0.320} 0.009 1133 0.718 Mindful -0.0829 (0.0918) [0.369] {0.395} Arousal Virtue -0.517*** (0.192) [0.009] {0.015} 0.005 575 0.371 Mindful -0.352** (0.155) [0.026] {0.040} Valence Virtue 0.0671 (0.137) [0.627] {0.655} -0.008 575 0.290 Mindful -0.116 (0.151) [0.444] {0.440} Fluid Intelligence Virtue -0.0334 (0.111) [0.765] {0.680} -0.005 575 0.092 Mindful 0.130 (0.124) [0.295] {0.290} Notes: This table shows the estimated impacts of each type of psychology-based curriculum from the social-emotional learning component. Panel A presents the estimated effects on behavior and academic performance outcomes. Panel B shows impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Virtue and Mindful are dummies equal to 1 if the student was enrolled in a school that was randomly assigned to the Virtue or Mindful interventions, respectively, and 0 if assigned to Clubs. In column (1), we present the estimated coefficients of the interaction between the treatment dummies and the “Post” indicator. Clustered standard errors, unadjusted p-values, and p-values adjusted using a wild bootstrap procedure (adjusted for small number of clusters) are shown in columns (2), (3), and (4), respectively. “Mean Clubs Group” in column (5) is the mean of the outcome for the group that only participates in Clubs, without any extra curricular activity. Sample size in each specification presented in column (6) varies according to the number of observations available for each outcome. Column (7) present the p-value for the test for differences between estimated coefficients for Virtue and Mindful presented in column (1). Estimations include all individual controls for which there is imbalance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1 Table 6: Heterogeneous Effects in the ASP’s Social-emotional Learning by Student Behavior at Baseline Effect on Well Effect on Diff. Badly vs. Mean Clubs Observations Behaved Badly Behaved Well Behaved Group (2)-(1) (1) (2) (3) (4) (5) Panel A: Behavior at School and Academic Performance Behavior at School 0.429*** 0.039 -0.389* -0.007 1334 (0.119) (0.163) (0.200) Math Grades -0.032 0.103 0.135 0.031 1332 (0.133) (0.129) (0.138) School Enrollment 0.138* -0.012 -0.150** 0.821 1133 (0.076) (0.040) (0.069) Panel B: SES and Emotion Regulation SES Index -0.019 0.148* 0.167 -0.006 1110 (0.154) (0.086) (0.158) Arousal -0.525** -0.394** 0.130 0.005 574 (0.232) (0.155) (0.259) Valence 0.069 -0.026 -0.096 -0.008 574 (0.177) (0.140) (0.188) Fluid Intelligence 0.182 0.002 -0.180 -0.005 574 (0.156) (0.120) (0.158) Notes: This table shows the heterogeneous impacts of the social-emotional learning component by student behavior at baseline. Panel A presents the estimated effects on behavior and academic performance outcomes. Panel B shows impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Columns (1) and (2) present the total effect of the social-emotional learning component on well or badly behaved children, respectively, relative to similar students in Clubs. Column (3) shows the difference between badly and well behaved treated students. “Mean Clubs Group” in column (4) is the mean of the outcome for the group that only participates in Clubs. The sample size of each specification presented in column (5) varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is imbalance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. P-values adjusted using a wild bootstrap procedure (adjusted for small number of clusters) are presented using the following notation: ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1. 44 Table 7: Heterogeneous Effects of Type of Curriculum by Student Behavior at Baseline Virtue Mindful Effect on Well Effect on Diff. Badly Beh.. Effect on Well Effect on Diff. Badly Beh. vs. Mean Clubs Obs. P-value Behaved Badly Behaved vs. Well Behav. Behaved Badly Behaved Well Behav. Group Virtue = (2)-(1) (5)-(4) Mindful (1) (2) (3) (4) (5) (6) (7) (8) (9) Panel A: Behavior at School and Academic Performance Behavior at School 0.236 0.108 -0.129 0.566*** -0.027 -0.593*** -0.007 1334 0.024 (0.152) (0.192) (0.273) (0.112) (0.164) (0.169) Math Grades -0.060 0.042 0.102 -0.012 0.162 0.173 0.025 1332 0.777 (0.137) (0.144) (0.146) (0.171) (0.146) (0.177) School Enrollment 0.170** -0.003 -0.172** 0.112 -0.023 -0.134* 0.821 1133 0.242 (0.079) (0.036) (0.072) (0.079) (0.058) (0.076) Panel B: SES and Emotion Regulation 45 SES Index -0.171 0.206+ 0.377** 0.109 0.091 -0.018 -0.004 1110 0.132 (0.195) (0.108) (0.189) (0.168) (0.092) (0.178) Arousal -0.387 -0.537** -0.150 -0.750*** -0.221 0.528* 0.006 574 0.131 (0.251) (0.232) (0.349) (0.254) (0.158) (0.251) Valence 0.085 0.061 -0.024 0.044 -0.133 -0.177 -0.009 574 0.898 (0.195) (0.152) (0.198) (0.290) (0.166) (0.303) Fluid Intelligence 0.049 -0.054 -0.103 0.398** 0.070 -0.328* -0.005 574 0.085 (0.182) (0.125) (0.191) (0.177) (0.138) (0.192) Notes: This table shows the heterogeneous impacts of the social-emotional learning component by student behavior at baseline, separated by type of curriculum. Panel A presents the estimated effects on behavior and academic performance outcomes. Panel B shows impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Columns (1) through (3) show the heterogeneity analysis using the sample of students enrolled in schools randomly assigned to Virtue relative to those assigned to Clubs. Columns (4) through (6) show same analysis but compare students in schools assigned to Mindful to students in Clubs. Columns (1) and (2) as well as (4) and (6) present the total effect of the social-emotional learning component on well or badly behaved children, respectively, relative to similar students in Clubs for each type of curriculum. Columns (3) and (6) shows the difference between badly behaved and well behaved treated students for each type of curriculum. “Mean Clubs Group” in column (7) is the mean of the outcome for the group that only participates in Clubs. The sample size of each specification presented in column (8) varies according to the number of observations available for each outcome. Column (9) shows the p-value for the test of equality between estimated coefficients in columns (3) and (6). Estimations include all individual controls for which there is imbalance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. P-values adjusted using a wild bootstrap procedure (adjusted for small number of clusters) are presented using the following notation: ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1, + p < 0.115. Table 8: Differences in Learning Channel (Clubs vs. Pure-Control Group) Effect on Well Effect on Diff. Badly vs. Mean Clubs Observations Behaved Badly Behaved Well Behav. Group (2)-(1) (1) (2) (3) (4) (5) Panel A: Behavior at School and Academic Performance Behavior at School 0.065 0.642*** 0.577* -0.013 1009 (0.202) (0.227) (0.305) Math Grades -0.058 -0.175 -0.117 0.044 1009 (0.167) (0.164) (0.183) School Enrollment 0.012 -0.005 -0.0173 0.794 820 (0.069) (0.048) (0.069) Panel B: SES and Emotion Regulation SES Index 0.094 0.084 -0.010 -0.002 860 (0.223) (0.097) (0.231) Arousal 0.459 0.338** -0.120 0.050 402 (0.293) (0.157) (0.327) Valence -0.507 -0.152 0.355 0.056 402 (0.343) (0.192) (0.339) Fluid Intelligence -0.085 -0.080 0.005 0.013 402 (0.182) (0.160) (0.230) Notes: This table shows the heterogeneous impacts of the protection component by student behavior at baseline. Panel A presents the estimated effects on behavior and academic performance outcomes. Panel B shows impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Columns (1) and (2) present the total effect of the protection component on well and badly behaved children, respectively, relative to similar students in Clubs. Column (3) shows the difference between badly and well behaved treated students. “Mean Clubs Group” in column (4) is the mean of the outcome of the group that only participates in Clubs. The sample size of each specification presented in column (5) varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is imbalance at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. P-values adjusted using a wild bootstrap procedure (adjusted for small number of clusters) are presented using the following notation: ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1. 46 Table 9: Comparison of Maternal and ASP Protection Effects on Effects on Diff. Not Protected Mean Protected Not Protected at vs. Protected Pure-Control Observations at Home at Home (2) - (1) Group (1) (2) (3) (4) (5) Panel A: Behavior at School and Academic Performance Behavior at School -0.248 0.448 0.696*** -0.013 1009 (0.299) (0.334) (0.246) Math Grades -0.322** 0.003 0.325** 0.044 1009 (0.123) (0.176) (0.116) School Enrollment -0.038 0.005 0.043 0.794 823 (0.055) 0.057 (0.048) Panel B: SES and Emotion Regulation SES Index 0.0617 0.102 0.0407 -0.002 860 (0.130) (0.118) (0.154) Arousal 0.324* 0.447** 0.123 0.050 402 (0.175) (0.175) (0.205) Valence -0.364* -0.071 0.294 0.056 402 (0.201) (0.251) (0.289) Fluid Intelligence 0.0512 -0.179 -0.230 0.013 402 (0.170) (0.176) (0.216) Notes: This table shows the heterogeneous impacts of the protection learning component by protection of children at home before the intervention. A student is protected if the mother is at home when the student returns from school. Panel A presents the estimated heterogeneous effects on behavior and academic performance outcomes. Panel B shows the heterogeneous impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Columns (1) presents the total effect of the ASP’s protection component on children whose mothers are at home relative to other protected students in the pure-control group. Column (2) shows the total effect of the ASP’s protection component on children who are “not protected” at home relative to other “not protected” children assigned to the pure-control group. Column (3) shows the difference between not protected and protected treated students. “Mean Pure-Control Group” in column (4) is the mean of the outcome for students in the pure-comparison group. Sample size in each specification presented in column (5) varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is imbalance in the non-experimental variation at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. P-values adjusted using a wild bootstrap procedure (adjusted for small number of clusters) are presented using the following notation: ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1. 47 Figure 1: Experimental Design 29 schools in Guatemala, Honduras and El Salvador Non-experimental Experimental variation variation Selection of 8 Random allocation of 21 schools schools (PSM) 7 schools 7 schools 7 schools 8 schools to EA to CS to MF to C This figure summarizes the experimental design that the authors developed to address this project’s main research questions. 48 Appendix For Online Publication Only Appendix 1 Outcome Descriptions A. Household Characteristics 1. Student lives with both parents: Dummy variable equals 1 if the student lives both mother and father or otherwise equals 0. 2. Student lives only with mother: Dummy equals 1 if the student lives only with mother or otherwise equals 0. 3. Student lives only with father: Dummy variable equals 1 if the student lives only with father or otherwise equals 0. 4. Student lives with other relatives: Dummy variable equals 1 if the student lives with a rela- tive (not with father or mother) or otherwise equals 0. 5. Student lives with unrelated adult: Dummy variable equals 1 if the student lives with an unrelated adult or otherwise equals 0. 6. No education: Dummy variable equals 1 if the student’s mother is not educated at all or otherwise equals 0. 7. Basic or intermediate education: Dummy variable that equals 1 if the student’s mother com- pleted elementary or high school or otherwise equals 0. 8. University or higher: Dummy variable equals 1 if the student’s mother completed univer- sity education or a postgraduate program or otherwise equals 0. B. Students Characteristics 1. Female: Dummy variable equals 1 if the student is female or otherwise equals 0. 2. Student’s age: A numeric variable that indicates the age of the student. 3. Student is enrolled in the evening shift: Dummy variable equals 1 if the student is enrolled in the evening shift or otherwise equals 0. Due to limited infrastructure and to serve most of the demand, schools in El Salvador operate in two shifts, one in the morning and another in the afternoon. 4. Student’s course: A numeric variable that indicates the course the student is enrolled in. 5. Travel time (home to school): A numeric variable that indicates the how long (minutes) it takes the student to walk to school. 6. Student has tried to emigrate to the USA: Dummy variable equals 1 if the student has tried to emigrate to the USA or otherwise equals 0. 7. Student works: Dummy variable equals 1 if the student works or otherwise equals 0. 49 C. Behavior and Academic Outcomes 1. Behavior at school: Reported by teachers and presented according to the following scale: Excellent (E), Very Good (VG), Good (G). This scale is continuous and comparable to American course grades. 2. Math grades: Variable that indicates performance in math. The grade is determined on a 0-100 scale, where 0 is the worst performance and 100 is the best. We have standardized these values from control groups at the grade level. 3. School dropout: Dummy variable that equals 1 if student dropped out in 2019 or otherwise equals 0. 4. School enrollment (in 2020): Dummy variable that equals 1 if the student is enrolled in 2020 or otherwise equals 0. 5. Fluid intelligence: Percentage of correct answers given after both negative and positive stimuli. Standardized relative to the comparison group. D. Mechanisms 1. SES Index: A standardized index of short-term perseverance, impulsiveness, and risk- taking behavior using inverse covariance weighting. 2. Short-term Perseverance: Dummy variable for the difficult game in round one. 3. Impulsiveness: Number of times the student responded correctly to the Go-NoGo stimu- lus. Standardized relative to the comparison group. 4. Risk-taking behavior: Average number of pumps only for the balloons that did not pop. Standardized relative to the comparison group. 5. Arousal: Average maximum arousal score using Reactiva. Standardized relative to the comparison group. 6. Emotion regulation (valence): Average maximum valence score using Reactiva. Standard- ized relative to the comparison group. 50 Appendix 2 Measuring Emotions Using an Artificial Intelli- gence Algorithm Why use computer vision to proxy emotional reactions? Many studies have shown that emotions play a relevant role in economics. For example, they affect decision-making, investment behaviors, and human capital accumulation, among others things (Weber and Johnson, 2009). The main objective of Reactiva is to measure emo- tional reactions to different emotion-laden stimuli based on artificial intelligence (AI)-powered computer vision in a context of social program evaluations taking place in the field. This tool was developed due to the need to proxy socioemotional dimensions of human capital in a more objective way than through self-reported tests (such as the Grit Test, Locus of Control Test, etc.). In particular, through this tool, we are able to estimate both positive and negative emotional reactions to stimuli based on videos from the GAPED database in the arousal and valence dimensions. We hypothesize that programs that aim to improve socio-emotional skills (SES) such as cognitive behavioral therapy would tend to reduce the valence and arousal of emotional reactions to negative stimuli, while maintaining the reaction to positive incitations. Moreover, we expect that the effect of negative stimuli will decrease. In order to test our hypothesis, we added raven-like matrices after both negative and positive stimuli to observe whether this impacted the subject’s capacity to provide the correct responses to these questions, which are a proxy for fluid intelligence. In other words, we expect to observe a decrease in the performance in these raven-like tests after subjects observe a video that incites negative emotion. Moreover, we expect that the effect of the stimuli (i.e. the negative video) should de- crease after the students attend the program, thereby demonstrating that the students are less sensitive to this kind of shock after the intervention. We argue that this decreased sensitivity could be interpreted as an increase in resilience or the capacity to regulate emotion. Reactiva Reactiva is a smartphone application that we created to collects data and conducts sim- ple experiments in the field. The name of the platform is a word play on the algorithm of the Re- activa application, which was developed by Affectiva and a team of researchers in MIT Media 51 Lab’s Affective Computing Group (founded by Rosalind Picard) where one of our co-authors underwent his postdoctoral training. Affectiva is company that spun-off from this research group that commercializes market-oriented versions of its AI-powered algorithms. Operating since 2009, Affectiva’s main focus is “to teach computers to understand human emotions.” The advertising sector uses the company’s solutions primarily to study individuals’ emotional re- actions to video advertisements by using a webcam and AI-powered algorithms to recognize and categorize emotions. Based on such observations, this sector can evaluate responses to the same ad among different demographic groups. The company offers its algorithms free of charge for research purposes. The origins of the use of facial expressions and affective computing to proxy emotions In 1872, Charles Darwin published a notable work in which he argued that most emo- tions are universal insofar as the human faces expresses them in the same way across races and cultures. In his study of human behavior, he explicitly states that universal facial expressions provide information about a person’s cognitive states. These states include boredom, stress, confusion, and others. After more than a century since Darwin conducted his own studies, the field of affec- tive computing (also called artificial emotional intelligence, or emotion AI) arose. While this field’s core ideas may be traced as far back as Darwin’s work and even early philosophical inquiries into emotion, the more modern branch of computer science originated with Rosalind Picard’s 1995 paper (Picard, 1995) on affective computing and her book Affective Computing published by MIT Press (Picard, 1997). Affective computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects. It is an interdisciplinary field spanning computer science, psychology, and cognitive science. The science behind affective computing Determining or recognizing emotions takes place in three stages: face detection, facial feature extraction, and expression classification. By using webcams as well as the front cam- eras of smartphones and tablets computer vision algorithms identify key landmarks on the face (e.g., corners of the eyebrows, tip of the nose, corners of the mouth, etc.). Machine learning algorithms (classifiers) then analyze pixels in those regions to classify facial expressions. Affec- tiva uses the Facial Action Coding System (FACS), which is the most common coding system cited in the literature, to classify facial expressions or Action Units (AUs). Combinations of 52 these facial expressions are then mapped to emotions. We were able to use Affectiva’s software development kit (SDK) for our research pur- poses. The SDK is built on Affectiva’s industry-leading patented science. The highly accurate classifiers of this kit have been trained and tested using Affectiva’s extensive emotion data repository—the world’s largest emotion database that includes analyses of more than 6.5 mil- lion faces from 87 countries. Affectiva identifies 7 emotions, 20 expressions, and 13 emojis (not considered here), and it includes classifiers for age and gender. We chose to to take advantage of Affectiva’s existing SDK because it includes all three phases of the emotion recognition process, works quickly, and is stable in real time. By using the classic web camera, the kit allowed us to detect facial land- marks on an image automatically. We were also able to take advantage of the SDK’s geometric feature-based approach for feature extraction. An attractive element SDK is its ability to measure the distance between landmarks on the face and to select the optimal set of features. The proposed system uses a neural network algorithm for classification, and it recognizes 7 facial expressions: anger, disgust, fear, happi- ness, sadness, surprise, and neutral. The metric values indicate the likelihood and the degree to which a particular emotion is being expressed. Therefore, an “intense” smile will produce a much higher measure than a subtle one. For our purposes, we looked at the max value for each metric. It is important to note that, on an individual level,the program is more likely to misread an input (e.g., respondent scratches his face, resulting in a false positive); thus, we ensure that the subject is observed long-enough for the software to capture a sufficiently intense response and interpret its corresponding emotion. How do we map facial expressions to emotions? The emotion predictors observe facial expressions as inputs, which are then used to cal- culate the likelihood of an emotion. Our facial expression to emotion mapping builds on EM- FACS mappings developed by Ekman and Friesen (1978). The facial expressions indicate to greater or lesser degrees the likelihood of the corresponding emotion. The following table shows the relationship between facial expressions and emotion predictors. 53 Table 10: Facial Expressions and Emotions Predictors Emotion Greater likelihood Lesser likelihood Brow Raised Joy Smiled Brow Furrowed Brow Furrowed Inner Brow Raised Lid Tightened Brow Raised Eye Widen Smiled Anger Chin Raised Mouth Opened Lip Sucked Nose Wrinkled Lip Sucked Disgust Upper Lip Raise Smiled Inner Brow Raised Brow Raised Surprise Brow Furrowed Eye Widen Jaw Dropped Inner Brow Raised Brow Raise Brow Furrowed Lip Corner Depressor Fear Eye Widen Jaw Dropped Lip Stretched Smiled Inner Brow Raised Brow Raised Brow Furrowed Eye Widen Lip Corner Depressor Lip Pressed Sadness Mouth Opened Lip Sucked Smiled Brow Furrowed Contempt Smiled Smirked 54 Furthermore, the SDK measures valence and engagement as alternative metrics for emo- tional experience. The kit measures engagement as facial muscle activation that illustrates the subject’s expressiveness, ranging in value from 0 to 100. More specifically, engagement or ex- pressiveness is a weighted sum of the following facial expressions: • Brows raised • Lips puckered • Brows furrowed • Lips pressed • Nose wrinkled • Mouth open • Lip corner(s) depressed • Lips sucked-in • Chin raised • Smile In addition, valence measures the degree to which an experience and emotion are either positive or negative on a spectrum, ranging in value from -100 to 100. Table 11 provides details regarding how valence is measured. Table 11: The Relationship Between Facial Expressions and Their Valence Measures Increase positive likelihood Increase negative likelihood Smile Inner Brow Raise Cheek Raise Brow Furrow Nose Wrinkle Upper Lip Raise Lip Corner Depressor Chin Raise Lip Press Lip Suck Using the metrics Emotion and expression metric scores indicate when users show a specific emotion or expression (e.g., a smile) along with the degree of confidence. See Figure 2 as a brief reference. The metrics can be thought of as detectors: as the emotion or facial expression occurs and in- tensifies, the score rises from 0 (no expression) to 100 (expression fully present). We also expose a composite emotional metric called valence, which provides feedback on overall experience. Valence values from 0 to 100 indicate a neutral to positive experience, while values from -100 to 0 signify a negative to neutral experience. 55 Figure 2: Facial Expressions and Detectors 56 Appendix 3 Cost-effectiveness Analysis We conduct the cost-effectiveness analysis of the ASP according to previously documented ap- proaches of Holla et al. (2021); Ganimian et al. (2021); Galasso and Wagstaff (2019) and the framework of Hendren and Sprung-Keyser (2020) by estimating the relationship between hu- man capital interventions and impacts on adult earnings on the marginal value of investments using public funds. More specifically, we estimate the intervention’s cost-benefit by consid- ering its indirect impact on reducing school dropout by improving behavior at school. The complete analysis is summarized in table A10 in the Appendix. We first use the estimated treatment effects of the intervention on bad behavior in the short run. We calculate the indirect effect of the ASP on dropout by comparing bad behavior and school dropout, average dropout rate from the pure-control group, and the estimated im- pact of the intervention on bad behavior. Our approximation of this indirect effect is a reduction on dropout by 1 percentage point, which is equivalent to 20 students who stay and complete at more than elementary school rather than dropping out of school (See panel A in table A10). Data on implementation costs by type of intervention were obtained from the NGO’s adminis- trative records. On average, the ASP costs US$296.5 per participant, ranging between US$269 for Clubs to US$327 for Virtue. See more details in panel B in table A10. To estimate the ASP’s benefits, we discounted the cost of the current interventions based on projected increases in income that participants will experience in their future wages because they did not drop out of school. Expected annual wages by education level completed were obtained from El Salvador’s 2018 Household Survey (Encuesta de Hogares y Propositos Multiples, EHPM).52 Using these wages, we estimate the net present value (NPV) of the potential earnings of an individual who has completed only elementary school, high-school, technical school, or college. For the NPV estimation, we assume that the individuals will work until the age of 55 years and discount the earning inflows at a rate of 5% with no increase in salary over time.53 As we show in panel C in table A10, estimated NPV ranges from US$35,000 for individuals who complete elementary school only and up to US$110,000 for those who graduate from college. Relative to the NPV of those who drop out of elementary school, we estimate that those who 52 The average salaries across El Salvador, Honduras, and Guatemala are very similar, so using the data from one country is not grossly different from using another country’s data. 53 Highly skilled individuals’ salaries usually increase at a higher rate than the salaries of less skilled workers. In this sense, the lack of increase in salary assumption will underestimate the differences in NPV between those who have completed high school or higher and those who have completed only elementary school. 57 complete at least high school make approximately US$19,000 more in NPV, and those who finish college can make up to US$75,000 more in NPV than those who drop out of elementary school. Considering that these interventions can reduce the total number of dropouts by 20 stu- dents, we estimate the net total of what students would earn if they do not drop out of school and complete their higher education. Then, assuming that these students could pay at least 1% of their income tax to fund this ASP,54 and using the total program cost per participant, we estimate that the ASP yields a benefit-cost ratio between 12.5 for students who complete up to high school and 50.2 for those who finish college. In this sense, investing in this program and implementing it as public policy is worthwhile because the intervention is likely to pay for itself even in the short run and has the potential to generate large additional welfare gains over time. 54 The average income tax rate in El Salvador, Guatemala, and Honduras is 10%. 58 Appendix Tables and Figures Table A1: Structure of the Virtue ASP Strength Activity Type Goals Duration Introduction to Wellness 1 Formative Introduce to the participants the theme and methodology that they will be working 60 min and Character Strengths on throughout the academic year. 2 Formative Explain what creativity is and why it is important? 60 min 3 Reflective Come up with a tool to help develop creativity. 15-30 min Creativity 4 Reflective Come up with a tool to help develop creativity. 15-30 min 5 Reflective Assess how creativity can be put into practice in the club. 15-30 min 6 Formative What is perspective and why is it important? 60 min 7 Reflective Come up with a tool to help develop perspective. 15-30 min Perspective 8 Reflective Come up with a tool to help develop perspective. 15-30 min 9 Reflective Come up with a tool to help develop perspective. 15-30 min 10 Formative What is courage and why is it important? 60 min 11 Reflective Come up with a tool to help develop courage. 15-30 min Courage 12 Reflective Come up with a tool to help develop courage. 15-30 min 13 Reflective Come up with a tool to help develop courage. 15-30 min Reflection: Am I putting 14 Formative Reflect on the importance of practicing character strengths every day. 60 min my character strengths into practice? 15 Formative What is perseverance and why is it important? 60 min 16 Reflective Set a goal toward which the student will have to work for one month. 15-30 min Perseverance 17 Reflective Come up with a tool to help develop perseverance. Monitor progress. 15-30 min 18 Reflective Come up with a tool to help develop perseverance. Monitor progress. 15-30 min 19 Formative Set life goals. Identify what skills students have learned in the club that will help 60 min them meet their goals. 20 Formative What is self-control and why is it important? 60 min Self- 21 Reflective Come up with a tool to help develop self-control. 15-30 min control 22 Reflective Come up with a tool to help develop self-control. 15-30 min 23 Reflective Come up with a tool to help develop self-control. 15-30 min 24 Formative What is social intelligence and why is it important? 60 min Social 25 Reflective Come up with a tool to help develop social intelligence. 15-30 min Intelligence 26 Reflective Come up with a tool to help develop social intelligence. 15-30 min 27 Reflective Reflecting on past events and ask: How empathetic have I been? 15-30 min 28 Formative What is hope and why is it important? 60 min 29 Reflective Come up with a tool to help develop hope. 15-30 min Hope 30 Reflective Come up with a tool to help develop hope. 15-30 min 31 Reflective Come up with a tool to help develop hope. 15-30 min Closure 32 Formative Reflect deeply on what one has learned and achieved. Plan the next (personal) steps 60 min students will take to continue building their character. 59 Table A2: Summary of Outcomes, Data Sources, and Instruments or Tasks Source Game or Instrument Type of Outcome (1) (2) (3) Panel A. Outcomes Behavior at School Administrative Records Teacher’s Report Main Math Grades Administrative Records Teacher’s Report Main School Enrollment Administrative Records Teacher’s Report Main Short-term Persistence SoftGames App Additions Game Mechanism Self-control SoftGames App Go-NoGo Task Mechanism Risk-taking Behavior SoftGames App Bartik Analog Risk Task Mechanism (Balloons Game) Perseverance Self-reports GRIT Instrument Mechanism Fluid Intelligence Reactiva App Raven Matrices Mechanism Arousal Reactiva App AI-based Emotion Detection Mechanism Valence Reactiva App AI-based Emotion Detection Mechanism 60 Table A3: Registration Form Question Response Options Full name Male Gender Female How old are you? In what municipality were you born? What is your current home address? Please include municipality and com- munity. With whom do you live? With both of my parents Only with my mother Only with my father With my mother and my stepfather With my father and my stepmother With my grandfather and/or grandmother With an aunt or uncle With another person known to my parents Other(s) Have you traveled to and from the United States? Yes No Do you work during your free time? For example, with your family in agri- Yes culture, as a store clerk, or on the street, etc. No What do you do for work? How many minutes does it usually take you to walk to school? What is the name of your responsible? Please write the full name. What is your responsible’s phone number? How are you related to your responsible? Mother Father Uncle/aunt Grandfather/grandmother Stepmother Stepfather Other What is your mother’s education level? If your responsible is someone other than your mother, what is your respon- sible’s education level? What does your responsible do to earn income? Has a steady job (works every day) Has his own business Works only sometimes. It depends on whether he gets work Would like to work but has not been able to find a job Is permanently disabled/chronically ill Is retired Other If your responsible has a stable job, what is his or her occupation? If your responsible has his own business, what is is occupation? If your responsible works a few times per week, what does he usually do for work? If your responsible has a stable job or his own business, how many days or Full day, every day per week hours does he work per week? Half a day, every day per week Whenever he gets work Which adult relative is at your house most often when you arrive home from Mother school? Father Uncle/aunt 61 Table A4: P-values of Differences Between Treatment and Control Groups School Characteristics Clubs vs. Clubs vs. Virtue vs. Clubs vs. Virtue Mindful Mindful Pure-Control School characteristics School is located in a very violent community 0.32 1.00 0.32 0.48 School is located in an urban area 0.06 1.00 0.06 0.50 Total enrollment (grades 1-6) 0.57 0.95 0.63 0.41 Total enrollment (grades 7-9) 0.93 0.43 0.42 0.44 School has its own building 0.55 0.55 1.00 0.47 School is connected to a water supply 1.00 1.00 1.00 0.35 School has computers 0.55 0.55 1.00 0.63 School has a health program for students 0.32 0.61 0.63 0.74 School has a food program for students 0.15 1.00 0.15 0.12 Notes: This table shows unadjusted p-values of balance tests for all available variables at baseline of table 1. 62 Table A5: Sharpened Two-stage Q-values and P-values of Differences Between Treatment and Control Groups at Baseline Individual Characteristics (1) (2) (3) (4) (5) (6) (7) (8) Experimental Variation Non-experimental Variation Sharpened Two- stage Q-values P-values Sharpened Two-stage P-values Q-values Clubs vs. Clubs vs. Virtue vs. Clubs vs. Clubs vs. Virtue Clubs vs. Clubs vs. Virtue Mindful Mindful Virtue Mindful Mindful Pure-Control Pure-Control Individual Characteristics Female 0.47 0.92 0.70 0.65 0.95 0.79 0.18 0.40 Student’s age 0.72 0.09 0.41 0.79 0.27 0.65 0.34 0.62 Student is enrolled in the evening shift 0.00 0.02 0.14 0.00 0.08 0.38 0.00 0.00 Student’s grade level 0.79 0.15 0.49 0.85 0.39 0.67 0.31 0.57 Travel time (minutes from home to school) 0.43 0.00 0.00 0.65 0.00 0.00 0.11 0.31 Student has tried to emigrate to the USA 0.40 0.61 0.76 0.65 0.72 0.84 0.77 0.84 Student works 0.05 0.19 0.84 0.17 0.40 0.91 0.47 0.65 Household Characteristics Student’s household composition Student lives with both parents 0.10 0.31 0.57 0.29 0.57 0.68 0.04 0.14 63 Student lives only with mother 0.07 0.31 0.17 0.23 0.57 0.40 0.02 0.08 Student lives only with father 0.69 0.50 0.63 0.79 0.68 0.73 0.92 0.95 Student lives with other relatives 0.99 0.87 0.87 1.00 0.92 0.92 0.78 0.85 Student lives with unrelated adult 0.72 0.09 0.02 0.79 0.27 0.08 0.81 0.87 Mother’s education No education 0.91 0.28 0.86 0.95 0.56 0.92 0.16 0.40 Elementary or secondary 0.09 0.68 0.17 0.27 0.79 0.40 0.00 0.00 University or post-graduate 0.09 0.20 0.07 0.27 0.40 0.23 0.11 0.31 Main Outcomes Behavior at School 0.40 0.39 0.16 0.65 0.65 0.40 0.01 0.05 Math Grades 0.55 0.15 0.37 0.68 0.39 0.64 0.46 0.65 School Enrollment (in 2020) 0.25 0.54 0.52 0.50 0.68 0.68 0.00 0.00 Mechanisms Fluid Intelligence 0.52 0.00 0.42 0.68 0.00 0.65 0.46 0.65 SES Index 0.00 0.00 0.77 0.00 0.00 0.84 0.96 1.00 Perseverance 0.04 0.19 0.64 0.14 0.40 0.73 0.02 0.08 Impulsiveness 0.62 0.30 0.01 1.00 0.57 0.05 0.00 0.00 Risk-taking Behavior 0.00 0.19 0.17 0.00 0.40 0.40 0.00 0.00 Arousal 0.27 0.00 0.32 0.55 0.00 0.58 0.00 0.00 Emotion Regulation (Valence) 0.92 0.00 0.30 0.95 0.00 0.57 0.14 0.38 Notes: This table shows adjusted sharpened two-stage q-values and unadjusted p-values of balance tests for all available variables at baseline of table 2. Table A6: Individual Characteristics and Attrition Attrition (=1 if no follow-up) (1) (2) (3) Treat 0.002 (0.011) 0.025* (0.014) -0.010 (0.055) Individual Characteristics Female -0.002 (0.007) -0.004 (0.009) Student’s age 0.001 (0.004) 0.000 (0.004) Student is enrolled in the evening education shift -0.001 (0.007) 0.003 (0.008) Student’s grade level -0.001 (0.004) -0.002 (0.004) Travel time (minutes from home to school) 0.000 (0.000) -0.000 (0.000) Student has tried to emigrate to the USA -0.010 (0.010) -0.029 (0.024) Student works 0.000 (0.002) 0.001 (0.004) Household Characteristics Student’s household composition Student lives with both parents -0.000 (0.007) 0.013 (0.018) Student lives only with mother -0.002 (0.009) 0.012 (0.019) Student lives only with father -0.002 (0.009) -0.004 (0.020) Student lives with other relatives 0.005 (0.007) 0.013 (0.016) Mother’s education level No education -0.003 (0.004) -0.002 (0.006) Elementary or secondary 0.003 (0.003) 0.001 (0.002) Main Outcomes Behavior at School 0.000 (0.002) -0.001 (0.002) Math Grades -0.001 (0.003) -0.000 (0.003) School Enrollment (in 2020) 0.019 (0.016) 0.018 (0.015) Mechanisms Fluid Intelligence -0.002 (0.002) -0.002 (0.002) SES Index -0.001 (0.002) -0.001 (0.003) Perseverance -0.003 (0.002) 0.004 (0.003) Impulsiveness 0.000 (0.002) -0.001 (0.003) Arousal 0.001 (0.002) -0.001 (0.002) Emotion Regulation (Valence) -0.003 (0.003) 0.001 (0.003) Observations 1,975 708 708 Interaction with Treatment No No Yes P-value for F Test Interactions 1.000 Notes: This table shows the estimated impacts of psychology-based interventions on student attrition rates. The dependent variable ”Attrition” in all columns is a dummy indicating that a person did not respond to the follow-up survey. Model 1 is only the impact of any treatment on the follow-up survey respondent; model 2 measures the impact of all variables; and model 3 interacts all variables with the psychology-based intervention. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. Clustered standard errors at the grade level are shown in parentheses. ∗ ∗ ∗p < 0.01, ∗ ∗ p < 0.05, ∗p < 0.1 64 Table A7: ASP Attendance and Dropout Attendance Sessions Sessions Dropped (% of Sessions) Completed Scheduled the ASP (1) (2) (3) (4) Virtue 0.181*** 9.761** 0.710 -0.098*** (0.045) (4.036) (3.154) (0.036) Mindful 0.093** 10.211*** 2.098 -0.064* (0.045) (2.901) (2.218) (0.035) P-value for Virtue = Mindful 0.0245 0.8915 0.5730 0.2225 Mean Club Group 0.6355 33.3134 42.7558 0.1993 Observations 1,393 1,393 1,393 1,393 Notes: This table shows the ASP attendance rates. The estimation sample includes all students treated in Clubs, Virtue, or Mindful. It compares attendance of participants randomly assigned to Virtue or Mindful to those treated in Clubs. The table includes attendance, which equals 0 for those who dropped out of the ASP. Column (1) shows the attendance rate as % of number of sessions completed (column 2). Column (3) lists the number of sessions scheduled. Column (4) depicts the share of students that dropped out of the ASP. “Mean Clubs Group” is the mean of the outcome for the group that participates in Clubs. All estimations include the relevant control variables at baseline. Bootstrapped standard errors at the course level are in parentheses. ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1 65 Table A8: Mother’s Education Level and ASP Protection Effects of More Effects of Less Diff. Less vs. Mean Educated Mothers on Educated More Educated Pure-Control Observations Mothers on Mothers on Mothers Group (2) - (1) (1) (2) (3) (4) (5) Panel A: Behavior at School and Academic performance Behavior at School -0.003 0.284 0.288 -0.013 1009 (0.304) (0.345) (0.298) Math Grades -0.058 -0.207 -0.149 0.044 1009 (0.140) (0.161) (0.102) School Enrollment -0.024 -0.008 0.015 0.794 823 (0.044) (0.063) (0.043) Panel B: SES and Emotion Regulation SES Index 0.190 0.006 -0.184 -0.002 860 (0.144) (0.115) (0.171) Arousal 0.475* 0.335* -0.140 0.050 402 (0.245) (0.184) (0.318) Valence -0.303 -0.105 0.198 0.056 402 (0.277) (0.212) (0.320) Fluid Intelligence -0.135 -0.047 0.088 0.013 402 (0.168) (0.163) (0.185) Notes: This table shows the heterogeneous impacts of the protection learning component by mother’s education level before the intervention. We define that a mother has “low education level” if she has completed elementary school or less. Panel A presents the estimated heterogeneous effects on behavior and academic performance outcomes. Panel B shows the het- erogeneous impacts on SES and emotion regulation outcomes. The descriptions of all dependent variables are available in Appendix 1. Columns (1) presents the total effect of the ASP’s protection component on children of more educated women , relative to similar students in the pure-control group. Column (2) shows the total effect of the ASP’s protection component on children of less educated women relative to other children assigned to the pure-control group. Column (3) shows the differ- ence between treated children from women with low vs. high-education levels. “Mean Pure-Control Group” in column (4) is the mean of the outcome for students in the pure-comparison group. Sample size in each specification presented in column (5) varies according to the number of observations available for each outcome. Estimations include all individual controls for which there is imbalance in the non-experimental variation at baseline. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. Wild bootstrap standard errors are shown in parentheses. ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1 66 Table A9: Maternal and ASP Protection for Child Living with Both Parents Panel A: Behavior at School and Academic Performance Total Effect Difference Total Effect Mean on Children by Maternal for Children with Pure- Observations with No Maternal Care with Maternal Control Care Care Group (1) (2) (3) (4) (5) Behavior at School -0.272 0.830*** 0.558 -0.009 645 (0.320) (0.290) (0.369) Math Grades -0.296* 0.299** 0.003 0.065 645 (0.169) (0.140) (0.186) School Enrollment -0.013 -0.010 -0.023 0.825 517 (0.047) (0.038) (0.056) Panel B: SES and Emotion Regulation SES Index 0.199 0.0119 0.211 -0.032 549 (0.178) (0.199) (0.133) Arousal 0.161 0.141 0.302 0.042 244 (0.303) (0.355) (0.246) Valence -0.613** 0.342 -0.271 0.038 244 (0.264) (0.372) (0.261) Fluid Intelligence 0.104 -0.259 -0.155 0.017 244 (0.284) (0.319) (0.195) Notes: This table shows the estimated impacts of maternal and ASP protection for a child living with both parents. ”Clubs” is a dummy equal to 1 if the student was enrolled in a school that was randomly assigned to the extracurricular activities, and 0 if assigned to the pure-control group. Mean Pure-Control Group is the mean of the outcome for the control group. The description of dependent variables is available in Appendix 1. All dependent variables are measured in standard deviations relative to the Clubs group. Sample size in each specification varies according to the amount of data available for each output. All estimations include the following individual controls: school shift, student lives with both parents, student lives only with mother, mother’s education level (elementary or higher), behavior at school, school dropout, school enrollment, short- term perseverance, impulsiveness, risk-taking behavior, and arousal. All regressions include randomization block (strata) fixed effects. Strata were defined as country and violence level (high or low) of the community where the school is located. Clustered standard errors at the course level are shown in parentheses and unadjusted p-values are shown in brackets. ∗∗∗ p < 0.01, ∗∗ p < 0.05, ∗ p < 0.1 67 Table A10: Cost-Benefit Analysis Description Values Notes and Assumptions Panel A. Indirect Impacts of the Intervention Correlation between bad behavior and dropout 0.05 Source: Study data School dropout from study sample 0.206 Estimated as 1 - Enrollment rate of the pure-control group. Source: Study data Intervention effect on bad behavior -0.19 Effect on bad behavior from the study, in score (-0.464 is the effect in sd, the sd = 0.398) Indirect effect of the intervention on dropout -0.01 Adjusting the effect on bad behavior using correlation with dropout Reduction in number of potential dropouts -20 Sample of study x’s indirect effect on dropout due to the program Panel B. Costs Annual cost per club (US$) Clubs 3,232.29 Source: Glasswing costing data Virtue 3,931.10 Mindful 3,509.45 Annual club cost per participant (US$) Clubs 269.36 Virtue 327.60 Annual cost per club divided by an average of 12 Mindful 292.46 participants per club Average 296.48 Panel C. Wages Annual wage of an individual who completed (US$) Elementary school 1,946.52 Source: Households Survey (EHPM) 2018 High school 3,056.88 Technical school 5,407.68 68 College 6,600.96 NPV of wages of an individual who completed (US$) Elementary school 35,070.54 High school 53,638.63 Assuming that they work until age 55, discount rate 5%, Technical school 91,944.35 no salary growth College 109,528.48 Difference in NPV relative to elementary school education (US$) High school 18,568.09 NPV of completing this education level - completing Technical school 56,873.81 basic education only College 74,457.94 Panel D. Intervention Effects on Total Earnings and Taxes Total earnings of potential dropouts who stay in school and complete (US$)... High school 371,361.8 Technical school 1137,476.2 College 1489,158.8 Taxes paid for those who completed at least (US$) High school 3,713.61 Assuming that individuals who might dropout will pay Technical school 11,374.76 1% of income tax for this ASP College 14,891.58 Benefit-cost ratio for those who complete at least... High school 12.53 Assuming that the government uses tax dollars to pay Technical school 38.37 for the intervention College 50.23 Notes: This table summarizes the assumptions and values used for the cost-benefit analysis of the ASP. More details regarding the analysis are presented in Appendix 3. Figure A1: Activity Sheet for the Virtue Curriculum—The Backpack Notes: This figure shows the activity sheet for The Backpack activity, which involves a reflexive session related to building perseverance in the Virtue curriculum. 69 Figure A2: Activity Sheet for the Virtue Curriculum–My Map 18 Notes: This figure shows the activity sheet used for the My Map activity, which involves a reflexive session to help build perseverance in the Virtue curriculum. 70 Figure A3: Manual for the Mindful Curriculum–Sun Breathing Activity CIERRE Notes: This figure shows the instructions for the “Sun Breathing” activity, which involves a breathing and relaxation exercise in the Virtue curriculum. 71 Figure A4: Project Timeline Identification of schools, agreements with prin- Mid-term follow- Baseline data collection up (dropout cipals, IRB submission and randomization Short-term follow-up Jan. 2019 and enrollment) Mar. 2019 Oct. – Nov. 2019 February 2021 Recruitment and en- Mid-term follow-up Program implementation rollment of participants Jul.-Aug. 2020 Apr. – Oct. 2019 mid-Jan – Feb. 2019 (cancelled) 72