Policy Research Working Paper 10511 Experimental Evaluation of a Financial Education Program in Elementary and Middle School Grades Caio Piza Isabela Furtado Vivian Amorim Development Economics Development Impact Evaluation Group June 2023 Policy Research Working Paper 10511 Abstract This paper investigates whether providing financial edu- mainly among middle school students, and suggestive evi- cation in elementary and middle school grades improves dence of improvements in short-term behavioral outcomes. students’ financial proficiency and actual behavior. It uses a However, the analysis indicates that the program did not cluster randomized control trial to evaluate a pilot program impact students’ school achievements in both the short and implemented in 101 Brazilian municipal schools in 2015. longer terms, which suggests that the program’s effects were The findings show positive impacts on financial proficiency, not strong enough to shift students’ behavior decisions. This paper is a product of the Development Impact Evaluation Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at caiopiza@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Experimental Evaluation of a Financial Education Program in Elementary and Middle School Grades∗ Caio Piza† Isabela Furtado‡ Vivian Amorim § JEL Codes: D14, I21, I25, O12, O15 ∗ Acknowledgements: We would like to thank Cláudia Forte, Alzira Silva, Thiago Nascimento, and Cláudia Donega from the Brazilian Association of Financial Education (AEF-Brasil), José Alexandre Vasco from the Securities and Exchange Commission of Brazil (CVM), and the staff from Municipal Secretariats of Education from Manaus and Joinville. We thank the seminar participants from the Sao Paulo School of Economics at Getulio Vargas Foundation, Insper Institute of Education and Research, the 39th grade Brazilian Econometrics Society Meeting (SBE), and the 3ie-IFPRI Seminar Series. We also thank Arianna Legovini, Bilal Zia, Miriam Bruhn, Florence Kondylis, and an anonymous referee for their comments and suggestions. Finally, we thank DIME i2i, DEC† Research Support Budget, and the Brazil Country Office for funding support. Development Impact Evaluation Unit (DIME) of the World Bank. Contact: caiopiza@worldbank.org ‡ Insper - Institute of Education and Research. Contact: isabelabf@insper.edu.br § Development Impact Evaluation Unit (DIME) of the World Bank. Contact: vamorim@worldbank.org 1 Introduction Financial literacy is at the center of the education debate in various developed and developing countries. For instance, the Organization for Economic Cooperation and Development (OECD) advocates that financial education should be integrated into basic school curricula (Lusardi and Mitchell, 2014).1 As individuals are being increasingly exposed to financial decisions at a young age, the sooner they learn to incorporate the costs of their choices in their decision-making process the better. Good financial habits at an early age are likely to benefit schooling, employment, and the standard of living through adulthood (Bruhn et al. (2016); Bruhn et al. (2022)). In fact, developed and developing countries worldwide have been implementing financial education interventions in schools aiming to develop financial skills that could be conducive to better decision-making throughout individuals’ life-cycle (Bover et al. (2018), Frisancho (2020), Gill and Bhattacharya (2019), Luhrmann et al. (2018)), Bruhn et al. (2016), Bhattacharya et al. (2016), De Beckker et al. (2021), Berry et al. (2018), Batty et al. (2020)). Financial education may improve the quality of intertemporal decision-making by increasing individuals’ knowledge about saving and borrowing instruments (Luhrmann et al. (2018)). Subjects with higher financial knowledge are more likely to make patient inter-temporal choices (Oberrauch and Kaiser (2022)). Also, financial literacy can be a factor in avoiding financial risk and taking advantage of economic opportunities (Miller et al. (2015)). In a recent meta-analysis, Kaiser et al. (2022) find that financial education interventions have, on average, a significant impact on financial knowledge (0.15-0.2 of a standard deviation) and on financial behaviors, such as borrowing, saving and investing, budgeting and planning, insurance, and remittances (0.06–0.1 sd).2 Effective financial education programs that impact children’s socio-emotional skills such as patience and self-control could potentially improve outcomes other than the financial-related ones. Research in psychology and economics literature shows that more impatient children and adolescents have lower human capital accumulation and are more likely to consume alcohol, 1 The OECD defines financial literacy as a combination of awareness, knowledge, skills, attitudes, and behaviors necessary to make sound financial decisions and ultimately achieve individual financial well-being. In this study, we use financial education and financial literacy interchangeably. 2 The literature review includes 76 randomized experiments. 1 develop obesity, get pregnant during adolescence, and commit a crime (Castillo et al. 2011; Moffitt et al. 2011; Sutter et al. 2013; Golsteyn et al. 2014; Kautz et al. (2014)). Socioemotional skills influence labor market earnings, participation in the labor market, probability of holding a job, school decisions, and a range of educational outcomes (Acosta et al. (2018); Santos et al. (2021); Alan et al. (2019); Kautz et al. (2014)). From a theoretical standpoint, programs aimed at developing the personality traits mentioned above should be prioritized in childhood in the face of the expected higher returns per dollar invested (Heckman et al. 2006; Cunha and Heckman 2007). There is increasing evidence that financial education programs targeting high school students are effective (Bruhn et al. (2016), Gill and Bhattacharya (2019); Frisancho (2022)). The available literature also suggests that in-school programs not only improve students’ financial knowledge but also change their behavior (Bover et al. (2018), Luhrmann et al. (2018), Frisancho (2022), Jamison et al. (2014), Bruhn et al. (2016), Stoddard and Urban (2020), Urban et al. (2020), and Harvey (2019)). Recent evidence suggests that these programs can have long-lasting effects on students’ human capital accumulation and on financial decisions and entrepreneurship (Frisancho (2020);Frisancho (2022); Bruhn et al. (2022)). There is also growing interest and evidence on the impacts of financial literacy interventions on elementary and middle school grades. Bhattacharya et al. (2016) find evidence of a 12 percentage points increase in the financial knowledge of eighth-graders in the United States. De Beckker et al. (2021) suggest a 0.46 sd increase in the financial knowledge as well as better financial behavior of eighth and ninth-graders in an experiment in Belgium. Batty et al. (2020) finds evidence of an increase in the financial knowledge of fourth-graders equivalent to 0.25 of a standard deviation in a pilot run in 20 schools in the United States. In an RCT with Turkish elementary schools, Alan and Ertac (2018) find that the program persistently reduces the incidence of behavioral problems in schools and increases students’ patience. Berry et al. (2018), on the other hand, do not find impacts of a financial literacy intervention implemented in Ghanaian elementary and junior high schools. In this paper, we use a cluster randomized trial to investigate the impacts of a financial literacy pilot implemented in 2015 in elementary and middle school grades in two Brazilian municipalities (Manaus and Joinville). We randomly assigned 101 municipal schools to receive the program, 2 and 100 to control. The intervention aimed to help young students to become more conscious of the trade-offs embedded in their daily decisions and more forward-looking. The evaluation focused on grades three, five, seven, and nine for reasons discussed later. We find positive effects on students’ financial knowledge of 0.07 of a standard deviation, on average. The effects are driven by middle school grades (0.1 SD) and are consistent with a recent summary of the financial education literature (Kaiser et al. (2022)). We also find suggestive evidence that the program helped students become more conscious about their consumption habits - i.e., more patient and risk-averse. We find suggestive evidence of changes in some contemporary behavioral outcomes such as the use of piggy banks, suggesting some linkages between actual change in behavior in the short-term. However, we do not detect impacts on both contemporary learning outcomes and human capital accumulation. We use a mediation causal analysis to investigate the potential causal link between gains in financial knowledge and change in behavioral outcomes. Our results do not point to a compelling causal linkage between formal knowledge and change in behavior, a result consistent with De Beckker et al. (2021). Our paper has four main contributions. First, we document positive impacts on financial proficiency, attitudes, and some behavioral outcomes of a large-scale pilot intervention implemented in elementary and middle school grades. Second, we show that financial education programs can potentially impact different skills according to students’ life cycles. Third, we test the overlooked hypothesis that increasing financial proficiency is a necessary condition to change individuals’ behavior. Finally, we use student-level data from the Brazilian national standardized exams to assess the program impacts on students’ contemporary and longer-term school progression and learning outcomes. Apart from this introduction, this paper is organized as follows. Section 2 describes the pilot program. Section 3 describes the data collection. Section 4 describes program’s implementation. Section 5 introduces the empirical strategy. Section 6 presents the main results. We then conclude in Section 7. 3 2 Financial Education Pilot in Elementary and Middle Schools To disseminate financial education in schools and empower the population to make better financial decisions, the National Strategy for Financial Education (ENEF) was launched in 2010 by the Brazilian Federal Government.3 In 2015, ENEF developed a financial educational pilot targeting elementary and middle school students. The Brazilian municipalities of Joinville and Manaus volunteered to participate and the intervention was implemented at the beginning of that school year on a sample of K-9 public schools under the management of the respective municipal governments. The initiative followed a successful financial literacy program that was tested in almost 900 public high schools in six Brazilian states. An experimental evaluation of this program shows significant improvements in students’ financial knowledge, intention to save, and financial autonomy, as well as greater participation in their household finances (Bruhn et al., 2016). The municipalities of Manaus and Joinville are very distinct from one another. Manaus, the capital of Amazonas state, is located in the north of the country, the second poorest region, whereas Joinville is a municipality of Santa Catarina, which belongs to the more affluent south region. In the year of the intervention, in comparison to Manaus, Joinville registered a 30% higher per capita income, a 10% higher human development index, and a difference of at least 30% in the Education Development Index (IDEB), the most important educational indicator in Brazil, for both primary and secondary education.4 Even though such differences might lead to regional variations in the quality of programs’ implementation, our setting increases the likelihood that our results can be informative to other Brazilian municipalities. ENEF has four financial education textbooks for elementary and middle school students. The first one is for first to fourth graders, the second one for fifth and sixth graders, the third one for seventh and eighth graders, and the fourth one for ninth graders. Each textbook is tailored to be adequate for students’ age groups and grades. Table B.2 shows the skills the intervention 3 ENEF stands for Estratégia Nacional de Educação Financeira. 4 IDEB stands for Índice de Desenvolvimento da Educação Básica and is used to evaluate the quality of primary and secondary education schools. The National Institute of Education and Research (INEP), an agency under the Ministry of Education, is in charge of calculating the index. For elementary education, for example, IDEB is calculated by multiplying the standardized fifth graders’ proficiency in Portuguese and Math (on a scale from 0 to 10) with the grade promotion from first to fifth grades (on a scale from 0 to 1). The index was created in 2005 and became the most important educational outcome for Brazilian educational policies, setting targets for schools, municipalities, and states. 4 aimed to develop. The curriculum is conceived to introduce the role of financial education in students’ daily life by developing financial concepts such as savings, consumption, and waste; making students able to identify situations related to financially responsible attitudes, estimate the budget necessary for financial projects, and find relevant information in the press media for decision-making in finance. With the development of these skills, the pilot is expected to increase students’ preference for future prospects and decrease present bias (Becker and Mulligan (1997); Luhrmann et al. (2018)). The textbooks’ content is then transversally introduced into each grade’s standard curriculum. 2.1 Sample Selection Due to budgetary limitations, we agreed with the implementing partner to identify approximately 200 schools that could participate in the pilot. We leverage existing administrative data to define the sampling frame of the experiment. According to the 2015 Census of Education, Joinville has 72 schools that offer elementary or middle school grades. We include them all in the experiment sampling frame.5 In Manaus has 302 schools and we randomly draw 129 to be part of the experiment sampling frame (Table B.1).6 For budgetary reasons, the pilot could not include all K-9 grades. In agreement with ENEF, we select grades 3, 5, 7, and 9 to generate evidence on the effects of each of the four textbooks developed for the pilot. This strategy allows us to assess whether the effect of the program varies by school grade and students’ like-cycle. It is important to emphasize that we are not testing the impact of the same material in different school grades. Instead, we are testing the impacts of school materials tailored to each specific grade. Within each treated school, not all the classes offering third, fifth, seventh, and ninth grade participate in the financial literacy intervention. At the beginning of the 2015 school year, the school principals from both treatment and control schools picked only one class of each grade to be included in the pilot. This strategy is adopted to minimize costs with data collection. In the end, our sampling frame includes 201 eligible schools for the pilot. To account for 5 Public schools that the municipal government manages. 6 The total of 302 schools does not include 53 schools in riverside communities in Manaus. 5 differences in both municipalities’ socioeconomic standards, and schools’ infrastructure, we stratify the sample by the municipality and three school types: schools that only offer the elementary school grades (first to fifth grades in Brazil), schools that only offer the middle school grades (sixth to ninth grades in Brazil), and schools that offer all elementary school grades, totaling six strata (Table B.1). 3 Data and Measurement Our research uses both administrative and survey data. We use administrative data to draw our sampling frame, to check the balance after randomization, and to track students in the Brazilian education system. To measure the impacts of the program on contemporary outcomes, we use survey data collected in the field at the end of the 2015 school year, right after the program implementation finished, in line with other financial education programs targeting elementary and middle school students (De Beckker et al. (2021), Alan and Ertac (2018), Berry et al. (2018)). 3.1 Survey Data The Center of Public Policy and Education Evaluation (CAEd), a Brazilian survey firm specializing in standardized proficiency tests and data collection, surveyed teachers and students included in the pilot. CAEd printed and handed out teachers’ and students’ questionnaires on the last school days of the 2015 school calendar. Students’ participation rate in the survey was very close to the average observed on a regular school day (around 80%).7 Teachers’ questionnaire investigates their wage level, teaching experience, and issues related to the incorporation of financial literacy textbooks in the curriculum, among other questions. Overall, there are no significant differences between teachers in treatment and control groups (Table B.6). 7 Table B.5 in section 4 shows the students’ participation rates. One may wonder whether there could be less student participation with the survey data being collected at the end of the school year as some students might have already gone on vacation or dropped out. On the one hand, the participation rate was close to the one observed on a regular school day. On the other hand, it is unclear if the absent students are the best ones, who had already succeeded in being promoted to the next grade even without final tests, or the worst students who had already dropped out or did not achieve the expected performance for grade-promotion. 6 Students had two hours to answer three questionnaires with multiple-choice questions suitable to each grade level and learning goal. The first one is based on the skills introduced in the financial literacy textbooks, as shown in Table B.2, and aims to assess students’ financial knowledge. The second questionnaire checks students’ attitudes towards savings and consumption, such as the use of piggy banks, their willingness to buy new things or conversations about financial themes with parents or friends. Table B.3 shows examples of questions. The last questionnaire asks about students’ socioeconomic background, such as the educational attainment of their mothers and whether they are beneficiaries of the national conditional cash transfer program, Bolsa Família.8 As expected, Table B.7 shows that there are no significant differences between the socioeconomic characteristics of treated and control students. Based on the student’s answers to the financial knowledge questionnaire, CAEd uses the Item Response Theory (IRT) to create a financial proficiency index. According to the IRT, proficiency is a latent trait (i.e., unobserved characteristic or attribute), and a set of questions (items), that is, observed performance, is the instrument used to measure students’ proficiency. Each question in an exam that uses IRT is accounted for based on its level of difficulty. This procedure allows the generation of an invariant measurement scale of the latent trait that is comparable across grades and over time. The financial proficiency index is normalized so that the treatment effect could be measured in terms of standard deviations (SD). The use of the standardized measure prevents the analysis from being scale-sensitive and allows for comparability between grades and with other studies. As in Bruhn et al. (2016), we use psychology-based indices computed by CAEd to capture students’ intertemporal consumption and saving choices in hypothetical scenarios. Table B.3 shows a subset of questions used by CAEd to develop the indices. The questions have four possible answers (I totally agree, I agree, I disagree, I totally disagree), classified on a 1 to 4 scale in which 4 represents the most forward-looking financial behavior. The consumption index 8 Fifth, seventh and ninth-graders answered all questionnaires in the school. For the financial knowledge exam, fifth, seventh, and ninth-graders answered 30, 32, and 36 questions, respectively. For the questionnaire on attitudes toward savings and consumption, fifth, seventh and ninth-graders answered 35, 41, and 41 questions, respectively. For all these grades, students answer a socioeconomic questionnaire with 24 questions. Third- graders answer the financial knowledge questionnaire in the school with the help of an enumerator, who read out the questions. Since third-graders are still acquiring reading and writing abilities, this strategy is intended to make the test easier for all students, regardless of their reading and writing skills. Still, for third-graders, the socioeconomic questionnaire and one of the students’ attitudes towards savings and consumption were sent to their homes so their legal guardians could fill out and return it the following day. Legal guardians were supposed to answer it based on how he/she believed his/her child would behave in a given situation. Schools prepared the legal guardians to answer these questionnaires by sending them a note prior to the test day. 7 captures the students’ preferences for planned expenditure (self-control) versus impulsive ones (impatience), whereas the savings index captures both risk-aversion. Both indices are computed using the principal component analysis and then standardized. Overall, our analysis has eight dependent variables from the survey data collected in the field. Three are the standardized indexes developed by CAEd, which are (i) financial proficiency, (ii) consumption (a proxy for patience and self-control), and (iii) savings (a proxy for risk-aversion). The other five are behavioral ones such as (iv) a dummy indicating whether a student talks to parents about financial subjects, (v) a dummy indicating whether a student talks to friends about the same subject, (vi) students’ use piggy banks (a proxy for saving), (vii) students’ access to financial services, and (viii) a dummy indicating whether a student receives an allowance from their parents. 3.2 Administrative Data Due to the limited budget, and the richness of publicly available administrative data, we did not collect baseline survey data. Prior to the program’s implementation, we use administrative data at the school level to check for balance between treatment and control schools. Since 1995, all private and public K-12 schools have participated in the annual Education Census. The Census is implemented by the National Institute of Educational Studies and Research (INEP), a research agency under the Brazilian Ministry of Education. The Census collects information on school facilities, school infrastructure, social services, teachers, and students. The data is collected at the student level, allowing us to track their sex, ethnicity, age, grade level, instruction time per day, class size, and student outcomes, such as grade promotion, retention, and dropout status. In 1990, INEP created the primary and secondary Education Assessment System (SAEB) aiming to assess students’ proficiency in Math and Portuguese.9 In 2005, the national standardized exam known as Prova Brasil was introduced within the scope of SAEB, which expanded the proficiency assessments to all public schools.10 The exam is applied every two years to fifth 9 SAEB stands for Sistema de Avaliação da Educação Básica. 10 Schools with at least 20 students enrolled in each of the evaluated grades. Students take the test at the end of the school year (between October and November). 8 and ninth-graders, and students in the last grade of high school. The exam assesses students’ proficiency in Portuguese and math. Since 1998, INEP has annually applied the National High School Proficiency Test (ENEM).11 ENEM is not mandatory and the students are the ones in charge of registering for the exam. ENEM participants are the students enrolled in high school, mostly in the last grade, and the ones that have already finished this level of education. Since 2009, the exam started being used by several Brazilian universities as one of their criteria to select their students. As a consequence, the number of participants significantly increased over time, from 115,000 in 1998 to 8.5 million in 2015. The exam aims to assess students’ learning levels in reading, math, human sciences, and natural sciences. All aforementioned administrative surveys are publicly available. INEP discloses identified data at the school level, allowing us to merge the pilot schools with Education Census, Prova Brasil, and ENEM. Table B.10 shows that, overall, treated and control schools do not have significant differences in average students’ proficiency in Portuguese and Math, grade promotion, dropout, retention rates, and age-grade distortion. Unidentified microdata at the student level is also publicly available and identified data only upon researcher request to access INEP’s facilities.12 As presented in section 2, principals’ choice of which classes would receive the intervention or would serve as a control is not necessarily random. In this sense, one may wonder whether school principals from treatment and control schools face different incentives to select the classes they did. In treated schools, principals could either select low-performer classes hoping to help their students with the intervention, or high-performer classes if they believe these students would benefit most from the program. In control schools, principals could select low performers in order to show that their schools should also receive the treatment in the future, or high performer classes if they intend to show that their schools do not need the program. In this sense, using administrative data at the school level to check the pre-intervention balance of students’ performance has one caveat since not all the students in these schools are included in the pilot. 11 ENEM stands for Exame Nacional do Ensino Medio. 12 We were not able to request access to students’ identified microdata by the beginning of 2015 when the pilot was implemented, and then check the balance between treated and control students. First, because we did not have the list of pilot students, and second because the request usually takes months to be processed and it was very restrictive at that time. 9 To check the balance between treated and control students, in 2019 we requested INEP to merge our survey data collected in the field with their administrative datasets. This task has to be performed by the agency since researchers are not allowed to access identified administrative data at the student level. The INEP technical team performs the merge using students’ names and the schools they are enrolled in. Table B.8 and Table B.9 show that there are no significant differences in the percentage of treated and control students found by the INEP technical team in its administrative datasets.13 Table B.4 shows the variables collected at the student level by INEP administrative surveys and we use them for two main tasks. First, for seventh and ninth graders included in the pilot, we compare their pre-intervention performance in Portuguese and math. The information is available for when these students were enrolled in fifth grade and, therefore, did the national proficiency exam (Prova Brasil ).14 For third and fifth graders, we do not have any administrative data on their pre-treatment performance in reading and math on standardized national exams. Hence, for these grades, we could not compare students’ proficiency between treatment and control classes.15 Second, the merged dataset allows us to investigate the impact of the financial literacy pilot on student outcomes only available in Education Census, Prova Brasil, and ENEM. As shown in Table B.4, we use the administrative data to assess the effects of the financial literacy pilot on (i) retention and dropout rates in the year of the pilot or up to three years later (between 2015-2018); on (ii) standardized proficiency test in math and Portuguese in the year of the pilot (for those enrolled in fifth and ninth grades in 2015) and two years after (those enrolled in third and seventh grades in 2015);16 and (iii) ENEM scores and high school completion rates for those enrolled in the ninth grade in the year of the pilot.17 Table B.9 shows that, for seventh- grade classes, there are no significant differences between treatment and control classes with regard to pre-treatment students’ performance. Nonetheless, for ninth-grade classes, the data show that students in control schools had a significantly higher pre-treatment performance in 13 INEP only authorizes access to the merged survey and administrative datasets using the computers in its facilities in Brasília. Therefore, all the analyses that include administrative data at the student level are run in INEP facilities. The agency only allows the extraction of the Stata do-files and the final outputs (tables or figures) as long as they do not contain sensitive information at the student level. No microdata can be extracted. INEP technical team masks students’ names so we work on an unidentified dataset. 14 For seventh graders who did not repeat any grade between the fifth and seventh grades, their fifth-grade proficiency is available in 2013. For ninth-graders who did not repeat any grade between the fifth and ninth grades, their fifth-grade proficiency is available in 2011. 15 No standardized proficiency test is applied by INEP before fifth grade. 16 For those who did not repeat between third and fifth grades and between seven and ninth grades, respectively. 17 For those who did not repeat between ninth grade and the last year of high school. 10 reading, which might indicate that the principals from control schools selected high-performer classes to be in the pilot evaluation. If this was the case, and if higher reading scores are correlated with better financial proficiency skills, our results would be biased downwards. 4 Program Implementation ENEF provided training on how to incorporate financial literacy textbooks into the school curriculum of the 2015 school year (that started in February and ended at the beginning of December). In Joinville, the training was held in February 2015 for the teaching supervisors of each school. In Manaus, supervisors of the regional education boards were trained at the end of March of the same year. Teachers in the schools that implemented the intervention were trained by those who attended the ENEF training. The pilot schools had the autonomy to decide when to introduce the textbooks into their classes. The programs’ supervisors designed by the Departments of Education of Joinville and Manaus were in charge of monitoring the pilot implementation. During the 2015 school year, the supervisors filled out questionnaires aimed to identify the major bottlenecks and keep track of the use of financial literacy textbooks. ENEF used the findings of the first two questionnaires applied during the first semester of the school year to improve the program’s implementation. At the beginning of the second semester, ENEF organized a meeting with teachers in charge of incorporating the pilot textbooks into their classes. This meeting sought to stimulate the use of the textbooks and to encourage teachers who were already using them to share their experiences with those who had had little contact with the material at that point. Insights into the programs’ implementation can also be obtained from the questionnaires applied to teachers and students at the end of the 2015 school year. Table B.5 shows that around 80% of pilot students answered the financial literacy assessment. With the exception of third graders, the socioeconomic questionnaire was answered by at least 97% of those that did the financial literacy test. We observe that there are no significant differences in response rates of treatment and control groups. Table B.5 shows that almost half of the third-grade teachers introduced financial literacy 11 textbooks during the whole school year. However, middle school teachers only introduced the textbooks in the second semester. Also, according to questionnaires applied to students, more than half of middle-school students received the financial literacy textbooks in the second semester. Slightly more than 20% of middle-school students had at least 80% of the financial literacy content covered. The late introduction of the textbooks is reflected in the percentage of the curricula actually covered. One can infer from the data on the program’s implementation that the treatment was not very intense, and its intensity differed across schools and grades. 12 5 Identification Strategy To estimate the causal impacts of the program on the outcomes of interest, we estimate the following reduced-form regression: 6 ∑︁ = + + + (1) =1 in which is the outcome variable (e.g., financial proficiency) of student in grade at school in municipality , is a binary variable that is equal to 1 if the student belongs to a school assigned to the program (treatment) and 0 otherwise, are strata fixed effects, and is the idiosyncratic error.18 The parameter of interest, , measures the intention-to-treat (ITT), the treatment effect on students of schools randomly assigned to receive the treatment. The standard error is clustered at the school level, the randomization unit. It is worth noticing that this regression does not distinguish the effect between grades. To assess the program’s effect for only elementary, only middle school, and for each grade included in the pilot, we separately re-estimate Equation 1 for each one of these samples.19 In addition to that, we follow Firpo et al. (2009) and estimate unconditional quantile intention-to-treat (UQITT) effects to check whether the program has distinct impacts across the financial proficiency distribution.20 18 Table B.1 shows that the cluster-randomized trial has six clusters. 19 We also estimate standard error clustering at the class-school level. The results are very similar and are available upon request. 20 We estimate the unconditional UQITT in two steps. We first run a regression of Y on constant and strata dummies and save the residuals. We then use the residuals as a dependent variable to estimate the UQITT. 13 6 Results 6.1 Financial Proficiency, Savings and Consumption Attitudes, and Behav- ioral Outcomes Table 1 shows that the intervention has a positive and statistically significant impact of 0.07 of a standard deviation on students’ financial proficiency, with the effect driven by the middle school grades. Larger impacts on students of more advanced grades are consistent with the hypothesis that financial education programs are more effective to increase the financial proficiency of the students more likely to experience financial decisions in daily life (Frisancho (2020), Bruhn et al. (2016)), and Zia (2023). Our results are in line with Jamison et al. (2014) as the authors find an impact of 0.08 of a standard deviation of a financial literacy pilot implemented among Ugandan youth groups, but are considerably smaller when compared to the ones reported by De Beckker et al. (2021) and Batty et al. (2020) who find an increase in the financial literacy of middle and elementary grade students of 0.46 and 0.25 of a standard deviation, respectively. To investigate whether students with better academic performance benefited from the intervention, we estimate the effect of the program at different points of the financial proficiency distribution. Figure 1 displays the effect of the pilot on elementary school students at 20 different quantiles of this distribution. The UQITT estimates show a positive and statistically significant of about 0.05 of an SD starting in the 45 percentile of the financial proficiency distribution. The results suggest that the program benefited a subset of elementary school students, but the effect on the subset was almost fully counterbalanced by no effects for the bottom 40 percentile. For middle-school students, the UQITT and ITT estimates are similar, pointing to a more homogeneous effect across the distribution in more advanced grades (see Figure C.2 in the appendix). 14 Figure 1: Quantile ITT estimates for financial proficiency, elementary students - 95%CI 0.30 0.25 0.20 0.15 Standard deviation 0.10 0.05 0.00 -0.05 -0.10 -0.15 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Quantiles of finacial proficiency Source: The financial proficiency is computed by CAEd based on students’ answers on a standardized exam that used the Item Response Theory (IRT). The index is normalized so that the treatment effect could be measured in terms of standard deviations (SD). We first run this financial proficiency score on the treatment dummy, the six strata dummies, and on a constant. We then take the residual of this estimation and run a simultaneous quantile regression, for each one of the quantiles shown in the Figure, in which the dependent variable is the residual of the first regression and the independent variable is the treatment dummy. 1000 bootstrap replications. 15 Table 1: ITT estimates for financial proficiency and hypothetical attitudes towards savings and consumption (1) (2) (3) (4) (5) (6) (7) Pooled Elementary Middle school 3rd grade 5th grade 7th grade 9th grade Financial Proficiency Treatment 0.072** 0.045 0.096** 0.049 0.041 0.090* 0.107* (0.033) (0.043) (0.043) (0.072) (0.043) (0.048) (0.056) pvalue 0.030 0.292 0.025 0.495 0.350 0.061 0.057 pvalue of RI 0.040 0.307 0.025 0.510 0.339 0.074 0.066 Obs. 14,655 7,641 7,014 3,635 4,006 3,640 3,374 Number of clusters 201 172 145 170 168 144 144 R2 0.154 0.149 0.182 0.127 0.173 0.208 0.168 Strata F.E. Yes Yes Yes Yes Yes Yes Yes Consumption index self-control Treatment 0.094*** 0.122*** 0.082*** - 0.122*** 0.080* 0.084** (0.026) (0.042) (0.031) - (0.042) (0.041) (0.037) pvalue 0.000 0.004 0.010 - 0.004 0.054 0.025 pvalue of RI 0.001 0.001 0.015 - 0.001 0.064 0.024 Obs. 10,220 3,648 6,572 - 3,648 3,409 3,163 Number of clusters 200 167 145 - 167 144 142 R2 0.089 0.119 0.076 - 0.119 0.100 0.055 Strata F.E. Yes Yes Yes - Yes Yes Yes Saving index risk-aversion Treatment 0.055** 0.087** 0.044 - 0.087** 0.066 0.018 (0.025) (0.041) (0.029) - (0.041) (0.040) (0.038) pvalue 0.031 0.036 0.138 - 0.036 0.102 0.644 pvalue of RI 0.036 0.046 0.153 - 0.048 0.108 0.686 Obs. 10,022 3,549 6,473 - 3,549 3,368 3,105 Number of clusters 200 166 145 - 166 144 142 R2 0.046 0.096 0.025 - 0.096 0.032 0.019 Strata F.E. Yes Yes Yes - Yes Yes Yes Note: Clustered standard errors at school level in parentheses. RI stands for Randomized Inference. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. The financial proficiency is computed by CAEd based on student’s answers on a standardized exam that used the Item Response Theory (IRT). The consumption and saving indices are based on students’ choices in questions containing four possible answers (I totally agree, I agree, I disagree, I totally disagree) and they are aimed to measure students’ self-control and risk-aversion, respectively. The financial proficiency, savings, and consumption indices are normalized so that the treatment effect can be interpreted in terms of standard deviations (SD). Our estimates also point to positive impacts on consumption and saving indices, the ones measuring students’ attitudes, particularly among fifth-graders (Table 1).21 These estimates should be interpreted as children becoming more conscious, self-controlled and forward-looking. Interestingly, the magnitude of the impact on saving attitudes declines steadily with school grades. This pattern suggests that older students are less risk-averse and potentially self- controlled. The larger point estimates for self-control and risk aversion among fifth graders might be associated with a few factors. Younger students might be less overconfident than teenagers and this can make them more responsive to interventions aimed to develop risk-aversion behavior. At the same token, it might be easier to change the behavior of younger students since their financial habits are not consolidated yet. We further investigate whether the program affected students’ behavioral outcomes by looking at more objective measures of change in behavior such as the use of piggy banks, and investments in human capital. We expect that more patient students would be more likely to invest in human capital whose returns happen later in life. Table 2 shows impacts on students’ likelihood to discuss financial subjects with their parents and friends, their likelihood to use a piggy bank (a proxy for actual savings), and their likelihood to receive a monthly allowance from parents (a proxy for students’ access to informal sources of credit and parents’ engagement). Finally, we test whether the treatment increases children’s access to financial services, such as debit and credit cards.22 ITT estimates show that fifth graders are 5.7 pp (or 11.6 percent) and 5.6 pp (or 28 percent) more likely to talk to parents and friends about financial subjects respectively, and 2.7 pp. more likely to use piggy banks (a proxy for actual savings). Overall, these results point to positive effects on awareness and saving outcomes. For middle school students, we find some indication that they become more likely to talk to friends about financial subjects. Unlike Bruhn et al. (2016), we do not find any impact of the program on students’ access to financial services. This is unsurprising in our context given that most students are young and do not deal with financial transactions in their 21 Due to characteristics of the research instruments for the third graders, the variables employed to set up the savings and consumption index are unavailable for this grade. Thus, column (2) estimates correspond to only fifth-graders. 22 To measure students’ access to financial services, we use a dummy variable that is one if a student has access to at least one service and zero otherwise. 17 daily lives. Our estimates show that the program had larger impacts on financial proficiency in later grades and larger effects on attitudes and behavior in earlier grades. The results then suggest that the association between financial proficiency and behavioral change might not be straightforward. Individuals might have formal knowledge about different subjects, such as financial proficiency, but the application of the knowledge in real life can be discouraged by the institutional environment they are inserted (Carpena and Zia (2018)). To formally test the hypothesis that learning mediates change in behavior, we follow Imai et al. (2011) and decompose the treatment effect estimates into the average causal mediation effect (ACME), and the average causal direct effect (ADE).23 The goal of this exercise is to assess the share of the change in behavioral outcomes that could be attributed to improvements in financial proficiency (ACME) and the share that comes directly from the program (ADE). The decomposition of ITT effects on ACME and ADE relies on the sequential ignorability assumption. The first part of the assumption assumes that conditioned on pre-treatment confounders the treatment assignment is orthogonal to the potential outcomes and the mediator outcome (financial proficiency). The second part of the assumption implies that the ‘observed mediator is ignorable given the actual treatment status and pre-treatment confounders (Imai et al., 2011). This second part of the assumption is the strongest, even in the presence of random assignment. It poses that conditional on the actual treatment (not treatment assignment) and a vector of pre-treatment confounders, the outcome is independent of the mediator factor. Note that, in our case, this implies an assumption that attitudinal and behavioral outcomes are orthogonal to financial proficiency once we control for the actual treatment and a vector of school characteristics. Since we did not design the experiment to obtain clean estimates of the ACME and ADE, we interpret our results as suggestive evidence. According to the estimates in Table B.13 and Figure C.5, while we find some evidence that financial proficiency mediates the improvement of middle students’ hypothetical attitudes in scenarios involving savings and consumption, it does not mediate increases in their behavior. For elementary school students, all of the effects seem to be direct, i.e., not mediated by changes in financial proficiency (Table B.13 and Figure C.4). In other words, the ADE and ACME estimates suggest that financial education programs affect students’ attitudes and behavior 23 See in the Appendix A a formal discussion of Casual Mediation Effects. 18 Table 2: ITT estimates for behavioral outcomes Pooled Elementary Middle Pooled Elementary Middle students students students students Talk to parents Talk to friends Treatment 0.012 0.057*** -0.012 Treatment 0.035*** 0.056*** 0.023** (0.010) (0.019) (0.012) (0.009) (0.014) (0.011) pvalue 0.216 0.003 0.306 pvalue 0.000 0.000 0.044 pvalue of RI 0.222 0.004 0.345 pvalue of RI 0.000 0.000 0.058 Obs. 10,748 3,875 6,873 Obs. 10,725 3,864 6,861 Number of clusters 200 167 145 Number of clusters 200 167 145 R2 0.006 0.003 0.001 R2 0.008 0.012 0.005 Strata F.E. Yes Yes Yes Strata F.E. Yes Yes Yes Piggy’s bank use Use of finantial services Treatment 0.022** 0.027* 0.017 Treatment 0.002 -0.002 0.006 (0.011) (0.015) (0.013) (0.009) (0.012) (0.013) pvalue 0.042 0.067 0.205 pvalue 0.783 0.860 0.662 pvalue of RI 0.046 0.063 0.236 pvalue of RI 0.773 0.872 0.638 Obs. 12,739 5,917 6,822 Obs. 12,538 5,718 6,820 Number of clusters 201 172 145 Number of clusters 201 172 145 R2 0.028 0.038 0.010 R2 0.005 0.004 0.006 Strata F.E. Yes Yes Yes Strata F.E. Yes Yes Yes Allowance Treatment 0.013 0.017 0.01 (0.008) (0.014) (0.010) pvalue 0.120 0.222 0.303 pvalue of RI 0.111 0.245 0.305 Obs. 10,699 3,862 6,837 Number of clusters 200 167 145 R2 0.009 0.012 0.007 Strata F.E. Yes Yes Yes Note: Clustered standard errors at school level in parentheses. RI stands for Randomized Inference. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. The dependent variables are equal to 1 if students talk to parents or friends about financial subjects, if they use a piggy bank, financial services, or receive an allowance from their parents, and 0 otherwise. through pathways other than the accumulation of formal knowledge. Overall, the estimates do not support the hypothesis that changes in financial proficiency are a necessary condition to change individuals’ behavioral outcomes, a finding consistent with Carpena and Zia (2018), and De Beckker et al. (2021). To assess whether the results on both financial proficiency, attitudes, and behavioral outcomes are picking up some heterogeneity in the data, we carried out a series of subgroup analyses. First, we compared the effects on boys and girls. The point estimates are larger for boys, but the difference in estimates between boys and girls is not statistically significant in most cases. Second, we tested whether the impacts varied by poverty status. We aimed to assess whether poorer kids - those participating in the Brazilian conditional cash transfer program - would respond less to an intervention intended to help kids become more forward-looking. In fact, the estimates on financial proficiency and consumption attitudes are larger in absolute terms 19 for the poor, but the differences in coefficients between poor and non-poor are not statistically significant for the most part. The heterogeneous effect analyses are available in the online appendix. 6.2 Robustness Check Back in 2015, we did not have access to administrative student-level data to check the balance between treatment and control groups and investigate the impacts of the program on students’ performance in Portuguese and math. As described in section 3, in 2020 and 2021 INEP granted the research team access to its facilities to complement our analyses using student-level data. These data allow us to rerun the balance tables at the student level and, more importantly, look at the program’s impacts on both contemporary and longer-term human capital outcomes. As discussed in section 2, school principals’ choice of the classrooms to participate in the pilot might not have been random. Consequently, if school principals strategically picked the best classrooms aiming to show the benefits of the program, or the worst classrooms betting on a potential complementarity between financial education and other school subjects, our estimates would be biased as we could be comparing treated and control students with a different pre- treatment level of skills. To assess students’ pre-treatment level of proficiency in Portuguese and math, we look at their scores in the Brazilian standardized exam when in grade 5. Indeed, the average test scores in grade 5 for students attending the seventh and ninth grades in 2015 were higher in the control group (Table B.9). To check whether this imbalance in pre-treatment skill level affects our results, we re-estimate Equation 1 for seventh and ninth graders controlling for their pre-treatment math and reading test scores.24 Controlling for past performance in math and reading leads to a substantial gain in precision. Among seventh graders, we observe a slight increase in the point estimate for financial proficiency scores (from 0.09 SD to 0.098 SD). For ninth-graders, we find a meaningful increase in financial proficiency scores, with the point estimate jumping from 0.11 SD to 0.15 SD (Table B.14), though the difference is not statistically significant. The updated estimates 24 Seventh and ninth-graders are the ones for which the pre-treatment performance in reading and math is available. For seventh-graders, we have their fifth-grade performance in reading and math in 2013. For ninth- graders, we have their fifth-grade performance in 2011. 20 indicate that we would underestimate the impacts of the pilot on financial proficiency if we did not control for an imbalance in pre-treatment proficiency in math and reading at the student level, particularly for ninth graders. Interestingly, controlling for past performance in math and reading scores does not change the point estimates for consumption and saving attitudes. 6.3 Implementation and IV Results The following analysis seeks to shed some light upon the role played by the quality of the program’s implementation on its impacts. This exercise is useful in that it avoids hasty conclusions that a given intervention did not have the expected effect when in fact the program was not carried out as planned. We use a variable that informs the percentage of the syllabus (use of the teaching material) covered during the school year. As discussed earlier, the material was delivered somewhat late at the treated schools, and that eventually hindered the exposure to the program and consequently impacted the intensity of treatment. Approximately one-third of the teachers claimed they covered up to 60% of the syllabus. We first carry out a correlation analysis testing whether the effects are associated with the quality of the program’s implementation. We do so by splitting the treatment group into two groups: schools with up to 60% of the syllabus covered, and schools with more than 60% of the syllabus covered. Since the quality of the program’s implementation is endogenous, we follow Frisancho (2020) and use the random assignment to treatment as an instrumental variable for the program’s implementation. The IV estimates provide the local average treatment effect (LATE) of the program by comparing the subgroup of treated schools that received the treatment (supervisor/teacher training and teachers who used the textbooks) with the control schools. We create a binary variable, , that is 1 for schools with at least 60% of the syllabus covered during the academic year, and 0 otherwise leveraging information gathered from teachers and students on whether the textbooks were incorporated into the classroom during the school year. Since the data show that some students and teachers from control schools apparently received and used the textbooks, we use a two-stage least squares to estimate the LATE: 21 First-Stage: = 1 + 2 ℎ + Σ + 1 , ℎ = 1, 2. (2) Second-Stage: = 1 + 2 + Σ + 2 (3) The parameter of interest, 2 , is the LATE. The IV estimates are shown in Table 3. The estimates are slightly larger and less precise than ITT ones, suggesting that the program did not have large effects even when better implemented. Table 3: IV estimates Pooled Elementary Middle school Financial Proficiency ITT LATE ITT LATE ITT LATE Estimate 0.073** 0.085** 0.047 0.055 0.099** 0.115** N. obs (0.034) (0.039) (0.046) (0.054) (0.042) (0.049) R2 14,655 14,655 7,641 7,641 7,014 7,014 0.167 0.166 0.15 0.149 0.188 0.189 Consumption Index Estimate 0.094*** 0.110*** 0.122*** 0.146*** 0.082*** 0.095*** (0.026) (0.031) (0.042) (0.050) (0.031) (0.036) N. obs 10,220 10,220 3,648 3,648 6,572 6,572 R2 0.089 0.087 0.119 0.117 0.076 0.076 Savings Index Estimate 0.055** 0.064** 0.087** 0.104** 0.044 0.051 (0.025) (0.029) (0.041) (0.049) (0.029) (0.034) N. obs 10,022 10,022 3,549 3,549 6,473 6,473 R2 0.046 0.046 0.096 0.095 0.025 0.025 Note: Clustered standard errors at school level in parentheses. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. Students are considered treated if their teachers answered that at least part of the content of financial education was covered in class. For 8.5% of the treated students and 7.5% of the control students, we do not have a teacher assigned. In these cases, we cannot infer actual treatment using teachers’ answers on the usage of the material. We then consider a student as treated if at least 50% of her peers in the classroom state they received the financial literacy textbook. 22 6.4 Longer-term Effects on Behavioral Outcomes: Investment in Human Capital In this section, we assess whether the pilot impacts students’ human capital investment decisions. The hypothesis is that if students understand the importance of education for their future earnings and become more patient and forward-looking following the pilot, they would be more likely to study harder, potentially improve their academic performance in Portuguese and math, and be less likely to drop out of school (Bruhn et al. (2016)). This has a direct policy implication as it would indicate whether offering financial education to elementary and middle school students could improve individuals’ outcomes other than their financial proficiency with potential benefits to their financial health and labor market outcomes in the longer term. We start by checking if the program has an immediate impact on the performance of fifth and ninth-graders in math and reading in 2015, the year the financial education program was piloted. footnoteFifth and ninth grades are the ones for which the performance in math and reading is available in 2015 due to their participation in Prova Brasil. See Table B.4. Figure C.1 shows a significant correlation between students’ financial proficiency score and their reading and math performance. We then investigate whether there is any causal relationship in place. Our results do not point to a contemporary effect of financial education on either math or reading skills for both grades (Table 4). Although the nil effects on math and reading proficiency of fifth graders could be explained by the fact that the intervention had a small effect on a subset of the students as suggested by the UQITT estimates, the lack of treatment effects for grade 9 is less expected. These results seem to suggest that teachers involved in this pilot evaluation did not prioritize financial education at the expense of other school subjects (e.g., teach to the test), a concern the research team and some school principals had back in the day. Overall, our estimates are in line with Batty et al. (2020) that also finds evidence of no substitution from the core school curriculum. 23 Table 4: ITT estimates for Human Capital Accumulation Outcomes 3rd grade 5th grade 7th grade 9th grade Only Strata FE Only Strata FE Only Strata FE Strata FE + Only Strata FE Strata FE + strata Financial strata Financial strata Financial Performance strata Financial Performance proficiency proficiency proficiency 5th grade proficiency 5th grade Repetition 0.017 0.010 0.029** 0.025* -0.007 -0.002 -0.008 0.016 0.031* 0.018 (0.017) (0.019) (0.014) (0.015) (0.014) (0.015) (0.016) (0.016) (0.017) (0.018) Dropout 0.005 0.007 0.011 0.000 0.006 0.002 0.002 0.033 0.028 0.020 (0.006) (0.006) (0.010) (0.008) (0.018) (0.014) (0.012) (0.020) (0.018) (0.019) Portuguese -0.029 -0.053 -0.017 -0.032 -0.004 -0.057 -0.005 -0.015 -0.063* 0.028 Prova Brasil (0.047) (0.051) (0.046) (0.031) (0.047) (0.041) (0.034) (0.042) (0.036) (0.036) Math -0.066 -0.096* -0.033 -0.045 -0.005 -0.060 -0.009 -0.077* -0.094** -0.020 Prova Brasil (0.053) (0.055) (0.049) (0.039) (0.051) (0.048) (0.038) (0.042) (0.037) (0.039) Finish -0.035 -0.035 -0.019 high school (0.025) (0.024) (0.023) Participation -0.025 -0.016 -0.012 in ENEM (0.024) (0.022) (0.023) Portuguese -0.046 -0.004 -0.025 ENEM (0.036) (0.05) (0.031) Math -0.028 -0.035 0.002 ENEM (0.047) (0.063) (0.045) Obs. 4650 3635 4755 4006 4697 3640 3137 4455 3374 2851 Note: Clustered standard errors at school level in parentheses. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. Grade retention assumes a value of 1 if the student is retained in the same grade at the end of the school year, and 0 otherwise. Dropout is equal to 1 if the student drops out at the end of the school year, and 0 otherwise. Finish high school is equal to 1 if the student finishes high school, and 0 otherwise. Participating in ENEM is equal to 1 if the students participate in the ENEM assessment, and 0 otherwise. Portuguese and Math are the proficiency scores in Prova Brasil on SAEB scale. Besides the contemporary effects on learning outcomes, the data allow us to test whether the program impacted students’ human capital accumulation. We first assess whether the pilot affected students’ probability of being retained or dropping out of school between 2015 and 2018. Second, for third and seventh graders, we assess their performance in math and reading two years after the intervention (2017) when they reached fifth and ninth grades, respectively.25 Finally, for ninth-graders, we assess whether the intervention affects their probability of finishing high school, taking the ENEM test, and the test takers’ ENEM proficiency score. We do not find any indication that the financial education program affects students’ academic achievements – test scores and school progression – in the longer term (Table 4). We interpret these findings as evidence that the program does not make students become more forward- looking with regard to their human capital accumulation decisions, which might be associated with poor program implementation as described in section 4. 7 Conclusion In this paper, we use a cluster randomized controlled trial to evaluate the impacts of a financial literacy pilot program in Brazilian elementary schools during the academic year of 2015. The pilot was implemented in 101 municipal schools and involved approximately 9,000 students. Our main findings showed that the program improved students’ financial proficiency, consumption (self-control) and saving (risk-aversion) attitudes, and some behavioral outcomes, such as the use of piggy banks. We find strong indications of heterogeneous effects though. While proficiency gains were stronger among middle school students, changes in attitudes and behavioral outcomes stood out among elementary education students. We tested the overlooked hypothesis in this literature that poses that change in formal knowledge precedes changes in actual behavior. We use a causal mediation effect analysis to test this hypothesis. Overall, our results seem to reject the hypothesis that claims that changes in knowledge are a necessary condition to change individuals’ habits. Finally, we used student test score data in math and reading in different school grades to 25 For those who did not repeat between third and fifth grades and between seventh and ninth graders, respectively. 25 investigate the program’s impacts on contemporary learning outcomes and on human capital accumulation. We do not find evidence that the program impacted grade progression, retention, dropout rates, or learning outcomes. Our reading of the results is that the program did not affect students’ intertemporal decision-making, confirming the preliminary results used by the implementing partner who decided to scale the program down in its current format. 26 References Acosta, P. M. et al. (2018): “The role of cognitive and socio-emotional skills in labor markets,” IZA World of Labor. Alan, S., T. Boneva, and S. Ertac (2019): “Ever failed, try again, succeed better: Results from a randomized educational intervention on grit,” The Quarterly Journal of Economics, 134, 1121–1162. Alan, S. and S. Ertac (2018): “Fostering patience in the classroom: Results from randomized educational intervention,” Journal of Political Economy, 126, 1865–1911. Batty, M., J. M. Collins, C. O’Rourke, and E. Odders-White (2020): “Experiential financial education: A field study of my classroom economy in elementary schools,” Economics of Education Review, 78, 102014. Becker, G. S. and C. B. Mulligan (1997): “The endogenous determination of time preference,” The Quarterly Journal of Economics, 112, 729–758. Berry, J., D. Karlan, and M. Pradhan (2018): “The impact of financial education for youth in Ghana,” World Development, 102, 71–89. Bhattacharya, R., A. Gill, and D. Stanley (2016): “The effectiveness of financial literacy instruction: The role of individual development accounts participation and the intensity of instruction,” Journal of Financial Counseling and Planning, 27, 20–35. Bover, O., L. Hospido, and E. Villanueva (2018): “The impact of high school financial education on financial knowledge and choices: Evidence from a randomized trial in Spain,” . Bruhn, M., G. Garber, S. Koyama, and B. Zia (2022): “The Long-Term Impact of High School Financial Education,” . Bruhn, M., L. d. S. Leão, A. Legovini, R. Marchetti, and B. Zia (2016): “The Impact of High School Financial Education: Evidence from a Large-Scale Evaluation in Brazil,” American Economic Journal: Applied Economics, 8, 256–295. Carpena, F. and B. Zia (2018): The Causal Mechanism of Financial Education: Evidence from Mediation Analysis, World Bank Policy Research Working Paper no. 8619. 27 Castillo, M., P. J. Ferraro, J. L. Jordan, and R. Petrie (2011): “The today and tomorrow of kids: Time preferences and educational outcomes of children,” Journal of Public Economics, 95, 1377–1385. Cunha, F. and J. Heckman (2007): “The technology of skill formation,” American Economic Review, 97, 31–47. De Beckker, K., K. De Witte, and G. Van Campenhout (2021): “The effect of financial education on students’ consumer choices: Evidence from a randomized experiment,” Journal of Economic Behavior & Organization, 188, 962–976. Fernandes, D., J. Lynch, and R. G. Netemeyer (2014): “Financial Literacy, Financial Education, and Downstream Financial Behaviors,” . Firpo, S., N. M. Fortin, and T. Lemieux (2009): “Unconditional quantile regressions,” Econometrica, 77, 953–973. Frisancho, V. (2020): “The impact of financial education for youth,” Economics of Education Review, 78, 101918. ——— (2022): “Is School-Based Financial Education Effective? Immediate and Long-Lasting Impacts on High School Students,” The Economic Journal. Gill, A. and R. Bhattacharya (2019): “The effects of a financial literacy intervention on the financial and economic knowledge of high school students,” The Journal of Economic Education, 50, 215–229. Golsteyn, B. H., H. Grönqvist, and L. Lindahl (2014): “Adolescent time preferences predict lifetime outcomes,” The Economic Journal, 124, F739–F761. Harvey, M. (2019): “Impact of financial education mandates on younger consumers’ use of alternative financial services,” Journal of Consumer Affairs, 53, 731–769. Heckman, J. J., J. Stixrud, and S. Urzua (2006): “The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior,” Journal of Labor economics, 24, 411–482. 28 Imai, K., L. Keele, D. Tingley, and T. Yamamoto (2011): “Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies,” American Political Science Review, 105, 765–789. Jamison, J. C., D. Karlan, and J. Zinman (2014): “Financial education and access to savings accounts: Complements or substitutes? Evidence from Ugandan youth clubs,” Tech. rep., National Bureau of Economic Research. Kaiser, T., A. Lusardi, L. Menkhoff, and C. Urban (2022): “Financial education affects financial knowledge and downstream behaviors,” Journal of Financial Economics, 145, 255– 272. Kautz, T., J. J. Heckman, R. Diris, B. Ter Weel, and L. Borghans (2014): “Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success,” . Luhrmann, M., M. Serra-Garcia, and J. Winter (2018): “The Impact of Financial Education on Adolescents’ Intertemporal Choices,” American Economic Journal: Economic Policy, 10, 309–32. Lusardi, A. and O. S. Mitchell (2014): “The Economic Importance of Financial Literacy: Theory and Evidence,” Journal of Economic Literature, 52, 5–44. Miller, M., J. Reichelstein, C. Salas, and B. Zia (2015): “Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature,” The World Bank Research Observer, 30, 220–246. Moffitt, T. E., L. Arseneault, D. Belsky, N. Dickson, R. J. Hancox, H. Harring- ton, R. Houts, R. Poulton, B. W. Roberts, S. Ross, et al. (2011): “A gradient of childhood self-control predicts health, wealth, and public safety,” Proceedings of the National Academy of Sciences, 108, 2693–2698. Oberrauch, L. and T. Kaiser (2022): “Cognitive ability, financial literacy, and narrow bracketing in time-preference elicitation,” Journal of Behavioral and Experimental Economics, 98, 101844. Santos, I., V. Petroska-Beska, P. Carneiro, L. Eskreis-Winkler, A. M. Munoz Boudet, I. Berniell, C. Krekel, O. Arias, and A. Duckworth (2021): 29 “Can Grit Be Taught? Lessons from a Nationwide Field Experiment with Middle-School Students,” . Stoddard, C. and C. Urban (2020): “The effects of state-mandated financial education on college financing behaviors,” Journal of Money, Credit and Banking, 52, 747–776. Sutter, M., M. G. Kocher, D. Glätzle-Rützler, and S. T. Trautmann (2013): “Impatience and uncertainty: Experimental decisions predict adolescents’ field behavior,” American Economic Review, 103, 510–31. Urban, C., M. Schmeiser, J. M. Collins, and A. Brown (2020): “The effects of high school personal financial education policies on financial behavior,” Economics of Education Review, 78, 101786. Zia, B. (2023): “When is financial education successful? Taking stock of the new wave of field evidence,” Handbook of Microfinance, Financial Inclusion and Development, 119–133. 30 Appendix A Mediation analysis The literature on financial education works with an implicit theory of change that poses that improvements in individuals’ financial proficiency should mediate changes in financial decision- making (Fernandes et al. 2014; Lusardi and Mitchell 2014). We use mediation causal effects to empirically test the hypothesis embedded in the theory of change. Carpena and Zia (2018) tested this relationship with microfinance borrowers in Gujarat, India. They find mixed results concerning the role of financial literacy as a mediator factor for behavioral change. Our set of estimates is the first attempt to isolate the effect of financial literacy on the attitudes and behavioral outcomes of elementary school students. We follow Imai et al. (2011) and decompose the treatment effect estimate into the average causal mediation effect (ACME), and the average causal direct effect (ADE). The goal of this exercise is to estimate the share of the treatment effect on attitudes and behavioral outcomes that are due to improvements in financial proficiency and the share that comes directly from the program. Since we do not design the experiment to obtain clean estimates of the ACME and ADE, our results should be seen as an attempt to shed light on this overlooked hypothesis. To assess whether the increase in financial proficiency impacts attitudes and behavior down the line, we rely on the sequential ignorability assumption. The first part of the assumption assumes that conditioned on pre-treatment confounders the treatment assignment is orthogonal to the potential outcomes and the mediator outcome (financial proficiency). The second part of the assumption implies that the ‘observed mediator is ignorable given the actual treatment status and pre-treatment confounders’ (Imai et al., 2011). In formal terms, the conditions are the following: { (′ , ), ()} ⊥ ⊥ | = , (4) 31 (′ , ) ⊥ ⊥ ()| = , = (5) This second part of the assumption is the strongest, even in the presence of random assignment. It poses that conditional on the actual treatment (not treatment assignment) and a vector of pre-treatment confounders, the outcome is independent of the mediator factor. Note that in our case this implies an assumption that attitudinal and behavioral outcomes are orthogonal to financial proficiency once we control for the actual treatment and a vector of observed school- level characteristics. We estimate the ADE and ACME running the following regression equations:26 = 1 + 2 + + Σ + (6) = 1 + 2 + + + Σ + (7) where is the financial proficiency and is the attitudinal/behavioral outcome variable of student in grade at school in municipality , and are defined as before, and is a vector of school characteristics that includes information on school infrastructure - whether computer lab, science lab, sports facilities, garbage collection, garbage recycling, and sewage connection - and management complexity.27 Coefficients 2 and 2 , correspond to the average total treatment effect (ITT) and the ADE respectively. Imai et al. (2011) shows that the ACME is given by the difference between 2 and 2 . We estimate the standard errors using bootstrap with 1,000 repetitions. To test whether the sequential ignorability assumption is likely to hold in our context, we 26 We use Hicks and Tingley (2011)’s medeff command in Stata 27 Management complexity is measured by a categorical variable that ranges from 1 to 3 where 1 refers to schools that offer one or two shifts and have up to 300 students, 2 represents schools that offer 2 or 3 shifts, have 1,000 students, and offer vocational training in high school, and 3 is for schools with 3 shifts, at least 500 students, and the option of vocation training in high school. 32 undertake a sensitivity analysis that tests for the correlation between the residuals of equations (6) and (7) (Figure C.3, Figure C.4, and Figure C.5). B Tables 33 Table B.1: Sample selected for the pilot study (A) Joinville (B) Manaus Total Manaus and Joinville I II III IV V VI VII Number of Randomization Randomization Number of Randomization Randomization Number of schools Sample Groups schools Sample Groups randomized schools Schools offering: Treatment Control Treatment Control Treatment Control Stratum A.1 Stratum B.1 1st-5th grades 20 All 10 10 202 36 sampled based on 18 18 28 28 2013 IDEB 1st-5th grades Stratum A.2 Stratum B.2 6th-9th grades 2 All 1 1 35 28 sampled based on 14 14 15 15 2013 IDEB 6th-9th grades Stratum A.3 Stratum B.3 1st-9th grades 50 All 25 25 65 All 33 32 58 57 34 Total 72 36 36 302 129 65 64 101 100 Notes: Number of schools managed by the municipal governments of Joinville and Manaus (Education Census, 2015). Following the guidelines of the Department of Education, 53 schools located in riverside communities in Manaus are not included in the pilot, due to difficulties in accessing some of these areas. In Joinville, all 72 schools are included in the randomization. In Manaus, 36 out of 202 schools offering first to fifth grade are randomized into treatment and control groups. The 36 schools are selected based on the distribution of the Educational Development Index for that grades in 2013. Also, in Manaus, 28 out of 35 schools offering sixth to ninth grade are randomized into treatment and control. The 28 schools are selected based on the distribution of the Educational Development Index of that grades in 2013. Finally, all schools offering first to ninth grade in Manaus are randomized into treatment and control. Overall, six strata are created in this randomization process (two municipalities and within the municipalities three types of schools). Table B.2: Descriptors and skills on the financial knowledge test 3rd grade 5th grade 7th grade 9th grade D01 Being able to identify the subject of texts whose topic explores socially yes yes yes yes responsible attitudes towards the environment D02 Being able to find information in texts about consumption – light, water, yes yes no no telephone bills, among others D03 Being able to identify the purpose of texts and text formats that include yes yes no no expenses, consumption, spending D04 Being able to recognize the purpose of text genres related to finances – no yes yes yes receipts, checks, invoices D05 Being able to recognize situations in which concepts related to finances no no yes yes are present: savings, expenses, consumption, spending, waste, risk, return, financial planning, and investment, among others. D06 Being able to identify situations related to financially responsible no yes yes yes attitudes D07 Being able to find information in graphs and tables that contain data yes no no no related to finances (purchases, sales, spending) D08 Being able to find information in texts that circulate in the financial yes yes yes yes world: classified ads, news features, among others D09 Being able to estimate values and/or procedures necessary for financial yes yes yes yes projects D10 Being able to distinguish remunerated from non-remunerated work. yes yes no no D11 Being able to identify the origin and destination of varied products yes no no no and/or those that can be recycled. D12 Being able to recognize socially responsible situations related to public no no no yes and private spaces. D13 Being able to identify advantages, disadvantages, and risks of cash and no yes no yes credit sales. D14 Being able to find implicit information in media texts that are relevant no yes yes yes for decision-making in finances. 35 Table B.3: Examples of questions on students’ behavior towards savings and consumption Answers’ scale Statement Totally Agree Disagree Totally Weight in index agree disagree Consumption questions I buy what I want, then I see how I can pay 1 2 3 4 0.45 I see no problem in owing money 1 2 3 4 0.44 If the brand is famous the product is of high 1 2 3 4 0.32 quality The best product is always the most expensive 1 2 3 4 0.33 I plan before spending my money. 4 3 2 1 0.1 It is worthless to plan because the money comes 1 2 3 4 0.43 from luck. Buy what I want is more important than have 1 2 3 4 0.45 planning Saving questions I think that saving money is important to avoid 4 3 2 1 0.41 problems in the future. I feel safer when I can save some money. 4 3 2 1 0.39 Saving some money is important to avoid debt 4 3 2 1 0.42 Buying everything I want is more important than 1 2 3 4 0.24 putting the money together. Avoiding waste is also a way to save money. 4 3 2 1 0.38 I try to use the products for longer. 4 3 2 1 0.35 Whenever I can, I save money. 4 3 2 1 0.37 I would rather spend the change on something I 1 2 3 4 0.19 want than save the money for later. Table B.4: Dependent variables collected in Prova Brasil, Education Census and ENEM. Survey Data Administrative Data We matched the pilot students with the administrative data Pilot Proficiency in Math and Grade-promotion/dropout ENEM students in 2015 Portuguese (Prova Brasil) (Education Census) 3rd graders in 2015 → Their 5th grade If they ever repeated or proficiency in 2017 dropped between 2015-2018 5th graders in 2015 → Their 5th grade proficiency in 2015 and If ever repeated or their 7th grade proficiency in 2017 dropped between 2015-2018 7th graders in 2015 → Their 5th grade proficiency in 2013 If ever repeated or and their 9th grade proficiency in 2017 dropped between 2015-2018 9th graders in 2015 → Their 5th grade If ever repeated ENEM average proficiency in 2011 or dropped between 2015-2018 score in 2018 and if finished high school Note: Retention rates, dropout, ENEM scores, and high school completion rates are available annually (Education Census and ENEM disclosed by INEP). Performance in Portuguese and Math is only available every two years for fifth and ninth- graders (Prova Brasil disclosed by INEP). 36 Table B.5: Program implementation, 2015 3rd grade 5th grade 7th grade 9th grade Students’ participation rate C T T-C C T T-C C T T-C C T T-C Mean 77.35 79.03 -1.68* 83.87 84.62 -0.75 76.68 78.33 -1.64 77.05 74.43 2.62* Standard error [0.86] [0.85] [0.75] [0.74] [0.87] [0.86] [0.89] [0.92] Number of obs 2,380 2,270 2,375 2,380 2,376 2,321 2,222 2,233 Response rate of socioeconomic and attitudinal questionnaires C T T-C C T T-C C T T-C C T T-C Mean 65.32 65.60 -0.28 97.39 99.11 -1.72*** 99.56 99.72 -0.16 97.72 99.33 -1.61*** Standard error [1.12] [1.13] [0.36] [0.21] [0.15] [0.12] [0.36] [0.20] Number of obs 1,808 1,756 1,992 2,014 1,822 1,818 1,712 1,638 Deliver of the financial literacy textbooks according to students, in % Treated students Mean Sd Mean Sd Mean Sd Mean Sd Beginning of the year - - 25.3 43.5 16.8 37.4 22.5 41.8 By the middle of the year - - 57.8 49.4 57.6 49.4 53.9 49.9 In the end of the year - - 11.2 31.5 15.9 36.5 18.0 38.4 Haven’t received - - 5.7 23.2 9.7 29.6 5.6 23.0 % of classes by the semester in which the teachers used the financial literacy textbooks Treated classes Mean sd Mean sd Mean sd Mean sd Only in 1st semester 24.7 43.4 17.7 38.4 17.9 38.6 20.3 40.6 Only in 2nd semester 27.2 44.8 41.8 49.6 52.2 50.3 57.8 49.8 All year 48.1 50.3 40.5 49.4 29.9 46.1 21.9 41.7 % of classes by the percentage of the financial literacy textbook covered by the teachers Treated classes Mean Sd Mean Sd Mean Sd Mean Sd Less than 40% 6.1 24.1 0.0 0.0 6.0 23.9 7.7 26.9 Between 40% and 60% 26.8 44.6 22.0 41.6 25.4 43.8 27.7 45.1 Between 60% and 80% 40.2 49.3 40.2 49.3 44.8 50.1 36.9 48.6 More than 80% 26.8 44.6 37.8 48.8 23.9 43.0 27.7 45.1 Notes: ENEF questionnaires were applied to teachers and students at the end of the 2015 school year. T = treatment; C = control. The value displayed for t-tests is the differences in the means across the groups. The covariate variable strata are included in all estimation regressions. The second block (response rate of socioeconomic and attitudinal questionnaires) refers only to students that answered the financial proficiency test. ***, **, and * indicate significance at the 1, 5, and 10 percent critical levels. Table B.6: Balance test for teachers’ characteristics 3rd 5th 7th 9th C T T-C C T T-C C T T-C C T T-C Teachers characteristics, in % Gender: male Mean 7.2 9.9 -2.6 10.0 20.9 -10.9* 48.5 33.6 14.8* 38.6 44.6 -6.0 SE [2.9] [3.3] [3.4] [4.3] [6.2] [4.6] [5.9] [5.0] Obs 83 81 80 91 66 107 70 101 Age: less Mean 27.7 26.8 0.9 26.3 30.4 -4.2 35.8 33.6 2.2 26.8 31.7 -4.9 than 35 yers SE [4.9] [4.9] [5.0] [4.8] [5.9] [4.6] [5.3] [4.7] Obs 83 82 80 92 67 107 71 101 Age: 36 to Mean 50.6 59.8 -9.2 60.0 53.3 6.7 44.8 54.2 -9.4 53.5 50.5 3.0 50 yers SE [5.5] [5.4] [5.5] [5.2] [6.1] [4.8] [6.0] [5.0] Obs 83 82 80 92 67 107 71 101 Age: older Mean 21.7 13.4 8.3 13.8 16.3 -2.6 17.9 11.2 6.7 19.7 16.8 2.9 than 51 years SE [4.6] [3.8] [3.9] [3.9] [4.7] [3.1] [4.8] [3.7] Obs 83 82 80 92 67 107 71 101 Color: white Mean 45.8 42.7 3.1 48.8 38.0 10.7* 43.3 35.5 7.8* 46.5 38.6 7.9 SE [5.5] [5.5] [5.6] [5.1] [6.1] [4.6] [6.0] [4.9] Obs 83 82 80 92 67 107 71 101 Experience: Mean 19.3 17.1 2.2 11.3 19.6 -8.3 17.9 15.0 3.0 14.1 14.9 -0.8 up to 5 years SE [4.4] [4.2] [3.6] [4.2] [4.7] [3.5] [4.2] [3.6] Obs 83 82 80 92 67 107 71 101 Experience: Mean 41.0 46.3 -5.4 42.5 41.3 1.2 40.3 57.0 -16.7** 52.1 45.5 6.6 6 to 15 years SE [5.4] [5.5] [5.6] [5.2] [6.0] [4.8] [6.0] [5.0] Obs 83 82 80 92 67 107 71 101 Experience: Mean 39.8 36.6 3.2 45.0 38.0 7.0 41.8 27.1 14.7** 33.8 39.6 -5.8 more than 16 years SE [5.4] [5.4] [5.6] [5.1] [6.1] [4.3] [5.7] [4.9] Obs 83 82 80 92 67 107 71 101 Wage: 3 to 4 Mean 30.1 19.5 10.6 31.3 22.8 8.4 10.4 27.1 -16.7*** 19.7 21.8 -2.1 minimum wages SE [5.1] [4.4] [5.2] [4.4] [3.8] [4.3] [4.8] [4.1] Obs 83 82 80 92 67 107 71 101 Wage: 4 to 5 Mean 31.3 25.6 5.7 33.8 38.0 -4.3 32.8 20.6 12.3* 21.1 28.7 -7.6 minimum wages SE [5.1] [4.8] [5.3] [5.1] [5.8] [3.9] [4.9] [4.5] Obs 83 82 80 92 67 107 71 101 Wage: 5 to 6 Mean 4.8 17.1 -12.3** 16.3 16.3 -0.1 19.4 23.4 -4.0 25.4 29.7 -4.4 minimum wages SE [2.4] [4.2] [4.2] [3.9] [4.9] [4.1] [5.2] [4.6] Obs 83 82 80 92 67 107 71 101 Wage: more than Mean 6.0 7.3 -1.3 6.3 9.8 -3.5 16.4 11.2 5.2 15.5 5.9 9.6** 6 minimum wages SE [2.6] [2.9] [2.7] [3.1] [4.6] [3.1] [4.3] [2.4] Obs 83 82 80 92 67 107 71 101 Notes: T = treatment; C = control. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. We run the balance test controlling for strata. All the variables shown in the table were collected using the questionnaires CAEd applied to teachers of pilot schools. 38 Table B.7: Balance test for households and students’ characteristics 3rd 5th 7th 9th C T T-C C T T-C C T T-C C T T-C Household characteristics, in % Access to Mean 68.7 66.8 1.9 67.6 67.4 0.2 71.5 69.4 2.1 73.7 75.1 -1.4 paved street SE [1.4] [1.5] [1.1] [1.1] [1.1] [1.1] [1.1] [1.1] Obs 1087 1028 1914 1983 1804 1802 1662 1616 Access Mean 99.6 99.1 0.5 95.9 96.0 -0.1 97.6 97.6 -0.0 97.9 97.0 0.9 to electricity SE [0.2] [0.3] [0.5] [0.4] [0.4] [0.4] [0.4] [0.4] Obs 1041 1005 1914 1980 1800 1803 1668 1621 Access to Mean 96.7 96.3 0.4 96.1 96.2 -0.2 96.5 96.7 -0.3 97.3 96.8 0.5 piped water SE [0.5] [0.6] [0.4] [0.4] [0.4] [0.4] [0.4] [0.4] Obs 1060 1023 1925 1988 1806 1808 1667 1618 Access to Mean 91.5 93.3 -1.8 85.7 84.3 1.4 85.2 88.7 -3.5*** 90.2 91.2 -1.1 garbage SE [0.9] [0.8] [0.8] [0.8] [0.8] [0.7] [0.7] [0.7] collection Obs 1050 1002 1907 1961 1794 1796 1668 1609 Beneficiary of Mean 36.2 36.1 0.2 45.4 44.9 0.4 40.9 42.9 -1.9 38.3 35.8 2.6** Bolsa Família SE [1.5] [1.5] [1.1] [1.1] [1.2] [1.2] [1.2] [1.2] Obs 1082 1059 1905 1958 1790 1788 1648 1604 Student characteristics, in % Adequate age Mean 87.9 86.6 1.3 82.3 80.6 1.7 79.9 80.1 -0.2 81.2 81.1 0.0 for their grade SE [1.0] [1.1] [0.9] [0.9] [0.9] [0.9] [1.0] [1.0] Obs 1038 997 1902 1952 1781 1781 1636 1602 Mother education/ Mean 26.1 28.2 -2.1 26.3 25.2 1.1 25.1 26.9 -1.8 24.1 22.8 1.3 Legal guardian: incomplete SE [1.3] [1.4] [1.2] [1.2] [1.2] [1.2] [1.2] [1.1] elementary school Obs 1081 1050 1272 1333 1347 1317 1364 1335 Mother education/ Mean 20.9 20.9 0.0 18.1 18.8 -0.7 22.3 18.4 3.9** 18.6 20.1 -1.5 Legal guardian: incomplete SE [1.2] [1.3] [1.1] [1.1] [1.1] [1.1] [1.1] [1.1] high school Obs 1081 1050 1272 1333 1347 1317 1364 1335 Mother education/ Mean 53.0 51.0 2.1 55.7 56.0 -0.4 52.6 54.7 -2.1 57.3 57.0 0.3 Legal guardian: at least SE [1.5] [1.5] [1.4] [1.4] [1.4] [1.4] [1.3] [1.4] high school degree Obs 1081 1050 1272 1333 1347 1317 1364 1335 Gender: male Mean 51.3 50.1 1.2 48.0 48.1 -0.1 49.5 49.6 -0.1 SE [1.1] [1.1] [1.2] [1.2] [1.2] [1.2] Obs 1905 1964 1798 1799 1660 1614 Color: white Mean 40.6 37.3 3.2* 36.5 34.9 1.6 37.0 35.5 1.5 SE [1.1] [1.1] [1.1] [1.1] [1.2] [1.2] Obs 1888 1945 1771 1787 1634 1585 Notes: T = treatment; C = control. ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. We run the balance test controlling for strata. All the variables shown in the table were collected using the questionnaires CAEd applied to students of pilot schools. For each grade, the number of observations in the treatment and control groups can change depending on the variable under evaluation. This happens when students did not answer that specific question. Table B.11 shows that there are no significant differences between the treatment and control groups regarding the percentage of these variables that are missing. Table B.12 shows the results of the multivariate test of means. Except for seventh grade, we do not reject the hypothesis that the means of students’ characteristics are equal in the treatment and control groups. 39 Table B.8: Pre-treatment balance test for third and fifth graders (1) (2) T-Test Control Treament Difference N Mean/SE N Mean/SE (1)-(2) Third grade Beneficiary of Bolsa Família 1082 36.23 1059 36.07 0.16 [3.10] [2.92] Adequate age for their grade 1038 86.99 997 85.66 1.34 [1.49] [1.76] Gender: male 2071 53.84 2009 51.57 2.27 [0.92] [1.22] Color: white 1624 38.92 1582 36.47 2.44 [3.80] [3.94] Mother education/Legal guardian: at least high school degree 1081 53.01 1050 50.95 2.05 [2.59] [2.26] Student found in Education Census 2015-2018 2380 87.02 2270 88.50 -1.49 [1.60] [1.51] Student found in Prova Brasil 2017 2380 59.03 2270 59.16 -0.13 [1.84] [1.89] Prova Brasil reading score in 5th grade 2017 1405 232.76 1343 230.43 2.33 [2.51] [2.45] Prova Brasil math score in 5th grade 2017 1405 243.09 1343 238.55 4.54 [2.92] [3.09] Fifth grade Beneficiary of Bolsa Família 1905 45.35 1958 44.94 0.41 [2.80] [2.65] Adequate age for their grade 1902 78.39 1952 77.61 0.78 [1.71] [1.58] Gender: male 2302 51.87 2307 50.59 1.28 [1.12] [1.10] Color: white 1888 40.57 1945 37.33 3.25 [2.52] [2.48] Mother education/Legal guardian: at least high school degree 1272 55.66 1333 56.04 -0.38 [1.87] [2.00] Student found in Education Census 2015-2018 2375 89.94 2380 90.04 -0.11 [1.52] [1.08] Student found in Prova Brasil 2015 2375 79.33 2380 81.60 -2.27 [2.22] [1.77] Prova Brasil reading score in 5th grade 2017 1884 220.58 1942 218.95 1.62 [2.92] [2.68] Prova Brasil math score in 5th grade 2017 1884 231.83 1942 229.17 2.66 [3.82] [3.33] Notes: ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. We run the balance test controlling for strata. All the variables shown in the table are from Prova Brasil, and Census of Education. 40 Table B.9: Pre-treatment balance test for seventh and ninth graders (1) (2) T-Test Control Treament Difference N Mean/SE N Mean/SE (1)-(2) Seventh grade Beneficiary of Bolsa Família 1790 40.95 1788 42.90 -1.95 [2.92] [2.87] Adequate age for their grade 1781 77.20 1781 76.81 0.39 [2.18] [1.85] Gender: male 1798 48.00 1799 48.08 -0.08 [1.45] [1.04] Color: white 2226 31.63 2186 32.20 -0.58 [4.09] [3.89] Mother education/Legal guardian: at least high school degree 1347 52.64 1317 54.75 -2.11 [2.01] [2.05] Student found in Education Census 2015-2018 2376 91.20 2321 91.12 0.08 [1.66] [1.11] Student found in Prova Brasil 2013 2376 67.05 2321 66.52 0.52 [2.25] [1.91] Prova Brasil reading score in 5th grade 2013 1593 211.03 1544 209.28 1.75 [3.16] [2.68] Prova Brasil math score in 5th grade 2013 1593 226.11 1544 224.70 1.41 [4.10] [3.66] Student found in Prova Brasil 2017 2376 62.50 2321 61.83 0.67 [2.04] [1.88] Prova Brasil reading score in 9th grade 2017 1485 273.81 1435 274.74 -0.93 [2.58] [2.31] Prova Brasil math score in 9th grade 2017 1485 269.66 1435 271.07 -1.42 [3.48] [3.01] Ninth grade Beneficiary of Bolsa Família 1648 38.35 1604 35.79 2.56* [3.07] [2.59] Adequate age for their grade 1636 76.96 1602 76.09 0.86 [1.80] [2.30] Gender: male 2161 49.56 2174 47.10 2.46* [1.05] [1.08] Color: white 2074 32.64 2087 30.09 2.55 [4.22] [3.95] Mother education/Legal guardian: at least high school degree 1364 57.26 1335 57.00 0.25 [1.75] [1.77] Student found in Education Census 2015-2018 2222 92.26 2233 92.79 -0.53 [0.98] [0.84] Student found in Prova Brasil 2011 2222 63.77 2233 64.22 -0.45 [72] [2.24] [72] [2.06] Prova Brasil reading score in 5th grade 2011 1417 206.10 1434 200.39 5.71** [71] [3.07] [72] [2.78] Prova Brasil math score in 5th grade 2011 1417 221.42 1434 215.72 5.69 [71] [3.80] [72] [3.59] Student found in Prova Brasil 2015 2222 82.36 2233 81.10 1.26 [72] [1.31] [72] [1.45] Prova Brasil reading score in 9th grade 2015 1830 265.01 1811 264.14 0.87 [70] [2.27] [70] [1.92] Prova Brasil math score in 9th grade 2015 1830 264.88 1811 260.73 4.15* [70] [2.73] [70] [2.74] Graduate in High School | students 9grade 2222 59.63 2233 56.02 3.61 [72] [1.58] [72] [1.91] Student in ENEM 2018 | students 9grade 2222 46.17 2233 43.62 2.56 [72] [1.54] [72] [1.83] Nota padronizada média ENEM-2018 793 0.13 723 0.12 0.01 [71] [0.04] [71] [0.05] Notes: ***, **, * Statistically significant at 1, 5, and 10 percent, respectively. We run the balance test controlling for strata. All the variables shown in the table are from Prova Brasil, and Census of Education. 41 Table B.10: Pre-treatment balance test with administrative data at school level Joinville Manaus Beta Obs Beta Obs IDEB 9th grade (2013) -2.87 51 1.81 82 Grade-promotion 9th grade (2013) -2.33 66 0.72 79 IDEB 5th grade (2009) -1.86* 68 0.01 79 Reading performance 5th grade (2011) -0.14 62 0.26 97 Math performance 5th grade (2013) -0.47 51 -0.68 86 Math performance 5th grade (2009) -0.18 45 0.86 100 Math performance 5th grade (2011) -6.52 51 1.40 69 Grade-promotion 3rd grade (2009) 0.49 62 -0.36 91 Reading performance 9th grade (2011) -0.16 51 -0.69 77 Math performance 9th grade (2009) -0.81 45 -0.71 85 Grade-promotion 7th grade (2009) -0.53 51 -0.02 74 Grade-promotion 7th grade (2011) -4.81 59 0.41 74 Grade-promotion 5th grade (2009) 0.93 60 -2.25 88 Grade-promotion 5th grade (2011) -3.22 59 0.02 86 Grade-promotion 3rd grade (2011) 0.54 60 0.74 97 Age grade distortion- 7th grade (2013) -0.48 51 3.31 89 Grade-promotion 7th grade (2013) -7.53 45 0.00 86 Math performance 9th grade (2013) -3.85 62 -2.83 97 Grade-promotion 5th grade (2013) -0.08 59 -0.00 69 Grade-promotion 3rd grade (2013) -3.41 51 -0.48 86 IDEB 9th grade (2011) -2.12 45 1.19 86 IDEB 5th grade (2013) 0.65 61 -0.03 85 Reading performance 5th grade (2009) 0.37 59 -0.04 82 Math performance 9th grade (2011) 1.75 60 0.43 97 IDEB 9th grade (2009) -0.00 70 -0.04 79 Grade-promotion 9th grade (2011) -4.27 51 -2.55 85 % teachers in 1st to 5th grade with undergrad -0.20 51 -2.88** 70 Grade-promotion 9th grade (2009) -2.70 51 -0.92 90 IDEB 5th grade (2011) -1.31 45 -0.02 88 Reading performance 9th grade (2009) -0.12 59 -2.65 76 Reading performance 5th grade (2013) -5.59 62 2.50 70 Age grade distortion 5th grade (2013) -2.94 59 3.73 100 Age grade distortion 3rd grade (2013) -3.04 59 -2.06** 100 % teachers in 1st to 5th grade with undergrad -2.53 50 -0.03 97 Age grade distortion- 9th grade (2013) -1.04 51 -2.00* 74 Reading performance 9th grade (2013) -1.10 51 0.06 69 Note: *** p<0.01, ** p<0.05, * p<0.1 The table shows the coefficients of the dummy variable that identifies whether the selected schools are assigned to the treatment or control group. These coefficients are obtained from regressions of the above-listed covariates on the treatment dummy, the six strata dummies, and a constant. Clustered standard errors at the school level are in parentheses 42 Table B.11: Regression of the percentage of students’ characteristics that are missing on treatment status Third-grade Fifth-grade Seventh-grade Ninth-grade Beta Obs Beta Obs Beta Obs Beta Obs Beneficiary of Bolsa Família -0.01 4,650 -0.03 4,755 0.00 4,697 0.02 4,455 Access to electricity -0.00 4,650 -0.03 4,755 -0.02 4,697 0.02 4,455 Mother education: at least high school degree -0.01 4,650 -0.03 4,755 -0.02 4,697 0.02 4,455 Age grade distortion -0.00 4,650 -0.03 4,755 -0.02 4,697 0.02 4,455 Mother education: incomplete elementary school -0.01 4,650 -0.02 4,755 0.00 4,697 0.02 4,455 Lives on one paved street 0.00 4,650 -0.03 4,755 -0.02 4,697 0.01 4,455 Access to garbage collection 0.00 4,650 -0.03 4,755 -0.02 4,697 0.02 4,455 Mother education: incomplete high school -0.01 4,650 -0.03 4,755 -0.02 4,697 0.03 4,455 Access to piped water -0.00 4,650 -0.03 4,755 -0.02 4,697 0.01 4,455 Gender: male -0.03 4,755 -0.02 4,697 0.01 4,455 Color: white -0.03 4,755 0.00 4,697 0.02 4,455 Note: *** p<0.01, ** p<0.05, * p<0.1. The table shows the coefficients of a dummy variable that identifies whether the students are in the treatment or control group. The dependent variables are the percentage of missing observations for each one of the above-mentioned variables. These variables are regressed on the treatment dummy, the six strata dummies, and a constant. Clustered standard errors at the school level are in parentheses. Table B.12: Multivariate test of means Test Statistic F df1 df2 pvalue Third-grade Lawley .006 1.32 8 1694 .224 Third-grade Pillai .006 1.32 8 1694 .224 Third-grade Roy .006 1.32 8 1694 .224 Third-grade Wilks .993 1.32 8 1694 .224 Fifth-grade Lawley .004 1.04 10 2330 .401 Fifth-grade Pillai .004 1.04 10 2330 .401 Fifth-grade Roy .004 1.04 10 2330 .401 Fifth-grade Wilks .995 1.04 10 2330 .401 Seventh-grade Lawley .007 1.84 10 2452 .047 Seventh-grade Pillai .007 1.84 10 2452 .047 Seventh-grade Roy .007 1.84 10 2452 .047 Seventh-grade Wilks .992 1.84 10 2452 .047 Ninth-grade Lawley .005 1.33 10 2479 .208 Ninth-grade Pillai .005 1.33 10 2479 .208 Ninth-grade Roy .005 1.33 10 2479 .208 Ninth-grade Wilks .994 1.33 10 2479 .208 Note: The table shows the results of multiple-sample multivariate tests on means. We test whether the means of several students’ characteristics are the same between the treatment and control groups. For third-graders, we test the variables: lives on one paved street, access to electricity, access to piped water, a beneficiary of Bolsa Familia, age-grade distortion, and mothers’ education. For fifth, seventh, and ninth graders, we also include students’ sex and the color of their skin. 43 Table B.13: Average casual mediation effects POOLED Consumption Saving Talk to parents Talk to friends Piggy’s bank use Use of financial services Allowance Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Total effect .030 .014 .046 .019 .008 .030 .002 .000 .003 -.00 -.00 .000 .000 -.00 .001 -.00 -.00 -.00 -.00 -.00 .000 ADE .060 .025 .095 .028 -.00 .066 .012 -.00 .031 .036 .019 .054 .018 .000 .035 .007 -.00 .022 .017 .000 .033 ACME .090 .051 .127 .047 .006 .086 .014 -.00 .033 .036 .019 .053 .018 .000 .035 .006 -.00 .020 .017 .000 .033 of total .336 .238 .591 .394 .211 1.78 .139 -.97 1.30 -.01 -.02 -.01 .023 .010 .123 -.18 -2.6 2.94 -.01 -.08 -.00 effect mediated Elementary education students Consumption Saving Talk to parents Talk to friends Piggy’s bank use Use of financial services Allowance Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI 44 Total effect .020 -.00 .046 .014 -.00 .035 -.00 -.00 .000 -.00 -.00 .000 -.00 -.00 .000 .001 .000 .003 -.00 -.00 .000 ADE .107 .047 .170 .071 .007 .137 .059 .028 .092 .058 .032 .085 .015 -.01 .042 .022 -.00 .047 .005 -.01 .024 ACME .128 .064 .193 .085 .019 .153 .059 .028 .091 .055 .030 .082 .015 -.01 .042 .024 -.00 .049 .003 -.01 .023 of total .161 .107 .322 .165 .091 .649 -.00 -.01 -.00 -.04 -.07 -.02 -.00 -.01 .015 .070 -.29 .446 -.13 -2.5 2.37 effect mediated Middle school students Consumption Saving Talk to parents Talk to friends Piggy’s bank use Use of financial services Allowance Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Mean 95% CI Total effect .037 .016 .058 .023 .009 .037 .003 .001 .005 .001 .000 .002 -.00 -.00 -.00 -.00 -.00 .000 .000 -.00 .002 ADE .036 -.00 .081 .006 -.03 .056 -.01 -.03 .009 .008 -.01 .035 .011 -.00 .033 .020 .000 .043 .023 .001 .047 ACME .074 .028 .119 .030 -.01 .078 -.01 -.03 .012 .009 -.01 .036 .009 -.01 .030 .020 -.00 .043 .024 .002 .048 of total .518 .314 1.33 .661 -5.3 9.88 -.20 -3.2 5.69 .078 -1.5 1.33 -.16 -3.2 2.58 -.01 -.12 .009 .032 .016 .232 effect mediated Note: The dependent variables are equal to 1 if students talk to parents or friends about financial subjects, if they use a piggy bank, financial services, or receive an allowance from their parents, and 0 otherwise. Sensitive analysis run using the command medeff in Stata. Standard errors are estimated with bootstrap with 1000 repetitions. ACME: Average Casual Mediation Effect. ADE: Average Direct Effect. Table B.14: ITT estimates on Proficiency and Attitudinal Outcomes 7th grade 9th grade Strata FE Strata FE + Strata FE + Strata FE + Strata FE Strata FE + Strata FE + Strata FE + gender color of the skin performance gender color of the skin performance 5th grade 5th grade Proficiency Treatment 0.090* 0.090* 0.092* 0.098** 0.107* 0.102* 0.106* 0.149*** (0.048) (0.047) (0.048) (0.042) (0.056) (0.055) (0.055) (0.055) pvalue 0.061 0.057 0.055 0.022 0.057 0.066 0.056 0.008 Obs. 3640 3637 3622 2538 3374 3362 3346 2251 R2 0.208 0.224 0.209 0.505 0.168 0.183 0.170 0.399 Consumption index Treatment 0.080* 0.080** 0.078* 0.070* 0.084** 0.083** 0.084** 0.087* (0.041) (0.040) (0.041) (0.042) (0.037) (0.037) (0.037) (0.044) RI pvalue 0.054 0.046 0.060 0.098 0.025 0.027 0.025 0.050 Obs. 3409 3407 3394 2384 3163 3160 3149 2115 R2 0.1 0.119 0.1 0.187 0.055 0.061 0.054 0.128 45 Saving index Treatment 0.066 0.064 0.070* 0.033 0.018 0.017 0.017 0.030 (0.040) (0.040) (0.040) (0.044) (0.038) (0.038) (0.038) (0.045) RI pvalue 0.102 0.112 0.081 0.456 0.644 0.660 0.663 0.508 Obs. 3368 3365 3352 2351 3,105 3,102 3,093 2080 R2 0.032 0.032 0.032 0.065 0.019 0.020 0.020 0.044 Note: *** p<0.01, ** p<0.05, * p<0.1. Clustered standard errors at the school level are in parentheses. RI stands for Randomized Inference. The financial proficiency is computed by CAEd based on students’ answers on a standardized exam that used the Item Response Theory (IRT). The consumption and saving indices are based on students’ choices in questions containing four possible answers (I totally agree, I agree, I disagree, I totally disagree). The answers are classified on a 1 to 4 scale, where 4 represents the most forward-looking financial behavior. The financial proficiency, savings, and consumption indices are normalized so that the treatment effect could be measured in terms of standard deviations (SD). C Figures Figure C.1: Financial Proficiency Score and IDEB 700 700 Financial Proficiency Score, 2015 Financial Proficiency Score, 2015 600 600 500 500 400 46 400 300 4 5 6 7 8 9 3 4 5 6 7 IDEB of fifth graders, 2015 IDEB of ninth graders, 2015 (a) Fifth graders (b) Ninth graders Note: The Figure shows on the y-axis the financial proficiency score calculated by CAEd for the sample of pilot schools and on the x-axis the Education Development Index (IDEB) calculated by the Brazilian National Institute of Education and Research. IDEB is the most important educational indicator in Brazil. The index is calculated by multiplying students’ proficiency in Portuguese and Math (on a scale of 0 to 10) by grade promotion (on a scale of 0 to 1). Figure C.2: Quantile ITT estimates for financial proeficiency 0.30 0.25 0.20 0.15 Standard deviation 0.10 0.05 0.00 -0.05 -0.10 -0.15 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Quantiles of finacial proficiency (a) Pooled, elementary and middle school 0.30 0.25 0.20 0.15 Standard deviation 0.10 0.05 0.00 -0.05 -0.10 -0.15 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 Quantiles of finacial proficiency (b) Middle school Source: The financial proficiency is computed by CAEd based on student’s answers on a standardized exam that used the Item Response Theory (IRT). The index is normalized so that the treatment effect could be measured in terms of standard deviations (SD). We first run this financial proficiency score on the treatment dummy, the six strata dummies, and on a constant. We then take the residual of this estimation and run a simultaneous quantile regression, for each one of the quantiles shown in the Figure, in which the dependent variable is the residual of the first regression and the independent variable is the treatment dummy. 1000 bootstrap replications. 47 Figure C.3: Sensitivity analysis: average causal mediation effect versus , pooled (elementary and middle school) Consumption index, pooled Saving index, pooled .2 .2 .1 .1 ACME(ρ) ACME(ρ) 0 0 -.1 -.1 -.2 -.2 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Talk to parents, pooled Talk to friends, pooled .1 .1 .05 .05 ACME(ρ) ACME(ρ) 0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Piggy's bank use, pooled Use of finantial services, pooled .1 .1 .05 .05 ACME(ρ) ACME(ρ) 0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Allowance, pooled .1 .05 ACME(ρ) 0 -.05 -.1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Source: The dependent variables are equal to 1 if students talk to parents or friends about financial subjects, if they use a piggy bank, financial services, or receive an allowance from their parents, and 0 otherwise. Sensitive analysis run using the command medeff in Stata. Standard errors are estimated with bootstrap with 1000 repetitions. ACME: Average Casual Mediation Effect. 48 Figure C.4: Sensitivity analysis: average causal mediation effect versus , elementary education Consumption index, Primary education Saving index, Primary education .3 .3 .2 .2 ACME(ρ) ACME(ρ) .1 .1 0 0 -.1 -.1 -.2 -.2 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Talk to parents, Primary education Talk to friends, Primary education .1 .1 .05 .05 ACME(ρ) ACME(ρ) 0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Piggy's bank use, Primary education Use of finantial services, Primary education .1 .05 .05 ACME(ρ) ) ACME(ρ0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Allowance, Primary education .1 .05 ACME(ρ) 0 -.05 -.1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Source: The dependent variables are equal to 1 if students talk to parents or friends about financial subjects, if they use a piggy bank, financial services, or receive an allowance from their parents, and 0 otherwise. Sensitive analysis run using the command medeff in Stata. Standard errors are estimated with bootstrap with 1000 repetitions. ACME: Average Casual Mediation Effect. 49 Figure C.5: Sensitivity analysis: average causal mediation effect versus , middle school Consumption index, Middle school Saving index, Middle school .3 .3 .2 .2 ACME(ρ) ACME(ρ) .1 .1 0 0 -.1 -.1 -.2 -.2 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Talk to parents, Middle school Talk to friends, Middle school .1 .1 .05 .05 ACME(ρ) ACME(ρ) 0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Piggy's bank use, Middle school Use of finantial services, Middle school .1 .1 .05 .05 ACME(ρ) ACME(ρ) 0 0 -.05 -.05 -.1 -.1 -1 -.5 0 .5 1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Sensitivity parameter: ρ Allowance, Middle school .1 .05 ACME(ρ) 0 -.05 -.1 -1 -.5 0 .5 1 Sensitivity parameter: ρ Source: The dependent variables are equal to 1 if students talk to parents or friends about financial subjects, if they use a piggy bank, financial services, or receive an allowance from their parents, and 0 otherwise. Sensitive analysis run using the command medeff in Stata. Standard errors are estimated with bootstrap with 1000 repetitions. ACME: Average Casual Mediation Effect. 50