USING BEHAVIORAL SCIENCE IN COMMUNICATION OUTREACH TO INCREASE FEMALE PARTICIPATION IN NATURAL RESOURCE MANAGEMENT IN MEXICO 1 Acknowledgements This document was written by Graciela Reyes-Retana, Environmental Economist Consultant; Gonzalo Pons, Behavioral Sciences Consultant; Katharina Siegmann, Environmental Specialist; and Zeina Afif, Senior Social Scientist, all with the World Bank. The Climate Investment Funds (CIF) greatly contributed to this study, with financing, time, and intellectual support. The authors would like to thank Anne T. Kuriakose, Hanna Karima Schweitzer, Joseph Dickman, Ines Angulo, Ezgi Canpolat, and Eirini Maria Pitta of the CIF Administrative Unit for their support throughout the preparation of this study. The Forest Carbon Partnership Facility (FCPF) contributed to this study with financing as well. The team thanks Valeria Hickey, Practice Manager of the World Bank Group’s Environment and Natural Resources Global Practice; Ximena Del Carpio, Practice Manager with the World Bank’s Poverty and Equity Global Practice; and Loli Arribas-Banos, Practice Manager with the World Bank’s Poverty and Equity Global Practice, for their support and comments to the report; as well as Jutta Kern, Operations Manager of the World Bank Mexico Office; and Gregor Wolf from the World Bank Mexico Country Management Unit, for their support. Margarita Gómez García and her team from Centro de Investigación y Docencia Económicas (CIDE), which includes Carmen Elena Castaneda Farill, Marina Benitez Kanter, and Pablo Ignacio Soto Mota, were responsible for the preparation and implementation of the experiment. Hugo Cardenas, World Bank Consultant, supported the fieldwork in Yucatán and provided valuable cultural, environmental, and social insights regarding the field regions. Survey research was completed by the firm Sistemas de Inteligencia en Mercados y Opinión (SIM). The team thanks Rainforest Alliance México-Alianza para Bosques A.C. (Rainforest Alliance) for providing data, comments on ideas and drafts, and their overall support. This document was edited by Lauren Manning and Abigail Dalton. Disclaimer This publication was commissioned by the Climate Investment Funds (CIF). However, the views expressed do not necessarily represent those of the CIF or the World Bank. While reasonable efforts have been made to ensure that the contents of this publication are factually correct, the CIF and the World Bank do not take responsibility for the accuracy or completeness of its contents and shall not be liable for loss or damage that may be occasioned directly or indirectly through the use of, or reliance on, the contents of this publication. 2 Acronyms CIF Climate Investment Funds CONAFOR National Forestry Commission DGM Dedicated Grant Mechanism for Indigenous Peoples and Local Communities E&L Evaluation and Learning eMBeD World Bank Mind, Behavior, and Development Unit FCPF Forest Carbon Partnership Facility FIP Forest Investment Program NRM Natural Resource Management RA Rainforest Alliance RCT Randomized Controlled Trial REDD+ Reducing Emissions from Deforestation and Forest Degradation 3 Table of Contents 1. Executive Summary ........................................................................................................ 6 2. Introduction.................................................................................................................... 9 3. About this report .......................................................................................................... 10 4. Behavioral Diagnostic Study ......................................................................................... 12 5. Methodology ................................................................................................................ 13 5.1 Intervention and experimental design ............................................................................... 13 Control (T0): Official Strategy .................................................................................................................. 14 Treatment 1 (T1): Expanded and simplified communication strategy .................................................... 16 Treatment 2 (T2): Behaviorally informed communication strategy ........................................................ 16 Local Allies................................................................................................................................................ 17 T1 and T2 Communication Channels ....................................................................................................... 18 5.2 Data sources ..................................................................................................................... 23 5.3 Sampling ........................................................................................................................... 24 5.4 Randomization & Balance.................................................................................................. 28 5.5 Survey............................................................................................................................... 29 5.6 Analytical approach........................................................................................................... 29 6. Results.......................................................................................................................... 30 6.1 Implementation ................................................................................................................ 30 Official communication channels ............................................................................................................. 30 Experimental variations (Additions to T1 and T2) ................................................................................... 31 6.2 Locality level analysis ........................................................................................................ 35 6.3 Individual level analysis ..................................................................................................... 39 Survey sample balance ............................................................................................................................ 39 Main outcome .......................................................................................................................................... 40 Sample description .................................................................................................................................. 41 6.4 Exploratory application-level analysis ................................................................................ 43 7. Limitations ................................................................................................................... 48 8. Discussion, conclusions, and policy implications ............................................................ 49 9. References .................................................................................................................... 53 10. Appendices................................................................................................................ 54 Appendix I: Text/WhatsApp Messages .......................................................................................... 54 Appendix II: Omnibus balance tests .............................................................................................. 58 Appendix III: Regression tables without covariates ........................................................................ 60 4 Appendix IV: No application evaluated vs. At least one application evaluated: Only localities from outside experimental sample........................................................................................................ 61 5 1. Executive Summary While a range of public programs in Mexico exist to incentivize communities to conserve and manage forest natural resources, a gender gap persists in the use of these initiatives. The experiment discussed in this report was commissioned by the Climate Investment Funds’ (CIF) Evaluation and Learning (E&L) Initiative to understand how to improve outreach to and encourage women to engage in productive natural resource management (NRM) programs. Following an earlier behavioral diagnostic study1, the World Bank and local partners conducted a Randomized Controlled Trial (RCT) to assess the effectiveness of behaviorally informed additional outreach and engagement measures for NRM programs in Mexico. The evaluation assesssed the impact of these additional measures in increasing the number of applications to a special call for proposals from the Dedicated Grant Mechanism for Indigenous Peoples and Local Communities (DGM) Forest Investment Program (FIP), a World Bank project delivered by Rainforest Alliance. The call specifically targeted women to implement activities related to Reducing Emissions from Deforestation and Forest Degradation� (REDD+). An increase in applications would suggest that there are ways to support potential benficiaries to overcome contextual and psychological barriers that kept them from applying to programs and from acting on their intentions to participate. A sample of 113 localities in Yucatán and Oaxaca states were randomly assigned to receive one of three communication strategies: a control group and two treatments2. The control group (T0) received the existing and official communication strategy about the request for applications. This was delivered by Rainforest Alliance, the national executing agency for the project, and already included a series of innovations to made the application process more attractive to women. The first treatment (T1) received the same official strategy, with additional communication activities, consisting of expanded communication channels, and simplified messages and processes.3 The second treatment (T2) received T0 and T1 activities, as well as modified and additional communication messages informed by behavioral science principles.4 The expanded communication activities for both treatment groups included a phone-based helpline, deployment of trusted local allies to share information and guidance, and community meetings to support at all stages of the application process. This complements responded to informational and behavioral barriers for women identified during the previous behavioral diagnostic study, such as lack of information or missinformation and complexity about the 1 http://pubdocs.worldbank.org/en/243311534947810543/Mexico-Forestry-Report-V14-DIGITAL.pdf 2 Note that these 113 localities were a subset of the localities that were able to participate in this call for proposals. Hence, the applications analyzed in this RCT are not all applications received as part of the call for proposals. 3 The additions of T1 and T2 were implemented by a private third party rather than the government. 4 T2 localities received differentiated behaviorally informed posters that used identity priming, calls to action, positive social norms to encourage participation, motivational reminders, proactive text messages, and a checklist. 6 process, low aspirations, limited agenecy and self-efficacy caused by traditional gender norms, distrust and intention to action gap. This experimental method allowed for the causal identification of the impact of the expansion of communication channels and simplification of the process, as well as the behaviorally- informed messaging. Outcomes were captured from administrative data of all submitted applications (from men and women), as well as a representative survey conducted among only local women. The additional measures to the official communication strategy had a strong effect on the number of applications submitted. While control group localities (T0) submitted few applications (only one was submitted from all 36 T0 localities), the treatment localities (T1 – 37 localities, T2 - 40) sent, on average, around 2.7 applications per locality5. The evidence suggests that the largest effect on the number of applications can be attributed to the expanded communication channels and simplified process. While T2 had slightly higher average number of applications than T1, this was not statistically significant, suggesting the components of T1 were the most impactful for this outcome. However, the messages informed by behavioral science showed a positive complementary effect on increasing the number of women applying as part of teams in these applications (T2 was significantly higher than T1 in this outcome). Survey results also suggest that women in T1 and T2 (compared to T0) were more likely to have heard of the call for proposals or to have recognized the posters deployed for their specific experimental group as part of the communication strategy. Nevertheless, only about 23 percent of surveyed women heard of the call for proposals or recognized the posters, suggesting the campaign’s reach was relatively low. Despite the increase in application numbers following the treatments that helped applicants overcome contextual and psychological barriers to applying, exploratory analysis suggests that additional support is required to translate this increase in the number of applications and recognition of the program into proposals strong enough to meet minimum technical criteria and actually win grant awards. None of the additional measures positively impacted the number of applications per locality that successfully passed the program’s first technical evaluation stage or were ultimately selected for the grant. While the research team can hypothesize the reasons for this, further research is needed to identify the type of programming that would help increase the technical quality of the applications. Ultimately, this experiment shows that increasing the number of communication channels, simplifying the application process, and using behaviorally informed messaging to address barriers such as beliefs and social norms can successfully increase the number of women that express concrete interest in productive natural resoure management. While additional assistance might be needed to combat the other barriers preventing women and other members from these localities from participating and winning grants, it demonstrates that target populations (i.e., women in rural areas) are interested in participating in productive are number of applications overall, not only from women. An application can include a team with both men 5 These and women. 7 natural resource management programs and seeking opportunities to access the needed funds to participate in these activities. 8 2. Introduction Mexico's forests contribute to both rural and urban economies through the provision of goods such as timber and value-added products, and important ecosystem services. These natural habitats support biodiversity, mitigate erosion, and help maintain soil fertility, water filtration, and the supply of raw material for vital, productive sectors. However, the sustainability of Mexico’s forests is threatened by socioeconomic stresses, unsustainable agricultural practices, and climate change. Drought, irregular rainfall, deforestation, and the degradation of natural resources all increase the risk of natural disasters and the loss of biodiversity. Many of the people living in these forests are poor and vulnerable: 88 percent of Mexico’s 12 million forest dwellers live in highly marginalized localities, and 62 percent live below the poverty line (INEGI, 2009; DOF, 2016). As a result, Mexico’s “Reducing Emissions from Deforestation and Forest Degradation� (REDD+) Strategy aims to conserve and manage natural resources while generating additional benefits for individuals and communities (World Bank Group, 2018). The varied effects of climate change also make women more vulnerable than men. Social, political, and economic inequalities result in differences in men’s and women’s adaptive capacity and resources to face climate change. Moreover, women and men use natural resources differently, and climate change further impacts access, use, and control over resources (World Bank Group, 2018). There is also a gender gap in the use of public incentives under Mexico’s REDD+ strategy. Of all subsidies allocated to individuals by the National Forestry Commission (CONAFOR) in 2016, only 25 percent were assigned to women (CONAFOR, 2017). Myriad structural barriers limit women’s involvement in forest sector programs. These institutional, legal, and economic barriers include longer working hours, poor public infrastructure to travel to submit official documents, weaker social status, poverty, lack of land tenure, and differences in health and educational attainment as compared to men (World Bank, 2018). For example, limited or lacking land rights and tenure prevent women from participating in most forest programs and community assemblies where REDD+ information is disseminated (Méndez López et al., 2015). Prevalent social norms and gender roles also impede women. Though women participate less in formal economic activities (38 percent of economically productive activity in Mexico is undertaken by women versus 62 percent by men), they are more likely to experience time poverty due to their range of domestic, informal, and unpaid activities (World Bank, 2018). Although the structural challenges that prevent women from participating in REDD+ initiatives have been identified in previous research, there is incomplete evidence on the social and psychological aspects associated with gender differences. Integrating a behavioral analysis can help uncover key psychological, cultural, social, and material restrictions on women’s participation. In recent years, behavioral sciences have provided a complementary lens to explain public policy challenges beyond structural barriers. Behavioral barriers—often unconscious and driven by biases, social norms, and mental models—are now increasingly 9 taken into consideration in the design and implementation of public programs in over 100 countries (World Bank, 2019). How a program is designed and implemented can disproportionately affect individual choices and actions. Identifying decision-making contexts influenced by mental shortcuts (i.e., thinking automatically), psychological biases, and mental models (i.e., predefined views and interpretation of ourselves and the world around us) allows the capture of psychological, social, cognitive, contextual, and external factors (World Bank, 2015). This can provide insight for better program design, especially in the context of rural communities where incentives are often driven by social and cultural norms. Thus behavioral science can provide interesting insights into the complex world that women live in—one governed by traditional role models, gender biases, stereotypes, self-concept, and the status quo. 3. About this report This report summarizes the findings of a field experiment commissioned by the Climate Investment Funds (CIF) Evaluation and Learning (E&L) Initiative, with additional financing from the Forest Carbon Partnership Facility (FCPF). The experiment was designed to help identify promising strategies to improve outreach to women in order to encourage them to engage in productive natural resource management programs. CIF was founded with the mandate to serve as a learning laboratory for scaled-up climate finance efforts. The CIF E&L Initiative is helping fulfill this mandate through strategic and demand-driven evaluations covering pressing challenges facing climate finance funders and practitioners. Drawing on experience from across CIF investments in clean energy, forests, and climate resilience in 72 developing countries, the E&L Initiative enables relevant, timely learning to inform decisions and strategies for CIF and the wider climate finance sector. The Forest Investment Program (FIP) is a CIF program designed to provide fast-track climate financing to reduce deforestation and forest degradation in tropical countries. The FIP Dedicated Grant Mechanism for Indigenous Peoples and Local Communities (DGM) is the largest global REDD+ initiative created solely for and by indigenous peoples and local communities. It is designed to enhance their capacity to engage in and contribute to local, national, and international REDD+ dialogue and to ensure sustainable forest-use practices led by indigenous peoples and local communities are supported, shared, and elevated to the global policy arena. FCPF is a global partnership of governments, businesses, civil society, and indigenous people's organizations focused on reducing emissions from deforestation and forest degradation, forest carbon stock conservation, the sustainable management of forests, and the enhancement of forest carbon stocks in developing countries. These activities are commonly referred to as REDD+. 10 In Mexico, a National Steering Committee representing indigenous peoples and local communities selected Rainforest Alliance México-Alianza para Bosques A.C. (Rainforest Alliance) as the National Executing Agency for the Mexico DGM. The Mexico DGM is a US$6 million project financed by FIP through the World Bank, from October 2017 to August 2022. The Project Development Objective (PDO) is to strengthen the capacity of forest-dependent people from selected states to participate in local, national, and international REDD+ related processes. Sub-projects in Mexico focus on conservation and integration of sustainable, productive activities into forest landscape management to improve community livelihoods, increase community participation in forest landscape management, support climate change mitigation and adaptation, and encourage the participation of women, youth, and those without land tenure. A dedicated sub-grant window seeks to foster the role of women in natural resource management and to ensure their access to finance and improvement of their livelihoods. We used that window to conduct the analysis described in this report. This dedicated sub-grant window addressed a series of structural barriers that commonly prevented women from being able to apply to forest sector programs. For example, land ownership was removed as a requirement to apply to this window, and women were given opportunities to share their project proposal with their respective communities based on an inicial endorsment. Thus, while communal support is always needed in order to implement activities and receive grants, it was no longer a pre-requisite for the application, allowing for a proposal to be pre-selected for funding before having to get the community´s final approval. Finally, acknowledging that fewer women had experience with these type of applications and therefore might have lower capacities, the application process was implemented in two phases. The first phase required a description of the intended project and documentation to back the legality of members. Although this required several details of the objectives and plans, it required less technical and methodological details than typical full proposals. Those that were selected for the second phase were then supported by the regional teams of Rainforest Alliance to further develop their proposals with all the usual technical details. All proposals selected in phase one were expected to be funded at the end of phase two, unless they dropped out, or similar. It is important to note, however, that the results of this report focus exclusively on phase one. Even though the window already addressed several structural barriers in the design that would usually prevent women from applying to the DGM and more broadly to other forest sector programs, a behavioral diagnostic study also uncovered a series of behavioral barriers that, if addressed as well, may further increase the involvement of women and typically underserved populations in productive REDD+ activities. Considering this, the team designed a Randomized Controlled Trial (RCT) to attempt to answer the following specific questions: 1. What is the impact of using simplified information and expanded communication channels on the number of women that apply for grants to participate in productive natural resource management programs? 2. What is the impact of messages informed by behavioral sciences (i.e., leveraging identity of participants, social norms, reminders, and principles of loss aversion) on the number of 11 women that apply for grants to participate in productive natural resource management programs? We also explore how outreach strategies informed by behavioral science may help achieve the important goal of increasing uptake by women of productive natural resource management programs. First, this report presents the methodology used for the intervention and experimental design. Then, it provides an analysis of the results at the locality and individual level. Finally, it provides conclusions and policy recommendations. 4. Behavioral Diagnostic Study The theoretical framework and design for this research were informed by the findings of a behavioral diagnostic study, titled “Closing the Gender Gap in Natural Resource Management Programs in Mexico,� commissioned by the FCPF in 2017 in Mexico (referenced in this document as the "behavioral diagnostic study"). The objective of that study was to understand the behavioral barriers, women's decision-making processes, motivations, influencing agents, and other factors women or men consider when deciding whether to participate in REDD+ related activities. Qualitative fieldwork for the study was undertaken in June and July 2017 in 31 communities in Yucatán and Oaxaca, Mexico. Three fieldwork methods were used, namely: women's and men's focus groups to learn about points of view, biases, perception, and norms; deep interviews with women and men—including influencers and community leaders—that informed case studies from different perspectives; and field observations to better understand context and behaviors. In total, 16 focus groups were held, 25 interviews completed, and 30 documents with field observations drafted. The total qualitative sample included 228 people from 15 localities. A literature review on behavioral science and the behavioral diagnostic study pointed to three overarching barriers that rural women face regarding their participation in productive NRM activities. These are: (i) scarcity of time and choices, given household chores and the limited activities they can take part of as a result of them, (ii) gender norms and stereotypes that default women to traditional labor roles, limiting their motivation, self-efficacy, self-advocacy, and aspiration to participate in new activities, and (iii) contextual barriers such as poverty, material limitations, and process complexity. One of the main conclusions from the study was the importance of improved choice architecture for rural women. When confronted with complex data, people tend to make more uninformed or irrational decisions than when information is clearly presented. In fact, one’s expectation that a decision-making process will require effort leads to the use of heuristics— increasing biases, low cognitive performance, or deterring decision-making altogether. But improving choice architecture can address barriers such as accessibility, program outreach, transportation, and illiteracy. This is true for both men and women, but it is especially important for individuals that are more constrained or less motivated to perform the behavior, which, in this context, given the afromentioned barriers, tend to be women in the REDD+ 12 context. Appropriate and simple choice architecture could encourage rural women to sign up for productivity programs. Another important finding of the behavioral diagnostic study was the importance of agents of change, such as communal leaders or role models, in building engagement among women and men. The behavioral diagnostic study demonstrated the feasibility of addressing both behavioral and structural barriers in tandem in the design and implementation of REDD+ related activities. Access to information, program take-up, and retention can all be improved with behavioral strategies, thereby increasing women's motivation and participation in forestry, agricultural, or other REDD+-related activities. These findings served as an input for the design of the Randomized Controlled Trial presented in this report. 5. Methodology This report summarizes the findings of the Randomized Controlled Trial (RCT) implemented in a sample of 113 localities in the states of Oaxaca and Yucatán, Mexico. Those states were selected since both are pilot states for FIP and FCPF and represent different ecosystems, i.e., those with tropical versus deciduous forests. To compare the effects of the interventions, localities in each state were randomly assigned to three different treatment groups. Both the qualitative background analysis and this study were designed and executed with the National Lab for Public Policies at the Mexican Center for Research and Teaching in Economics (CIDE). 5.1 Intervention and experimental design This study utilized a RCT to compare three types of outreach strategies, testing whether behaviorally informed additional measures to the project outreach strategy had a causal effect on the number of grant applications, as well as on the number of women applying. Additional measures included: 1) The expansion of communication channels and simplification of messages and processes 2) The use of messages informed by behavioral science principles Even though many different principles were used to complement the project outreach strategy, there were not enough resources to test all possible combinations. Hence, the team chose an incremental approach, staggering the additional measures across the treatment groups, as shown below. Table 1 summarizes the combination of principles used to augment each experimental group. 13 Table 1. Main components in each experimental group. Control (T0): Project Official Strategy The control group (T0) received the official dissemination from Rainforest Alliance for the DGM-Mexico project. This outreach was already innovative, as it targetted marginalized groups that are usually under-represented in community decision-making bodies and related fora. Rainforest Alliance designed an inclusive communication strategy featuring dissemination through partner organizations, radio, and outreach with community leaders. The strategy included posters (see Figure 1); brochures (see Figure 2); meetings in central localities; information on the DGM website; social networking;6 and engagement through civil society organizations and universities. Meetings took place in central localities to which people in surrounding localities could commute. These meetings presented the DGM program, the characteristics of the dedicated sub-grant window, the phases and eligibility criterias, and the requirements for the application. They also left ample time at the end for questions. Communal leaders from localities of the area 6Communication was deployed on social networking sites including the Official DGM-Mexico Facebook page (https://www.facebook.com/MDEBosquesMexico/) and official digital networks from other civil society organizations and universities. 14 were invited and encouraged to also invite women from their localities. After the meeting, attendees were given posters and brochures to distribute in their localities. Likewise, attendees sometimes collected contact information of the officials conducting the meetings, which allowed them to call and send messages with questions. Since approval from the community was needed to receive final funding for this call for proposals (although not to apply)7, the Rainforest Alliance outreach focused on targeting local governance through communal leaders, so they could liaise with their communities to participate. The expectations was that this approach would encourage a more participatory development of the applications, involving the entire community. This outreach effort was equivalent in all three treatment groups of localities. However, localities in T0 received only this information, and not the activities described in the following sections. Figure 1. Poster T0 Figure 2. Brochure T0 7 This is not a unique requierement of this project but of any project in forest landscapes in Mexico. As land is communical the communal assembly needs to endorse all activities implemented in their land. This is specifically challanging for women as they are mostly not represented in assemblies and hence can not easily present and defend their project proposals. 15 Treatment 1 (T1): Expanded and simplified communication strategy Treatment Group 1 (T1) received the same official communication as T0, plus simplified information and formatting of the call for proposals. People in the T1 localities who received the information needed to understand the requirements, benefits, and process of the call for proposals. As the behavioral diagnostic study showed the importance of simplification and reducing the ‘hassle factor’ that people faced when attempting to access information, this information was presented using familiar and easy-to-understand terms, and was distributed via diverse communication channels in order to better reach potential applicants. However, the content transmitted did not differ from the one distributed to T0. Information was delivered through expanded communication channels. Some examples used in T1 included: (i) posters with simplified information (Figure 3), (ii) an informative meeting and follow up with local allies, (iii) audio-messages in public spaces explaining the steps to participate in the program, and (iv) a helpline to answer general questions. More details about these mechanisms are presented below. All materials and information used in the intervention were tested in the field prior to finalizing them. Treatment 2 (T2): Communication strategy with messages informed by Behavioral Sciences Treatment Group 2 (T2) received the same elements as T1, which already included the official dissemination of T0. In addition, they received: (i) posters with behavioral messages to address psychological bottlenecks including identity, social norms,8 and loss aversion9 (Figure 4); (ii) SMS/WhatsApp messages with reminders about the call for proposals; and (iii) a checklist to help interested individuals complete the application process (Figure 5). We can interpret the differences in outcomes between T1 versus T0 groups as the effect of simplification and expanded communication mechanisms, and T2 versus T1 as the added effect of including messages based on behavioral sciences principles and proactive text messaging. Table 2 highlights the difference in communication channels between T1 and T2. 8 The poster images and verbal messages normalized the participation of women in activities outside of the home, setting an example of accepted behavior. 9 Posters and some text messages used loss aversion framing by telling possible participants not to miss out on the opportunity to participate. 16 Table 2. Differences in communication channels between T1 and T2 CHANNELS T1 T2 With simplified information on the steps to With simplified information on the steps to participate and calls to action participate and calls to action Posters and + messages informed by Behavioral brochures Sciences (constructed identity, social norms, loss aversion) Brochures were shared by local authorities Brochures were shared by local authorities. Meeting held by local allies to give the general Meeting held by local allies gives the information of the call in which the simplified Informative general information of the call in which the formats are delivered + a checklist with all meeting simplified formats are delivered the steps to follow and the dates to carry them out Meeting held by local allies to check the applications were complete and signed and to collect Second meeting them Audio messages Homogeneous message about the call, the requirements, and steps to participate To answer general questions about the call Helpline: + to send reminders about their progress To answer general questions about the call in the application process and encouraging SMS/WhatsApp → Reactive response them to engage. Messages informed by messages Behavioral Sciences. → Reactive and proactive Local Allies The behavioral diagnostic study revealed the importance of having a field-based point of contact to build trust with potential applicants. With the help of local civil society organizations, we identified and assigned 13 local allies to a cluster of localities. Allies were not related to the government but did have previous experience working in the region. The advantages of this approach are that their services are not as expensive as traditional agricultural extension agents, they had already established trust among local populations and authorities, and their familiarity allowed them to play an active and key role before and during implementation. However, local allies never provided technical assistance, they also did not guide groups on technical elements of their proposal. Their role was simply limited to support outreach and build trust. Among their many activities, local allies: • Visited the localities and became the first point of contact with the local authorities; 17 • Strategically placed posters announcing the program and delivered information materials to the local authorities; • Delivered the recorded audio message to be transmitted through public speakers in the locality; • Organized and delivered information sessions about the program and requirements; • Retrieved contact and basic demographic information of potential applicants interested in receiving more information and support; • Encouraged potential applicants to seek advice and information using the helpline; • Collected the applications with all the required documentation. These activities, and related communication channels, are outlined in the following section. T1 and T2 Communication Channels Information meetings Local allies organized meetings to deliver information about the program and the application process, including requirements and important deadlines. The meetings were held in places that were easily accessible to women, such as buildings belonging to the agrarian or non-agrarian authorities or in schools. Attendees also received the basic printed materials that they would need in order to complete the application process. These meetings were held between 22 November and 22 December 2018. Local allies also visited some localities between 23 February and 6 March 2019 for a second series of meetings. They confirmed and collected completed applications before delivering them to provisional offices established by Rainforest Alliance in Oaxaca and Yucatán between 23 and 29 March 2019. For those who could not attend the meeting or had not yet completed their applications at this time, some applications were received as photographs through WhatsApp and delivered to Rainforest Alliance electronically.10 Posters Informational posters were displayed in pairs in sites frequented by women. Using strategic sites— schools, clinics, agrarian offices, municipal buildings, Diconsa stores11, tortillerías or mills, kiosks, community centers, and drinking water stations—ensured that women saw the posters frequently. One poster contained general information about the call for applications. A second shared information about the first meeting and outlined the steps for applying for funds. Both displayed contact information for the helpline. To increase trust, the second poster included the photograph and name of the person in charge of the helpline so applicants would know who they would be speaking to. Posters for both T1 (Figure 3) and T2 (Figure 4) provided simple information about the DGM and invited people to attend the meeting and call the helpline. Posters for T2 presented information in a way that reduced the influence of behavioral barriers by leveraging the identities of potential 10 61 applications were submitted electronically by the team runing the RCT. 11 Community-owned stores that sell food and other products. 18 participants, engaging their loss aversion bias, and using social norms. For example, as seen in Figure 4, the language used in the poster for T2 suggested there were already women working in these activities and that more should join them, actively reaching out to women and challenging the idea that this is uncommon or unacceptable. It also leverages identities of motherhood, suggesting that working in these activities can bring better lives to their children, which was found in the diagnostic study as both a common motivation and worry. Figure 3: Posters for T1 localities Figure 4: Posters for T2 localities 19 Audio message An audio message contained general information about the program and other channels of communication, such as the posters and meetings. These messages were disseminated only through the local loudspeakers in the community, rather than through a private service. To avoid possible contamination between localities in different treatment groups inside the same municipality, the message was the same for both T1 and T2 groups. Checklist In addition to the simplified forms received during the information meetings, people in T2 localities received a detailed checklist outlining important deadlines and the steps they needed to follow to develop their proposal and complete their application (Figure 5). Figure 5. Checklist received by attendees of the T2 information meeting 20 Helpline, Phone calls, SMS and WhatsApp The helpline (set up in addition to the already established Rainforest Alliance helpline), staffed Monday to Friday from 9:00 to 17:00, was dedicated to answering calls and sending and responding to messages from people in both T1 and T2 localities: • Phone calls: For both groups, the phone line was available in a reactive way. Individuals could receive help filling out their application forms if they called the line directly. • SMS and WhatsApp: In T1 localities, communication was again reactive. Text message support was responsive to initial contact from a potential participant. After opting in, interested people also received four messages with information about the second series of meetings and the submission process for their applications. In T2 locations, people opted in to receive communication by providing a cell phone number at the first informational meeting or by using the helpline. To increase the agency of women and reduce the possibility that even those interested would not participate, people in T2 localities that demonstrated interest received proactive SMS or WhatsApp messages with reminders connected to the checklist (Figure 5) and with information about their progress in the application process. They also received encouraging messages (Figure 6) informed by behavioral sciences (identity, social norms, call to action, and loss aversion) to motivate them to complete their application. Figure 6. Example of messages sent to T2 21 Local authorities and brochures Local authorities were provided with an information kit so that they could answer basic questions and a brochure (Figure 7 and 8) to give to the those requesting information about the program. The authorities were also given a form to register the names and telephone numbers of those asking for information. Local allies collected these forms when they returned to localities for the second information meeting. Figure 7. Brochure T1 Figure 8. Brochure T2 22 Each of the above measures were designed to directly respond to the behavioral bottlenecks identified (Figure 9) during the behavioral diagnostic study. Figure 9. Behavioral bottlenecks and response in the experiment BEHAVIORAL BOTTLENECKS RESPONSE IN THE INTERVENTIONS Messenger: Local allies, local 1. Distrust authorities, and showing the name of the person in charge of the helpline More communication channels: 2. Lack of Information and posters, meetings, audio messages, misinformation local authorities, and helpline 3. Complexity of the process Simplification and reduction of hassle and friction costs factors (call to action) Identity priming, harness social 4. Low aspirations, limited influence, challenge gender roles, and agency and self-efficacy actively reach out to women as a group caused by traditional gender to highlight their relevance in the norms and roles program 5. Intention to action gap Reminders and checklist 5.2 Data sources First, the team used administrative data from the Instituto Nacional de Estadística, Geografía e Informática (INEGI) and CONAFOR to extract the socio-demographic variables of localities. This data is primarily from the INEGI Population and Household Census of 2010, INEGI Land Use and Vegetation Series VI 2016, National Population Commission (CONAPO) marginalization index by 23 locality 2010, and a private dataset of phone coverage provided by CIDE. Outcome data was primarily extracted from the application forms received by the Rainforest Alliance. Finally, the team performed a representative survey to gather individual-level data of women in the selected localities (See Section 5.5). 5.3 Sampling The experiment took place in two of the five Mexican states where the DGM project is implemented: Yucatán and Oaxaca.12 The team chose a sample of 113 localities13 with characteristics relevant to the program and experiment, such as the presence of forests. From the 25,631 localities in the official catalog for these states, 12,880 had locality-level socio- demographic data available in the census dataset.14 The team applied the filters outlined in Table 3 to those with available information as part of the sampling process.15 A convenience sample was taken to reach 113 localities, accounting for feasibility considerations of the local implementer. Table 3. Sampling process and localities remaining in each step. Sampling step Oaxaca Yucatán Total Official list (INEGI census 2010) 13,936 11,695 25,631 Available locality level data (INEGI census 2010) 10,418 2,462 12,880 In agrarian unit with forest coverage (INEGI Land use and 636 238 874 Vegetation 2017) Without land conflict (INEGI census 2010) 542 178 720 With GSM cellphone coverage (private data) 207 115 322 10 adult women (18+) or more in a locality (INEGI census 2010 – also excludes those without available data on 124 39 163 women population) Convenience reduction because of logistical limitations 82 31 113 Table 4 uses 2010 census data to compare all localities in Oaxaca and Yucatán versus the localities sampled for the study. The team also included a comparison with rural localities in the states that have forest coverage. An important caveat of this analysis is that the census does not include 12The project is also implemented in the states of Campeche, Quintana Roo, and Jalisco. 13Localities are the smallest census units in Mexico. They vary widely in size and demographic characteristics. In the sample, the smallest had an adult population of 16 and the largest of 552, although localities can have many thousands. Agrarian units (Nucleos agrarios) can be comunidades or ejidos. Comunidades are long-standing rural population centers with formal ownership of their traditional or customary lands. Ejido refers to a portion of land titled to a rural population nucleus that was formed more recently or relocated from another area. Community members with full ownership rights are legally recognized as ejidatarios or comuneros. Both types of community groups are governed by a similar structure. 14 The localities not included in the census have no population or have populations larger than 4,999 inhabitants. 15 For example, localities that are not part of an agrarian unit are dropped when looking for forest coverage. Also, localities without data for the number of women are dropped when looking for adult women (The census does not include subgroup information of the population for localities with one or two houses). 24 subgroup information of the localities’ population (e.g., the population of men versus women, population that speaks the local indigenous language) for localities with only one or two houses. This affects 30 percent of the localities of the selected states. In the census data, these localities appear as missing values when subgroup information is used as variables for comparison. Additionally, filtering for localities with more than ten women excluded any locality without this information. Localities in the sample are, on average, larger in terms of inhabited houses and population than rural localities with forest coverage. This is likely due to the aformentioned exclusion of localities with one or two houses. Those in the sample are also marginally smaller than the states’ average size, but not significantly. In addition, localities in the sample are similar to rural localities with forest coverage – both feature agriculture as the main economic activity and a similar proportion of people above three years of age speaking indigenous languages.16 These characteristics are less prominent in the selected states as a whole. The sample is slightly more educated than other rural localities with forest coverage, and marginally lower than the state, although these differences are not statistically significant. According to available demographic variables, females make up about half of the population in the localities, on average, for all comparison groups. But localities in the sample have fewer women- headed households than the state average, and about the same as rural localities with forest coverage. Finally, for localities with data on the marginalization index of 201017, we show descriptive statistics in Table 4 using the official groups “Very High� and “High� based on official cutoffs of the continuous index. Most localities in the sample are marginalized, with 68% in the “High� group and 29% in the “Very High� group. We can also assess the differences between localities using the continuous marginalization index: The sample localities (M =.36, SD = .78) are, on average, less marginalized than other rural localities with forest coverage (M =.72, SD = .87); but virtually equally marginalized with those across the state (M =.33, SD = .85). Another important aspect of the sample is the number of grants awarded from CONAFOR prior to this new call for proposals. Localities in the sample received on average 1.48 (min = 0, max = 22) forest grants from CONAFOR between 2007-2017. However, 67 percent of the sampled localities did not receive any grants, indicating inexperience with these processes18. Unfortunately, we do not have access to this indicator for localities outside of the sample. Nevertheless, this figure highlights how this experiment targeted localities that typically had not benefited from these grants before. 16 Localities in these two states had, according to the 2010 census, at least 27 indigenous languages spoken. Our sample had at least 10 of these indigenuous languages. Rural localities with forest coverage had at least 15. Nevertheless, in terms of proportion of people speaking indiginous languages, the sampled localities had a higher average proportion of people speaking an indigenous language (0.62) than the average locality in these two states (0.45) . Rural localities with forest coverage had 0.59, which is statistically equivalent to our sampled localities. 17 CONAPO (2010) Indice de marginalización por localidad 2010. Taken from http://www.conapo.gob.mx/es/CONAPO/Indice_de_Marginacion_por_Localidad_2010 The marginalization index combines a series of socio-economic indicators: percentage of population 15 years or older illiterate or without completing primary education, precentage of inhabited houses without a toilet, electricity, refrigerator or piped water, or with dirt floor, and average number of inhabitants per room. 18 14% of localities had received 1 grant, 16% between 2-10, and 2.6% had received 17 or more. 25 In terms of external validity, the differences between the localities suggest that the results of this experiment may not be applied directly to the entire state, or even other rural localities with forest coverage, without some adaptation. 26 Table 4. Comparison of study sample vs. rural localities with forest coverage vs. all localities in the states. Experimental Sample Rural Localities with Forest Coverage All localities in states N Mean or % SD Min Max N Mean or % SD Min Max N Mean or % SD Min Max Total inhabited 113 37.96 34.88 5 209 870 20.69 35.69 1 309 12,880 53.73 124.32 1 1296 houses Total population 113 172.22 173.38 28 1230 870 92.67 163.25 1 1513 12,880 221.46 501.75 1 4939 Agriculture as main economic activity 112 89.29% 780 91.03% 11,641 78.76% (dichotomic) With population that speaks indigenous 113 76.11% 794 75.94% 11,826 61.19% language (dichotomic) Without information of subgroups in locality (less than 3 113 0.00% 870 30.57% 12,880 31.23% households in locality – dichotomic) Proportion female 113 0.5 0.04 0.39 0.61 604 0.5 0.07 0 0.78 8,857 0.51 0.06 0 1 Proportion of female-headed 113 0.16 0.12 0 0.5 604 0.15 0.13 0 0.75 8,839 0.20 0.13 0 1 households Proportion that speaks indigenous 113 0.62 0.41 0 1 604 0.59 0.4 0 1 8,839 0.45 0.41 0 1 language for 3 years or older Average grades of schooling for 15 113 4.89 1.31 1.88 10.56 604 4.7 1.43 0 11.47 8,839 5.09 1.67 0 18.23 years or older Marginalization index = Very High 113 29.20% 604 42.05% 8,839 28.72% (dichotomic) Marginalization index = High 113 68.14% 604 56.62% 8,839 65.79% (dichotomic) 27 5.4 Randomization & Balance Randomization was completed in multiple steps. First, a k-mean algorithm was implemented to create 10 clusters of localities based on four characteristics: i) proportion of women aged three years or older who spoke an indigenous language in 2010; ii) marginalization index by locality in 2010; iii) number of CONAFOR grants for forest activities received by the locality between 2007 and 2017; and iv) proportion of adult women out of the total adult population of the locality in 2010. Second, one-third of each cluster was randomly assigned to each of the three experimental groups, ensuring a more balanced sample. Finally, localities that were part of an agrarian unit that was comprised of localities in different experimental groups were re-randomized as a cluster to the same experimental group. Most components of the intervention were implemented at the locality level, but some aspects utilized the infrastructure of the agrarian unit, including rooms to have the informational meetings or to hang posters. The re-randomization at the cluster level prevented contamination of these cluster-level intervention components between experimental groups. The outcome of this randomization is shown in Table 5 disaggregated by state. Table 5. Localities by experimental group and state State T0 T1 T2 Total Oaxaca 27 28 27 82 Yucatán 9 9 13 31 Total 36 37 40 113 With this design, the power calculations indicate that the minimum detectable effect (MDE) of the experiment was 32 percentage points with a power of 0.80. This means we would not be able to detect statistical significance of differences that are lower than 32 percentage points. It should be noted that power might be too low to statistically detect feasibly small differences between treatment arms, especially between T1 and T2. Table 6 shows the balance test for five variables at the locality level. The team chose these variables because they capture different aspects of localities. Past CONAFOR grants help account for existing forest management capacity, the marginalized locality index accounts for socio-economic level, the adult population for the number of potential applicants, and the proportion of adult women and indigenous language speakers accounts for potentially underrepresented groups in forest management. The p-values were computed from the F-statistics of independent Ordinary Least Square (OLS) models that used each selected variable as dependent variables and the treatments as independent variables. Robust standard errors were clustered by agrarian unit. All p-values are higher than 0.5, which shows they are not statistically different. However, as some differences between groups might be considered economically important, we control for these variables in our models. Table 6. Balance test at locality level 28 Variable T0 T1 T2 p-value Marginalized Locality Index 2010 0.28 0.37 0.42 0.830 Grants awarded from CONAFOR 0.97 1.97 1.48 0.548 between 2007-2017 Proportion of adult women in adult 51.7% 51.4% 50.5% 0.503 population Proportion of women 3 years or older that speak indigenous 68.4% 53.7% 64.8% 0.551 language Adult Population 88.28 97.14 92.20 0.904 Values are product of independent OLS regressions with each variable as dependent variable and treatment as independent variable, clustering robust standard errors by agrarian unit. Descriptive statistics are predicted values and p- values are from the model’s F-statistic. Additionally, the team ran independent omnibus tests for each comparison (i.e., T0 vs. T1, T1 vs. T2, T0 vs. T2) in which the treatment variable was the dependent variable of a logistic regression, and the five variables, plus state entity, were independent variables. These regressions also clustered robust standard errors by agrarian unit.19 All three models had Wald χ2 p-values > 0.79 and all independent variables in the model had p- values > 0.196, meaning they are statistically jointly equivalent. Complete models are available in Appendix 2. 5.5 Survey The team also conducted a survey to gather representative individual-level data from women in the experiment. The survey took place in 52 localities of Oaxaca and Yucatán, where data were gathered from 1,485 women between 18 and 45 years old. This sample was constructed using a probabilistic systematic stratified sampling method (with the strata being the state and experimental group), with a margin of error of +/- 2.9 percent and a confidence level of 95 percent. The instrument, which may be found in the online Appendix20, had the following sections: socio- economic information, knowledge about CONAFOR and forest management, women’s participation and social cohesion, knowledge of DGM-Mexico, use and consumption of mass media, and evaluation of DGM materials. For this report, the team focused on the knowledge of DGM-Mexico and DGM materials (posters)—deemed the most relevant to the outcomes of the experiment and those with more complete data.21 5.6 Analytical approach 19The team cluster by agrarian unit to account for the randomization, in which localities of the same agrarian unit were assigned to be in the same experimental group. 20 Click here to view survey. 21While additional questions are relevant as well, in many cases, the sample of respondents to those questions ended up being too small. Descriptive statistics of other questions in the survey are shown in the online Appendix. Click here. 29 The main outcomes of our research are captured at the locality level of analysis. They include the number of applications sent by locality, the number of women part of a team by locality, and the number of applications led by a woman. Additionally, the survey data provide us with a few individual level secondary outcomes, namely whether the women had heard about DGM-Mexico and whether they recognized any of the posters for each particular experimental group. The team used OLS regressions to compute the main ordinal and continuous outcomes and logistic regressions for binary (dummy) outcomes. The main estimates of interest were the coefficients for the treatment variable. For the locality-level regressions, the team controlled for the total adult population in the locality in 2010, the proportion of adult woman in the locality in 2010, the proportion of woman three years or older speaking an indigenous language in 2010, the localities’ marginalization index in 2010, the number of grants awarded from CONAFOR between 2007 and 2017, and a dummy variable for state entity. The team clustered the standard errors by agrarian unit to account for the randomization, in which localities of the same agrarian unit were assigned to the same experimental group. For the individual-level regression, computed using survey data, the team controlled for age and included dummy variables for women married or in a partnership, completed primary education or less, whether they have children, whether they speak an indigenous language, and the state they live in. The team clustered standard errors by locality to account for sampling and randomization. The team also performed descriptive analyses to explore differences between the applications from those in the sample and the applications from the rest of Oaxaca and Yucatán states, as well as between evaluated applications and those that were not evaluated. 6. Results 6.1 Implementation The section describes the implementation of the interventions. As the additional measures to the official project outreach in T1 and T2 were implemented by CIDE, more data and details were available about this process as more data was collected. Quantitative and qualitative evidence was gathered for the project communication mechanism (T0). Official communication channels According to information provided by Rainforest Alliance, there were 17 official meetings in Yucatán and Oaxaca, involving 256 participants from 198 different localities. Regarding our specific sample, at least one person of 30 localities in our study attended these meetings (12 localities from T0, 9 from T1, and 9 from T2). The participants received flyers and posters related to the project that they could share in their communities. Through other official communication channels, Rainforest Alliance and some of the project stakeholders also provided printed materials to some of the localities. The project was 30 promoted through community speakers and indigenous radio messaging. While the exact number of and places where material from the official channels was shared is unknown (other than the ones mentioned before related to meetings), we can assume it was homogeneously distributed between T0, T1, and T2 localities due to the randomization process. Experimental variations (Additions to T1 and T2) 77 randomized localities received the expanded and simplified outreach strategy.22 Graph 1 shows descriptive statistics regarding the implementation of the different components: • While the experiment tried to reach all 77 localities, approximately 15.6 percent of localities in these two experimental conditions were unreachable or rejected the program. • Almost all reached localities agreed to schedule a meeting, receive materials (e.g., posters, flyers), and to present materials in public spaces. However, only about 52 percent of localities had the first meeting to discuss the program. The rest of the first meetings could not be scheduled, or no one attended. • 57.5 percent of localities in T2 had the first meeting, compared to 46 percent of T1. Even though this difference is not statistically significant (N = 77, chi2(1) = 1.0279, p-value = 0.311), it is important to mention that posters were delivered to localities before these meetings took place. If these were hung, the messaging used to encourage attendance would represent a differentiation between treatments, since the posters in T2 were informed by behavioral sciences. • About 45.5 percent of localities had a meeting during the second round of site visits. • The posters were delivered and supposed to be hung before the two meetings. During the last visit made by local allies to the localities, we were able to check if the posters were, in fact, hung. This visit was made either during the first or second meeting; about 32 percent of localities had hung a poster during the time of the last visit to the locality. • 68 percent of localities had at least one person that interacted with the helpline through phone, WhatsApp, or text messaging. o 53 percent of localities had at least one person receive a text reminder for the second meeting (60 percent T2 vs. 46 percent T1) from the helpline; o 60 percent of T2 localities had at least one person receive regular text messages informed by behavioral sciences principles. Graph 1. Percentage of localities with implemented activities for T1 and T1 22 T1 and T2 had 37 and 40 localities, respectively. 31 100% 90% 84% 85% 84% 84% 83% 83% 80% % of localities in T1 and T2 68% 68% 68% 70% 60% 60% 58% 60% 53% 52% 50% 46% 46% 45% 45% 46% 40% 32% 33% 32% 31% 30% 20% 10% 0% 0% Contacted Approval to Had first Had second Poster hung by Interacted by Received Received hang poster meeting meeting time of last visit phone, reminders of behavioral to locality WhatsApp or meetings messages text T1 T2 Total Table 7 shows the number and proportion of attendees by meeting and treatment group. On average, more people went to the first meeting than the second (t = 3.04, df = 24, p-value = 0.0056). T2 had slightly more people attend their meetings than T1 (p > 0.3221) and saw a slightly higher percentage of women attend the second meeting (t = -1.14, df = 30, p-value = 0.2620) than the first. This value is not statistically significant, however, as the sample was very small at this point. Table 7: Attendees to meetings T1 T2 N Mean SD Min Max N Mean SD Min Max Attendees 16 25.00 16.77 5 75 23 35.48 49.37 1 200 Meeting 1 Proportion women 16 0.58 0.15 0.38 0.86 22 0.58 0.18 0.30 0.92 Attendees 17 11.71 17.27 1 75 18 18.72 23.36 1 103 Meeting 2 Proportion women 15 0.53 0.23 0.2 1 17 0.61 0.18 0.25 0.81 While Graph 1 shows the percentage of localities that participated in each round of meetings, Graph 2 shows the details of wether they participated in one or both. As noted in Graph 2, about 18 percent of localities in T1 and T2 only held the first meeting, 12 percent only held a meeting during the second round of site visits, and 34 percent held both meetings sequentially. Differences in attendance may be because of variations in the typology of the two meetings. The first meeting was designed to be more informative and motivational. The second, closer to the application deadline, was aimed at channeling any technical doubts to Rainforest Alliance and collecting completed applications to be submitted. When localities only held one meeting, the content was more informative, rather than technical—even if the meeting was held closer to the application deadline. 32 Graph 2. Percentage of localities by treatment and meeting number. 100% % of localities in T1 and T2 90% 32% 35% 34% 80% 70% 60% 14% 10% 12% 50% 14% 18% 23% 40% 30% 20% 41% 36% 33% 10% 0% T1 T2 Total No meetings Only meeting 1 Only meeting 2 Both meetings Table 8 shows the predictive margins of an OLS regression with the number of applications submitted by the locality as a dependent variable. The categorization of number of meetings is the independent variable, controlling for the size of population and number of grants received from CONAFOR between 2007 and 2017. While this comparison is only correlational and the sample size is small, these predictions provide some indication that the second meeting may have had greater influence on the number of applications submitted. Localities with only the second meeting or both meetings submitted significantly more applications than those that had no meetings or only the first meeting. An alternative conclusion is that it was easier to arrange multiple meetings with localities that were more enthusiastic about the program,23 suggesting greater motivation to apply. Table 8. Predictive margins of OLS regression Marginal prediction of Standard Error number of applications (SE) No meetings 0.20 0.78 Only meeting 1 0.91 1.09 Only meeting 2 6.27 1.38 Both meetings 5.36 0.79 Note: OLS regression with a number of applications submitted as a dependent variable and categorization of number of meetings as an independent variable, controlling for the size of population and number of grants received from CONAFOR between 2007-2017. 23Although the regression controlled for previous grants, and number of applications was higher for “second meeting alone� as well. 33 An estimated 1,733 individuals were reached by this intervention (about 29.6 percent of the adult population in these localities in 2010). Graph 3 disaggregates this reach by different activities. Graph 3: Estimated individuals reached by each activity 2000 1800 1600 1400 Number of people 1200 1029 921 1000 800 600 400 431 704 358 631 200 143 239 200 100 0 96 44 100 0 Reached Meeting 1 Meeting 2 Attended 2 Interacted by Received Received meetings phone, reminders of behavioral WhatsApp or meetings messages text T1 T2 Graph 4 provides the cumulative number of first contacts to the helpline per day and treatment group, a timeline of how long this intervention lasted, and important milestones. Graph 4. Cumulative number of first contacts per day, by treatment and gender 34 Note: Bars correspond to number of people that contacted the helpline that day. The grey shaded areas correspond to periods in which the first and second meetings took place. The dashed green vertical lines correspond to the dates when behavioral WhatsApp and SMS messages were sent to the people interested in T2 locations. The solid brown line (March 29th) was the deadline to submit the application. We can draw a few conclusions from this graph. First, T2 localities (light blue lines) had more people interacting with the help center than T1. Secondly, more women (light blue and orange dotted lines) than men interacted with the help center, on average. This difference is more pronounced for T2 than T1. Third, and not surprisingly, the days with the most contacts to helpline (identified by the gray bars) coincide with the days in which T2 received behavioral science SMS messages (dashed green vertical lines). This is especially true when those messages included actionable steps that one was asked to perform and those asking the prospective applicant to call to helpline or send pictures if they needed help. Finally, while T2 had a constant increase in the cumulative number of people interacting with the help center, T1 cummulative contacts increase mainly after the first meeting and before the second meeting, when the help center sent reminders to those that signed up, suggesting the proactive messaging of the help center was key to promoting interaction. 6.2 Locality level analysis 441 applications were received by Rainforest Alliance in Yucatán and Oaxaca for this call for proposals. 215 of these applications were sent by people in the 113 localities that were part of our study. However, only one application came from T0 localities. The other 214 were sent by people from localities in either T1 or T2. One locality in T0, 14 in T1, and 18 in T2 submitted at least one application (80 localities had no applications submitted). Graph 5 shows descriptive statistics of the number of applications, disaggregated by treatment and gender of the team leader. 35 Graph 5. Number of applications submitted, by gender of the leader and treatment 90 82 80 Number of applications 70 60 48 50 44 40 40 30 20 10 1 0 0 T0 T1 T2 Woman Men Note: Absolute numbers do not adjust for the fact that T2 has slightly more localities. This is considered in the regressions in Tables 9 and 10 by using averages. Additionally, Graph 6 shows the percentage of the adult population of the localities in the study that is part of a team applying to the DGM-Mexico, disaggregated by gender and treatment. Graph 6. Percentage of the adult population of localities in the study (based on INEGI, 2010) that is part of a team applying to the DGM, by gender and treatment. 30% 23.4% 25% (434) Percentage of adult population 20% 15.1% (277) 15% 9.1% 9.5% 10% (164) (170) 5% 0.7% 0.7% (11) (11) 0% T0 T1 T2 Women Men Note: Absolute numbers don’t adjust for the fact that T2 has slightly more localities. This is considered in the regressions in Tables 9 and 10 by using averages. The absolute numbers are listed in parentheses. Table 9 shows the estimates of T1 and T2 as compared to T0 for different outcome variables, and Table 10 shows additional estimates of T1 vs. T2 for the same outcomes. 36 Table 9. Treatment effects of locality level variables, as compared to T0. All- Female Total Total Applications women Evaluated Selected leader people women group T1 2.308 1.113 0.124 0.044 -0.002 7.110 3.508 (0.667)*** (0.422)*** (0.106) (0.051) (0.007) (2.945)** (1.676)** [0.001] [0.010] [0.247] [0.391] [0.765] [0.018] [0.039] T2 2.792 1.889 0.556 0.057 0.023 15.578 9.889 (0.748)*** (0.588)*** (0.197)*** (0.060) (0.023) (4.954)*** (3.115)*** [0.000] [0.002] [0.006] [0.340] [0.320] [0.002] [0.002] Adj Base Mean .159 .0821 .001 .036 .001 1.671 .740 N 113 113 113 113 113 113 113 Specification OLS OLS OLS OLS OLS OLS OLS Base T0 T0 T0 T0 T0 T0 T0 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by agrarian unit in parenthesis. Covariates include: adult population in the locality, the proportion of adult woman in the locality, the proportion of woman of 3 years or older speaking an indigenous language, the localities’ marginalization index of 2010, the number of grants awarded from CONAFOR between 2007-2017, and a dummy of state. On average, localities in T1 submitted 2.308 more applications and T2 localities submitted 2.792 more applications than T0. The difference between T1 and T2 submission rates was not statistically significant. Given the adjusted base mean of T0 was 0.159 applications (only one application was submitted from the 36 T0 localities), this increase is quite substantial in relative terms. It represents a 1,452 percent increase in application rates for T1 localities and a 1,756 percent increase for T2. Given the strong change in the number of applications, it is not surprising that other related indicators also show an effect. In terms of number of applications that had a female leader, there was a 1.113 and 1.889 increase for T1 and T2, respectively, as compared to T0. There was again no significant difference between T1 and T2. In terms of applications that had all women as part of the group, T1 did not have a statistically significant effect compared to T0. T2, however, showed an increase of 0.433 and 0.556, compared to T1 and T0, respectively. It is important to keep in mind that the team that submitted the singular application in T0 was not composed by all women and did not have a woman leader. Thus, the unadjusted base means of T0 for these outcomes was 0. Even though the objective of the experiment was specifically to increase the number of applications submitted, especially by women, the team also looked into the applications that ultimately received a grant. As shown in Table 9, the increase in the number of applications did not translate into a higher number of applications that were evaluated (after getting through the initial filter) or ultimately selected to receive grants. Additional information about this individual analysis may be found in section 6.4. Table 10. Treatment effects of locality level variables, as compared to T1. 37 All- Female Total Total Applications women Evaluated Selected leader people women group T0 -2.308 -1.113 -0.124 -0.044 0.002 -7.110 -3.508 (0.667)*** (0.422)*** (0.106) (0.051) (0.007) (2.945)** (1.676)** [0.001] [0.010] [0.247] [0.391] [0.765] [0.018] [0.039] T2 0.485 0.776 0.433 0.013 0.025 8.469 6.381 (0.910) (0.606) (0.190)** (0.070) (0.025) (5.154) (3.186)** [0.596] [0.204] [0.025] [0.851] [0.323] [0.104] [0.048] Adj Base Mean 2.466 1.195 .125 .080 8.781 4.248 N 113 113 113 113 113 113 113 Specification OLS OLS OLS OLS OLS OLS OLS Base T1 T1 T1 T1 T1 T1 T1 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by agrarian unit in parenthesis. Covariates include: adult population in the locality, the proportion of adult woman in the locality, the proportion of a woman of 3 years or older speaking an indigenous language, the localities’ marginalization index of 2010, the number of grants awarded from CONAFOR between 2007-2017, and a dummy of state. 38 As people can submit an application as a group, the team also analyzed the total number of people that were part of the applications. This figure was also disaggregated by gender. As in previous indicators, there was a significant increase in terms of total people and, specifically, the number of women that applied for both T1 and T2 localities. Both indicators were higher in T2 than T1. Although “total people� only barely missed the 10 percent significance threshold (p-value = 0.104), “total women� was significant at the 5 percent confidence level. Additionally, Graph 7 shows the proportion of women in teams of the submitted applications of T1 and T2, demonstrating a clear increase in proportion of women in T2. Graph 7. Proportion of women in teams of submitted applications. by treatment. Note: T0’s only application had an even split between women and men The tables in Appendix 3 show the regressions without controls, where standard errors were still clustered by agrarian unit. Conclusions from this analysis remain unchanged. 6.3 Individual level analysis (survey) Survey sample balance Table 11 shows the balance test for seven variables at the individual level. The p-values are from F-statistics of independent OLS models that used each variable as a dependent variable and each treatment as an independent variable, clustering robust standard errors by locality. Only “has children� is statistically unbalanced (F[2, 51] = 7.97, p-value = 0.001) with T1 having significantly more parents than the other groups. Omnibus tests of the joint significance of each comparison (i.e., T0 vs T1, T0 vs T2, T1 vs. T2) were run using logistic regressions and including a dummy for state entity to account for the sampling design. Only the model of T0 vs. T1 was 39 statistically significant (Wald χ2(8) = 23.56, p-value = 0.0027), mainly driven by “has children� and “speaks indigenous language�. The other omnibus tests were not statistically significant (p-value > 0.288). These variables were used as covariates in the main models to control for this potential unbalance. The other two omnibus tests were not statistically significant (T0 vs. T2: Wald χ2(8) = 4.66, p-value = 0.7936; T1 vs. T2: Wald χ2(8) = 9.68, p-value = 0.2882; complete models in Appendix 2). Still, results from these individual-level regressions should be interpreted with caution given the potential unbalance. Table 11: Balance test at individual level. T0 T1 T2 p-value N Married or in a partnership (dummy) 0.741 0.794 0.763 0.275 1,483 Educational level of complete primary or lower 1,486 0.358 0.416 0.424 0.423 (dummy) Age 31.969 32.429 31.361 0.319 1,486 Has children (dummy) 0.808 0.903 0.824 0.001 1,486 Household size 4.633 4.343 4.641 0.186 1,486 Identifies with indigenous group (dummy) 0.833 0.779 0.841 0.437 1,472 Speaks indigenous language (dummy) 0.465 0.292 0.489 0.218 1,486 All variables from survey. Main outcome The team identified the following questions to help assess the potential outcomes of the program: whether an individual had heard about DGM-Mexico and whether they recognized any of the posters for their particular experimental group. The treatment estimates of these outcomes are in Table 12. Table 12: Individual-level outcomes Models 1, 3, and 5 use T0 as comparison/base group, while 2, 4, and 6 use T1. Heard of Heard of Recognizes Recognizes Recognizes posters Recognizes posters DGM DGM posters posters or heard of DGM or heard of DGM (1) (2) (3) (4) (5) (6) T1 0.047 0.097 0.081 (0.025)* (0.045)** (0.048)* [0.060] [0.032] [0.091] T2 0.036 -0.010 0.108 0.011 0.084 0.003 (0.023) (0.023) (0.044)** (0.056) (0.044)* (0.057) [0.120] [0.650] [0.013] [0.837] [0.058] [0.959] T0 -0.047 -0.097 -0.081 (0.025)* (0.045)** (0.048)* [0.060] [0.032] [0.091] Adj Base 0.050 0.097 0.111 0.207 0.152 0.233 Mean N 1,480 1,480 1,483 1,483 1,483 1,483 Specification Logit Logit Logit Logit Logit Logit Base T0 T1 T0 T1 T0 T1 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by localities in parenthesis. Coefficients of logits are marginal effects. Covariates include: age and dummy variables for women married or in a partnership, complete primary education or less, whether they have children, whether they talk an indigenous language, and state entity. 40 Approximately 5 percent (adjusted) of women in T0 had heard about DGM-Mexico. There was a marginally significant effect of T1, as compared with T0. Being in T1 was associated with a 4.7 percentage point (pp) increase in the percentage of women surveyed that had heard of DGM- Mexico. Additionally, approximately 11.1 percent of women in T0 recognized at least one of the posters from their experimental group. Being in T1 was associated with a 9.7 pp increase in the percentage of women recognizing the posters compared to T0. T2 was associated with a 10.8 pp increase. Taken together, approximately 15.2 percent of women in T0 had heard about DGM-Mexico or recognized at least one poster. Models 5 and 6 in Table 12 were run with this combined outcome variable. Compared to T0, T1 and T2 saw a respective 8.1 pp and 8.4 pp increase in women that had either heard of DGM or recognized at least one poster, significant at the 10 percent confidence level. This suggests that more women in T1 and T2 were aware of the campaign, especially regarding poster recognition. Sample description After weights were applied, 69.2 percent of the surveyed women lived in Oaxaca and 30.7 percent in Yucatán. The average age was 31.9 (SD = 8.21), with no significant difference between the two states. The distribution of ages was relatively uniform, except for an unexpected spike in the number of people reporting as 45-years-old. As this is the upper limit for the survey, the survey monitoring agency suggests this may be caused by social desirability from people wanting to participate in the survey. Thus a few older women may have answered the survey. Household size was on average 4.6 persons (SD= 1.9), with Yucatán having a higher average (mean = 5.1, SD = 2.4) than Oaxaca (mean = 4.3, SD = 1.6). As shown in Table 13, Yucatán had a much higher percentage of married women (76.2 percent vs. 39.1 percent), although Oaxaca showed a higher percentage of women in a partnership (9 percent vs. 34 percent). Taken together, three-quarters of the sample were in some kind of partnership. Additionally, 84.4 percent of the sample had children. 9.4 percent of those surveyed with children were single mothers, although this groups represents less than one percent in Yucatán, while 12.9 percent of Oaxaca. Table 13: Oaxaca Yucatán Sample Percentage of Has children Has children Has children women surveyed by civil status and children Civil Status Yes No Total Yes No Total Yes No Total Single 12.87 9.89 22.75 0.74 12.85 13.6 9.42 10.9 20.32 Married 37.46 1.68 39.14 69.65 6.53 76.17 46.37 2.96 49.33 Free Union 32.32 1.83 34.14 7.98 1.08 9.05 25.37 1.68 27.05 Divorced 1.7 0 1.7 0.22 0.22 0.43 1.35 0.07 1.41 Widow 2.02 0 2.02 0.74 0 0.74 1.68 0 1.68 No answer 0.25 0 0.25 0 0 0 0.2 0 0.2 41 Total 86.61 13.39 100 79.33 20.67 100 84.39 15.61 100 Values are percentages within each state and the sample. Table 14 shows the eduactional level of the women surveyed. 18.4 percent of the sample has at least an educational level of completed high school or higher. 21.7 percent of the sample from Oaxaca had at least an educational level of completed high school or higher, while Yucatán 10.8 percent. Educational level is relevant in that it proxies human capital that can support completing quality proposals and applications. Table 14: Percentage by educational level of women surveyed Educational level Oaxaca Yucatán Total None 4.2 3.94 4.12 Incomplete primary 13.67 16.14 14.43 Complete primary 19.45 28.23 22.15 Incomplete middle school 5.48 6.1 5.67 Complete middle school 30.75 33.06 31.46 Incomplete high school 4.71 1.7 3.79 Complete high school 14.66 10.4 13.35 Incomplete technical career 0.79 0.22 0.61 Complete technical career 0.88 0 0.61 Incomplete bachelor 2.04 0 1.42 Complete bachelor 3.29 0.22 2.35 Graduate degree 0.08 0 0.05 Total 100 100 100 Only 2.3 percent of the surveyed women listed forest activities as their first or second activity (in terms of time invested or occupation), with a higher percentage in Oaxaca than Yucatán. Other than home activities or chores, the most common occupation in Oaxaca were selling products. In Yucatán, the production of handicrafts was the most common after the home activities. Table 15: Percentage of women surveyed that reported activity as first or second activity Oaxaca Yucatán Total Home activities 88.36 93.25 89.86 Sale of products 41.88 5.88 30.82 Craft production 8.71 42.64 19.14 Agricultural activities (agricultural, livestock, fishing) 14.09 8.49 12.37 Employed in the private sector 6.95 0.54 4.98 Student 3.82 2.03 3.27 Forest Activity 3.20 0.42 2.34 Seamstress 0.40 5.80 2.06 Government activities 1.60 0.82 1.36 I do not have a job 0.45 0.00 0.31 Other 1.64 1.05 1.46 Percentages combine two variables (first and second activitiy), and so percentages do not add up to 100 42 95.6 percent of women in Yucatán and 75.6 percent in Oaxaca identify as part of an indigenous group. However, when asked if they knew an indigenous language, the percentages were 90.7 percent and 23.4 percent, respectively. Table 16: Percentage of surveyed women that identifies with indigenous origing or speaks language Oaxaca Yucatán Total Considers herself part of an indigenous group 75.6 95.6 81.1 Speaks indigenous language 23.4 90.7 41.9 6.4 Exploratory application-level analysis To provide a holistic view of the application and selection processes, the team analyzed all applications received by Rainforest Alliance in Oaxaca and Yucatán, not limited to the experimental sample localities. The below analysis will focus specifically on comparing successful applications, as well as comparing applications from our experimental sample with those from other localities. It is important to highlight that this analysis is exploratory and correlational rather than causal, as the team did not have an experimental variation or comparable groups for this analysis. Given T0 only had one application (which was evaluated but not selected), comparing it to applications from T1 and T2 (which had hundreds of applications) could be very noisy and not meaningful. While we cannot derive conclusive evidence from this exploratory analysis, it can help generate hypotheses as to why the experimental sample had a low number of applications that were selected for funding, and develop future diagnostic work and interventions to address this. The evaluation and selection process followed a structured and homogeneous system, allowing for comparability between the experimental sample’s applications and applications from other localities. Figure 10 shows this process. First, staff from the Rainforest Alliance received all applications and run them through a completeness check. This completeness check did not include any technical evaluation per se. Proponents were given an extra time to provide any missing information if that were the case. All complete applications were then run against an exclusion list and checked for adminitrative requierements, mainly safeguards-related. All proposals that passed this filter were submitted for technical evalution to community leaders in the region. Because of different reasons that are not recorded in our data (e.g. teams dropping, missing key documentation or permits, etc.), a small number did not pass to the evaluation phase, noted as “Eligible but not evaluated.� Finally, representatives from indigenous groups and local communities from the regions ranked the applications (“Evaluated�) and selected the best proposals (“Selected�). Those selected would subsequently receive technical assistance by Rainforest Alliance to elaborate a full project proposal and ultimatly receive funding. This second phase is not covered by the experiment nor did it occur during that time. Figure 10: Selection process of applications 43 Step 1: Receive proposal and completeness check Rainforest Alliance received all applications and run them through a completeness check Step 2: Eligible but not evaluated Rainforest Alliance filtered the applications against the exclusion list and safeguards requierements. Step 3: Evaluated Rainforest Alliance processed the applications to be evaluated by representatives from indigenous groups and local communities. Step 4: Selected Representatives from indigenous groups ranked and selected the proposals. As Graph 8 shows, most of the applications submitted to this call for proposals from localities in the sample did not provide the minimum technical information for the grants. Hence, they were not evaluated or selected. While localities outside of the experiment’s sample submitted a similar number of applications (215 in the sample and 226 outside), the rate of applications that were not technically eligible was much lower (48 percent vs. 93 percent), suggesting that technical qualifications were the main barrier hindering applications in our sample. Out-of-sample applications were also evaluated and selected at a higher rate (38 percent and 11 percent, respectively). We, unfortunately, do not have further data to assess the objective “quality� of the out-of-sample proposals or to isolate the specific issue that made applications technically ineligible. Graph 8. Status of applications coming from experimental sample vs. localities outside the sample 44 100% 1 7 26 7 90% 80% % of total applications 70% 85 60% 50% 6 200 40% 30% 109 20% 10% 0% Out of sample In sample Not technically elegible Elegible but not evaluated Evaluated Selected Regarding the comparisons between applications from our sample and from outside, it is important to recall that we are comparing groups of localities and populations with different characteristics, since our experimental sample was selected through a series of filters. As Table 17 shows, there were significant differences between characteristics of applications from our experimental sample and those from out-of-sample localities. For example, evaluated applications outside the sample were typically projects involving larger land areas and team sizes, hosted a lower percentage of “Comuneros� in the team, a higher percentage of “Avecindados,� and a lower percentage of men. Applications from our experimental sample were smaller in average land surface proposed for the project and team size, and involved a higher percentage of “Comuneros,� lower percentage of “Avecindados,� and higher percentage of men when compared to applications from outside the sample.24 Other variables have more ambiguous patterns. For example, the percentage of indigenous people within teams was higher in evaluated applications coming from outside the sample. However, this is also a prevalent characteristic of both the evaluated and non evaluated applications coming from teams in our experimental sample, and a general characteristic of the localities we targeted in this experiment. 24 Ejidos and Agrarian Communities: Social property of land that covers most of the surface in the Mexican countryside. Ejidatario/Ejidataria: Person who forms part of the ejido and has a legal certificate of agrarian rights, parcel certificate or common rights, or the resolution of the agrarian tribunal. Comunero/Comunera: Holder of rights within an agrarian community, which allows him/her to use and enjoy his/her land and the transfer of its rights, as well as the use and benefit of the assets of common use. Avecindado: Person who lives in the Ejidos or Agrarian Communities without being ejidatarios or comuneros 45 Table 17. Characteristics and team composition of applications from experimental sample vs. outside the sample, and of evaluated vs. not evaluated applications Not Evaluated Evaluated Not in In Not Not in In Not in In Evaluated Sample Sample Evaluated Total Sample Sample Sample Sample Surface of land for 182.28 10.92 736.09 19.25 454.29 11.24 73.09 687.9 240.89 project* Total People* 11.30 4.83 14.54 9.38 12.89 5 7.14 14.19 9.05 Includes Women 96% 94% 97% 100% 96% 94% 94% 97% 95% Includes Men* 74% 87% 68% 100% 71% 87% 82% 70% 79% Includes 54% 35% 96% 100% 75% 37% 42% 97% 56% Avecindados/as* Includes 7% 51% 9% 38% 8% 51% 35% 11% 29% Ejidatarios/as* Includes 50% 68% 32% 25% 41% 67% 61% 32% 54% Comuneros/as* Includes youth* 83% 81% 95% 88% 89% 81% 82% 95% 85% Includes people 97% 95% 92% 100% 94% 95% 96% 92% 95% older than 32 Includes 35% 79% 43% 75% 39% 79% 63% 45% 59% Indigenous* Female group leader 53% 59% 66% 38% 60% 59% 57% 64% 59% Agroforest activity 56% 52% 50% 38% 53% 51% 53% 49% 52% Climate Smart 18% 22% 14% 25% 16% 22% 20% 15% 19% Agriculture activity Silvo-pastoral 6% 17% 4% 25% 5% 17% 13% 5% 11% activity* OBSERVATIONS 115 207 111 8 226 215 322 119 441 * p<0.05 for Not in Sample vs. In sample and Not evaluated vs. Evaluated Similarly, comparisons of localities that sent applications was conducted. While a similar number of applications were submitted by localities inside and outside the experimental sample en masse, the number of localities that submitted these applications differs. 33 localities submitted the 215 applications in the sample, while 148 localities submitted the 205 applications from outside of the sample.25 Hence, the average number of applications per in-sample locality was much higher (mean from inside sample = 6.52, mean from outside sample = 1.39, t = -10.81, df = 179, p-value < 0.001), with 76 percent of localities that submitted an application from outside the sample submitting only one application. Table 14 compares localities with at least one application evaluated to those that sent applications but had none accepted (includes localities from both in and outside our sample). The localities that 25We were not able to identify a specific locality for 21 applications, due to ambiguous naming or the application’s reference to another geographical unit rather than the locality. 46 had at least one application evaluated are larger on average, have a smaller proportion of the population speaking indigenous languages, and have more years of schooling. Similar patterns arise when this same comparison is restricted to localities outside of the experimental sample (see Appendix 4). Table 18. No application evaluated vs. at least one application evaluated No evaluation At least one application evaluated Variable Obs Mean Std. Dev. Min Max Obs Mean Std. Dev. Min Max Total inhabited houses** 91 143.76 192.51 2 1,084 90 228.34 270.30 1 1,246 Total population** 91 594.30 794.50 6 4,939 90 909.16 1,057.04 1 4,763 Proportion female 90 0.51 0.04 0.39 0.64 89 0.51 0.04 0.29 0.56 Proportion adult 90 0.59 0.09 0.39 0.94 89 0.61 0.08 0.42 0.82 Proportion of 3 years or older that speaks indigenous language *** 90 0.59 0.37 0 1 89 0.39 0.37 0 1 Average grades of schooling for 15 years or older*** 90 5.44 1.03 3.16 9.54 89 5.92 1.16 2.08 9.68 Marginalization index = Very High+ 90 8.89% 89 6.74% Marginalization index = High+ 90 87.78% 89 78.65% Proportion of 12 years or older unemployed 90 0.01 0.01 0 0.08 89 0.01 0.02 0 0.15 ** p<0.05; *** p<0.01 using t-tests. + a test with the continues marginalization index showed a significant difference p<0.05 in which localities with at least one evaluated application where less marginalized. Table 15 compares the demographic characteristics of localities in the sample that submitted applications with those outside the sample that submitted applications as well. The localities from the sample that sent applications are smaller on average, have a larger proportion of the population that speaks indigenous languages, have fewer years of schooling, and have a slightly smaller proportion of women and adults in the population when compared to those outside of the sample. Interestingly, regarding the variables used as filters, only 12 out-of-sample localities that submitted applications (13.5 percent) were identified as having forest coverage, suggesting that this filter was not as relevant as anticipated.26 Other rural activities, such as sustainable ranching and eco- agriculture, that do not require direct management of forests were also eligible for the call for proposals. Additionally, only two out-of-sample localities that submitted applications (1.35 percent)had less than three households. This criterion is necessary for the census to capture subgroup information in locality-level census data. This is a reason why all sampled localities had at least three houses, meaning out-of-sample localities that submitted applications were similar to our sample in that respect. 26 Recall all localities in the sample had forest coverage by design. 47 Table 19. Localities with application submitted from in sample vs. out-of-sample In sample Not in sample Variable Obs Mean Std. Dev. Min Max Obs Mean Std. Dev. Min Max Total inhabited houses*** 33 48.76 40.43 6 174 148 216.38 252.21 1 1,246 Total population*** 33 218.39 184.88 28 773 148 869.58 1004.09 1 4,939 Proportion female** 33 0.50 0.03 0.40 0.59 146 0.51 0.03 0.29 0.64 Proportion adult** 33 0.57 0.11 0.39 0.80 146 0.61 0.08 0.41 0.94 Proportion of 3 years or older that speaks indigenous language** 33 0.61 0.39 0 1 146 0.47 0.38 0 1 Average grades of schooling for 15 years or older*** 33 5.10 0.78 3.18 7.16 146 5.81 1.15 2.08 9.68 Marginalization index = Very High+ 33 15.15% 146 6.16% Marginalization index = High+ 33 84.85% 146 82.88% Proportion of 12 years or older unemployed 33 0.01 0.02 0 0.08 146 0.01 0.02 0 0.15 ** p<0.05; *** p<0.01 using t-tests. + a test with the continues marginalization index showed a significant difference p<0.01, with in sample localities being more marginalized. 7. Limitations This RCT had four main limitations related to design and implementation. First, the intervention was designed and evaluated as a package of activities, instead of having multiple treatment arms for each independent activity (e.g. improving poster, adding messages based on behavioral science to poster, simplifying form, sending messages informed by behavioral science). Therefore, the impact of each individual activity cannot be decoupled from the group of activities performed under each treatment group. Secondly, the survey was conducted in the middle of the experiment in February 2019. Therefore, variables that come from this survey should not be considered as outcome variables but as output measures. Moreover, the survey may have had some effects in the localities where it was applied (for example, as a reminder). However, as it was applied to a sample that represented T0, T1, and T2 localities, the potential effect should be equivalente across groups. Third, the experiment intentionally targeted localities that appeared relevant for the research questions (i.e. with forest coverage) based on existing data, had a wide presence of women and indigenous persons, and had characteristics necessary for the implementation (e.g., reachable location, phone coverage). This targeting broadly achieved the objective of reaching groups that are typically not represented in productive forest resource management projects, at least as much as possible given the available information and budget. However, the experiment’s sample ended up being less representative of typical participants, as mentioned, where results are not directly transferable to all localities that the grant serves. 48 Finally, the DGM outreach for the project and the experiment were initially delayed. As outreach for T1 and T2 was conducted before Christmas, the project required implementation in a time period longer than that in similar programs and during a period with more distractions. A potential effect is that communication campaign may have been diluted due to this time period, where the impact of the intervention is understated. 8. Discussion, conclusions, and policy implications The strategies used to increase the number of applications were effective and simple. They may be easily replicated by government and non-profit organizations to reach their stakeholders, with potentially significant effects. The additional measures implemented to the official outreach strategy had a strong effect on the number of applications submitted, including those from women. While the control group localities barely submitted applications (one application received from all 36 localities), the treatment groups sent, on average, more than two applications per locality. The evidence shows that the largest effect on the number of applications can be attributed to expanded communication channels and simplification process (T1)27. While the localities that received the behavioral messages (T2) had a higher average number of applications compared to T1, this effect was not statistically significant28. Similarly, increased knowledge of DGM or recognition of DGM posters by women can be attributed mostly to T1 measures. Unsurprisingly, this strong increase in number of applications translated into many more applicants, including women. The first major change was related to expanding the communication channels—personal outreach to each locality, setting up two meetings to explain the call for proposals and process, and setting up a help line. While the official method reaches out to communities through meetings as well, localities that are less accustomed to responding to calls for proposals (as our sample appeared to be) may be less motivated to go out of their way to access the necessary information, especially if it involves traveling to a different locality. Additionally, T1 included a simplification process for information. Having easy to understand materials may be especially important in contexts of lower educational levels, language barriers, and lower economic outcomes, as our sample showed. These strategies help prevent “friction costs,� that is, the seemingly small but detrimental barriers that keep individuals from engaging with a service, and which reduce the cognitive bandwidth required to grasp the information. Finally, another aspect of critical importance is the support of local allies, which improve trust issues between localities and outsiders. While we cannot isolate which of these strategies had the greatest impact, we generally believe the increase stems from making it easier for individuals to learn about and apply to the program. The behavioral messages (T2) seem to have had an independent, positive, and statistically significant effect on the number of women applying29, complementing the effect of the expanded 27 B = 2.308, SE = 0.667, p-value = 0.001. 28 B = 0.485, SE = 0.910, p-value = 0.596. 29 B = 6.381, SE = 3.186, p-value = 0.048. 49 channels and simplification process. In particular, T2 seems to have increased the proportion of female team members, and the number of applications with teams composed fully of women in consequence30. T2 addressed social norms that may have historically prevented women from applying to natural resource management programming. It also highlighted aspects of their identities in the context of this program (e.g., mentioning that this would help provide children with a better life), which could motivate more women to apply as compared to T1, where messaging lacked components that specifically target women. Reminders, checklists, and use of loss aversion principles were additional components of T2, which we hypothetized would have increased the number of applications overall. As mentioned, we saw a higher average number of applications in T2 compared to T1, but the differences were not statistically significant. It may be that we lacked the power to detect statistical significance or that, following the simplification and increased outreach, the additional strategies employed were more effective at making teams feel comfortable with being composed of mostly women, rather than at encouraging additional applications. It is notable that, counter to our expectations, the additional components of T2 (i.e., posters and messages informed by behavioral science, checklist, proactive messaging) did not appear to increase knowledge or recognition of DGM materials by women. This may be because the groups in the survey seemed slightly different before the intervention started (unbalanced) or that the informational barrier was already overcome with the adaptations made through T1. Thus, T2’s impact was seen in making an explicit call to women and addressing the behavioral barriers they faced. While the main objective of the experiment was achieved by motivating more people, especially women, to apply to productive natural resource management programs, we did not find evidence that these interventions were enough to achieve more participation. We could not detect an increase in the average number of applications selected to receive grants31. Exploratory analysis suggests that most applications sent were not “technically eligible,� meaning the proposals lacked important minimum technical information required to be strong enough to qualify. We do not have enough experimental variation to causally isolate the reason for the low success rate, but we can formulate a few hypotheses. First, by targeting localities and populations with low participation in previous calls for proposals, we might have applied the treatment to localities with less capacity in terms of infrastructure, resources, and knowledge. This would understandably lead to proposals with less potential or of lower quality. Indeed, our exploratory analysis showed that localities in our sample that submitted applications, when compared to other localities that submitted applications, had lower levels of education, higher levels of marginalization, and a larger presence of indigenous languages (which could suggest higher language barriers). As mentioned, technical assistance would be provided to DGM applications in phase two. However, these results seem to suggest that support is needed earlier to ensure they qualify. This finding suggest that many communities that might have never participated in any government program would require 30 B = 0.433, SE = 0.190, p-value = 0.025. 31 T0 vs T1: B = -0.002, SE = 0.007, p-value = 0.765, T0 vs T2: B = 0.023, SE = 0.023, p-value = 0.320. 50 technical assitance even before articulating a project idea, but at stages such as community governance, establishing working groups and identifying suitable activities. Second, the additional and simplified communications measures may have lacked the technical detail required to support successful applications, particularly in view of the likely low baseline capacity in these localities. Working with local allies had several advantages, especially related to establishing trust of local populations. However, direct support with more technical skills or training may be needed to help people who lack experience in submitting such applications. Additionally, Rainforest Alliance approach is focused on targeting local governancethrough community leaders, and participatory development of proposals. In this sense, it’s possible that by trying to reach more individuals directly and simplify the process, the ideation of projects may have lacked a more participatory and meticulous process. Third, and related to the previous point, there may be something inherently less appealing about the design of the proposals from our sample. The exploratory analysis shows that applications in our sample featured smaller teams and smallers tracts of land available for the project, as well as a higher number of applications per locality. This could suggest more partitioning of projects within localities of our sample. We also see team compositions with a higher proportion of men and “comuneros� in our sample32. These audiences do not represent the prioritized audience of this special call for proposals which was intended to target women in particular. We, unfortunately, lack sufficiently detailed information to determine whether these hypotheses are true. The results of this experiment suggest that increasing the outreach channels and simplifying messages and forms can effectively increase recognition of the program and the number of applications to productive forest resources management programs. This suggests that the informational barriers, cognitive costs, and time burden associated with getting informed and applying to the program may be a significant barrier for potential applicants. The results also suggest that women, who have been traditionally under-represented within these programs, may be encouraged to apply when the program addresses important behavioral barriers by challenging gender norms, prevailing views about women’s strengths and abilities, and low aspirations. This may suggest that these under-represented populations, and women in particular, may be interested in participating in natural resources management programs but might not do so in the face of structural or behavioral barriers. Likewise, while these populations seem willing to apply once these barriers are addressed (e.g. the drastic increase seen from T0 to T1 and additional increase of women applying in T2 localities), they may lack the capabilities (e.g. educational, economic, and information required to implement especially productive or innovative activities), or resources (e.g. land, technology, infrastructure) to present proposals strong enough to be eligible. This is evidenced by fact that an increase in proposals was not matched by an increase in selected grants among the treatment localities. Even though all localities in this study were allowed to access any resources, people from localities within the experiment hardly participated in the official project outreach, suggest that the 32Even if T2 increased the number of women in teams, applications from our overall sample seem to have higher proportion of men than out-of-sample applications. 51 experiment localities represents a population that is under-reached by these resources. At the same time, an outreach through community leaders as done by the project seem to have significant impacts on local governance for project idea preparation. The team concludes that further experiments or projects should take up relevant elements from the experiment embedded in existing and functional outreach mechanisms and outreach should be linked closely and at very early stages with sound technical assitance. Future research could also include analyzing the specific effects of each individual treatment activity. Rather than assessing the intervention as a package of activities, as this study does, the impact of each activity could be decoupled from the group of the activities performed under each treatment to better distill impact (e.g., some localities receiving only expanded mechanisms, others only simplications, others only communications challenging gender roles, others only proactive messages, etc., and others receiving different combinations of those activities). In closing, while active participation of women in natural resources management programs in Mexico is relatively low compared to men, this experiment demonstrated that rural women are interested in participating in such programs. First, the outreach and engagement of these programs would be more effective when it explicitly targets rural women through accessible channels and relatable messaging. However, the experiment also demonstrated that applicants may require technical assistance to identify and formulate project ideas, even in the early stages, and to provide them with the knowledge and tools necessary for a meaningful engagement in calls for proposal. Additionally, one of the findings of the earlier behavioral diagnostic report was that women participating in productive activities tend to play a leadership role within the community. Given that the experiment suggested high interest from women-led groups, this could be expected to eventually shift to women’s increased participation in communal governance bodies in Mexico over time. Even though this experiment was tested in a non-governmental context, the findings are applicable to other programs (that is, governmental, nongovernmental, private sector) that are interested in reaching rural women. By taking a human-centered approach to understand barriers that women or marginalized groups face in the takeup and sustainability of natural resources management programs, it is possible to support gender inclusion by making the communications and outreach strategy fit better the needs and motivations of these groups. 52 9. References Comisión Nacional Forestal (CONAFOR) (2017). Informe de Medio Término Proyecto Bosques y Cambio Climático. CONAFOR. Mexico. Diario Oficial de la Federación (DOF) (2016). Acuerdo por el que se emiten las Reglas de Operación del Programa de Fomento a la Economía Social para el ejercicio fiscal 2017. DOF, México. Instituto Nacional de Estadística, Geografía e Informática (INEGI) (2009). Estados Unidos Mexicanos, Censo Agropecuario 2007, VIII Censo Agrícola, Ganadero y Forestal, Aguascalientes. INEGI, México. Méndez-López, M.E., E. García-Frapolli, I. Ruiz-Mallén, L. Porter-Bolland, and V. Reyes-Garcia. (2015). From Paper to Forest: Local Motives for Participation in Different Conservation Initiatives. Case Studies in Southeastern Mexico. Environmental Management 56: 695. World Bank Group (2015). World Development Report 2015: Mind, Society, and Behavior. Washington, DC: World Bank. doi: 10.1596/978-1-4648-0342-0 World Bank Group (2018). Closing the Gender Gap in Natural Resource Management Programs in Mexico. Washington DC. USA. World Bank Group (2019). Behavioral Science Around the World: Profile of 10 Countries. World Bank. Washington, DC, USA. 53 10. Appendices Appendix I: Text/WhatsApp Messages MESSAGE DATE Welcome! Thank you for your interest in the DGM program. Don't miss out After each of the first meetings on this opportunity! Call 01 800 5223553/WhatsApp 5521930605. Remember that the DGM will provide resources for your productive activities in order to give a better life to your family. Don't miss out on this After each of the first meetings opportunity! Call 01 800 5223553/WhatsApp 5521930605. Thank you for contacting us! You can find us at our working hours: Mon-Fri Out of office message 9-17 hrs. 01 800 5223553/WhatsApp 5521930605. Hi [name], thanks for your interest. Enjoy the holidays and remember that there is an opportunity waiting for you! Join the DGM and give your family a December 22 better quality of living. See you in January! Happy new year! Remember that the DGM helps you with resources for your productive activities in order to give your children a better life. Call us January 2 beginning on January 7: 01 800 5223553/WhatsApp 5521930605. Remember that the DGM will provide resources for you to work and give a better life to your family. Contact us beginning at January 7! 01 800 January 4 5223553/5521930605. If your new year's resolution is to give you family a better life, the DGM program can help you accomplish it. Call us and participate! 01 800 January 4 5223553/5521930605. The telephone line of the DGM is 01 800 5223553/WhatsApp 5521930605. January 4 We are waiting for you! 54 We are back! Let's start your journey in the DGM together. Don't miss out on this opportunity! It's only 10 steps! Use the orange sheet that they gave you January 7 at the meeting as a guide. We'll start with the third step of the orange sheet. Give us a call us if you don't have it! Hi, you can communicate with us at these numbers: 01 800 January 7 5223553/WhatsApp 5521930605. We are waiting for you! Step 3: Have an idea and form a team of 2 or more people for the DGM. Fill the list of participants. They can be from your family and benefit together! January 8 Don't miss out on this opportunity! Step 4: Choose your representative. In order to participate you will need a copy of their official ID and CURP. If you don't have your CURP, we can help January 9 you out. Have you formed your team of 2 or more people for the DGM? Make a list and search for a copy of the ID and CURP of your representative. Tell us if you got January 11 it! Step 5: Develop your idea alongside your team. Write it out! Remember that January 14 you can call us with your ideas and solve any questions. How is the development of your idea going? You can call us for support and January 16 for any inquiry: 01 800 5223553/WhatsApp 5521930605. Step 6: Fill your DGM format! More people like you are already doing it. Start January 18 with sections A and B. Give us a call! We'll help you out. We recommend you contact us on the phone or WhatsApp to help you better January 18 with the format. Call us! 01 800 5223553/WhatsApp 5521930605. 55 Most of the people interested are filling sections A and B of their format. Remember that we can help you, call us at 01 800 5223553/WhatsApp January 21 5521930605. Today, almost everyone has already filled sections A and B of their format. January 23 How are you doing? Call us if you need help! Step 7: Now fill sections C and D of your DGM format! Don't miss on the January 25 opportunity to participate! Call us, we'll help you out! Most of the interested people are filling sections C and D of their format. Remember that we can help you, call us at 01 800 5223553/WhatsApp January 28 5521930605. Remember that the DGM will provide resources for your productive activities. Don't miss out the chance to participate and give your family a January 28 better life! Do you have your full DGM format? You are still in time and you have made January 30 great progress. If you have any questions, we will help you! We have worked really well together. In order to take the next step, send us a February 1 photo of your format to confirm that it is filled. Thank you very much! Step 8: Complete and sign the power of attorney with the data of the February 5 members of the team. You will need a copy of the ID of all the members. Remember that you can call us via phone or WhatsApp if you need help with February 5 the power of attorney. Call us! 01 800 5223552/WhatsApp 5521930605. Step 9: Prepare for the meeting. You will need: list of members, CURP and ID of the representative, the DGM format and the power of attorney with a copy February 6 of all of the member's ID 56 The finish line of your DGM application is getting closer! Keep going and call February 6 us if you have any questions. Many people like you are doing it. Give us a call! Do you have your full DGM format? You are still in time and you have made February 7 great progress. If you have any questions, we will help you out! Remember that the DGM will provide resources for your productive activities. Don't miss out the chance to participate and give your family a February 11 better life! The finish line of your DGM application is getting closer! Keep going and have February 14 your documents ready. Many people like you are doing it. Hi! We will be receiving the formats until tomorrow, so that we can look through them before the meeting. WhatsApp +0152 5521930685, February 18 ucex.lnpp@cide.edu. Thank you! The meeting to pick up the documents for the DGM will be on __ at __ in __ Before each meeting The documents you will need are: ID, CURP, filled application format, list of Before each meeting participants and the power of attorney. We have delivered your DGM application! Congratulations on the effort and March 29 the accomplishment of the goal, be aware of the results! The DGM results will be out on May 2019. You have achieved the first goal! May 5 Remember that this effort is made to give your children a better life. Our accompaniment finishes here. If you have any questions, please call the Rainforest Alliance at 018009530057. You can also check the webpage May 5 www.DGM-mexico.org 57 Appendix II: Omnibus balance tests Omnibus Balance test using locality-level sample and variables. T0 vs T1 T0 vs T2 T1 vs T2 Yucatán (dummy) 0.196 0.541 -0.060 (0.767) (0.730) (0.715) [0.799] [0.459] [0.933] Marginalized Locality Index 2010 -1.261 -0.886 0.793 (0.975) (0.980) (1.000) [0.196] [0.366] [0.428] Proportion of women 3 years or 0.408 0.459 -0.082 older that speak indigenous language (0.491) (0.479) (0.413) [0.406] [0.338] [0.843] Grants awarded from CONAFOR 0.077 0.067 -0.015 between 2007-2017 (0.105) (0.090) (0.070) [0.461] [0.456] [0.829] Proportion of adult women in -1.953 -6.956 -4.068 adult population (6.780) (5.770) (5.381) [0.773] [0.228] [0.450] Adult Population 0.004 0.001 -0.002 (0.003) (0.003) (0.003) [0.275] [0.639] [0.603] Constant 1.185 3.731 1.897 (3.531) (3.033) (2.821) [0.737] [0.219] [0.501] Chi2 2.997 3.142 1.649 p-value 0.8092 0.7908 0.9490 Pseudo R2 0.0577 0.0448 0.0235 N 73 76 77 Specification Logit Logit Logit Base T0 T0 T1 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by agrarian unit in parenthesis. Coefficients are log odds. 58 Omnibus Balance test using survey sample and variables. T0 vs T1 T0 vs T2 T1 vs T2 Married or in a partnership 0.056 -0.012 -0.068 (dummy) (0.162) (0.179) (0.188) [0.732] [0.946] [0.719] Educational level of complete 0.317 0.437 0.108 primary or lower (dummy) (0.218) (0.256)* (0.270) [0.145] [0.087] [0.690] Age -0.010 -0.025 -0.015 (0.010) (0.015)* (0.015) [0.356] [0.095] [0.308] Has children (dummy) 0.816 0.273 -0.474 (0.307)*** (0.230) (0.292) [0.008] [0.234] [0.105] Household size -0.073 -0.019 0.058 (0.056) (0.048) (0.053) [0.193] [0.692] [0.271] Identifies with indigenous group -0.119 -0.025 0.076 (dummy) (0.244) (0.235) (0.261) [0.627] [0.916] [0.771] Speaks indigenous language -1.162 -0.238 0.921 (dummy) (0.560)** (0.574) (0.563) [0.038] [0.678] [0.102] Yucatán (dummy) 0.764 0.525 -0.257 (0.917) (0.874) (0.873) [0.405] [0.548] [0.768] Constant 0.076 0.593 0.440 (0.539) (0.628) (0.543) [0.887] [0.344] [0.418] Wald Chi2 23.5597 4.6559 9.6796 p-value 0.0027 0.7936 0.2882 Pseudo R2 0.0546 0.0155 0.0416 N 937 1,008 993 Specification Logit Logit Logit Base T0 T0 T1 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by localities in parenthesis. Coefficients are log odds. 59 Appendix III: Regression tables without covariates Applications Female All-women Evaluated Selected Total Total leader group people women T1 2.459 1.189 0.135 0.053 8.416 4.127 (0.708)*** (0.436)*** (0.051)** (0.056) (2.702)*** (1.310)*** [0.001] [0.008] [0.010] [0.340] [0.003] [0.002] T2 3.022 2.050 0.550 0.072 17.164 10.544 (0.846)*** (0.664)*** (0.209)*** (0.063) (5.821)*** (3.625)*** [0.001] [0.003] [0.010] [0.255] [0.004] [0.005] Adj Base .0278 0 0 .0278 .611 .306 mean N 113 113 113 113 113 113 Specification OLS OLS OLS OLS OLS OLS Covariates No No No No No No Base T0 T0 T0 T0 T0 T0 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by agrarian unit in parenthesis. Applications Female All-women Evaluated Selected Total Total leader group people women T0 -2.459 -1.189 -0.135 -0.053 -8.416 -4.127 (0.708)*** (0.436)*** (0.051)** (0.056) (2.702)*** (1.310)*** [0.001] [0.008] [0.010] [0.340] [0.003] [0.002] T2 0.564 0.861 0.415 0.019 8.748 6.418 (1.103) (0.795) (0.215)* (0.074) (6.358) (3.829)* [0.611] [0.282] [0.057] [0.799] [0.173] [0.098] Adj Base 2.486 1.189 .135 .081 9.027 4.432 mean N 113 113 113 113 113 113 Specification OLS OLS OLS OLS OLS OLS Covariates No No No No No No Base T1 T1 T1 T1 T1 T1 * p<0.1; ** p<0.05; *** p<0.01. P-values in squared brackets. Robust standard errors clustered by agrarian unit in parenthesis. 60 Appendix IV: No application evaluated vs. At least one application evaluated: Only localities from outside experimental sample. No evaluation (Not in sample) At least 1 application evaluated (Not in sample) Variable Obs Mean Std. Dev. Min Max Obs Mean Std. Dev. Min Max Total inhabited houses 66 179.92 214.05 2 1,084 82 245.72 276.99 1 1,246 Total population 66 736.89 885.85 6 4,939 82 976.38 1,083.54 1 4,763 Proportion female 65 0.52 0.04 0.39 0.64 81 0.51 0.03 0.29 0.56 Proportion adult 65 0.60 0.08 0.41 0.94 81 0.61 0.07 0.43 0.82 Proportion of 3 years or older that speaks indigenous language 65 0.60 0.37 0 1 81 0.36 0.35 0 1 Average grades of schooling for 15 years or older 65 5.51 1.13 3.16 9.54 81 6.04 1.12 2.08 9.68 Marginalization index = Very High 65 9.23% 81 3.70% Marginalization index = High 65 86.15% 81 80.25% Proportion of 12 years or older unemployed 65 0.01 0.01 0 0.06 81 0.01 0.02 0 0.15 61