Policy Research Working Paper 10416 Improving Reading Abilities, Attitudes and Practices during COVID Results from a Home-Based Intervention of Supplementary Texts for Young Readers in Cambodia Michael F. Crawford David Rutkowski Leslie Rutkowski Education Global Practice April 2023 Policy Research Working Paper 10416 Abstract This paper provides results from the randomized control show that providing books in isolation was not enough. trial project, Promoting Development and Home Reading Rather, books in conjunction with a network of reading of Supplementary Texts for Young Readers in Cambodia. supports was found to be an effective means to boost One control and three treatment groups were assessed on reading outcomes, including reading proficiency measures, how literacy and reading habits changed when households frequency of reading, and attitudes toward reading. The were provided a variety of high-quality and low-cost early results highlight the need for at-home reading materials in reading materials along with varying degrees of encourage- poor households as an integral step to improve early reading. ment toward building better reading habits. The findings This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at mcrawford1@worldbank.org, drutkows@io.edu, and lrutkows@iu.edu. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Improving Reading Abilities, Attitudes and Practices during COVID: Results from a Home-Based Intervention of Supplementary Texts for Young Readers in Cambodia Michael F. Crawford, The World Bank David Rutkowski, Indiana University Leslie Rutkowski, Indiana University This paper was authored Michael F. Crawford, David Rutkowski, and Leslie Rutkowski and received substantial contributions from Amberine Huda and Ariana Maria Del Mar Grossi. JEL: C93, I20, I21, I28, O15 Keywords: Quality education, literacy, early grade reading, parental involvement, teaching and learning materials, supplementary reading materials, reading ability, learning poverty, field experiment, Cambodia, improvement in literacy, reading at home, reading habit, reading attitude. Introduction The ability to read is a central and defining feature of literacy, along with the ability to use oral language, listen, and to write (National Institute of Child Health and Human Development, 2000). Reading ability is a complex skill requiring several years of instruction and practice (Castles et al., 2018), with a large share of that work needing to take part outside the classroom (Shanahan, 2022). In addition, successful reading acquisition requires easy access to age-appropriate books, which is associated with more reading practice by children, and improved attitudes toward reading, and greater enjoyment of reading (Willingham, 2015). Further, more access to books results in more reading and more reading leads to better literacy development (Krashen, 2012; Merga, 2016). A key component to reading outside the classroom is access to high-quality, age-appropriate reading material. Not surprisingly, a lack of linguistically stimulating home environments, which is often associated with children from poor households, is associated with lower literacy skills (Can & Ginsburg-Block, 2016). Moreover, at the macro level, in low- and middle-income countries (LMICs), many households lack sufficient children’s books (UNICEF, 2017), posing a clear barrier to improved reading literacy. According to the World Bank (2021), by the end of primary school 53 percent of children in low- and middle-income countries cannot read or understand a simple story. Further, as noted by Wagner (2018), many of those children cannot read a single word even after attending school for two or three years. For example, a 2010 study in Cambodia, where the current study takes place, found that 48% of children were unable to read a single word (RTI International, 2015). Although the situation has somewhat improved within Cambodia since 2010, UNICEF estimated that only 27 percent of 3-5 year old’s are developmentally on track in literacy (UNICEF, 2020). Finally, the impact of the COVID-19 pandemic only exacerbated learning challenges. In fact, 29 of 100 million children that the UN regards as falling behind due to the pandemic live in Eastern and South-Eastern Asia (United Nations, 2022). According to the OECD, even in school, only about half of 15-year-olds are in schools where every student has at least one Khmer textbook (MoEYS, 2018). In the same study, results indicated that just 8% of 15-year-olds in school and 2.1% of all 15-year-olds in the Cambodian population are minimally proficient readers. Book access and availability has been constrained in Cambodia by supply shortages of both school textbooks and supplementary reading materials, with an acute lack in children’s storybooks. Initial technical meetings conducted by World Bank staff with non-governmental organizations (NGOs) in Cambodia suggested that close to 1,000 children’s titles exist in Cambodia, with fewer than 200 titles available for children in Grade 2 and below. In contrast, in the United States industry research shows over a billion dollars in annual revenue from sales of children’s books. The lack of unique book titles may be attributed to the lack of demand for early grade reading materials, which in turn can contribute to low reading skills and a dampened culture of reading. Further, as Cambodia is a middle-income country with large amounts of poverty, even a small increase in price at any level of distribution affects the demand for children’s books. As one possible way to reduce barriers to reading resources and improve reading literacy in Cambodia, the current study employed an experimental intervention where high-quality but low-cost books were distributed to early grade readers to examine if book recipients’ early literacy improved. Improving literacy in Cambodia will require investments of time and resources, especially in schools. For example, the Cambodian government currently spends just 2.2 percent of its GDP on education, which is lower than both the regional average as well as the average for its income group (World Bank, 2020). Further, per student spending in Cambodia is the lowest among 81 countries that participated in an international assessment of math, science, and reading (MoEYS, 2018). Importantly, the levels of spending in Cambodia are considered to be insufficient for meeting the UN Sustainable 2 Development Goal in Education to achieve minimum proficiency in reading by 2030 (UN Economic and Social Commision for Asia and the Pacific, 2022). Further, most Cambodian children also lack adequate educational resources at home, which are often seen as necessary for educational success (Mullis et al., 2016). The current study focused on home reading resources and whether providing young Cambodians with high quality reading resources improves their reading proficiency. The Study In response to the low supply of quality children’s books and low early grade literacy skills across Cambodia, in March 2020, with funding from the Results in Education for All Children (REACH) grant and implemented by World Education Inc. (World Education) the World Bank, in cooperation with the Cambodian Ministry of Education, Youth, and Sports (MOEYS), launched the Promoting Development and Home Reading of Supplementary Texts for Young Readers in Cambodia also known as the Home-Based Reading (HBR) intervention. This randomized controlled trial study aimed to assess how literacy and reading habits changed when households were provided with a variety of high-quality supplementary reading materials and varying degrees of parental and child involvement or encouragement towards building better reading habits. Specifically, the experiment set out to see if access to high-quality reading materials and outside encouragement to engage with those materials leads to improved reading skills. The Home-Based Reading (HBR) intervention focused on students in Grades 1 and 2 across three districts in the Kampong Thom and Battambang provinces of Cambodia. These provinces were chosen based on findings from the USAID All-Children Reading-Cambodia intervention which found that children across Battambang and Phnom Penh provinces were underdeveloped in their pre-literacy skills and oral language development. For example, Grade 1 Early Grade Reading Assessment (EGRA) results in Cambodia revealed less than 30% accuracy in students’ ability to recognize and identify Khmer letters, suggesting a disconnect between instruction and at-home practices (DeStefano, Pressley, & King, 2018). During the initial home visit, home reading friends (HRFs) obtained consent from all participants in the study per household, and administered the baseline EGRA, child survey and caregiver survey for the control and all treatment groups. They also provided caregivers with instructions and reminders on how to encourage their child to read and record their experience reading a particular book. During this initial house visit the HRF provided the caregiver with a phone credit incentive for reporting on books read. During each subsequent house visit, HRFs reaffirmed consent from the caregiver and child, did each child and caregiver survey, and provided reminders on how to document reading of books from the package. In the final house visit, HRFs administered the endline surveys and the endline EGRA assessment. Research Design A four-group experimental design study was developed with one control group and three treatment groups (see Figure 1). The first group (control) did not receive any books or encouragement but were administered a baseline and endline assessment and background survey. All treatment groups were administered this baseline and endline assessment as well as the background survey. Each treatment group was given a different treatment. Treatment group 1 (T1) received a series of book packages across three house visits but did not receive any encouragement to read the books. Treatment group 2 (T2) students received the same book packages as T1 and their caregivers received weekly text message reminders to encourage their students to read. Treatment group 3 (T3) students received the same book packages as T1 and T2, received the same text messages as T2, and received phone calls and home visits from a home reading friend who was tasked with managing the data collection and encouraging participants to read. Each child in the treatment groups was asked to place stickers on each book they finished. In addition, caregivers were asked to take a photo of the child holding the book and send it to 3 the home reading friend via Facebook Messenger. Caregivers were also provided with a small phone credit to cover data charges. Figure 1: Study design Control Group • Baseline and endline home visit • Baseline and endline EGRA Treatment 1 • Baseline, two midline, and endline home visits • Baseline and endline EGRA • Book packages • Written instructions for caregivers Treatment 2 • Baseline, two midline, and endline home visits • Baseline and endline EGRA • Book packages • Written instructions for caregivers • Weekly automated text messages encouraging caregivers to read with their child Treatment 3 • Baseline, two midline, and endline home visits • Baseline and endline EGRA • Book packages • Weekly automated text messages encouraging caregiver to read with their child • Real-time access to and weekly phone calls from project staff Sampling Design The sample was drawn from the Banan, Kamong Svay and Santuk districts (see Figure 2). The sample included 504 households (126 in the control group and 378 households across three treatment groups). Between the beginning of the project in February of 2020 and the end in December of 2020, a total of 32 households dropped from the project, leaving 472 households at the end of the study, of which 114, 117, 120, and 121 were in each of the treatment groups, described subsequently. The sample of households that was maintained throughout the intervention included 107 students from the Banan district of Battambang province, 92 students from the Kampong Svay district, and 109 students from the Santuk districts of Kampong Thom province. 4 Figure 2: Map of selected districts for the intervention in Cambodia Source: HBR Project, World Bank World Education conducted the process of selecting all participating households for the intervention. Using data from the Cambodian Ministry of Education, all primary schools across the selected districts of Banan, Kamong Svay and Santuk were used (see Figure 2). Once schools were randomly chosen, an equal number of boys and girls were selected from each school. School Directors worked with World Education and invited each of the selected students’ caregivers to the school to inform them about the study and obtain consent when applicable. Requirements for participation included that no other participating household was located within 100 meters and if the caregiver of the household owned a smartphone, had a Facebook account, or could access Facebook messenger. Households who agreed to participate were then randomly placed into the Control, Treatment 1 (T1), Treatment 2 (T2), or Treatment 3 (T3) groups. Implementation In what follows, we first explain how the study progressed despite the disruptions caused by the COVID-19 pandemic. We then provide a detailed description of study design and implementation. The timeline was interrupted by the COVID-19 pandemic (see Figure 3). Figure 3 provides a summary of all household visits which began in March 2020. Cambodia initiated a national lockdown in mid-March which delayed parts of the study. As can be seen in Figure 3, the baseline and subsequent visits were staggered for the pre-lockdown sample (which refers to the part of the sample that received the first visit of the intervention before the lockdown started, and which comprises a total of 174 households), and the second one designated the main sample (which refers to the part of the sample that received their first visit after the COVID-19 outbreak while they were in lockdown, and which comprises a total of 308 households). Table 1 provides an overview of the number of students for the pre-lockdown and main sample groups at baseline and endline. Notable here is some imbalance due to the pandemic. There is also some attrition between baseline and endline. 5 Figure 3: Data collection timeline Source: HBR Project, World Bank Table 1: Number of students interviewed for pre-lockdown and main sample by treatment group Baseline Control T1 T2 T3 Total Pre-Lockdown Sample 15 74 44 41 174 Main Sample 111 52 82 85 330 Total 126 126 126 126 504 Endline Control T1 T2 T3 Pre-Lockdown Sample 14 69 41 40 164 Main Sample 100 48 79 81 308 Total 114 117 120 121 472 Study Design and Implementation For all groups, the study featured baseline and endline EGRAs as well as student and caregiver questionnaires. In all treatment groups, caregivers and children also responded to a survey during home visits 2 and 3. These questions were geared toward reading behaviors associated with books delivered as part of the intervention. In addition, the study intervention featured home reading friends (HRFs), who conducted the home visits to establish a rapport with caregivers, administered surveys and assessments, and delivered packages of books to the treatment groups. In the treatment groups, they also instructed children to read the book package, preferably out loud with their caregiver for 10-15 minutes at a time. 6 They reminded caregivers to post photos to the designated Facebook messenger group of their children holding any book they finished reading. In what follows we explain the instruments used in the study. Books The objective of book selection was twofold: (i) determine the most appropriate and enjoyable books to provide to Grade 2 Khmer students, and (ii) determine how many books should be included in each package. A field test was conducted with six Grade 2 students (three boys and three girls), who were chosen by their teachers as average readers from a school in the Kampong Savay district. One caregiver also participated in the field study. A total of 77 books were chosen to be candidates for inclusion in the book packages. Fifty-nine (59) books were chosen to be included in three roughly equal-sized packages. The final set of books included a variety of NGO-produced storybooks. Close to half the books were titles that had been used by USAID in a Grade 1 school-based reading intervention in these provinces conducted prior to the pandemic. As a result, students, who were Grade 2 age, would have been exposed to roughly at least half of the books in their classrooms as first graders. Packages of books were delivered to children in all treatment groups during the baseline, home visit 1 and home visit 2. The packages included 24, 22, and 13 books respectively. Stickers, Photos, and Incentives Several tools were used to collect data on student reading. First, each child was given a set of stickers and asked to place one on the book upon completion to show that they had finished reading it. They were asked to choose either a happy or sad faced sticker according to how they enjoyed the book. Children were allowed to read the book more than once, but no data was collected on the second reading. Home reading friends counted the number of stickers on each book during their home visits and asked the students how many books they read to triangulate the results. In addition, caregivers were provided with a phone credit of $4-$8 throughout the study so that they could post a photo of their child with each book upon completion to a private Facebook group created for the study. Assessment For all groups, a baseline and endline Early Grade Reading Assessment (EGRA) was administered. EGRA assesses a core set of reading subskills that constitute early grade literacy. EGRAs have been used in over 60 low- and middle-income countries since the invention of the EGRA protocols in 2005. The Khmer EGRA instrument used for this assessment was adapted from existing assessments created through collaborations of the MoEYS and several non-governmental organizations. The EGRA dimensions are provided in Appendix 1. Surveys At baseline, all participating children were asked several questions about their reading attitudes and behaviors. During home visits 2, 3, and the endline, children in all treatment groups were asked a series of questions about the books they received as part of the intervention (i.e., which books they read, which books they liked, with whom they read the books). In addition, several questions from the baseline survey were repeated during the endline. In addition to the children surveys, at baseline, one caregiver per family was asked several questions about basic demographics, their child’s reading behaviors and attitudes toward reading, and questions about home storybook ownership and the caregiver’s disposition toward buying storybooks. Caregivers in the treatment groups were asked a series of questions about reading attitudes and behaviors. Caregivers were also asked about how well they adhered to instructions for the study, whether owning storybooks was a good idea, and the caregiver’s perceptions about reading improvement during the study. 7 Results In Table 2, we include basic counts of the sample demographics across all the study control and treatment groups. Here, we can see that the samples are balanced across sex, province, and district and that there is a distribution of ages across each of the study groups. Table 2: Basic baseline demographic counts across treatment and control groups Control T1 T2 T3 Total Sex Female 63 63 63 63 252 Male 63 63 63 63 252 Age in years 6 1 6 3 1 11 7 22 25 35 31 113 8 41 53 55 54 203 9 29 21 21 26 97 10 33 21 12 14 80 Province Battambang 42 42 42 42 168 Kampong Thom 84 84 84 84 336 District Banan 42 42 42 42 168 Kampong Svay 42 42 42 42 168 Santuk 42 42 42 42 168 In Table 3, we also include baseline results for selected reading attitudes and behavior measures, as measured from the child. Here, we can see that although most children read textbooks at home at least a few times a week, most never read storybooks or did not respond. Table 3: Baseline reading attitude and behaviour measures Control T1 T2 T3 Total How often do you read school textbooks at home? Every day 26 21 31 26 104 A few times a week 68 56 59 56 239 One time a week 3 13 4 20 40 Never 0 0 0 0 0 Other 0 1 1 1 3 Missing 29 35 31 23 118 How often do you read storybooks at home? Every day 0 2 1 2 5 A few times a week 4 5 4 2 15 One time a week 0 3 2 2 7 Never 93 81 88 96 358 8 Other 0 0 0 1 1 Missing 29 35 31 23 118 What amount of reading would you like to do in the future? More reading of storybooks at home 83 93 88 87 351 The same amount of reading as now 30 24 28 27 109 Less reading 11 5 9 11 36 Missing 2 4 1 1 8 Finally, we include a summary of the caregiver response to the question “If there was a nice storybook available for your child that cost 4,000 riel [approximately $1 USD], would you buy it?” Except for two T1 and 1 T2 caregivers, all others (n = 501) responded “yes.” In what follows missing values do not distinguish between households that dropped from the study and those that chose not to respond to a given question or questions. As such, the total sample size sums to 126 in each treatment group, with some reflecting non-response and some reflecting attrition. In Tables 4 and 5, we include responses from the caregiver questionnaire about the frequency of reading school textbooks and storybooks. Results are reported for baseline and endline in both control and treatment conditions. Note, however, that the wording of the question about reading frequency is different for endline treatment caregivers. For baseline caregivers (all conditions) and the control caregivers, the questions are worded as follows: “How often does your child read school textbooks (storybooks) at home?” For endline treatment caregivers, the question is worded as: “In the last two months since I last saw you, how often did your child read school textbooks (storybooks) at home?” Notable changes between baseline and endline for the frequency of reading school textbooks are that substantial proportions in the treatment groups reported never reading schoolbooks at home. This can likely be explained by the pandemic-associated school closures. We also note that the percentage of missing responses for the control group doubles whereas missing percentages fall for all treatment groups. 9 Table 4: Caregiver responses about frequency of reading school textbooks Baseline Control T1 T2 T3 Every day 29% 22% 26% 26% A few times a week 52% 45% 43% 49% One time a week 6% 10% 10% 11% Never 0% 0% 0% 0% Don't know 0% 0% 0% 0% Other 0% 0% 0% 1% Missing 13% 22% 21% 13% N 126 126 126 126 Endline Control T1 T2 T3 Every day 20% 18% 16% 13% A few times a week 36% 40% 40% 50% One time a week 16% 19% 13% 14% Never 0% 15% 26% 18% Don't know 0% 0% 0% 0% Other 1% 1% 0% 1% Missing 28% 7% 5% 4% N 126 126 126 126 For caregiver response to the frequency of reading storybooks at home, we note several interesting findings. First, at baseline, of those that responded, most caregivers across all groups report that their children never read storybooks at home. By contrast, at endline, nearly all treatment group caregivers reported that their children read from one time per week to every day (T1 = 92%, T2 = 95%, T3 = 96%). The control group continued to report high percentages of never. We also note that missing rates dropped substantially for all treatment groups at endline, whereas the missing rate doubled in the control group. 10 Table 5: Caregiver response to frequency of reading storybooks Baseline Control T1 T2 T3 Every day 1% 1% 1% 2% A few times a week 2% 2% 2% 2% One time a week 1% 4% 2% 4% Never 84% 71% 75% 79% Don't know 0% 0% 0% 0% Other 0% 1% 0% 1% Missing 13% 22% 21% 13% N 126 126 126 126 Endline Control T1 T2 T3 Every day 1% 20% 22% 26% A few times a week 6% 50% 52% 59% One time a week 1% 22% 21% 11% Never 65% 0% 0% 0% Don't know 0% 0% 0% 0% Other 0% 1% 0% 0% Missing 28% 7% 5% 4% N 126 126 126 126 In Tables 6 and 7 we report children’s response to the question around frequency of reading school textbooks and storybooks at home. Again, the questions are worded differently for baseline (all groups) and endline (control) compared to treatment endline. For baseline children (all conditions) and the control endline, the questions are worded as follows: “How often do you read school textbooks (storybooks) at home?” For endline treatment children, the question is worded as: “Since I last saw you, how often have you read school textbooks (storybooks) at home?” Notable findings from Table 7 show substantial shifts to less frequent reading for all groups, with a large percentage of children in all treatment groups reporting an increase in never reading schoolbooks and a doubling of the percentage of missing responses in the control group. These shifts likely reflect the impact of the pandemic-related school closures. 11 Table 6: Child response to frequency of reading textbooks Baseline Control T1 T2 T3 Every day 21% 17% 25% 21% A few times a week 54% 44% 47% 44% One time a week 2% 10% 3% 16% Never 0% 0% 0% 0% Other 0% 1% 1% 1% Missing 23% 28% 25% 18% N 126 126 126 126 Endline Control T1 T2 T3 Every day 10% 0% 0% 0% A few times a week 34% 33% 35% 40% One time a week 10% 23% 13% 14% Never 0% 25% 34% 29% Other 0% 1% 0% 0% Missing 47% 18% 17% 17% N 126 126 126 126 Table 7 shows that, at baseline, most children report never reading storybooks. By contrast, at endline, most children in all treatment groups report reading from one time per week to every day (T1 = 89%, T2 = 93%, T3 = 95%). This finding is especially notable in contrast to the control group, where most children responded never or did not respond to this question. 12 Table 7: Child response to frequency of reading storybooks Baseline Control T1 T2 T3 Every day 0% 2% 1% 2% A few times a week 3% 4% 3% 2% One time a week 0% 2% 2% 2% Never 74% 64% 70% 76% Other 0% 0% 0% 1% Missing 23% 28% 25% 18% N 126 126 126 126 Storybooks Control T1 T2 T3 Every day 0% 18% 24% 25% A few times a week 5% 50% 48% 56% One time a week 0% 21% 21% 14% Never 48% 3% 2% 1% Other 0% 1% 0% 0% Missing 47% 7% 5% 4% N 126 126 126 126 We now turn to a summary of changes in reading duration, as reported by caregivers. Caregivers at baseline in all groups and at endline in the control group were asked “How long does your child usually read each time they read school textbooks (storybooks)?” whereas caregivers at endline in treatment groups were asked “How long did your child usually read each time they read school textbooks (storybooks)? Tables 8 and 9 present caregiver results for duration of reading schoolbooks and storybooks. Table 8 shows that missing rates in all groups except T1 increased between baseline and endline. Otherwise, there did not appear to be systematic shifts in the duration of reading schoolbooks between baseline and endline across these groups. By contrast, there are several striking findings related to changes in the duration of reading storybooks. Table 9 shows that the majority of caregivers did not respond to this question at baseline. At endline, the percentage of missing response remained very high in the control group whereas most treatment caregivers reported that their children read between five and 30 minutes (T1 = 84%, T2 = 80%, T3 = 74%). Also notable is that 20% of T3 caregivers reported that their children read longer than 30 minutes. 13 Table 8: Caregiver response to child reading duration of schoolbooks Baseline Control T1 T2 T3 Below 5 minutes 2% 2% 2% 4% 5-15 minutes 34% 31% 39% 46% 15-30 minutes 40% 37% 33% 34% Over 30 minutes 11% 8% 5% 3% Don't know 0% 0% 0% 0% Missing 13% 22% 21% 13% N 126 126 126 126 Endline Control T1 T2 T3 Below 5 minutes 0% 2% 2% 3% 5-15 minutes 24% 41% 37% 35% 15-30 minutes 25% 29% 25% 25% Over 30 minutes 23% 2% 4% 13% Don't know 0% 2% 1% 2% Missing 28% 22% 31% 22% N 126 126 126 126 Table 9: Caregiver response to child reading duration of storybooks Baseline Control T1 T2 T3 Below 5 minutes 0% 2% 0% 2% 5-15 minutes 2% 4% 2% 2% 15-30 minutes 1% 2% 2% 3% Over 30 minutes 0% 0% 0% 1% Don't know 0% 0% 0% 2% Missing 97% 93% 96% 91% N 126 126 126 126 Endline Control T1 T2 T3 Below 5 minutes 0% 3% 4% 2% 5-15 minutes 4% 43% 48% 35% 15-30 minutes 2% 41% 33% 39% Over 30 minutes 2% 5% 10% 20% Don't know 0% 1% 1% 1% Missing 93% 7% 5% 4% N 126 126 126 126 14 Attitudes toward reading In what follows, we report changes in several variables that measured children’s attitudes toward reading. We begin with changes in the caregivers’ response to the question “How much does your child seem to like reading storybooks?” which is reported in Table 10. At baseline, nearly all caregivers did not respond to this question. Further, at endline, a similar percentage of control caregivers also did not respond. In contrast, and strikingly, an overwhelming majority of caregivers in the treatment groups reported that their child likes/loves reading storybooks or feels OK/so-so about reading storybooks (T1 = 91%, T2 = 93%, T3 = 95%). Table 10: Caregiver response to child attitude toward reading storybooks Baseline Control T1 T2 T3 I like it a lot/love it 1% 6% 2% 1% It's OK/So-so 2% 2% 3% 5% I don't like it much 0% 0% 0% 0% I don't like it at all/I hate it 0% 0% 0% 0% Missing 97% 92% 94% 94% N 126 126 126 126 Endline Control T1 T2 T3 I like it a lot/love it 3% 62% 55% 66% It's OK/So-so 2% 0% 0% 0% I don't like it much 0% 0% 0% 0% I don't like it at all/I hate it 0% 0% 0% 0% Missing 95% 38% 45% 34% N 126 126 126 126 Table 11 presents children’s responses to a similar question about liking reading storybooks. All children in all groups at baseline and endline were asked “How much do you like reading storybooks?” Similar to caregiver responses, most children across all groups did not respond to this question at baseline. The preponderance of control children at endline also did not respond. Of treatment children at endline that responded to this question, they universally endorsed the option “I like it a lot/love it.” That said, substantial percentages of missing responses remain for endline treatment children. 15 Table 11: Child response to attitude toward reading Baseline Control T1 T2 T3 I like it a lot/love it 1% 6% 2% 1% It's OK/So-so 2% 2% 3% 5% I don't like it much 0% 0% 0% 0% I don't like it at all/I hate it 0% 0% 0% 0% Missing 97% 92% 94% 94% N 126 126 126 126 Endline Control T1 T2 T3 I like it a lot/love it 3% 62% 55% 66% It's OK/So-so 2% 0% 0% 0% I don't like it much 0% 0% 0% 0% I don't like it at all/I hate it 0% 0% 0% 0% Missing 95% 38% 45% 34% N 126 126 126 126 Next, we present the results of responses to endline treatment caregivers and children about their perceived improvement in reading ability. Caregivers were asked “First of all let’s talk about if you think your child is better at reading. Do you think your child is:” with response options “Better at reading before the Home Reader project (before we first met you and gave you the books)”; “The same as before”; “Worse at reading”; or missing. Table 12 shows results for caregiver responses to this question, where large proportions of caregivers indicated that their child is reading better than before participating in the study, with the largest proportion of T3 caregivers endorsing this response. 16 Table 12: Caregiver response to perceived reading improvement T1 T2 T3 Better at reading than before 74% 83% 88% The same as before 13% 9% 5% Worse at reading 6% 3% 4% Missing 7% 5% 4% N 126 126 126 Table 13 presents children’s responses to a similar question: “First of all let’s talk about if you think you are better at reading. Do you think you are?” with response options that are essentially identical to caregivers’. Again, large proportions of children perceived themselves as better readers than before participation, with the largest percentage in T3 (76%). Also interesting was that 25% of T1 and 21% of T2 children felt that they read the same as before, whereas just 11% of T3 children felt that their reading ability had not changed. Table 13: Child response to perceived reading improvement T1 T2 T3 Better at reading than before 60% 69% 76% The same as before 25% 21% 11% Worse at reading 7% 5% 10% Missing 7% 5% 4% N 126 126 126 Caregivers and children were also asked about changes in reading confidence. Caregivers responded to the question: “Now let’s talk about if you think your child feels more confident about reading. Does your child feel:” with response options “More confident about reading”; “The same as before”; “Less confident than before”; or missing. Children were asked “Now let’s talk about if you feel more confident (brave) about reading. Do you feel?” with response options that are identical to caregiver options. Table 14 shows results of both caregivers and children to the question about reading confidence. In general, both caregivers and their children perceived improvements in the child’s reading confidence; however, higher percentages of caregivers endorsed this category. Further, percentages were highest in treatment group 3. Interestingly, 29% of children in T1 felt that their reading confidence was the same as before. 17 Table 14: Caregiver and child response to perceived reading confidence at endline T1 T2 T3 Caregiver Child Caregiver Child Caregiver Child More confident at reading than before 72% 61% 83% 71% 90% 82% The same as before 18% 29% 12% 21% 6% 14% Less confident than before 2% 3% 1% 4% 0% 0% Missing 7% 7% 5% 5% 4% 4% N 126 126 126 126 126 126 Next, we present results to a question posed to both caregivers and children around changes in enjoyment of reading. Although this question is similar to the results in Table 11 (How much do you like reading storybooks), this question targets reading in general and also how enjoyment has changed over the duration of the study. The caregiver question reads “Finally, let’s talk about if your child enjoys reading. Does your child:”, with response options “Enjoy reading more now compared to before we gave you any books”; “Feel the same”; and “Enjoy reading less than before we gave you books.” The children’s question reads “Finally, let’s talk about if you enjoy reading. Do you?” with response options that are identical to caregivers. Results are reported in Table 15 for both caregivers and children. As with other reading attitude measures, both caregivers and children resoundingly agreed that children enjoyed reading more than before participating in the study. We note some discrepancies, however, in these proportions, where fewer children in T1 and T2 felt like they enjoyed reading more as a result of the study. In T3, by contrast, similar percentages of caregivers and children agreed that the children enjoyed reading more as a result of study participation. 18 Table 15: Caregiver and child perceived changes in reading enjoyment T1 T2 T3 Caregiver Child Caregiver Child Caregiver Child Enjoy reading more than before 81% 64% 79% 69% 83% 85% Feel the same 12% 26% 15% 24% 13% 10% Enjoy reading less than before 0% 2% 1% 2% 0% 1% Missing 7% 7% 5% 5% 4% 4% N 126 126 126 126 126 126 Finally, we present results from caregiver and child responses to a question around future desires for reading. Caregivers were asked “In future, would you like to do:” with response options “More reading at home with your child”; “The same amount of reading with our child as now”; and “Less reading with your child.” Children were asked “In the future, would you like to do?” with response options “More reading of storybooks at home”; “The same amount of reading as now”; and “Less reading.” Tables 16 and 17 present the baseline and endline results for caregivers and their children, respectively. At baseline, most caregivers in all groups wanted more reading with their child. At endline, a proportion of treatment caregivers shifted their responses from more reading to the same amount of reading as now. This likely reflects their satisfaction with the intervention. Similar to caregivers, at baseline most children in all groups wanted to read storybooks more. And like the caregivers’ responses, a substantial proportion of treatment children shifted their responses from more reading to the same reading. Again, this shift likely reflects the context of the study – that children were happy with the intervention and its impact on increased reading, which the children hoped to sustain. Table 16: Caregiver future reading intentions Baseline Control T1 T2 T3 More reading at home with your child 83% 82% 84% 85% Same amount of reading 17% 15% 16% 15% Less reading with your child 1% 3% 0% 0% Missing 0% 0% 0% 0% N 126 126 126 126 Endline Control T1 T2 T3 More reading at home with your child 81% 71% 69% 79% Same amount of reading 10% 21% 26% 17% Less reading with your child 0% 0% 0% 0% Missing 10% 7% 5% 4% N 126 126 126 126 19 Table 17: Child future reading intentions Baseline Control T1 T2 T3 More reading of storybooks 67% 74% 70% 69% Same amount of reading 24% 19% 22% 21% Less reading 9% 4% 7% 9% Missing 0% 3% 1% 1% N 126 126 126 126 Endline Control T1 T2 T3 More reading of storybooks 70% 54% 50% 55% Same amount of reading 15% 29% 35% 35% Less reading 5% 9% 10% 6% Missing 12% 8% 5% 4% N 126 126 126 126 Changes in EGRA Results Since the EGRA measures were not normally distributed, nonparametric methods were utilized in this analysis. This analysis returns 3 ANOVA type statistics: A main effect of group (do the groups differ on the outcome, averaged over time), a main effect of time (does the outcome differ by time, averaged over groups), and a group time interaction (do the groups have different changes across time, (i.e., different change rate for groups). This analysis results in a p-value, where a p-value smaller than 0.05 indicates that the null hypothesis of no treatment effect can be rejected. In other words, since the interaction effect between group and time was of primary interest, statistical significance would suggest that some groups experienced a different change in the outcome variable between time 1 and 2. After a significant overall group time interaction was discovered, pairwise comparison of groups was used to establish which group(s) significantly differ from other group(s). The following section describes the results of these analyses for all 10 EGRA measures of interest. Table 18 describes the EGRA results per treatment group. The means of the baseline groups (excluding participants who did not participate in the endline), means of the endline groups, and the differences between baseline and endline are noted. Significantly different rates of change were indicated, but only for treatments that grew significantly more than control or other treatments, as this is the research question of interest. In the table, C, T1, T2, and T3 correspond to control, treatment group 1, treatment group 2, and treatment group 3, respectively. 20 Table 18: Subtest EGRA results per treatment group Subtask Group Type Mean Score Baseline Endline Gain Consonants (TLNC) C 27.88 27.91 0.04 T1 25.81 26.02 0.21 T2 25.56 28.67 3.11,1 T3 27.19 29.45 2.26,1 Vowels (TLNV) C 17.61 18.04 0.44 T1 17.28 16.33 −0.95 T2 17.10 18.50 1.401 T3 17.56 19.11 1.55,1 Correct letter per C 26.22 27.49 1.27 minute (advanced T1 26.55 27.4 1.19 naming, TLNA) T2 23.02 29.85 6.83,1 T3 26.52 32.39 5.87,1 Correct familiar C 16.23 19.96 3.73 words per minute T1 15.49 19.01 3.52 (TFWR) T2 14.61 23.42 8.82,1 T3 17.36 26.07 8.72,1 Oral reading fluency C 24.34 31.11 6.76 (180 seconds, T1 25.91 27.62 1.71 TORF_180Seconds) T2 23.12 36.72 13.59,1 T3 30.68 41.09 10.411 Reading C 1.37 2.03 0.661 comprehension T1 1.42 1.62 0.20 (TCOM) T2 1.23 2.17 0.951 T3 1.71 2.36 0.651 Dictation (TD) C 3.64 3.59 −0.05 T1 3.74 3.97 0.23 T2 3.16 5.10 1.94,1 T3 3.95 5.50 1.56,1 Correct familiar C 12.89 17.15 4.25 words per minute T1 12.31 16.14 3.83 (additional subtest, T2 11.28 20.02 8.75,1 TFWRA) T3 14.51 22.82 8.31,1 Oral reading fluency C 21.01 25.95 4.94 (additional subtest, T1 22.80 26.21 3.41 TORFA_180Seconds) T2 20.64 34.57 13.93,1 T3 27.90 38.67 10.77,1 Reading C 0.81 1.20 0.39 comprehension T1 0.89 1.26 0.37 (additional subtest, T2 0.88 1.52 0.63 TCOMA) T3 1.14 1.92 0.78,1 Note: Differences between groups were tested at the 0.05 level. If a group grows significantly more over time than another group, this is indicated by a superscript of the group letter/number. E.g. C indicates a group grew more than control, 1 indicates a group grew more than T1, etc. 21 In Table 18, several effects can be observed. The following sections describes these effects, mentioning both the overall ANOVA-type test statistic, the group pairwise comparisons, and a graph depicting the interaction for every variable. Consonant Letter Identification (TLNC) A significant group time interaction exists for TLNC ( () = 6.15 = 2.93, ≤ .01). Further pairwise comparisons of groups reveal that the change in treatment group 1 and the control group over time differ significantly from the change in treatment group 2 and treatment group 3. This difference is plotted in Figure 4. Notably, the control and treatment group 3 demonstrate essentially no change over time while treatment groups 1 and 2 experience positive growth. Figure 4: TLNC Scores over time per group As can be seen in Figure 4, treatment group 2 and treatment group 3 increase significantly more over time than the control group and treatment group 1. Vowel Letter Identification (TLNV) Similar to TLNC, a significant overall group time interaction exists for TLNV ( = 4.29, = 2.89 , ≤ .01). Pairwise comparisons between groups reveal a significant difference between the treatment group 1 and treatment groups 2 and 3. Notably, the control group only differs significantly from treatment group 3, and not treatment group 2. Figure 5 depicts the directions of these interactions. 22 Figure 5: TLNV Scores over time per group As we can see in Figure 5, treatment group 3 increases significantly more than the control group. Treatment group 2 also increases more than control, but the difference does not reach significance. Notably, treatment group 1 seems to decrease quite drastically over time. Letter Identification (TLNA) Again, a significant group time interaction exists for the TLNA variable ( = 6.22, = 2.56, ≤ .01). Pairwise comparisons reveal the same pattern as the TLNC variable, where treatment groups 2 and 3 differ significantly from treatment group 1 and control. Figure 6 depicts this trend. 23 Figure 6: TLNA Scores over time per group Figure 6 leads to similar conclusions as Figure 5: Both treatment groups 2 and 3 increase more over time than treatment group 1 and control. Familiar Words Reading (TFWR) Once more, a significant group time interaction emerges for TFWR ( = 6.07, = 2.68 , ≤ .01). Pairwise comparisons show the same patters as discovered earlier for TLNA and TLNC; treatment groups 2 and 3 differ from the control group and treatment group 1. Figure 7 depicts this trend. 24 Figure 7: TFWR Scores over time per group Figure 7 shows a very similar trend to Figure 6; T2 and T3 improve more than C and T1. Oral Reading Fluency (TORF_180Seconds) A significant group time interaction emerges for TORF_180Seconds ( = 4.47, = 2.80, ≤ .01). Treatment groups 2 and 3 differ from treatment group 1, but only treatment group 2 differs from the control group. The difference between group 3 and control does not reach significance. Figure 8 depicts this trend. 25 Figure 8: TORF_180Seconds scores over time per group As can be seen in Figure 8, both T2 and T3 improve more than T1. Only T2 improves significantly more than control. Again, there seems to be more growth for the control group than for T1, which is somewhat surprising. Reading Comprehension (TCOM) Again, a significant group time interaction emerges ( = 5.04, = 2.92, ≤ .01). Pairwise comparisons indicate treatment groups 2 and 3 differ from group 1. Notably, only treatment group 1 differs significantly from the control group. Figure 9 depicts this trend in more detail. 26 Figure 9: TCOM scores over time per group As can be seen in the figure, T2 and T3 grow significantly more than T1. T1 differs significantly from control, but surprisingly grows less than the control group. Dictation (TD) TD, the final main variable of interest, again shows a significant group time interaction ( = 12.94, = 2.80, ≤ .01). Pairwise comparisons reveal the familiar pattern of treatment groups 2 and 3 differing from control and group 1. Figure 10 depicts this trend. 27 Figure 10: TD scores over time per group Figure 10 shows the familiar pattern of change, with treatment groups 2 and 3 increasing significantly more than group 1 and control. Additional Subtests: Familiar Words Reading (TFWRA); Oral Reading Fluency (TORFA_180Seconds); Reading Comprehension (TCOMA) Since the last three variables are additional analyses, we only included results that broke the familiar trend of treatment group 2 and treatment group 3 increasing more than treatment group 1 and control. This is only the case for the TCOMA variable. Only group 3 differs significantly from group 1 and control here, as depicted in Figure 11. Plausible Explanations To better understand the possible explanations for differences across treatment groups, we examined several possibilities. Based on a comparison of means across the treatment groups, we found that the mean number of books did not show a treatment effect. We also considered how scores changed between the baseline and endline for each treatment group. To do so, we plotted the score changes on the seven original EGRA measures. These plots can be found in Figures 12 to 18. Notably, treatment group 1 experienced a decline between the baseline and endline for a subgroup of participants. The deterioration in scores can be seen in each figure as a distribution of change scores below zero. Figures 13 to 16 exemplify this finding. Plausible explanations for this deterioration include the intervention itself – that only the delivery of books was not enough to boost scores relative to the other treatment groups. 28 Importantly, all findings from this study must be viewed in the context of the pandemic associated lockdowns. For example, it might be that all groups were negatively impacted by the pandemic; however, those in the treatment groups were protected in some way from more severe learning losses. Figure 11: TCOMA scores over time per group 29 Figure 12: Change in total letter naming, consonants Figure 13: Change in total letter naming, vowels Figure 14: Change in total letter naming 30 Figure 15: Change in reading familiar words Figure 16: Change in total oral reading fluency Figure 17: Change in total reading comprehension 31 Figure 18: Change in total dictation Summary and Conclusion In this paper, we summarize the results of a randomized controlled trial to improve the early grade reading of 504 Khmer children. The study focused on providing high-quality, age-appropriate storybooks along with a variety of supports for readers and their caregivers. Our findings suggest that providing books alone is not enough. Rather, books in conjunction with a network of reading supports is an effective means to boost reading outcomes, including reading proficiency measures, frequency of reading, and attitudes toward reading. The provision of a significant number of storybooks to households with 2nd grade children, along with reminders and encouragement for home reading, correlates with improvements in readers’ frequency, duration, enjoyment, and abilities in reading. Intensity of reminders and direct contact with a trained “home reading friend” appear to increase positive outcomes. The intervention caused households where children previously did not own or read storybooks to resemble households in higher-income countries where children own and read books regularly. Despite the absence of instruction (because of COVID-related school closures), children in two of the three treatment groups increased their reading abilities and continued to progress as readers. These results suggest several policy-relevant conclusions that merit consideration for expansion in Cambodia and elsewhere. The intervention distributed existing storybooks and as a result was able to move from design to implementation rapidly. Although costs and cost-effectiveness were not formally analyzed, direct costs per book were significantly less than 1 USD equivalent per copy, so direct costs of the three book packages total about USD 30 per household. Other associated costs of administering the intervention raise this figure, but reading outcomes were improved over a period of months with a modest investment of financial and human resources. The intervention was designed to complement school-based reading instruction, but school closures meant this instruction was not provided. It is possible that home reading may further accelerate and raise home reading quantity, enjoyment, and ability when combined with appropriate instruction. The intervention also presents a potential model for improving reading ability among out-of-school 32 children, and children whose schooling is interrupted for other reasons (for example, internal displacement and refugee status). Further research could disaggregate the effects of improved reading attitudes (especially increased reported enjoyment of reading) on reading amount and ability to establish potential directions or causality and interactions among factors. Further research could also seek to isolate the effectiveness of using only books to which children have been previously exposed in a school setting, combinations of familiar and new books (as was the case here), and only novel books on the factors of interests. As a result of COVID-related school closure and inadequate alternative instruction, global rates of learning poverty—the inability to read and understand an appropriate text by age ten—are estimated to be close to 70% in low- and-middle-income countries (World Bank, 2022). Governments are well advised to consider how interventions that furnish households with appropriate supplementary reading materials and systems for the encouragement of home reading may improve literacy outcomes and other important educational goals. Provision of home reading material at scale may profitably become a feature of overall efforts to ensure all children learn to read and gain the foundational skills they require for success in building their human capital over their lifetimes. 33 References Can, D. D., & Ginsburg-Block, M. (2016). Parenting stress and home-based literacy interactions in low- income preschool families. Journal of Applied Developmental Psychology, 46, 51–62. Castles, A., Rastle, K., & Nation, K. (2018). Ending the reading wars: Reading acquisition from novice to expert. Psychological Science in the Public Interest. https://doi.org/10.1177/1529100618772271 DeStefano, J., Pressley, J., & King, S. (2018). All Children Reading – Cambodia Student Performance in Early Literacy: Baseline Report. Washington, DC: USAID. https://earlygradereadingbarometer.org/pdf/Cambodia_2018_ACR.pdf Krashen, S. (2012). Direct instruction of academic vocabulary: What about real reading? Merga, M. K. (2016). “I don’t know if she likes reading”: Are teachers perceived to be keen readers, and how is this determined? English in Education, 50(3), 255–269. MoEYS. (2018). Education in Cambodia: Findings from Cambodia’s experience in PISA for Development. MoEYS. Mullis, I. V., Martin, M. O., & Sainsbury, M. (2016). PIRLS 2016 reading framework. PIRLS, 11–29. National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel: Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups (NIH Publication No. 00-4769). National Institute of Child Health and Human Development. RTI International. (2015). Assessment of Early Grade Reading in the Education Sector in Cambodia (Task Order 15 (DEP-AME); EdData II). USAID. https://pdf.usaid.gov/pdf_docs/PBAAF436.pdf Shanahan, T. E. (2022, March 31). How to provide effective reading instruction [Text/HTML]. World Bank. https://documents.worldbank.org/en/publication/documents- reports/documentdetail/099620103312223967/P17425203c5a110520a5c3004086a91b687 UN Economic and Social Commision for Asia and the Pacific. (2022, March 16). Sustainable development goal 4 profile. ESCAP 75. https://www.unescap.org/kp/2022/sdg-4-goal-profile UNICEF. (2017). State of the Worlds Children 2017-Children in a Digital World. Unicef, 2017. UNICEF. UNICEF. (2020). Cambodia education. Cambodia. https://www.unicef.org/cambodia/education United Nations. (2022). Sustainable Development Goals Report 2021: SDG 4 Indicators. https://unstats.un.org/sdgs/report/2021/goal-04/ Willingham, D. T. (2015). Raising kids who read: What parents and teachers can do. John Wiley & Sons. World Bank. (2020). Cambodia: Human capital index 2020. https://databankfiles.worldbank.org/data/download/hci/HCI_2pager_KHM.pdf?cid=GGH_e_hcp external_en_ext World Bank. (2021). What is learning poverty? [Brief]. World Bank. https://www.worldbank.org/en/topic/education/brief/what-is-learning-poverty World Bank. (2022). The State of Global Learning Poverty: 2022 Update. Washington, DC. https://thedocs.worldbank.org/en/doc/e52f55322528903b27f1b7e61238e416- 0200022022/original/Learning-poverty-report-2022-06-21-final-V7-0-conferenceEdition.pdf 34 Appendix A Consonant letter identification (TLNC)—33 Khmer consonant letters were randomly arranged, and students were asked to correctly identify each. The assessor moved to the next letter if students did not know the answer. The number of correct answers are reported. Vowel letter identification (TLNV)—23 Khmer vowel letters were randomly arranged, and students were asked to correctly identify each. The assessor moved to the next letter if students did not know the answer. The number of correct answers are reported. Letter identification (advanced; TLNA)—students were given a 10 row by 10 column grid of letters and asked to identify as many letters as possible in 60 seconds. Student scores were then converted into a correct letters per minute rate. For this subtest, if a child could not identify or read the first few designated items, the subtest was discontinued. Familiar words reading (TFWR)—students were given a 5 row by 10 column grid of words and asked to read as many as they could in 60 seconds. Their score was then converted into a correct words per minute rate. The words chosen for this subtask were frequently used in the Grade 1–3 textbooks. For this subtest, if a child could not identify or read the first row, the subtest was discontinued. Oral reading fluency (TORF_180Seconds)—students were given a simple reading passage (61 words) and given up to three minutes to read. Their score was converted to a correct words per minute rate. For this subtest, if a child could not identify or read the first few designated items, the subtask was discontinued. Reading comprehension (TCOM)—students were given up to three minutes to completely read a simple reading passage and then asked five reading comprehension questions to assess their ability to understand the meaning of the text they had just read. Dictation (TD)—students were read two short sentences and asked to write them on paper. Student results were scored as ‘correct,’ ‘partially correct,’ or ‘incorrect’ by the assessors. Familiar words reading (additional subtest; TFWRA)—students were given a 5 row by 11 column grid of words and asked to read as many as they could in 60 seconds. Their score was then converted into a correct words per minute rate. The words chosen for this subtask were selected from the storybooks that will be provided to Grade 2 students in Kampong Thom and Battambang as part of the Home-Based Reading project. This subtest was discontinued if a child could not identify or read the first row. Oral reading fluency (additional subtest; TORFA_180Seconds)—students were given a simple reading passage (51 words), which was taken from one of the provided storybooks. Students were given up to three minutes to read the passage. Their score was converted to a correct words per minute rate. For this subtest, if a child could not identify or read the first few designated items, the subtask was discontinued. Reading comprehension (additional subtest; TCOMA)—students were given up to three minutes to completely read a simple reading passage and then asked three reading comprehension questions to assess their ability to understand the meaning of the text they had just read. 35