EVIDENCE January, 2011 Number 4 from to POLICY a note series on learning what works, from the Human Development Network 59368 Can Computers Help Students Learn? Policymakers and development experts seeking to improve tion experts understand how technology may boost the qual- the quality of education are interested in the role technology ity of education, the World Bank supported a two-year study can play. Not only do they want to use technology to directly of a program in Colombia that places computers in public aid learning, but they also want to ensure that students in de- schools. The study failed to find that the computers led to veloping countries ­ and poor communities everywhere ­ get any measurable increase in student test scores. Researchers the same exposure, and same education benefit, from technol- suggested this could be because teachers and students mainly ogy as do their counterparts in wealthier parts of the world. used the computers to learn how to use computers, instead Bringing computers into the schoolroom is seen by experts as of using them as a part of the teaching process. The results one way to do this. But just making technology available may do not mean that computers and other information and not be enough. Policymakers and development experts need communications technologies cannot raise educational qual- to know how to ensure the technology is used effectively. ity. But it does offer a cautionary note to those seeking to The World Bank is at the forefront of helping developing increase the availability of such technology tools: being able countries provide their students with the best educational op- to access technology is not always enough ­ people may also portunities, while working to meet the United Nations Mil- need training in how to use the technology to reach the stated lennium Development Goal of ensuring primary education educational goals. for every child. To assist educators, policymakers and educa- Case Study Colombia A public-private partnership in Colombia, called Com- bished computers and provides training so teachers learn puters for Education, was created in 2002 to increase the how to use the computers in class. To be considered, availability of computers in public schools for use in edu- schools must have a separate room that can be refitted to cation. The national program gives public schools refur- place the computers. Schools picked are linked to a local university, Did You Know... which designs a teaching strategy and helps implement 20 months of training in the schools for the teachers. Ownership of home computers in Colombia more than doubled Training covers computer installation, classroom man- between 2007 and 2009, reaching 13 percent. agement strategies and a separate phase focused on But this is still low compared with Chile (57 percent), Brazil (51 using the computer to strengthen student skills, with percent) and Mexico (32 percent) a focus on Spanish language reading and writing and Television is more commonplace. Almost every household math. Teachers are introduced to and taught how to has one. use educational software in classroom teaching. Since ­The Nielson Company starting, the program has installed more than 73,000 computers in over 6,300 public schools in more than 1,000 municipalities. By 2008, over 2 million students Researchers focused on students who were in grade three and 83,000 teachers had taken part. through nine at the baseline period. By focusing on lower To test the effect of the program, researchers randomly grades, they avoided including students who were more assigned 100 schools eligible for the program into either a likely to drop out or graduate before the study was over. treatment or a control group. A baseline survey was con- Teachers were asked about their knowledge and use of ducted before the program started, followed by another computers in class, while school heads were asked about survey two years later. Researchers collected data from stu- the school in general. dents in grades three to 11, the math and Spanish teach- A total of 8,216 students began in the study, with an ers, and the principal. A follow-up was conducted two even distribution of students across grades and genders. years later. In both cases, the survey for students includ- Two years later, 37 percent of all students in the study had ed questions about socioeconomic background, school dropped out of school, likely because the schools were in attendance, study habits and grades, attitude toward rural areas where migration was high. The drop-out rate school and use of computers. A short version of the na- did not affect the distribution of students in the treat- tional Colombian Saber assessment exam was also given. ment and control groups. The Findings Students in schools that received the computers Researchers did not find any difference in test and teacher training did not do measurably scores when they looked at specific components better on tests than students in the control of math and language studies, such as algebra group. Nor was there a positive effect on other and geometry, and grammar and paraphrase measures of learning. ability in Spanish. The program's stated goal was to train teachers to use com- There simply was no difference in how well students did, re- puters in specific subjects, with a focus on incorporating the gardless of whether or not they were in a school that had the computers into classroom teaching of Spanish and math. In additional computers. Overall, the study showed that the pro- Spanish classes, students on average correctly answer about gram had little or no effect on everything ranging from stu- 40 percent of test questions. Among students in the schools dent test scores to the probability that a student liked school. that received computers, test scores were 1.7 percent higher than students in the control group, a statistically insignifi- Part of the reason may be that although cant result. In math, another subject included in the pro- computers were available, teachers did not use gram, there also was no positive ­ or negative ­ difference them very often. in test scores between students in the treatment and control groups. This was consistent regardless of grade, subject and Less than half of language teachers in schools that received type of student. computers and training reported that they used a computer This bulletin summarizes the results of the research paper "The use and misuse of computers in education: Evidence from a randomized experiment in Colombia," by Felipe Barrera-Osorio and Leigh L. Linden. Financial support for the research study also was provided by the Colombia Computers for Education program. Full study can be found at: http://econ.worldbank.org. Most of the increase was in school, which makes sense because the schools had received additional computers. ...but they used the computers mainly in classes in which they were taught how to use a computer, not in math and Spanish. Most of the increased usage among students in the treat- ment group came from using the computers for their com- puter classes. Only three to four percent of students, in both the treatment and control groups, reported using computers as a learning aid in Spanish classes, for example. in the previous week, compared with 17 percent of teachers Overall, the program did meet its goal of in the control group. Math teachers reported similar usage. putting computers in schools, increasing the number in the treatment schools by almost And when teachers did use the computer, it 300 percent. generally was not as a teaching aid or for other classroom activities. In the control group, schools had an average of 5.1 comput- ers, compared with 13.4 in schools in the program. (Each Teachers in schools that received computers and training school had 80 or more students.) reported using the computers about half a day more a week than those in the control group. But they did not report It also successfully provided special computer that they used the computers as teaching aids during math training. and language classes. Researchers suggested that teachers might be using the computers to help prepare for class ac- Some 95 percent of teachers in the treatment group did receive tivities, instead of for teaching. training, compared with 8 percent in the control group. The higher computer use by teachers also was But teachers did not implement the final, and concentrated among those teaching grades three key part, of the program, which was to through five. By sixth grade, the computer usage incorporate the technology into their teaching. in treatment schools declined to the same levels of that in the control group schools. Studies on the relationship between using computers in the Schools in the program reported a 20 percent increase in classroom and improved test scores in developing countries give use of computers by 3rd grade Spanish teachers, declining mixed results: A review of Israel's Tomorrow-98 program in the to 16 percent for 4th grade teachers, and further declining mid-1990s, which put computers in schools across the country, in higher grades. did not find any impact on math and Hebrew language scores.* But in India, a study of a computer-assisted learning program showed a significant positive impact on math scores.** One Similarly, students did report using computers thing researchers agree on, more work is needed in this field. more often than those in the control group... * Angrist, J. and Lavy, V. (2002) "New Evidence on Classroom Computers and Pupil Learning". Economic Journal, 112, pp. 735-765. About 66 percent of students in schools in the program ** Linden, L., Banerjee, A. and Duflo, E. (2003), "Computer-Assisted Learning: reported using a computer in the previous week, com- Evidence from a Randomized Experiment", Poverty Action Lab Paper No. 5, October. pared with 41 percent of students in the control group. Conclusion Making policy from evidence Getting new technology into schools to aid student learn- gram. Programs may need to include extra components ­ ing is a key goal of many development organizations, pri- such as incentives for teachers who use computers in the vate groups and educators. But as this study to measure classroom, or extra instruction in specifics of computer- the impact of computers in the classroom shows, provid- assisted teaching ­ to make sure the technology into the ing technology and training is not necessarily enough to education process. ensure that the technology is incorporated into the class- Technology, and especially computers, may help stu- room experience. dents learn better and help teachers be more effective. Policymakers and researchers need to keep this in But as this study shows, getting to that point may require mind as they pursue programs to create a better educa- policymakers to do more than make technology available. tional environment through increased use of technology. Successful implementation may require a more hands-on Teachers may need more monitoring or training to en- approach to following teachers, and students, as they ad- sure they use the technology as envisioned by the pro- just to these new educational tools. The Human Development Network, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help alleviate poverty. The goal is to collect and build empirical evidence that can help govern- ments and development organizations design and implement the most appropriate and effective policies for better educational, health and job opportunities for people in developing countries. For more information about who we are and what we do, go to: http://www.worldbank.org/hdchiefeconomist THE WORLD BANK, HUMAN DEVELOPMENT NETWORK 1818 H STREET, NW WASHINGTON, DC 20433 Produced by Office of the Chief Economist, Human Development Network, Communications/Aliza Marcus