050-2791 vietnam vol1.qxd 8/24/04 9:24 AM Page 1 Working Paper Series No. 2004-4 VIETNAM READING AND MATHEMATICS ASSESSMENT STUDY VOLUME 1: HOW MEASURING ACHIEVEMENT DISTRIBUTION INFORMS EDUCATION POLICY DEVELOPMENT Human Development Sector Unit East Asia and the Pacific Region The World Bank June 2004 The findings, interpretations and conclusions expressed in this paper are entirely those of the authors and should not be attributed in any manner to the World Bank, to its affiliated organizations, to members of its Board of Executive Directors, or to the countries they represent. Vietnam Reading and Mathematics Achievement Study CONTENTS Abstract iii Acronyms and Other Abbreviations iv Preface and Acknowledgments v I. Introduction 1 II. Study Methodology 2 III. Main Study Findings 4 IV. Policy Development 6 A. Teacher Subject Knowledge 6 B. Performance & Resource Standards 9 C. Targeted Programs 11 D. Formula-Based Funding 13 E. Innovation, Autonomy & Accountability 16 F. Measuring Pupils' Achievement 18 V. Conclusion 21 Appendix 1 Variation Among Schools Within Countries in Reading Comprehension 23 Appendix 2 Reading Skill Levels and Mathematics Skills Levels 24 ii Vietnam Reading and Mathematics Achievement Study ABSTRACT This paper is the first of three volumes presenting the Vietnam Reading and Mathematics Assessment Study conducted in 2001. The present volume sets the study in its country context, describes its genesis, outlines the methodology, and summarizes the results. The main purpose of this volume, however, is to show how the analysis of the study results points to specific policy interventions. The strong link between a teacher's subject knowledge and the performance of her pupils argues for teacher upgrade programs, while the high variation of pupil performance between schools in the same province calls for local programs that improve school effectiveness. This first volume is followed by a second volume detailing the study results, and a third providing a full description of the study methodology. iii Vietnam Reading and Mathematics Achievement Study ACRONYMS AND OTHER ABBREVIATIONS ASEAN Association of South East Asian Nations AusAID Australian Agency for International Development CIDA Canadian International Development Agency DfID Department for International Development (UK) EFA Education for All EPA Educational Priority Area FSQL Fundamental School Quality Level GDP Gross domestic product HLM Hierarchical Linear Model (http://www.ssicentral.com/hlm/hlm.htm) IEA International Association for the Evaluation of Educational Achievement IIEP International Institute for Educational Planning KEDI Korean Education Development Institute MDG Millennium Development Goals MLn Multilevel Modeling (http://multilevel.ioe.ac.uk/) MoET Ministry of Education and Training NGO Non-governmental organization NIED National Institute of Education Development NIES National Institute of Educational Science NORAD Norwegian Agency for Development Cooperation OECD Organization for Economic Cooperation and Development PEDC Primary Education for Disadvantaged Children PIRLS Progress in Reading Literacy Study SIP School Improvement Program TIMSS Trends in International Mathematics and Science Study UNESCO United Nations Education, Scientific and Cultural Organization UNICEF United Nations Children's Fund iv Vietnam Reading and Mathematics Achievement Study PREFACE AND ACKNOWLEDGMENTS In 2000, the Vietnam Ministry of Education and Training (MoET) launched a large-scale monitoring study of primary education, the first of its kind in the country. The study would test pupils and their teachers in the last grade of primary education (Grade 5), using a sample cross- sectional survey in two key subject areas, reading comprehension in Vietnamese and mathematics. The MoET set up a committee to define the key research questions that guided the whole survey and the subsequent analysis. In 2001, the year in which the tests were administered, over 90 percent of the relevant age group entered Grade 1 and about 80 percent reached Grade 5. Thus about 20 percent of the relevant age group was not covered by this study. This volume is the first of three. Each volume contains different aspects of the study carried out in Vietnam in 2001. Volume 1 presents the study, summarizes the main issues and proposes policy recommendations under six themes: (a) raising student achievement by improving teacher subject knowledge; (b) improving system effectiveness by applying performance and resource standards; (c) closing the gap between the top and the bottom through compensatory funding; (d) counterbalancing emerging social differences with formula-based funding of education; (e) further improving learning achievement through increased innovation, autonomy and accountability; and (f) measuring the achievement of pupils and schools through regular testing. Volume 2 goes on to present the detailed research results and makes many suggestions for future action, while Volume 3 reports on the full technical aspects of the design and conduct of the study. The issues raised and recommendations suggested in Volume 1 can be used for a debate on the sequencing and construction of a critical path of the different policy options that could be adopted by Vietnam. These policy options will require considerable effort and will also require the creation of a critical mass of national specialists with the technical skills needed to design, conduct, and analyze achievement studies such as this one. They will require strong domestic political leadership and commitment. A number of areas would also benefit from technical assistance and development finance from donors and development organizations. Indeed, many donors ­ multilateral, bilateral and NGOs1 -- have made significant contributions to the schools sector in Vietnam, and future work should build on those efforts. The study was conducted by the MoET in Vietnam. Many people were involved in the study. Thanks are due to the following persons: the late Vice-Minister Le Vu Hung, who provided the oversight and guidance for this study; Dang Huynh Mai, Vice-Minister responsible for schooling and primary education; senior directors and heads of departments of the MoET; National Education Institutes, which provided specialists and researchers to conduct this study; provincial and district education offices, whose staff collected the data from over 3,600 schools throughout Vietnam in an exemplary fashion; Nguyen Quoc Chi, National Manager of the Vietnam Primary Education Project, who was instrumental in conducting the study and who provided special insights into the problems of primary education; Professor Dang Ba Lam, Director of the National Institute of Educational Development (NIED), who provided a member of the data team; Professor Tran Kieu, the Director of the National Institute of Educational Science (NIES), who 1Multilateral (e.g. Asian Development Bank, European Commission, International Development Agency, UNESCO, UNICEF); bilateral (e.g. Australia, Canada, France, Japan, Norway, UK); NGOs (e.g. Oxfam). v Vietnam Reading and Mathematics Achievement Study furnished researchers for the questionnaire committee, and also provided Professor Do Dinh Hoan and Dr. Do Cong Vinh to work on the study (several persons on this team were also from the NIES). Professor Hoan and his team were responsible for the test development. Dr. Vinh organized the data editing, entry, cleaning, and analysis. Members of the international support team included Dr. Kenneth Ross, Dr. Mioko Saito and Stephanie Dolata from IIEP/UNESCO; Professor Patrick Griffin of the Assessment Research Centre of Melbourne University; Miyako Ikeda who guided the data entry, cleaning and merging of files for nearly 75,000 records and who conducted many of the subsequent statistical analyses, and finally the principal architect of the study who guided and helped the international advisory team and the national researchers, Emeritus Professor Neville Postlethwaite of Hamburg University. This volume was authored by Christopher Shaw and Neville Postlethwaite, reviewed by Luis Benveniste, Elizabeth King and Christopher Thomas, and edited by Mai Thi Thanh, Jeffrey Waite and Juliana Williams. Christopher Shaw and Mai Thi Thanh World Bank in Vietnam June 2004 vi Vietnam Reading and Mathematics Achievement Study I. Introduction day schooling4 and, as resources permit, expansion of full-day schooling is being Vietnam's education system is in transition. encouraged in place of unregulated after Once a system that nurtured a high- school tuition. In addition, the school year performing academic elite, it is now aiming has recently been lengthened to match the to cater for all -- including remote and roll-out of a new curriculum for primary minority populations, while providing a school. quality education. This "education transition" mirrors Vietnam's shift to a At the same time as its primary education market-based, service-oriented economy that sector is growing, Vietnam is making rapid requires a broad human capital base. This progress in universalizing lower secondary change in the coverage in education offers education and expanding other levels, in critical support to the broader economic order to meet the target of Education for All transition: international studies (TIMSS (EFA) and meet the Millennium 19952 and Barro 20013) have found links Development Goals (MDG). The most between education quantity (increases in the recent population census (1999) showed that average years of schooling of the the primary school-age population had population) and education quality (increases stabilized and was expected to decline in the average student performance on during the decade 2000-2010. The prospect international assessments) on the growth of a fall in demographic pressure, coupled rate of real per capita GDP. with a projected growth in GDP and an increase in public spending on education At the beginning of the school year 2001-02, during the same period, allowed Vietnam ­ Vietnam had almost achieved its national for the first time ever ­ to broaden its focus target of universal access to primary beyond access to schooling, to address the education: over 90 percent of the relevant quality of mass basic education. In order to age group is enrolled in primary education. make the transition from increasing access The Government is now moving to address to improving quality, the authorities urgently the more difficult issue of enrolling the last needed a deeper understanding of the out-of-school children from remote areas underlying factors that influence the quality and disadvantaged circumstances (less than of primary education in Vietnam. 10 percent of the 6-10 age cohort), and ensuring that the one fifth of students who In addition to its international obligations currently drop out before completing under the EFA and MDG initiatives, primary school stay to the end of the cycle. Vietnam was motivated to improve the quality of its education system by a desire to The triple-shift system in primary schools increase its competitiveness and to further has been eliminated and a school building integrate its economy with the world. Eager program is replacing `leaf' or `non- for a more active learning approach, better permanent' school buildings in rural areas suited to the demands of globalization, with durable, permanent facilities. The time continual rapid change and a market-based dedicated to instruction is being increased economic system, the Government has with a growing proportion of pupils in full- continued to pursue education reforms on various fronts. 2International Study Center, 1997, Performance Assessment in IEA's TIMSS, Boston. 3Barro, Robert J., 2001. "Human Capital and Growth." American Economic Review 91 (2): 4 At present, one-third of pupils are in full-day 12-17 schooling, and paying for the afternoon session. 1 Vietnam Reading and Mathematics Achievement Study In 2002, new curricula and textbooks for II. Study Methodology primary schools were introduced and are gradually working their way up the Availability of information on education in elementary cycle. This curriculum reform Vietnam increased during the 1990s. aims to modernize the content and methods Following a UNESCO sector report5 in the of teaching and to promote a more dynamic, early 1990s, the Government and the World flexible approach to learning. Bank jointly conducted a financing study in the mid-90s,6 which examined the costs and Further elements of Vietnam's financing of the education and training modernization of education include a system and answered questions such as: (i) program of teacher professional how much does education cost? (ii) who development during 2002-2005 to support pays for education? and (iii) do they get the use of the new curriculum and to good value for money? Further analytic introduce modern methods of instruction, work has been undertaken more recently, and a large program of support for primary including: (a) a review of the primary education of disadvantaged children for the education curriculum content and period 2004-2009. To start this process the instructional methods; (b) an updated review MoET wished to have a `picture' of key of the costs and financing of primary learning outcomes in the last grade of mass education undertaken as part of a broader education, Grade 5, before such major analysis of public expenditures; and (c) an reforms took place. Having an initial analysis of the 1993 and 1998 household benchmark of learning outcomes would expenditure surveys that linked poverty and allow a comprehensive assessment of the education and identified some educational effectiveness of external support to basic disparities.7 None of these studies however education and the subsequent estimation of directly addressed education quality and value-added of the reforms expressed in performance or attempts to define the terms of increased student learning and outcomes of the education system in terms achievements. of students' learning achievement. For all these reasons (change of focus in Indeed, before the present study was primary education from access to quality, undertaken, little was known about the greater impact of quality improvements on overall quality of primary education in economic and social development, need for Vietnam. Four fifths of those who enter a baseline measurement of learning primary schools complete the full cycle of performance, etc.,) it was decided that the five years and among them a small elite present study would look in-depth at the regularly participate with great success in quality of the system, taking student international mathematics Olympiads or achievement in primary education as its similar competitions. But this narrow focus focus. ignored the fact that the remaining group ­ around one fifth of those who enter a primary school ­ drop out before the end of 5 UNESCO, 1992, Vietnam Education and Human Resources Sector Analysis, Hanoi. 6 World Bank, 1997, Vietnam Education Financing, Washington DC. 7 National Foundation for Education Research, 2000, Vietnam Primary Curriculum Review, London; World Bank, 2000, Vietnam: Managing Public Resources Better, Washington DC; World Bank, 2002, Trends in the Education Sector 1993-98, Washington DC 2 Vietnam Reading and Mathematics Achievement Study their schooling. While this study does not measure pupil retention per school, literacy Questionnaires were designed to answer the studies in other countries suggest that many MoET's questions. The pupil tests reflected adults with less than the full cycle of both the old and new curricula; the teacher primary education remain functionally tests allowed for a wider range of illiterate and innumerate for the rest of their achievement while placing teachers and lives. For this reason, the proportion of pupils on the same scale. Once tested and children not completing primary school revised, the final set of instruments education is a matter of concern. consisted of: a pupil reading test; a pupil mathematics test; a pupil questionnaire; a From 1998 to 2000, the MoET's National teacher reading test; a teacher mathematics Institute for Educational Sciences (NIES) test; a teacher questionnaire; and a school conducted a small study of primary pupil head questionnaire.8 achievement at Grades 3 and 5 in fourteen schools in each of five (out of 61) provinces Given the highly decentralized nature of of Vietnam. Given the limitations of the primary education management, the study research design and the scale of this work ­ collected provincial as well as national data; which was to provide training in research cost and size precluded analyzing data at the methodology rather than carry out a district or sub-provincial level. A representative scientific study ­ it is not probability sample of schools (proportional possible to generalize from these results to to the enrolment of Grade 5 pupils) was all provinces. The results were nevertheless drawn such that the sampling error for any able to show large disparities in learning province was between 7.5 percent and 10 achievement both between provinces and percent for a percentage, and about 0.10 to between schools within provinces. Grade 3 0.13 of a standard deviation for the estimate pupils in the capital, Hanoi, had levels of of a mean. This gave a sampling error for achievement in mathematics and reading the national estimates that was very small comprehension in Vietnamese that were indeed (between 0.03 and 0.05 for a higher than Grade 5 pupils in the other four, percentage and 0.003 of a standard deviation more rural, provinces. These few facts for an achievement score); it required testing suggested that parts of the primary school in 60 schools in each province making a system did not yet provide the high quality total of 3,660 schools for the nation. A and equitable learning conditions needed for random sample of 20 pupils from all classes successful learning achievements by all in Grade 5 was selected within each school, pupils. in order that this first comprehensive achievement study provide a good estimate The MoET opted for a study of Grade 5 of the within-school variance.9 The target achievement in reading and mathematics population ("desired") was split into two and its determinants, in order to have groups. Schools with fewer than 20 Grade 5 systematic and detailed information that pupils were excluded from the study, on the could be used for decision making at the grounds that data collection costs could be national and provincial levels. To ensure its reduced substantially without causing major relevance, the MoET provided a list of about distortions in the study population. Pupils in one hundred research questions to which it the remaining schools make up the wanted answers. The questions concerned "defined" population. The numbers of the levels and variation in achievement in the two key subject areas, materials and 8 textbooks in the classrooms, human and See Vol. 2, Ch 1, for further details of research material resources in the schools, and design, and Vol. 3, for full details of the test development and calibration. selected process variables including school 9See Vol. 3, Ch. 4, for full details of sample inspection and teaching methods. design. 3 Vietnam Reading and Mathematics Achievement Study schools and pupils in the desired, defined III. Main Study Findings and excluded populations are shown in Table 1. Overall results indicate that while Vietnam's average score in reading comprehension is Table 1: Survey Population (Desired, comparable with the results of similar Defined and Excluded) studies conducted in a range of developed and developing countries.11 Figure 1 Schools Pupils compares the reading comprehension Desired 14,642 1,812,053 achievement of Vietnam's Grade 5 pupils in Defined 14,173 1,808,089 2001 to the 1991 results of Grade 3, 4 or 5 Excluded 469 3,964 pupils in a selection of European and OECD countries. Testing took place on April 12-13; Grade 5 pupils were randomly selected on the first Figure 1: Reading Scores day and the testing proper was carried out on 500 the second day. The survey response rate for 450 the whole country for schools, pupils, 400 350 teachers, and school heads is given in Table Score 300 250 2. 200 East Germany Hungary 3 Vietnam 5 in Greece 4 New Zealand 5 USA 4 3 2001 Table 2: Survey Response Rate Country and grade level (all countries except Vietnam refer to 1991) Learning achievement, however, varies Planned Response % widely across the pupil population. Number Number Response Variation in achievement across schools, in Schools 3,639 3,635 99.89 the country as a whole and also among Pupils 73,200 72,660 99.26 schools within provinces, is surprisingly Teachers 7,278 7,178 98.63 large, as shown in Table 3.12 Heads 3,639 3,631 99.78 Table 3: Equity of Achievement Scores The response rates in this study were much Among Provinces and Schools higher than in similar sample surveys in many other countries. The few schools and Proportion of pupils that were missing were due to pupils' Variance Available absence on the day of testing or schools Reading Math having been flooded. The following sections Provinces 11% 14% present the results of the survey and present Schools 56% 64% recommendations for policy action.10 Tables 4a and 4b shows the percentage of pupils at different skill levels (increasing 11To compare learning achievement in Vietnam with that of other countries, reading comprehension scores for Vietnam measured by the present study were calibrated to the scale used in the 1991 IEA Reading Literacy Study. See Vol. 3, Ch. 3, for full details of the international benchmarking of the 2001 Vietnam 10 See Vol. 2, for detailed study results, and Vol. data with that of the 1991 IEA Reading Literacy 3, for technical details of the study methodology study. and analysis. 12See Vol. 2, Ch. 7. 4 Vietnam Reading and Mathematics Achievement Study from 1 to 6)13 by school location (isolated, Table 5b: Proportion of Pupil Variance of rural or urban). Subject Scores (Mathematics) Table 4a: Percentage of Pupils by Skill MLn HLM Level (Reading) Pupil 35% 34% School 65% 66% Skill Level 1 2 3 4 5 6 This inter-school variation is much larger I 9.7 21.1 25.5 17.9 17.6 8.2 than that observed in most other systems; R 4.7 15.4 24.5 20.5 23.8 11.2 the 2001 Grade 4/5 IEA PIRLS study U 1.5 7.6 17.5 20.7 30.9 21.7 indicated a 13 percent variation among T 4.6 14.4 23.1 20.2 24.5 13.1 schools in Sweden and a 27 percent I(solated), R(ural), U(rban), T(otal) variation in the USA.15 The policy implications of this level of variation, taken Table 4b: Percentage of Pupils by Skill up later in this paper, relate to the need to Level (Mathematics) identify and address quality issues at the school level: teachers, management, Skill Level learning climate, etc. 1 2 3 4 5 6 I 0.4 6.4 17.0 32.6 25.2 18.5 Not surprisingly, these disparities in pupils' R 0.1 3.6 12.4 29.8 29.1 25.0 learning achievements reflect differences in U 0.0 1.3 5.4 20.6 34.2 38.5 inputs and process across schools. While T 0.2 3.5 11.5 28.2 29.7 27.1 overall levels of human and material I(solated), R(ural), U(rban), T(otal) resources are increasing in schools, these resources are not distributed equitably As expected, home background is important across schools within provinces.16 One third in predicting pupil learning achievement. of the variation in school resources is However, in several provinces, more attributable to differences among provinces, variation in pupil's achievement is due to while two thirds of variation is attributable differences among schools rather than to differences among schools within differences between pupils. Tables 5a and provinces. There are also disparities in the 5b show the maximum amount of variance instructional hours that pupils receive (with in achievement that can be explained by the most pupils enrolled in half-day schools and pupil level or by the school level.14 only some in full-day schools). Teacher quality varies across schools, with teachers Table 5a: Proportion of Pupil Variance of demonstrating different levels of subject Subject Scores (Reading) knowledge in reading and mathematics. This is important since teachers' scores on MLn HLM the mathematics test are highly correlated Pupil 42% 42% with pupils' mathematics test scores, at both School 58% 58% the national and provincial levels. 17 There tends to be little difference in the kinds of 15 See Vol. 1, App. 1, for the range of variation 13See Vol. 1, App. 3, for definitions of the among schools found in the 2001 IEA Study. different skills levels for reading comprehension, 16 See Vol. 2, Ch. 7, Tables 7.1 and 7.2. and Vol. 2, Ch 2, for further details on the ways 17 Simple and partial correlations of teacher in which student achievement is recorded (skill subject knowledge in mathematics and pupil level, benchmark and total score). achievement in mathematics are 0.29 and 0.25 14See Vol. 1, App. 2, and Vol. 2, Ch. 9 and App. with the effect of home background of pupils 9.2. removed 5 Vietnam Reading and Mathematics Achievement Study teaching methods that teachers say they IV. Policy Development employ.18 This section proposes a critical path of Disparities are often associated with policy change, based on the policy messages isolation or poverty: isolated schools that emerge from the present study and typically have fewer resources, poorer centered on six main themes:20 teachers, and insufficient textbooks. It is in the capital province, Hanoi, where the best · Raising student achievement -- by resources and teachers are available and it is improving teacher subject knowledge; in the remote border and mountainous areas · Improving the effectiveness of the and some Mekong Delta provinces where system -- by applying performance and the major deficiencies are to be found.19 resource standards; · Closing the gap -- through targeted In short, the study paints a picture of a programs; polarized school system. At one end of the · Counterbalancing emerging social system, the children from well-off families differences -- with formula-based funding; and high socio-economic backgrounds gain · Further improvements in achievement -- access to well-resourced schools and are through increased innovation, autonomy and taught in full-day sessions by well-qualified accountability; teachers. In contrast, those from poor · Measuring achievement of pupils and families -- often in isolated or remote areas - schools -- through regular testing. - may well attend single-shift sessions in semi-permanent buildings and be taught by Between some of the themes there is some staff, many of whom are not at the national overlap and without careful sequencing of standard. The standard of education policy changes there is a risk of services provided to this second group is of contradiction if actions are implemented in poor quality and as a result the learning an unstructured or unplanned manner. outcomes and pupils' achievement are Where possible, for each theme, comments limited. are made on: (a) changes in policy and choice of strategy; (b) changes in process; In summary: (c) changes in inputs; and (d) changes in the · Almost all children of primary school- roles and responsibilities of the various age are enrolled in school, but about one actors at different levels in the system. fifth do not complete the full primary cycle; · On average, pupils in remote rural areas A. Teacher Subject Knowledge attend for only part of the day in schools with fewer resources, including qualified The largest predictor of pupil achievement teachers, and achieve less than pupils in in reading and in mathematics was the urban schools; relevant teacher subject knowledge.21 · The variation in student achievement is Indeed, some Grade 5 pupils performed due mainly to differences among schools, better than some Grade 5 teachers. The and these differences among schools are reading comprehension and mathematics sharper in some provinces than in others; score distributions for pupils and teachers · Analyses of schools that are more are shown in Figures 2 and 3. In each chart effective may provide guidance for the flatter score distribution shows pupil improving school quality and equity in scores, while the steeper distribution shows Vietnam. 20 See Vol. 2, Ch. 10, for the full list of policy 18See Vol. 2, Ch. 4. recommendations. 19See Vol. 2, Ch. 2, Table 2.5. 21 See Vol. 2, Ch. 9. 6 Vietnam Reading and Mathematics Achievement Study teacher scores.22 In reading comprehension, The implication for immediate action is the top 12 percent of pupils scored better clear: ensure that all teachers' subject than the bottom 30 percent of teachers. knowledge meet minimum standards. The national policy on teacher qualifications sets Figure 2: Reading Score Distributions (% standards, but not all primary teachers meet of candidate pupils [flatter curve] these minimum standards. & teachers [steeper curve]) 6 Figure 4: Relationship between Teacher & Pupil Provincial Mean Reading Scores 5 4 580 560 3 540 520 2 mean 500 1 teacher 480 ovince 460 0 Pr 440 420 400 400 Figure 3: Math Score Distributions 420 440 460 480 500 520 540 560 580 600 Provincepupil mean (% of candidate pupils [flatter curve] & teachers [steeper curve]) Figure 5: Relationship between Teacher & 6 Pupil Provincial Mean Mathematics Scores 5 590 4 n 540 3 mea 490 2 eachert 440 1 Province 390 0 340 400 420 440 460 480 500 520 540 560 580 600 Provincepupilmean The detailed analysis also showed that there were considerable differences among While the national MoET oversees the provinces in teacher subject knowledge and teacher training system, primary teacher this in turn was closely linked to the training colleges are operated by provincial differences among provinces in pupil education authorities. The resource base is achievement. Figures 4 and 5 show the not comparable across provinces, and relationship between the teacher (vertical) different local characteristics and needs lead and the pupil (horizontal) provincial mean each provincial education authority to adjust scores in reading comprehension and in the qualifications offered by their local mathematics. teacher college. As a result, primary teacher training colleges produce teachers with different levels of subject knowledge. This approach enabled the education system to expand towards universal primary 22See Vol. 2, Ch. 3. education, especially in remote and rural 7 Vietnam Reading and Mathematics Achievement Study provinces where the intake of trainees was knowledge. Work is already under way in sometimes below the desired level of ten pilot provinces, with support from general education. However, this pragmatic external donors, on the redefinition of approach has now led to a heterogeneous standards for primary teachers, the renewal teaching force with multiple qualifications of pre-service courses, the preparation of and different levels of subject knowledge professional development training for and teaching skill. serving teachers, and the renewal and strengthening of provincial primary teacher Given that Vietnam has almost reached its training facilities. Once the pilot phase of goal of universal primary education and that this program is seen to be working, there a decline in the school age population is will be an opportunity to revise and expand anticipated over the next few years, the the program to the national scale . country now has an opportunity to renew and update its teaching force. All teachers For new teachers, the teacher training should met defined standards of subject content of the pre-service professional knowledge in order to be able to teach in training courses could be modified to take primary schools, at least above the pupils' account of minimum standards, and standard top standard level. This requires strict national assessment procedures adopted for testing of subject knowledge of teachers to teacher certification . Once the new courses ensure that they have reached the standards have been launched, no exemptions should before being certified as teachers, with be allowed in the certification of new perhaps opportunities for re-testing later in teachers. A national quality assurance group their career. could construct teacher assessment instruments and link teacher certification to The combination of a continuing fall in the these instruments, thus ensuring that size of the primary school-age cohort and differences across the provincial teacher the large proportion of under-qualified training colleges are eliminated and that all teachers calls for an increased focus on in- teacher training colleges meet minimum service professional development rather than course standards and deliver comparable further expansion of pre-service training. qualifications to their trainees. Teacher upgrading and certification should be a priority at both the national and Once certified, teachers need to be deployed provincial levels. First, there needs to be to schools in an efficient and equitable opportunities to raise all teachers already manner. The study has shown an uneven working in schools to the new standard, distribution of teacher subject knowledge through professional development such as among schools. First, teachers with fewer upgrading and retraining courses. Second, years of academic education and teaching pre-service training courses must be revised experience are over-represented in isolated in a way that ensures that all new teachers and rural schools. Not surprisingly, these meet the required standard. differences show up as differences between provinces and in pupils' learning For teachers currently in the system, performance. The following table shows especially those whose subject knowledge is some of the teachers' characteristics broken low, professional development opportunities down by school location (isolated, rural or and incentives should be made available. urban). At present, teachers search for their School directors and local education own teaching job and most teach in the inspectors can assist in identifying province where they did their teacher candidates. A first step may be to target training. The few teachers that move from courses at specific schools or to offer their province of origin do so for personal correspondence or distance learning courses reasons or for promotion to posts of in order to increase teachers' subject responsibility. 8 Vietnam Reading and Mathematics Achievement Study Table 6: Selected Teacher Characteristics School Mean age Ethnic Mean years Mean years Mean years location (years) % Female group academic professional teaching % Kinh education education experience Isolated 30.5 57 80 11.6 1.9 9.1 Rural 33.2 73 95 11.9 2.1 12.0 Urban 37.9 86 97 12.2 2.2 15.8 Total 33.7 73 93 11.9 2.1 12.2 Newly qualified staff, with little or no attempts made to reduce the disparities in experience or lower academic and the quality of school staffs among schools. professional qualifications, rarely obtain posts in urban schools. They tend to be B. Performance & Resource Standards appointed in the more difficult, under- resourced schools in isolated or rural areas The study shows that some schools produce of their province. Hence, the current teacher graduates with very low levels of deployment process does not compensate for achievement in reading and mathematics. In the disadvantages that already have an countries that have highly developed impact on the quality of education in such education systems, the governance and schools. A more rational system of teacher promotion of effective national education deployment might be obtained if decision- delivery systems have three centrally makers ­ provincial and district officials, as managed components: (i) national goal well as head teachers -- had access to better setting and policy development; (ii) information on the characteristics of selecting and applying system performance applicants for teaching positions. This would standards; and (iii) coordinating and require an improved personnel data system, regulating actions by service delivery units. with information available by school. It The second of these components -- selecting would also have to be supported by a system and applying system performance standards of incentives that encouraged an adequate -- can be implemented along complementary proportion of good teachers to go to and stay tracks. First, the administration can define in hardship posts. The basic elements of minimum operational standards (outcomes) such a system are being developed with the that all locations are required (and assistance of external donors in ten pilot resourced) to meet. Second, it can adopt a provinces. If successful, this could be set of international standards that reflect generalized to the rest of the country to national aspirations, but are often beyond obtain a better match between needs and the reach of poor locations. candidates. Selecting and applying performance Teachers with low subject knowledge, standards are normally done at the national should be encouraged to undertake level in order to ensure close professional development along the lines of correspondence between international the Government's on-going Primary standards, national goals and operational Teacher Development Project. For new targets. Few provincial or municipal teachers, there should be a national governments have sufficient resources to standard for subject knowledge. The research and develop comprehensive deployment of teachers to schools should be performance standards as this can be costly. conducted on a more rational basis and It is critical however that decentralized levels of government in the development of 9 Vietnam Reading and Mathematics Achievement Study national standards be involved, not only other East Asian countries (Korea, Thailand because local participation is in itself or Singapore) or to OECD standards. desirable, but also because it can ensure that standards are adapted to local constraints. Since late 2003 the MoET has developed the concept of a `minimum service standard' as Traditionally in Vietnam, the MoET has set a pragmatic transition point on the journey national standards for inputs in absolute towards the high national standards. The terms, and it is only just beginning to define idea of a minimum service standard has also outputs or levels of achievement. The been linked with a regular school audit. The present study defined skill levels of minimum service standard describes not achievement for Grade 5.23 This is a first only expected results in terms of learning step in having teachers think in terms of skill achievement but also lists of inputs and levels and conducting regular classroom processes. The present study showed that testing throughout the year, so that the many classrooms and schools were poorly teacher knows which pupil is at which level resourced.24 Resource inputs were of two and can determine the next steps to take. To kinds: material (classroom-specific and reinforce this approach the unit responsible school-wide) and human (classroom and for the curriculum should re-examine the head teachers). Isolated schools were the curricula and new materials in terms of skill least well-resourced, rural schools somewhat or competency levels, and should feed these better resourced and urban schools the best concepts into the in-service and pre-service resourced. Consequently, provincial teacher training programs and to the unit school audits or reviews of resources can be responsible for testing. The testing unit used to target programs of supplementary could produce classroom tests for regular resource allocation. testing of pupils, and guide teachers in their regular assessment of pupil learning The Government's program "Primary performance throughout the school cycle. Education for Disadvantaged Children" (PEDC), started in 2003 with substantial During the 1990s, the MoET concentrated multi-donor support,25 is testing this process. on increasing primary school enrollment to In pilot areas, results from school audits are meet the Government target of universal being discussed by the provincial and education. As Vietnam now edges towards national authorities, so that the national its goal of universal primary schooling, it is authorities can take action to even out time to move beyond the purely quantitative differences among provinces where dimension and to pay attention to the quality necessary. The `minimum service standard' of educational achievement in primary is now part of the Government's education. Most central and provincial Fundamental School Quality Level (FSQL) education administrators have not monitored program in primary education. As the learning performance using nationally contents of the minimum service standard standardized measures. The methodology are tested and updated, the Government is used in the current study allows policy applying this approach nationally, makers to understand the variables that accompanied by a core package of contribute to learning achievement and the resources. The package of inputs and distribution of knowledge across provinces, processes described in the minimum service schools and individuals. Given Vietnam's standard will need to be provided to every economic aspirations, these policy makers school so that all are resourced at least at the may even wish to consider linking national standards for learning outcomes to those of 24See Vol. 2, Ch. 4 and 5. 25AusAID (Australia), CIDA (Canada), DfID 23See Vol. 1, App. 2. (UK), NORAD (Norway), and the World Bank. 10 Vietnam Reading and Mathematics Achievement Study minimum level. This should narrow listed in Table 7.29 Factors marked with an disparities among schools. asterisk show large differences among provinces; reweighting these critical factors It is suggested that outcome performance be across provinces will likely require central specified in terms of minimum standards MoET intervention. For others factors, it is using the skill competency level26 approach enough for the provincial authorities to take adopted for the current study. It is also action. suggested that the material resources FSQLs be revised. Provincial education Table 7: Major Factors Associated with authorities should be responsible for Large Achievement Differences Among reporting on compliance with minimum Pupils/Schools resource standards, through a regular auditing process. Home factors Private corner for study at home C. Targeted Programs Meals per day* Days absent last month This study shows that the difference between the mean provincial scores in Pupil Materials reading comprehension and mathematics in Use own Vietnamese textbooks the highest and lowest provinces is 1.5 Use own mathematics textbooks standard deviations.27 This is equivalent to Borrow books three years of schooling, a large gap between the top and bottom. The difference between the 25th and 75th percentile of School and Staff Factors schools is about 1.0 standard deviation in School resources index* both reading and mathematics.28 One Staff Teacher training* standard deviation, equivalent to two years Temporary teachers of schooling, still constitutes a large gap Total number of pupil problems across the middle 50 percent of schools. Total teacher problems Teacher sex This confirms the results of an earlier study Teacher training that showed Grade 3 pupils in the capital Classroom furniture* outperforming Grade 5 pupils in some other Teacher's reading score rural provinces. Most education systems Teacher's math score throughout the world try to minimize differences among provinces so that children Teaching/Learning behaviors receive a common standard of education no Story telling competition matter where they live. However, the study Percent of parents met shows that some children live in areas mathematics homework assigned without a quality school, and that this will Math homework corrected likely exacerbate provincial inequality. Reading homework assigned Reading homework corrected The main factors associated with large differences among pupils and schools are 26See Vol. 1, App. 2. 27See Vol. 2, Ch. 2. 29See Vol. 2, Ch. 8 & 9. Factors listed do not 28Test score differences were 97 for reading and include economic and user charges, or specific 108 for mathematics on the standardized scale details on the hours of instruction, as this study with a mean of 500 and standard deviation of did not provide reliable information on these 100. matters. 11 Vietnam Reading and Mathematics Achievement Study Factors in the home: A private corner in the project, and the success of the pilot scheme home was associated with higher showed the way forward. Through this achievement even after taking account of pilot, teachers are now familiar with the home background. Similarly the regularity procedures needed to operate a book lending of meals and days absent were very different scheme for all pupils. between the highest and lowest achieving schools and pupils. By advising parents Many schools have few books either in about the usefulness of providing a private classroom or school libraries. Where pupils corner at home as well as the need to were able to borrow books, schools minimize their children's absence from performed better. As some of the titles from school, the MoET can influence factors in the original list of 29 become optional, they the home. Compensatory measures, such as could be made available through the school school feeding programs or conditional cash or classroom library. Other research shows transfers to poor households, may help in that having access to many books and being areas where the limited number of meals per able to read them is closely linked to day affects pupils' learning. improvement in reading. There have been very successful `book flood' programs in Pupil materials and books: In a few schools, several ASEAN countries; a similar program pupils still did not have a pencil, notebook could be introduced in Vietnam. or other learning materials. It is suggested that each school have at its disposal a School staff: The level of teacher training of number of items such as pencils, erasers and the entire school staff proved important. writing supplies to provide to pupils in need. This often has to do with the aura of a However the important factor that shows up school and what is sometimes referred to as from this study was access to and use of the school's culture and work/learning textbooks. The official MoET book list has ambience. Having fewer temporary teachers 29 textbooks and accompanying books that was also a feature of better achieving pupils are meant to have. Of these, eight schools. were for the Vietnamese language and five were for mathematics. The study showed Teaching and learning behaviors: Higher that in nearly all cases, pupils had the key scores were registered in schools where the books for Vietnamese and mathematics but head teacher reported fewer behavioral only 20 to 50 percent of pupils had the other problems amongst the teaching staff.30 books. In some poor homes, no books were Teachers' behavioral problems were also available. associated with behavioral problems amongst pupils. In schools where teachers With the renewal of the primary school met with a higher percentage of their pupils' curriculum, it is time to review whether all parents, the pupils had higher achievement of the books are really necessary. As a scores. It was either difficult for the matter of policy it would seem that a shorter, teachers to meet parents in the poorer more realistic list of what pupils must have achieving schools or they were not making a should be drawn up by the curriculum sufficient effort. The need for teachers to group. The authorities should then ensure meet parents frequently, and not just when that every child has the books. This will test or examination results are announced, is mean subsidizing the books for the very something that should be stressed in teacher poor families, an action that the MoET is training programs. Finally, where already following for minorities and for homework was given and corrected in both disadvantaged children. A book lending subject areas, schools performed better. scheme has already been piloted in four provinces under a recently completed externally supported primary education 30 See Vol. 2, Ch. 5. 12 Vietnam Reading and Mathematics Achievement Study How might the Government go about co-ordination of the work. One approach to reducing this achievement gap between the targeted programs that is discussed below is top and bottom? The first policy that of a school needs-based funding recommendation, to raise everyone's formula. achievement by improving teacher subject knowledge, will improve achievement of If differences in school achievement are to those at the bottom but may not necessarily be evened out, it will be important first to reduce the gap between the top and bottom. equalize inputs across schools, and second, Indeed, as the national average of to introduce a program of additional actions performance in learning outcomes increases, targeted at schools that are performing the amount of variance in learning outcomes below the mean. The program should is also likely to increase, thus leading to a concentrate on selected inputs over and widening in the achievement gap between above the new minimum standards for the top and the bottom. The second policy schools. These measures should include a) recommendation, setting and applying a new having a realistic list of books for all pupils minimum standard and ensuring that every and then ensuring that the books are school meets or exceeds that minimum available to all children; b) providing better standard by equalizing inputs with the teachers to needy schools; c) introducing a provision of a core package of inputs and book flood program; and d) introducing actions in all schools, will provide a much school intervention programs in selected needed boost to the schools that are at the areas. bottom. However, equalizing inputs and guaranteeing the minimum service standard D. Formula-Based Funding by all, as the first step on the way to eventually meeting the ideal `national Differences in the pupils' family standard' or an agreed international target, backgrounds are often highly related to how may not be enough for schools with large well the schools achieve in the different numbers of disadvantaged or minority subjects. As pupils leave school and enter students, or those with performance society and the world of work, differences in difficulties. Thus, the third policy achievement lead to variations in successes recommendation that is made is for a and differences in wealth. Over time, and targeted education program for such schools. without corrective measures, society itself may gradually become less equal. Since The program could include a variety of scholastic achievement has such actions and policies that remove or alleviate implications for future social and economic the pedagogical obstacles to performance success, it is critical that education policy be (such as limited duration of instruction, designed to compensate, at least in part, for absence of learning materials, or below- the disadvantage that results directly from standard teachers) as well as non-pedagogic poverty and isolation. The most tangible and economic barriers (such as fees for the asset that poor individuals and families have full-day sessions and other user charges that is their potential human capital, and policies are often beyond the capacity of poor rural that enhance their human capital have high families). returns in terms of poverty alleviation. Ensuring that poor and isolated children The cumulative impact of the actions have access to a good quality basic recommended above will be considerably education will enable them later to greater if, instead of being carried out in an participating in the growing economy, and uncoordinated manner, they are included in ensure that a lack of educational opportunity a coherent targeted education program. To does not become a barrier to mobility. do so would require a senior member of the MoET to take special responsibility for the 13 Vietnam Reading and Mathematics Achievement Study Most schools recruit their pupils from the province was much larger than in other immediate surrounding geographical area. countries where similar research had been These `catchment' areas are chosen in order conducted. In over one half of the total to limit the distance or travel time from sixty-one provinces of Vietnam, the home to school, and they are also expected variation in reading scores was due more to to have the effect of encouraging each differences between schools in those school to serve its own neighborhood. particular provinces than to the differences However, families that live in a particular between pupils in the schools. In about neighborhood tend to have similar fifteen of these provinces the relationship characteristics, and therefore the resulting between the school reading scores and the composition of a school's pupil body tends socio-economic composition of the schools' to be more homogeneous than would be pupil bodies was strong. Thus, in these expected if pupils were randomly allocated fifteen provinces there would seem to be an among schools. As a result, in an isolated, emerging issue of social segmentation. poor, rural location, the pupils tend to come Furthermore, in ten of these fifteen from families with agricultural and other provinces the provincial average reading relatively poor backgrounds. In contrast, in scores are below the national average. So in an urban setting, populated mainly by this final group of ten provinces, not only families with relatively wealthy were there large differences and a strong backgrounds, it would be expected that relationship between the reading score and children from similarly wealthy the schools' socio-economic composition, backgrounds would attend the same schools. but the provincial average reading scores As a result, individual schools are often were low. In these ten provinces31 it would relatively homogeneous internally in terms seem that there may be an emerging issue of of their social composition, while the social segmentation. differences in social composition of student populations of schools in different locations "Formula funding" is commonly used to tend to be quite large. address the problem of inequitable outcomes that are associated with pupils' socio- As well as the information on student and economic characteristics. Formula funding teacher learning achievements collected is based on the premise that schools catering through tests of reading comprehension and for socio-economically disadvantaged pupils mathematics, this study collected need more resources to reach the same information on pupils' family backgrounds outcomes as schools serving pupils from in terms of the number of possessions in the wealthier families. In other words, unequal home and other characteristics. Using unit inputs can, when allocated equitably, statistical techniques it has been possible to result in equal unit outputs. Under this examine the relationship between pupils' approach, schools are funded according to a achievement scores and their home formula that, roughly speaking, multiplies backgrounds, and to see to what extent the its pupil enrollment by a unit cost that varies variation in pupil achievement was in accordance with criteria that reflect the associated with the differences in socio-economic characteristics of the wealth/poverty differences in family community served by the school. Census backgrounds; and, whether in a particular data are typically used to determine the province schools had intakes of pupils that socio-economic index of the school's were different in terms of their families' catchment community; the lower the index economic backgrounds and that are related to the schools' achievement in reading. 31 The ten provinces are : An Giang, Binh Dinh, Ca Mau, Can Tho, Cao Bang, Gai Lai, Hoa The study shows that in Vietnam the spread Binh, Lang Son, Kon Tum, Yen Bai. in reading scores among schools in a 14 Vietnam Reading and Mathematics Achievement Study (i.e. the poorer the community), the higher poorly performing schools to emulate the the unit cost. The technology for this kind better schools in order to attract voucher- of approach is well developed and has been bearing pupils. It is clear that voucher used successfully in several countries.32 systems are more easily applied in urban settings, where it is reasonable to expect that "School improvement programs" (SIP) can parents have a choice of schools. be used alongside, or instead of, "formula funding". SIPs require schools that are Finally, some public school systems change identified as catering for disadvantaged the socio-economic mix of pupils within pupils to develop proposals to improve schools, as a way of offering disadvantaged learning; these proposals call for resources pupils the same quality education afforded over and above the base funding that the to pupils from wealthier families (thereby school normally receives. Like formula avoiding the poor quality education that is funding, the SIP approach ensures that generally associated with "ghetto" additional resources are directed towards neighborhoods). This is done by managing schools with the most needy pupils. SIPs the location of businesses and enterprises in have the advantage, over formula funding, a way that brings different social groups into of ensuring that the school management has neighborhoods, or sometimes, by already had to develop a plan linking the transporting pupils from one school additional resources to improved learning. catchment area to another. Formula funding, on the other hand, has the advantage of lower transaction costs, both Some of these ideas have already been taken for the school and for the funding agency, up in the recommendation on closing the since there are no separate funds to gap that was made earlier in this volume. administer. Vietnam is already using targeted SIPs with the recently launched primary education "Voucher" systems are an alternative to program for disadvantaged children. formula funding. Formula funding treats the However, as the management of public school as the funding unit, since the per- finance and budget systems is modernized pupil unit cost is a community average that and the linkage between the budget and an is typically calculated only once every few education medium-term expenditure years. One pupil entering or leaving the framework is strengthened, an approach that school will not affect the unit cost. In includes elements of formula funding could theory, a voucher is assigned to an become a promising way of moving further individual pupil and has a monetary value forward. that varies according to that pupil's socio- economic characteristics. (In fact, in most Planning and funding of schools in Vietnam cases where this approach is applied, follows an approach that is based on vouchers have a fixed value, but are standardized norms. While there are norms restricted to disadvantaged pupils.) The that take into account differences in pupil is able to "redeem" her voucher at the provinces, the approach does not yet fully school of her parents' choice, with the address all issues such as the financial voucher then being honored by the funding implications of providing quality basic agency. Voucher systems are built on the services to those children with assumption that, in seeking out the best disadvantages or disabilities, or the extra schools, disadvantaged pupils will not only cost of transporting basic supplies and other receive a better education, but will drive resources to poorer schools, often located in remote and hard to access places, and thus 32Ross KN and Levacic R, 1999, Needs-based more costly than supplying large cities. resource allocation in education via formula Moving to an approach based upon funding of schools. Paris: IIEP/UNESCO minimum service standards, providing 15 Vietnam Reading and Mathematics Achievement Study additional or differential inputs for schools economic make-up of the pupil body. It is with special characteristics and recommended that subsequently a formula disadvantaged pupils, and allowing local funding approach be adopted in order to level flexibility to diverge from standardized reduce differences in academic achievement national norms could address these issues. among schools in selected provinces. At the provincial level there must be flexibility to provide resources to schools so E. Innovation, Autonomy & Accountability as to reflect the background and needs of the children, where they live, local climatic and Can further improvements in achievement communication difficulties etc., and to be made even after the actions already strengthen the link to outcomes and to suggested (improving teachers' subject meeting the minimum service standards for knowledge, applying performance and all pupils. Since differential amounts of resource standards, reducing the gap resources will be needed especially by between the top and bottom by first isolated schools, data systems can be used to equalizing inputs to all schools and identify shortfalls in the required personnel subsequently introducing a targeted or those schools in need of extra help. As education program for schools performing the system matures and greater below the mean) are carried out? Once all responsibility for planning and finance is children are in school and the nation's decentralized, increased local-level resources are being distributed to schools autonomy, even down to the level of the equally and efficiently, attention turns away school, can be supported through demand- from issues of access and availability to driven funding windows that are targeted to those of quality, performance and results. In support local innovations. the process, the education system needs to adapt and change. A central part of this In summary, the Government may wish to change is the move from concept of equity include a `needs-based' school funding of inputs to one based on equity of outcomes formula in its budget and public financial or learning achievement. But equalizing resource allocation systems. Such a needs- achievement often requires different based school funding formula would inter amounts of inputs to schools and thus, this alia define school resource needs in terms of recommendation may appear to be in the socio-economic composition of the contradiction with those made earlier which school's pupil population. Through such talk of equalizing inputs. This contradiction strategies, if the Government sees the can be resolved by carefully sequencing the education system as an important instrument recommended actions. to address the effects of family wealth differences, it would be able to redress Reference has already been made to the emerging socio-economic differences. need for equalizing inputs to all schools to ensure that all reach at least the minimum In some provinces it can be seen that there performance standard, and to the idea of a were large differences among schools in targeted education program for low reading scores and that they were related to performing schools. Vietnam has still some socio-economic differences in the pupil way to go before it can say that inputs have bodies of the schools. This can undermine been equalized or that it has a well-targeted social cohesion. Where this was the case, program that addresses the needs of weakly provinces were identified and it has been performing or disadvantaged schools. In suggested that the results be checked and, this section then, attention is paid to the where necessary, programs be created that provision of extra resources to schools, are targeted at specific schools in specific classes and individual pupils in need of provinces to reduce the relationship between increasing their achievement. In order to school achievement and school socio- raise the level of the lowest schools, policy 16 Vietnam Reading and Mathematics Achievement Study makers may for example need to attract very good relationship between the teacher `excellent' teachers and managers from and their pupils. elsewhere in the province to work in those selected poorly achieving schools and When only one method of teaching has been districts. prescribed, it can militate against the required flexibility and initiatives that Does the Vietnamese education system teachers may need to take. Given the allow sufficient flexibility to start moving standardized, pre-defined approach to beyond the original concept of equity of teaching methods that has been followed in inputs? As with many countries, in order to Vietnam prior to the introduction of the new build and to reinforce its national identity, curriculum, it is recommended that the Vietnam harmonized its various curricula MoET review the need for a policy and programs and, to ensure uniformity and statement on the desirability of innovation standardization, it adopted a tightly defined and autonomy, accompanied of course by pedagogical approach typified in the `model suitable accountability measures. This can school'. All schools attempt to follow the also be reflected in the content of the teacher teaching methods and operating procedures pre- and in-service training programs. In laid down for the fully resourced model addition, pupils with low achievement will school. But the resources needed to operate need a great deal of extra support if they are as a model school are beyond those to catch up with their peers. These pupils are available to most schools. In addition, the usually from the isolated schools and from standardized and uniform approach, while it disadvantaged regions. They have accrued has brought all schools onto a single an accumulated deficit over time. national program, makes little allowance for Mobilizing and delivering the extra support local adjustments and gives very limited that is needed may be done best through autonomy for a school to follow a more local level innovations that adopt a flexible flexible process that corresponds to local approach to learning. needs. Thus for all except the model school itself, the standardized, uniform, `model' School culture and extra-curricula approach that is followed is restrictive and activities: There is an intangible factor in does not foster innovation, allow flexibility, schools that contributes to their success. It is nor does it provide a mechanism to deliver sometimes called the `culture' of the school basic service standards to all pupils in all and is often attributed to the style and schools. dynamism of the head teacher and district leaders. Inspectors sometimes say that when Teaching methods: It is well-known to they walk into a school it is possible to `feel' teachers that pupils are different and that if the children are busy, committed and one must vary the methods of instruction for eager to learn. One of the questions in the different educational objectives to different study in part dealt with the kinds of factors pupils. Although frontal teaching is good for that are sometimes associated with such a some educational objectives, it is small learning culture. It asked about the children group work or individual work that is better producing their own school magazine and for other objectives. Teaching is not only holding story-telling competitions and concerned with raising the cognitive level of general knowledge quizzes. It was seen, for achievement but also with producing an example, that only 25 percent of pupils were interest in learning the subject, in learning in in schools where there was a school general and a liking for school. In some magazine. But one might have asked about cases it might even be necessary to guide the other extra-curricula activities such as children how to behave in school and mix reading clubs, chess clubs, mathematics with the other pupils. These tasks require a clubs, school theatrical plays or having a school orchestra. It is this diversity of 17 Vietnam Reading and Mathematics Achievement Study activities and the children's commitments to teachers can introduce innovations and them that are associated with good many school activities that are linked to schooling. To achieve these kinds of learning, such as school magazines and activities a certain amount of autonomy is reading clubs and the pupils having a small needed, together with the flexibility to drama club and so on. introduce local innovations. Many school directors may need to be encouraged to F. Measuring Pupils' Achievement initiate these kinds of innovative activities. For some of these extra-curricula activities With the completion of this study the and clubs they may wish to invite village Vietnamese Government and academic artisans into the schools. They must be free community have completed a tremendous to do this without seeking multiple or task and they deserve to be congratulated. bureaucratic permissions, often connected The information collected through this study with form-filling. School management and analyzed by the national researchers autonomy should be matched by greater constitutes the first comprehensive, accountability to parents, to the community representative, scientific study of education and to the administrative hierarchy. quality in Vietnam. By carrying out this study, Vietnam now has the basic building One problem in some of the isolated areas is blocks, methodology, procedures, and a that girls, often from traditional or minority large part of the professional and scientific family groups, do not go to school or drop knowledge necessary for building and out of school early. School intervention operating a national system of monitoring programs have been effective in raising and evaluation at all levels of the education girls' school attendance. Such programs system. Indeed, a repeat on a smaller scale first encourage parents to change their of such a study is already planned later in behaviors toward their children's schooling. this decade when the primary curriculum has For example, programs where the mothers been fully implemented and the pilot were encouraged to ask the children to read program of professional development to them or to tell the story that they have just programs for serving teachers have been been reading were found to have a direct and completed. So what should now be the next positive effect on improving their children's steps to capitalize upon the knowledge and reading achievement. Second, these skills built through this study? programs encourage mothers to send their daughters to school and to persuade their Earlier this report reviewed the functions neighbors to do the same. This kind of that support the governance and promotion mobilization campaign is already familiar to of effective education, such as policy Vietnam and has been used in the past to development, selecting and applying good effect. The MoET and the provincial performance standards, and coordinating offices may consider targeted campaigns of and regulating action. A further essential this type in certain zones and link these to group of activities needed to ensure effective other interventions mentioned in the education delivery systems relates to previous section on `closing the gap'. monitoring, evaluation and accountability. This second group of activities spans three It is suggested that the MoET introduce broad areas: measuring performance; more flexibility in the teaching process analyzing performance (to see divergence rather than having the `standard' approach, between standards and performance); more flexibility for provinces to move communicating performance (informing resources among schools, and more school- stakeholders of the system performance). level autonomy -- balanced by greater accountability -- and flexibility in the Measurement and analysis of performance `culture of the school' such that head are required to produce valid and reliable 18 Vietnam Reading and Mathematics Achievement Study results for those who must take decisions should be produced in key subject areas and about the educational achievement in the the analysis will be aimed at the schools of the country. identification of emerging system-wide problems and broad issues related to The measurement of performance has three curriculum development, related changes in strands. The first is frequent measurement of instructional methods, and at whether the performance throughout the academic year education system's overall goals and as each module of instruction is completed objectives, especially in terms of quality, are by a class (often two weekly or monthly being met throughout the country. The modules of work in the class group). This is sector ministry will also wish to look at the often done through short tests of pupils relative quality of education across administered by teachers for different provinces and areas within provinces to combinations of learning modules. The identify those with low scores and therefore second strand of performance measurement those in need of help. is often the annual measurement of performance at the district or more likely the The use of teacher administered tests will provinces, often undertaken at the end of the represent a new approach in the classrooms. academic year. The third stand is less The overall aim is to furnish information to frequent and looks at the overall system the teachers for them to know where each of performance at the national level where it is their pupils is at regular intervals throughout probably sufficient to have national the school year. One important point will be monitoring of educational quality conducted at the beginning of the school year. This every three to five years and based on a will allow the teacher to know what he or sample survey rather than a complete pupil she has to do in order to have all of the or school census. None of these three pupils reach the higher levels of strands of performance measurement are achievement. The curriculum group will "high-stakes" or are used to certify or need to work together with a testing center document individual performance with in order to produce the teacher administered diplomas or certificates. tests. This kind of approach certainly worked very well in South Korea when the The analysis of performance will be slightly Korean Education Development Institute different for each of the three strands. For (KEDI) produced these kinds of tests the first strand, it is the teachers who must together with remedial and enrichment look at the results. They will be looking for materials for all key subject areas. It was the percentage of their class pupils who have found that even within classes the pupils mastered each objective and skill level. were at very different levels. This implies They will also see the kinds of errors made that the teacher may have to use different by the pupils (this assumes that the test instructional approaches to have everyone developers will construct diagnostic items). learn to the next level. As was mentioned in Finally, the teachers will be able to see the previous section, pupils are different and which pupils are ready to move to learn in different ways. The approach of instruction at the next skill level and where requiring teachers to follow a single remedial work is needed with some pupils. standardized way of teaching for all pupils For the second strand, the provincial will not necessarily work. Hence, there is a officials will be interested in the need for both remedial and enrichment identification of lowly achieving pupils, materials and tests that allow discrimination schools and districts so that they will know between pupils' abilities. what kinds of extra resources and work will be required. They may be able to identify The communication of results will be for the particular errors being made in groups of MoET to decide. Teachers will schools. For the third strand, national tests communicate individual results to pupils and 19 Vietnam Reading and Mathematics Achievement Study aggregate class results to the school head as Monitoring and evaluation of education well as to the parents. Provincial authorities performance is often done either by a will communicate the results to the district specialized unit of the education ministry or offices as well as to other parts of the by an independent education quality provincial administration and the national assurance agency. The MoET is already education ministry. At the national level it moving in this direction through the creation will be important to communicate the results of the testing and accreditation section and to the various divisions of the Ministry, to the reorganization of NIED and NIES. the education community, to teachers and to These changes present a good opportunity to the public at large. Special feedback of formalize testing and monitoring and information will often be passed from the evaluation functions and to build a modern national authorities to the curriculum group. professional capacity in this specialization. Specialized item analysis, or error analysis Most skills are available in Vietnam but are of test items, provided to the curriculum dispersed across different institutions or the teams can assist them to improve the staff qualified to work in evaluation and textbooks at the appropriate time. The testing are presently working on other improvement is usually in terms of topics. The bulk of the technical expertise sequencing the material as well as about necessary to conduct thorough evaluations typical errors that have been allowed to exist of performance will need to be assembled at in the teaching. the national level within the MoET or eventually in an independent education Where should the function of monitoring inspection agency. and evaluation of education performance be located? This particular study was a one- This national quality assurance group would off, freestanding study, financed through an doubtless need some technical help for a externally supported project with loans and three to five year period in test construction, grants that provided external technical test scaling, and large-scale data collection, assistance and advisory services. The as well in data analysis and reporting to substantive work was carried out by education authorities and school Vietnamese researchers and educators. communities. It will be important to have Nevertheless, the survey and analysis were personnel at the national and provincial not integrated into the regular annual levels who are able to read the research program of the MoET, nor were they results and understand them and be able to anchored to a particular institutional home. take decisions on remedial action. Such The study has been instrumental in building persons will have to be identified and the core technical competencies needed to trained and again external partners will be undertake future studies of monitoring and needed to assist in this capacity building evaluation of education performance. But in exercise. order to mainstream this function, and ensure its sustainability and permanence, It must be recognized that in information further capacity building and linkage to an gathering, the above types of studies will not institutional location need to be made. The necessarily furnish all of the different kinds skills required are in instrument of information that are useful to a system of construction, probability sampling of education. For example, it was seen that schools and pupils, data collection, entry, there were many pupils in schools where the cleaning, and weighting as well as file school head perceived behavioral pupil and merging and finally in the statistical teacher problems. In such cases, it is often analyses of the data in order to provide wise to conduct small additional studies compact information. focusing only on one aspect of the research, in this case the behavioral problems. Other vexing problems are the impact of extra 20 Vietnam Reading and Mathematics Achievement Study tuition on achievement, the role played by V. Conclusion user charges and other fees, or the kinds of teaching approaches that pay off with To raise the achievement level of different kinds of pupils. Earlier research disadvantaged students in poorly performing suggested that pupils at different levels of schools is a major challenge. It requires a lot skills required different instructional of work and great commitment by all methods;33 further experimental work on concerned at each level of the system. As this issue would be helpful. To avoid more information becomes available through duplication of effort and to ensure a scientific, representative surveys, the MoET coherent approach that minimizes the is beginning to examine this new empirical burden on teachers and school directors, the data more and more and use such MoET could designate a unit responsible for information to guide and instruct their coordinating all of the studies and decision-making process. Informed summarizing all of the results into periodic decision-making based on solid facts and reports. figures is always to be preferred to non- informed or intuitive decision-making. The The quality assurance capacity of the research study reported here set out to bring Vietnamese education system needs to be information to the authorities that was a built up, consolidated and experts assembled direct response to the questions raised by and located in one place in a unit that could senior education managers. The study has become the core of a national center. The provided a comprehensive and scientific consolidated unit should construct three baseline measurement of educational quality types of diagnostic tests (rather than in primary education not only for the nation certificative tests). The first will be tests for as a whole but also for each of the teachers to use in school during the course provinces. of their year's teaching in order to see what steps to take depending on the skills that So, what should be the next step and where their pupils have and have not mastered. should the Government start? First, the The second will be for provinces for the study has provided two immediate provincial authorities to identify schools and opportunities; one to disseminate the districts where achievement is poor and information and results to provincial where remedial action will be required. education managers and second to use the Finally, it will produce tests to be used in methodological volume and instruments to national quality assurance programs that strengthen further national capacity and train will not only identify weak schools but also experts in the field of learning assessment. provinces where extra help might be needed. Beyond this, central and the provincial The members of the center will need to work levels of government will no doubt wish to in cooperation with the curriculum address some of the issues that have been developers and the group should be highlighted and the recommendations made nurtured and supported with assistance to in this report. A list of the major factors that develop the necessary skills and grow into a are associated with large achievement national center. differences among pupils and schools has been presented. Individual interventions have been discussed and the need for a combination of these measures into a coherent holistic package has been recommended. The areas with large 33 proportions of non-Kinh and also with large Assessment Research Center, 2003, Instructional methods in Vietnam Pupil achievement differences between the Kinh Achievement: supplementary analyses, and non-Kinh pupils are: Lang Son; Hoa Melbourne: University of Melbourne. Binh; Lai Chau; Kon Tum; and Tuyen 21 Vietnam Reading and Mathematics Achievement Study Quang. These areas could be designated that have been suggested in this paper, Educational Priority Areas (EPA), and could measurement and analysis of education be where the first efforts are made. quality and performance should be expanded with the application of the techniques used Meanwhile, in parallel with the in this study to other levels and types of improvement and compensatory programs education and training in Vietnam. 22 Vietnam Reading and Mathematics Achievement Study Appendix 1: Variation among schools (rhos) within countries in reading comprehension How did the Vietnam intra-class correlations (rhos) compare with the rhos in other countries? In 2001 the IEA undertook a large survey or reading in Grade 4 in many countries. The rhos for those countries have been given below in Table 1.1 Table 1.1: Intra-class correlations for Grade 4/5 in the IEA PIRLS Study (data collection 2001)34 Argentina 0.418 Latvia 0.213 Belize 0.348 Lithuania 0.214 Bulgaria 0.345 Moldova 0.395 Canada35 0.174 Morocco 0.554 Colombia 0.459 Netherlands 0.187 Cyprus 0.105 New Zealand 0.250 Czech Republic 0.157 Norway 0.096 England 0.179 Romania 0.351 France 0.161 Russian Federation 0.447 Germany 0.141 Scotland 0.179 Greece 0.221 Singapore 0.586 Hong Kong (SAR) 0.295 Slovakia 0.249 Hungary 0.222 Slovenia 0.087 Iceland 0.084 Sweden 0.132 Iran 0.382 Turkey 0.271 Israel 0.415 Macedonia 0.424 Italy 0.198 United States 0.271 Kuwait 0.334 It can be seen that the among-school variation ranged from .084 in Iceland to .586 in Singapore. Most were in the region of .10 to .25 and this is what is generally considered normal for primary school. However, this study showed that among-school variation in Vietnam, with a rho of .66, was larger than any of the PIRLS study countries. 34In many of the IEA countries an intact class was taken thus underestimating the within school variance that in turn results in an overestimation of the between school variance. 35Canada consisted only of the provinces of Ontario and Québec. 23 Vietnam Reading and Mathematics Achievement Study Appendix 2: Reading skill levels and mathematics skill levels Reading Skill Levels Level 1 Matches text at word or sentence level aided by pictures. Restricted to a limited range of vocabulary linked to pictures Level 2 Locates text expressed in short repetitive sentences and can deal with text unaided by pictures. Type of text is limited to short sentences and phrases with repetitive patterns. Level 3 Reads and understands longer passages. Can search backwards or forwards through text to for information. Understands paraphrasing. Expanding vocabulary enables understanding of sentences with some complex structure. Level 4 Links information from different parts of the text. Selects and connects text to derive and infer different possible meanings. Level 5 Links inferences and identifies an author's intention from information stated in different ways, in different text types and in documents where the message is not explicit. Level 6 Combines text with outside knowledge to infer various meanings, including hidden meanings. Identifies an author's purposes, attitudes, values, beliefs, motives, unstated assumptions and arguments. Mathematics Skill Levels Level 1 Reads, writes and compares natural; numbers, fractions and decimals. Uses single operations of +, -, x and / on simple whole numbers; works with simple measures such as time; recognizes simple 3D shapes. Level 2 Converts fractions with denominator of 10 to decimals. Calculates with whole numbers using one operation (x,-,+ or : ) in a one-step word problem; recognizes 2D and 3D shapes. Level 3 Identifies place value; determines the value of a simple number sentence; understands equivalent fractions; adds and subtracts simple fractions; carries out multiple operations in correct order; converts and estimates common and familiar measurement units in solving problems. Level 4 Reads, writes and compares larger numbers; solves problems involving calendars and currency, area and volume; uses charts and tables for estimation; solves inequalities; transformations with 3D figures; knowledge of angles in regular figures; understands simple transformations with 2D and 3D shapes. Level 5 Calculates with multiple and varied operations; recognizes rules and patterns in number sequences; calculates the perimeter and area of irregular shapes; measurement of irregular objects; recognizes transformed figures after reflection; solves problems with multiple operations involving measurement units, percentage and averages. Level 6 Problem solving with periods of time, length, area and volume; embedded and dependent number patterns; develops formulae; recognizes 3D figures after rotation and reflection and embedded figures and right angles in irregular shapes, data from graphs and tables. 24 Working Paper Series 2004-1 Disability Issues in East Asia: Review and Ways Forward 2004-2 Skills and Growth in Cambodia 2004-3 Evaluating the Performance of SGP and SIP: A Reivew of the Existing Literature and Beyond 2004-4 Vietnam Reading and Mathematics Assessment Study (Volume 1)