80100 Vietnam SABER Country Report STUDENT ASSESSMENT 2009 Key Policy Areas for Student Assessment Status 1. Classroom Assessment Guidelines for classroom assessment practices are provided in formal, system- level documents that are publicly available. These documents provide information on the type and number of assessments to be conducted by classroom teachers during the school year. Teachers primarily develop their classroom assessment expertise on the job. There has been a concerted effort to improve teachers' skills in classroom assessment, in part through special projects funded by donors. There are insufficient resources currently available to more systematically support teachers' development of skills in classroom assessment. Only ad-hoc mechanisms are being used to monitor the quality of classroom assessment practices, which are generally considered to be weak. 2. Examinations The examination system has been operating regularly under the guidance of a formal, publicly-available policy document. The Examinations office is part of the Department of Testing and Accreditation, which is accountable to the Ministry of Education. Regular funding for core examination activities is allocated by the government, although the quality of implementation of those activities varies. There are no mechanisms in place to ensure the quality of the examinations. 3. National Large-Scale Assessment (NLSA) The NLSA program has been operating on an irregular basis. Although there is no policy document that authorizes the NLSA, there is a general understanding that it will take place. The program only has irregular funding, provided by donors, that covers some core NLSA activities. The NLSA office must draw on staff from the National Institute of Educational Sciences and universities to carry out (with the support of international consultants) NLSA activities, and thus is inadequately staffed to effectively carry out these activities on a long-term basis. 4. International Large-Scale Assessment (ILSA) To date, Vietnam has never participated in an ILSA, but has taken concrete steps to participate in one in the near future by registering for PISA 2012. THE WORLD BANK VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Introduction SABER-Student Assessment methodology In 2009, Vietnam joined the Russia Education Aid for The SABER-Student Assessment framework is built on Development (READ) Trust Fund program, the goal of the available evidence base for what an effective which is to help countries improve their capacity to assessment system looks like. The framework provides design, carry out, analyze, and use assessments for guidance on how countries can build more effective improved student learning. As part of the READ Trust student assessment systems. The framework is Fund program, and in order to gain a better structured around two main dimensions of assessment understanding of the strengths and weaknesses of its systems: the types/purposes of assessment activities existing assessment system, Vietnam participated in a and the quality of those activities. formal exercise to benchmark this system under The World Bank’s Systems Approach for Better Education Assessment types and purposes Results (SABER) program. SABER is an evidence-based program to help countries systematically examine and Assessment systems tend to be comprised of three strengthen the performance of different aspects of their main types of assessment activities, each of which education systems. serves a different purpose and addresses different information needs. These three main types are: What is SABER-Student Assessment? classroom assessment, examinations, and large-scale, system level assessments. SABER-Student Assessment is a component of the Classroom assessment provides real-time information SABER program that focuses specifically on to support ongoing teaching and learning in individual benchmarking student assessment policies and systems. classrooms. Classroom assessments use a variety of The goal of SABER-Student Assessment is to promote formats, including observation, questioning, and paper- stronger assessment systems that contribute to and-pencil tests, to evaluate student learning, generally improved education quality and learning for all. on a daily basis. National governments and international agencies are Examinations provide a basis for selecting or certifying increasingly recognizing the key role that assessment of students as they move from one level of the education student learning plays in an effective education system. system to the next (or into the workforce). All eligible The importance of assessment is linked to its role in: students are tested on an annual basis (or more often if (i) providing information on levels of student the system allows for repeat testing). Examinations learning and achievement in the system; cover the main subject areas in the curriculum and (ii) monitoring trends in education quality over usually involve essays and multiple-choice questions. time; (iii) supporting educators and students with real- Large-scale, system-level assessments provide feedback time information to improve teaching and on the overall performance of the education system at learning; and particular grades or age levels. These assessments (iv) holding stakeholders accountable for results. typically cover a few subjects on a regular basis (such as every 3 to 5 years), are often sample based, and use multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities. VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional The indicators are identified based on a combination of and organizational structures for designing, carrying criteria, including: out, or using results from the assessment; the availability of sufficient and stable sources of funding; • professional standards for assessment; and the presence of trained assessment staff. • empirical research on the characteristics of effective assessment systems, including analysis of the System alignment refers to the extent to which the characteristics that differentiate between the assessment is aligned with the rest of the education assessment systems of low- versus high-performing system. This includes the degree of congruence nations; and between assessment activities and system learning • theory—that is, general consensus among goals, standards, curriculum, and pre- and in-service experts that it contributes to effective teacher training. assessment. Assessment quality refers to the psychometric quality of Levels of development the instruments, processes, and procedures for the assessment activity. It covers such issues as design and The World Bank has developed a set of standardized implementation of assessment activities, analysis and questionnaires and rubrics for collecting and evaluating interpretation of student responses to those activities, data on the three assessment types and related quality and the appropriateness of how assessment results are drivers. reported and used. The questionnaires are used to collect data on the Crossing the quality drivers with the different characteristics of the assessment system in a particular assessment types/purposes provides the framework country. The information from the questionnaires is and broad indicator areas shown in Table 1. This then applied to the rubrics in order to judge the framework is a starting point for identifying indicators development level of the country’s assessment system that can be used to review assessment systems and in different areas. plan for their improvement. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the development level of its assessment activities compared to best or recommended practice in each area. For each indicator, the rubric displays four development levels—Latent, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Emerging, Established, and Advanced. These levels are trained teachers in the country is high at 97 percent, and artificially constructed categories chosen to represent key the student to teacher ratio is relatively low at 20 students stages on the underlying continuum for each indicator. per teacher. Each level is accompanied by a description of what performance on the indicator looks like at that level. Vietnam spent about 5.3 percent of GDP on education in 2008 (latest available data) and approximately 20 • Latent is the lowest level of performance; it percent of the national budget was allocated to represents absence of, or deviation from, the education. desired attribute. • Emerging is the next level; it represents partial Vietnam has made important progress in improving its presence of the attribute. education system since the 1990s, including increasing • Established represents the acceptable minimum the educational attainment of the Vietnamese standard. population, introducing universal enrollment in primary • Advanced represents the ideal or current best education (the Law on Universal Primary Education that practice. was adopted in 1991 required compulsory basic education (grades 1-5) for all children), significantly A summary of the development levels for each increasing completion rates, and improving student assessment type is presented in Appendix 3. achievement levels. In reality, assessment systems are likely to be at At the same time, Vietnam has identified a number of different levels of development in different areas. For areas of focus for further development, including example, a system may be Established in the area of improving access to and the quality of education for examinations, but Emerging in the area of large-scale, lower-income students and ethnic minorities; improving system-level assessment, and vice versa. While intuition learning outcomes, especially for children from suggests that it is probably better to be further along in disadvantaged groups; improving pedagogical practices; as many areas as possible, the evidence is unclear as expanding access to school resources; and improving to whether it is necessary to be functioning at school governance. Advanced levels in all areas. Therefore, one might view the Established level as a desirable minimum outcome to Detailed information on Vietnam’s student assessment achieve in all areas, but only aspire beyond that in those system was obtained from the country’s self-diagnosis areas that most contribute to the national vision or report that was prepared as part of the READ Trust priorities for education. In line with these considerations, Fund program in 2009. At the time of the review, the ratings generated by the rubrics are not meant to be SABER-Student Assessment questionnaires had not additive across assessment types (that is, they are not been developed. Self-diagnosis information was meant to be added to create an overall rating for an analyzed using the SABER-Student Assessment rubrics. assessment system; they are only meant to produce an It is important to remember that the SABER-Student overall rating for each assessment type). The methodology Assessment rubrics primarily focus on benchmarking a for assigning development levels is summarized in country’s policies and arrangements for assessment Appendix 4. activities at the system or macro level. Additional data would need to be collected to determine actual, on-the- Education in Vietnam ground practices in Vietnam, particularly by teachers and students in schools. The following sections discuss the findings by each assessment type, accompanied by Vietnam is a lower middle income country in East Asia. suggested policy options. The suggested policy options GDP per capita (current US$, 2009) is $1130, with annual were determined in collaboration with key local growth of approximately 4.2 percent. At the primary level, stakeholders based on Vietnam’s immediate interests gross school enrollment is 104 percent; at the secondary and needs. Detailed, completed rubrics for each level, it is 76 percent. Gross enrollment for boys and girls is assessment type are provided in Appendix 5. approximately equal at both levels. The percentage of SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Classroom Assessment In general, classroom assessment practices are considered to be weak. While there are clear guidelines for conducting classroom assessment activities, primary Level of development school teachers in particular need additional training in how to assess students in all subject areas. Secondary In Vietnam, the official curriculum sets out assessment school teachers need assistance in how to assess their policies for teachers to apply in their classrooms. The own subject areas. Donor-supported projects provide Ministry of Education and Training provides regulations assistance in monitoring and building teacher capacity governing the type and number of assessments to be to improve the quality of classroom assessment conducted by teachers during the school year. This practices. document is in the form of a Ministry of Education and Training decision, which has been communicated to all Mechanisms are in place to monitor the quality of schools. classroom assessment practices, such as moderation efforts. However, these tend to be conducted on an ad There is an official curriculum or standards document hoc basis. that specifies what students are expected to learn, but the level of performance required is not clear. Classroom assessment information is required to be Specifically, the curriculum and outcome standards are disseminated to key stakeholders, such as school listed, but tend to be general in nature and hence rarely administrators, education managers, and parents. At used by schools for devising assessment tasks. the same time, there are limited required uses of classroom assessment information to support student There are scarce system-wide resources available to learning. teachers to help them engage in classroom assessment activities. This is because the education budget is Suggested policy options: primarily built around teacher salaries, and therefore limited resources are available for other things. 1. Make available system-level resources for teachers to enable them to engage in quality There are some system-level mechanisms to ensure classroom assessment practices. that teachers develop skills and expertise in classroom assessment. Limited training in assessment is provided 2. Introduce more systematic mechanisms to during pre-service teacher training. In general, teachers monitor and provide feedback on the quality of develop most of their expertise on the job. In the last classroom assessment practices. decade, there has been a concerted campaign to upgrade teachers' skills in assessment, which has been achieved through a set of special projects funded by donors such as Asian Development Bank and the World Bank. Short courses of two to three days have been offered to thousands of teachers in the last three years. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Examinations practices are learned on the job. Teams of teachers who undertake the marking of test papers receive special training to standardize procedures. Level of development There is a clear understanding of what the examinations The examinations program in Vietnam includes grade 9 measure. The examination questions are based on an examinations (used for school cycle certification), the examination framework for each subject that tends to School Leaving Examination at grade 12 and the follow the relevant textbook in use in schools rather University Entrance Examination. The School Leaving than the official curriculum. The School Leaving Examination and the University Entrance Examination Examination is comprised only of multiple choice are required by national law. Grade 9 examinations are questions. The University Entrance Examination incudes set by each province. Not all provinces are required to a variety of item types, including open ended questions. have grade 9 examinations as some make their The questions, however, are not pilot tested. The decisions based on students' achievement records in University Entrance Examination is perceived as more grades six through nine. The results of the School rigorous and valid than the grade 12 test and hence is Leaving Examination are used mainly for entry to used by universities for their admission decisions. All vocational/professional colleges. They are not used students can take the examinations; there are no extensively by employers, and are not used by language, gender, or other equivalent barriers. universities. The School Leaving Examination and the University Entrance Examination guidelines are There are no mechanisms in place, such as pilot testing disseminated every year before the examination. or pre-testing, to ensure the quality of the examinations. Most of the work in developing the Regular funding is allocated for the examinations by the examinations is undertaken by a team of teachers who government. This funding covers all core examination are solely responsible for ensuring the validity and activities, although not all activities are actually carried quality of the examination results. Some technical out to specification or standards. Funding does not documentation on the examinations is prepared, but it cover research and development, and no systematic is not in a formal report format. research on the validity and reliability of the examination results has been done. Currently, there are no mechanisms in place to monitor the consequences of the examinations. The Examinations office, GDETA, belongs to the Department of Testing and Accreditation and is Suggested policy options: accountable to the Ministry of Education and Training. The office has 35 full-time professional staff whose 1. Enhance public confidence in the quality of the main functions with regard to examinations are examinations by making publicly available a organizational and supervisory. Vietnam does not offer high-quality, independent technical report, and opportunities that prepare individuals for work on the introducing systematic and transparent examinations. Very few of the GDETA staff have had any mechanisms to ensure quality at key stages of special training in assessment policies and practices, the examinations process. and they do not actively participate in setting or monitoring the examinations. On-the-job training is the 2. Monitor the consequences of the examinations primary way of gaining expertise in assessment. by funding independent research on their Teachers develop and review the examinations. impact, creating a permanent oversight However, there are no up-to-date courses or workshops committee, or conducting regular quality on examinations available to teachers. Although reviews. teachers receive some training in assessment in their pre-service years, the content is more general and theoretical, and most contemporary assessment SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 National Large-Scale Assessment The NLSAs are developed on the basis of the national curriculum. Anchor items were selected from the first (NLSA) NLSA exercise and used in the second NLSA exercise for purposes of comparison. The tests results, when Level of development presented publicly, were not opposed, although some questioned the difficulty levels. There are also regular internal reviews of the NLSA to ensure that it measures National assessments of student achievement levels at what it is intended to measure. grade 5 were conducted in 2001 and 2007. The Asian Development Bank recently funded a project to carry There are some mechanisms in place to ensure the out a national assessment of grade 6 and grade 9 quality of the NLSA. In regards to the 2001 and 2007 students. While there is no policy document pertaining assessments, the anchor items were pretested, and to NLSA, there is a general understanding that the NLSA questions appear to have been given careful scrutiny to will take place. eliminate ambiguities and other potential weaknesses. The NLSAs included items assessing students' ability to The NLSA office is a temporary agency or group of make creative applications, and items measuring people. Specifically, the surveys are done by teams from problem-solving skills. The tests and questionnaires the Vietnam National Institute of Educational Sciences included questions from recent international surveys to (VNIES) and local universities, with support from provide a basis for comparing results with other international consultants. countries. The tests were trialed and revised. However, the final tests omitted several key areas in the There is irregular funding allocated to the NLSA. The curriculum, and appeared to include many relatively VNIES staff is funded through the government’s easy questions. education budget, while expenses pertaining to the national assessments of student achievement levels There is a comprehensive, high-quality technical report have been covered by external donors. The funding that was made available to the public. However, NLSA covers core NLSA activities (design, administration, results are poorly disseminated. analysis, and reporting), as well as research and development activities. Some mechanisms are in place to monitor the consequences of the NLSA, such as workshops and Almost all student groups are included in the NLSA expert reviews. sample. Only very hard to reach areas of the country are excluded. In addition, student with disabilities are Suggested policy options: not provided any special accommodations. 1. Institutionalize the NLSA by developing a formal Vietnam offers some opportunities to prepare policy document that authorizes the NLSA individuals for work on the NLSA. More than 20 NLSA program, and by ensuring regular government team members have attended training courses on funding to cover all core NLSA activities. national large-scale survey projects organized by the World Bank and the Asian Development Bank. These 2. Create a permanent, adequately-staffed office team members are now the Vietnamese experts in the responsible for NLSA activities and accountable areas of test design, sampling, data processing, and to an external body. other technical matters. The training of NLSA team members in the implementation of NLSAs has been mainly provided by donors and foreign experts. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 International Large-Scale Assessment (ILSA) Level of development Suggested policy options: Vietnam has not participated in an ILSA in the last 10 years. However, it has registered for PISA 2012. At the 1. Provide a stable organizational structure within moment, there is no policy document that addresses Vietnam for the ILSA program, and introduce a participation in ILSAs, and there is no funding for policy document that addresses participation in, participation in ILSAs. and the allocation of government funds for, ILSA activities. A team and a national coordinator have been identified to carry out ILSA activities. The ILSA office is newly 2. Build capacity within Vietnam to carry out ILSA established and thus has only a few permanent staff. activities, including ensuring that the national The ILSA team has attended some international coordinator is fluent in the international workshops and meetings. Some opportunities to learn language of the assessment (English, in the case about ILSA have been made available in the country to of PISA) and providing the necessary training to ILSA team members only. ILSA staff so that they can carry out ILSA activities effectively and with minimal issues. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education system opportunities instruction grade/age system at to the next (or level(s), and to particular into the monitor trends in grade/age workforce) learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the system where the system basis (such as basis (such as allows for allows for every 3-5 years) every 3-5 years) repeats repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a few confined to one subject areas subject areas subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide institutional wide institutional support and ensure the capacity to support and capacity to support and capacity to support and quality of classroom ensure the quality of ensure the quality of ensure the quality of assessment practices. classroom assessment classroom assessment classroom assessment practices. practices. practices. CLASSROOM ASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and and a need to develop There is institutional institutional capacity and institutional capacity to capacity and some strong mechanisms to run the examination. The limited mechanisms to monitor it. The EXAMINATIONS examination typically is monitor it. The examination is of high of poor quality and is examination is of quality and is perceived perceived as unfair or acceptable quality and is as fair and free from corrupt. perceived as fair for corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA There is a stable NLSA place. NLSA in place and a in place. There is in place and institutional need to develop institutional capacity and capacity and strong institutional capacity to some limited mechanisms to monitor run the NLSA. mechanisms to monitor it. The NLSA is of high NATIONAL (OR SYSTEM- Assessment quality and it. The NLSA is of quality and its LEVEL) LARGE-SCALE impact are weak. moderate quality and its information is ASSESSMENT information is effectively used to disseminated, but not improve education. always used in effective ways. There is no history of Participation in an ILSA There is more or less There is stable participation in an ILSA has been initiated, but stable participation in an participation in an ILSA nor plans to participate there still is need to ILSA. There is and institutional capacity in one. develop institutional institutional capacity to to run the ILSA. The capacity to carry out the carry out the ILSA. The information from the INTERNATIONAL LARGE- ILSA. information from the ILSA is effectively used SCALE ASSESSMENT ILSA is disseminated, to improve education. but not always used in effective ways. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 4: Methodology for Assigning 4. A preliminary level of development is assigned to Development Levels each quality driver. 1. The country team or consultant collects information 5. The preliminary development level is validated using about the assessment system in the country. expert judgment in cooperation with the country team and The World Bank Task Team Leader. 2. Based on the collected information, a level of development and score is assigned to each dimension in For scores that allow a margin of discretion (i.e., to the rubrics: choose between two levels of development), a final decision has to be made based on expert judgment. For • Latent = 1 score point example, the aforementioned hypothetical country has • Emerging = 2 score points an ‘Enabling Context’ score of 2.33, corresponding to a • Established = 3 score points preliminary level of development of ‘Emerging or • Advanced = 4 score points Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the 3. The score for each quality driver is computed by country team chooses ‘Emerging’ as the most aggregating the scores for each of its constituent appropriate level. dimensions. For example: 6. Scores for certain key dimensions under ‘Enabling The quality driver, ‘Enabling Context,’ in the case of Context’ (in the case of EXAM, NLSA, and ILSA) and ILSA, has 3 dimensions on which a hypothetical country under ‘System Alignment’ (in the case of CLASS) were receives the following scores: Dimension A = 2 points; set as ceiling scores, i.e., the overall mean score for the Dimension B = 2 points; Dimension C = 3 points. The particular assessment type cannot be greater than the hypothetical country’s overall score for this quality score for these key dimensions. These key variables driver would be: (2+2+3)/3 = 2.33. include formal policy, regular funding, having a permanent assessment unit, and the quality of assessment practices. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 5: SABER-Student Assessment Rubrics for Vietnam This appendix provides the completed SABER-Student Assessment rubrics for each type of assessment activity in Vietnam. In each row of the rubric, the relevant selection is indicated by a thick border and an asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection (as indicated by a thick border and an asterisk). The explanation or justification text can be located in the “Development level rating justifications” section at the end of each rubric. If a row includes a superscript, but not a thick border and an asterisk, this means that insufficient information was available to determine the relevant selection in the row. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 13 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 VIETNAM Classroom Assessment SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment There is no system-level document that There is an informal system-level There is a formal system-level document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. 1 assessment. classroom assessment. assessment. * 2 This option does not apply to this This option does not apply to this The availability of the document is The document is widely available. dimension. dimension. restricted. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with system learning goals There are no system-wide resources for There are scarce system-wide resources There are some system-wide resources There are a variety of system-wide 3 teachers for classroom assessment. for teachers for classroom assessment. for teachers for classroom assessment. resources available for teachers for classroom assessment. * There is no official curriculum or There is an official curriculum or There is an official curriculum or There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn or students are expected to learn, but the students are expected to learn and to to what level of performance. level of performance required is not what level of performance. 4 clear. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: Having effective human resources to carry out classroom assessment activities There are no system-level mechanisms This option does not apply to this There are some system-level There are a variety of system-level to ensure that teachers develop skills dimension. mechanisms to ensure that teachers mechanisms to ensure that teachers and expertise in classroom assessment. develop skills and expertise in classroom develop skills and expertise in classroom * 5 assessment. assessment. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis, and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are 6 from widespread weaknesses or there is known to be weak. known to be of moderate quality. known to be generally of high quality. no information available on classroom assessment practices. * There are no mechanisms to monitor the There are ad hoc mechanisms to monitor There are limited systematic mechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the 7 practices. practices. assessment practices. quality of classroom assessment * practices. ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required to be disseminated to some key required to be disseminated to all key 8 stakeholders. stakeholders. stakeholders. * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessment to support student learning. classroom assessment to support classroom assessment to support classroom assessment to support 9 student learning. student learning, excluding its use as an student learning, including its use as an * input for external examination results. input for external examination results. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Classroom Assessment: Development level rating justifications 1. The official curriculum sets out assessment policies for teachers to apply in their classrooms. MOET has a set of regulations governing the type and number of assessments to be made by teachers in the school year. 2. The document is in the form of MOET's decision which has been communicated to all schools. 3. The budget for education is primarily to cover teacher salaries; hence, little is left for professional development and teacher support. 4. The curriculum and outcome standards are specified, but tend to be general and hence rarely used by schools for devising assessment tasks. 5. While limited training in assessment is provided during pre-service teacher training, teachers develop their expertise mostly on the job. In the last decade, there has been a concerted campaign to upgrade teachers' skills in assessment, which has been achieved by a set of special projects funded by donors such as ADB or the World Bank. Short courses of 2-3 days in duration have been offered to thousands of teachers in the last three years. 6. While there are clear guidelines on conducting classroom assessment, primary school teachers in particular need additional training in how to assess all subject areas. Secondary teachers need assistance in how to assess their own subject areas. Donor-supported projects provide assistance in monitoring and building teacher capacity to improve the quality of classroom assessment. 7. There are some moderation efforts but they not systematic. 8. School administrators, education managers and parents receive students' grades. 9. There are clear guidelines on the processes of conducting classroom assessment and on using the assessment results to support student learning. Classroom assessment, however, is not an input for external examination results at the Grade 12 level, while it is for the Grade 9 examinations. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 VIETNAM Examinations SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which assessment activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies No standardized examination has taken The standardized examination has been The examination is a stable program that This option does not apply to this place. operating on an irregular basis. has been operating regularly. dimension * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this 1 authorizes the examination. document that authorizes the authorizes the examination. dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this 2 dimension. the public public. dimension. * This option does not apply to this This option does not apply to this The policy document addresses some The policy document addresses all key 3 dimension. dimension. key aspects of the examination. aspects of the examination. * ENABLING CONTEXT 2: Having strong leadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the the examination or are indifferent to it. examination. examination. examination. * There are no attempts to improve the This option does not apply to this There are independent attempts to There are coordinated attempts to examination by stakeholder groups. dimension. improve the examination by stakeholder improve the examination by stakeholder 3 groups. groups. * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this not welcomed by the leadership in dimension. generally welcomed by the leadership in dimension. charge of the examination. charge of the examination. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this 4 examination. the examination. examination. dimension. * This option does not apply to this Funding covers some core examination Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. 5 processing or reporting. processing and reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and 6 dimension. development. dimension. development. * ENABLING CONTEXT 4: Having strong organizational structures The examination office does not exist or The examination office is newly The examination office is a stable This option does not apply to this 7 is newly established. established. organization. dimension. * The examination office is not This option does not apply to this The examination office is accountable to This option does not apply to this accountable to an external board or dimension. an external board or agency. dimension. 8 agency. * Examination results are not recognized Examination results are recognized by Examination results are recognized by Examination results are recognized by by any certification or selection system. certification or selection system in the one certification or selection system in two or more certification or selection 9 country. * another country. system in another country. The examination office does not have The examination office has some of the The examination office has all of the The examination office has state of the the required facilities to carry out the required facilities to carry out the required facilities to carry out the art facilities to carry out the examination. 10 examination. examination. examination. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment 11 examination, issues are pervasive. effectively, with minimal issues. effectively, with no issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on 12 examination. examination. the examination. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning examinations with learning goals and opportunities to learn It is not clear what the examination This option does not apply to this There is a clear understanding of what This option does not apply to this 13 measures. dimension. the examination measures. dimension. * What the examination measures is This option does not apply to this What is measured by the examination is This option does not apply to this 14 questioned by some stakeholder groups. dimension. largely accepted by stakeholder groups. dimension. Material to prepare for the examination There is some material to prepare for the There is comprehensive material to * There is comprehensive material to is minimal and it is only accessible to examination that is accessible to some prepare for the examination that is prepare for the examination that is 15 very few students. students. accessible to most students. accessible to all students. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the examination There are no courses or workshops on There are no up-to-date courses or There are up-to-date voluntary courses There are up-to-date compulsory courses examinations available to teachers. workshops on examinations available to or workshops on examinations available or workshops on examinations for 16 teachers. to teachers. teachers. * Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in most examination-related tasks. examination-related tasks. examination-related tasks. examination-related tasks. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the assessment meets quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring quality There is no technical report or other There is some documentation on the There is a comprehensive technical There is a comprehensive, high quality documentation. examination, but it is not in a formal report but with restricted circulation. technical report available to the general report format. public. * There are no mechanisms in place to This option does not apply to this There are limited systematic mechanisms There are varied and systematic 17 ensure the quality of the examination. dimension. in place to ensure the quality of the mechanisms in place to ensure the examination. quality of the examination. * ASSESSMENT QUALITY 2: Ensuring fairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the 18 examination process is high. examination process is moderate. examination process is low. examination process is marginal. * The examination results lack credibility The examination results are credible for The examination results are credible for This option does not apply to this 19 for all stakeholder groups. some stakeholder groups. all stakeholder groups. dimension. * The majority of the students (over 50%) A significant proportion of students A small proportion of students (less than All students can take the examination; may not take the examination because of (10%-50%) may not take the examination 10%) may not take the examination there are no language, gender or other 20 language, gender, or other equivalent because of language, gender, or other because of language, gender, or other equivalent barriers. barriers. equivalent barriers. equivalent barriers. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Using examination information in a fair way Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all 21 proper way by all stakeholder groups. stakeholder groups in a proper way. stakeholder groups in a proper way. stakeholder groups in a proper way. 22 * Student names and results are public. This option does not apply to this Students’ results are confidential. This option does not apply to this dimension. dimension. * ASSESSMENT QUALITY 4: Ensuring positive consequences of the examination There are no options for students who There are very limited options for There are some options for students who There is a variety of options for students do not perform well on the examination, students who do not perform well on the do not perform well on the examination. who do not perform well on the 23 or students must leave the education examination. examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination. examination. examination. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Examinations: Development level rating justifications 1. The Grade 12 examinations are required by law, and are therefore part of the legal framework. Grade 9 examinations, however, are set by each province, and not all provinces are required to have the grade 9 examinations (as some provinces make their selections entirely based on the students' achievement record in grades six through nine). 2. Grade 12 and university entrance exam guidelines are produced and disseminated every year before the exam date. 3. There are independent attempts to improve the examinations, including by policy makers and the media. There is a lot of debate on how best to improve the examinations. 4. Regular funding is allocated for the examination by the government. 5. Funding covers all core examination activities, although not all activities are actually done to specification/standards. 6. No systematic research on the validity and reliability of the exam results has been done. 7. The Examination office belongs to the Department of Testing and Accreditation. 8. GDETA is part of MOET, to which it is accountable. Therefore, GDETA is not accountable to an external board or agency. 9. In Vietnam, the results of the Grade 12 School Leaving Examination are used mainly for entry to vocational/professional colleges, are not used extensively by employers, and are not used at all by universities. It is unclear whether the examination results are recognized in another country. 10. The Examination office has scanning and computing capacity for administering and recording exam results. 11. The examination office has 35 full-time professional staff whose main functions with regard to examinations are organizational and supervisory. Staffing is stable, with only two staff leaving in the last ten years. 12. "Very few of the staff of GDETA, however, have had any special training in assessment policies and practices, and they do not participate actively in setting or monitoring the examinations. On-the-job training is the primary way of gaining expertise in assessment and teams of teachers are depended upon to develop and review the examinations. " 13. The examination questions are based on an examination framework for each subject that tends to follow the relevant textbook in use in schools rather than the official curriculum. Additionally, the Grade 12 School Leaving Examination is comprised of only multiple choice questions, and topics not readily examined SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 through multiple-choice questions, such as oral expression, listening, writing, practical skills, and project work, are not included in the examination. The University Entrance Examination, however, incudes a variety of item types, including open ended questions. The questions, however, are not pilot tested and time limits are not tested. 14. Few stakeholders question the exam content. 15. While nearly all schools have access to the required textbooks, many students from minority ethic groups (of which there are 54) are disadvantaged because the examinations have to be taken in the Vietnamese language. These students are also handicapped because their teachers tend to be less qualified and their resources are leaner. 16. Although teachers do receive some training in assessment in their pre-service years, the content is more general and theoretical, and most current assessment practices are learned on the job. Additionally, teams of teachers who undertake the marking of test papers receive special training to standardize procedures. The training varies from three days to a week. 17. Most of the work in developing the examinations is undertaken by a team of teachers who are solely responsible for ensuring the validity and quality of the examination. No trial testing or pre-testing is done. 18. While the examinations are carefully supervised, cheating does occur and many staff have minimal faith in the security policies in place for the Grade 12 examinations. 19. The University entrance exam is perceived as more rigorous and valid than Grade 12 graduation and hence is used by universities for their admission decisions. 20. All students study in Vietnamese from Lower secondary onwards; the gender ratio seems to be balanced amongst exam takers. 21. Examinations results are used for certifying high school leaving and making university admission decisions. 22. Student names and results are published on the school bulletin board. 23. There is a provision for recounts where students feel that they deserve a better score on the examination. Additionally, students may retake the examination the following year. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 VIETNAM National (or System-Level) Large-Scale Assessment (NLSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which NLSA activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the NLSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for NLSA No NLSA exercise has taken place. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this 1 irregular basis. been operating regularly. dimension. There is no policy document pertaining There is an informal or draft policy * There is a formal policy document that This option does not apply to this to NLSA. document that authorizes the NLSA. authorizes the NLSA. dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. There is no plan for NLSA activity. This option does not apply to this There is a general understanding that the There is a written NLSA plan for the dimension. NLSA will take place. coming years. * ENABLING CONTEXT 2: Having strong public engagement for NLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the NLSA. 2 the NLSA or are indifferent to it. NLSA. NLSA. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for NLSA There is no funding allocated to the There is irregular funding allocated to There is regular funding allocated to the This option does not apply to this 3 NLSA. the NLSA. NLSA. dimension. * This option does not apply to this Funding covers some core NLSA Funding covers all core NLSA activities: This option does not apply to this dimension. activities: design, administration, analysis design, administration, analysis and dimension. 4 and reporting. reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development activities. dimension. development activities.5 * ENABLING CONTEXT 4: Having strong organizational structures for NLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this 6 team. group of people. institution or unit. dimension. * This option does not apply to this Political considerations regularly hamper Political considerations sometimes Political considerations never hamper 7 dimension. technical considerations. hamper technical considerations. technical considerations. * This option does not apply to this The NLSA office is not accountable to a The NLSA office is accountable to a This option does not apply to this 8 dimension. clearly recognized body. clearly recognized body. dimension. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources for NLSA There is no staff allocated for running an The NLSA office is inadequately staffed The NLSA office is adequately staffed to The NLSA office is adequately staffed to 9 NLSA. to effectively carry out the assessment. carry out the NLSA effectively, with carry out the NLSA effectively, with no minimal issues. issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities to The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the opportunities to prepare individuals for 9 10 NLSA. NLSA. work on the NLSA. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the NLSA is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against This option does not apply to this 11 curriculum or learning standards. dimension. curriculum or learning standards. dimension. * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned What the NLSA measures is largely 12 questioned by stakeholder groups. dimension. by some stakeholder groups. accepted by stakeholder groups. * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSA to ensure that it measures what it dimension. measures what it is supposed to intended to measure. is intended to measure. measure. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the NLSA There are no courses or workshops on There are occasional courses or There are some courses or workshops on There are widely available high quality 13 the NLSA. workshops on the NLSA. the NLSA offered on a regular basis. courses or workshops on the NLSA offered on a regular basis. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the NLSA meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA No options are offered to include all This option does not apply to this At least one option is offered to include Different options are offered to include 14 groups of students in the NLSA. dimension. all groups of students in the NLSA. all groups of students in the NLSA. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in 15 ensure the quality of the NLSA. dimension. ensure the quality of the NLSA. place to ensure the quality of the NLSA. * There is no technical report or other There is some documentation about the There is a comprehensive technical There is a comprehensive, high quality documentation about the NLSA. technical aspects of the NLSA, but it is report but with restricted circulation. technical report available to the general 16 not in a formal report format. public. * ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA NLSA results are not disseminated. NLSA results are poorly disseminated. NLSA results are disseminated in an This option does not apply to this effective way. dimension. * NLSA information is not used or is used This option does not apply to this NLSA results are used by some NLSA information is used by all in ways inconsistent with the purposes dimension. stakeholder groups in a way that is stakeholder groups in a way that is or the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the assessment. * assessment. There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in 17 monitor the consequences of the NLSA. dimension. monitor the consequences of the NLSA. place to monitor the consequences of the NLSA. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 National (or System-Level) Large Scale Assessment (NLSA): Development level rating justifications 1. National assessments of student achievement levels at Grade 5 were conducted in 2001 and in 2007. The Asian Development Bank recently funded a project to carry out a national assessment of Grade 6 and Grade 9 students. 2. There is no opposition to date although some provinces were anxious when seeing their results below the national average. 3. The VNIES staff are funded through the Education budget (paid for by the government), while expenses pertaining to the national assessments of student achievement levels have been covered by external donors, such as the World Bank, ADB, and Inrec/Mekong Economics. 4. The funding from donors covers some core NLSA activities. 5. Funding covers research and development. For example, funding covers design and anchoring between surveys and data analysis. 6. The surveys are done by teams from the National Institute of Educational Sciences and universities, with support from international consultants. 7. Political considerations never hamper technical considerations. For example, for the Grade 5 survey, the sample was constructed so that it was representative at both the national and provincial levels. 8. The NLSA office has not been officially established. 9. Internally, assistance for conducting the NLSA was provided by GDETA staff, Primary Education Department officials, Department staff in the provinces, and foreign consultants. The tests and questionnaires used in the survey were developed by teams of local educators, assisted by foreign consultants. After marking, recording and cleaning, the merging of results and further analyses were undertaken largely by foreign consultants. Some of the VNIES staff were trained in Europe to assist with the analyses. While the main implementers of the project were VNIES staff, the final reports were largely written by World Bank staff. 10. More than 20 of the staff have attended training courses organized by the World Bank and the ADB, especially for national large-scale survey projects, and these staff are now the Vietnamese experts in the areas of test design, sampling, data processing, and other technical matters. The training of human resources for implementation of national large-scale assessments has been primarily provided externally by donors and foreign experts. 11. The NLSA was developed on the basis of the National Curriculum. Anchor items were selected from the first NLSA exercise and used in the second NLSA exercise for purposes of comparison. The tests results, when presented, were not opposed though some questioned the difficulty levels. 12. The World Bank, IIEP, University of Melbourne and others provided training to Vietnam on large-scale surveys. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 13. Almost all student groups are included in the NLSA sample. Only very hard to reach areas are not included and student with disabilities are not provided any special accommodations. 14. The anchor items were pretested, and questions appear to have been given careful scrutiny to eliminate ambiguities and other potential weaknesses. The 15. NLSA included items assessing students' ability to make creative applications, and items measuring problem-solving skills. The tests and questionnaires included questions from recent international surveys to provide a basis for comparing results with other countries. The tests were trialed and revised. However, the final test omitted several key areas in the curriculum, and appeared to include many relatively easy questions. 16. A volume on technical aspects of the surveys was compiled and made public. 17. Workshops and expert reviews are held. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 VIETNAM International Large-Scale Assessment (ILSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which ILSA takes place in a country or system and the extent to which that framework is conducive to, or supportive of, ILSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for ILSA The country/system has not participated This option does not apply to this The country/system has participated in The country/system has participated in in an ILSA in the last 10 years. dimension. at least one ILSA in the last 10 years. two or more ILSA in the last 10 years. * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concrete steps to participate in an ILSA in dimension. steps to participate in at least one ILSA in dimension. 1 the next 5 years. the next 5 years. * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this addresses participation in ILSA. document that addresses participation in addresses participation in ILSA. dimension. ILSA. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. ENABLING CONTEXT 2: Having regular funding for ILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at There is regular funding approved by law, ILSA. donors. discretion. decree or norm. * This option does not apply to this Funding covers some core activities of Funding covers all core activities of the Funding covers all core activities of the dimension. the ILSA. ILSA. ILSA. Funding does not cover research and This option does not apply to this This option does not apply to this Funding covers research and development activities. dimension. dimension. development activities. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective human resources for ILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. activities. activities. activities. * This option does not apply to this The national/system coordinator or The national/system coordinator is fluent This option does not apply to this dimension. other designated team member may not in the language of the assessment. dimension. be fluent in the language of the This option does not apply to this assessment. * The ILSA office is inadequately staffed or The ILSA office is adequately staffed or The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, 2 effectively. with minimal issues. with no issues. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Providing opportunities to learn about ILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this international workshops or meetings. international workshops or meetings. workshops or meetings. dimension. * The country/system offers no This option does not apply to this The country/system offers some The country/system offers a wide range 3 opportunities to learn about ILSA. dimension. opportunities to learn about ILSA. of opportunities to learn about ILSA. * This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition 4 team members only. to the country's/system's ILSA team * members. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of ILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical The country/system met all technical published. standards to have its data presented standards required to have its data standards required to have its data beneath the main display of the presented in the main displays of the presented in the main displays of the international report or in an annex. international report. international report. The country/system has not contributed This option does not apply to this This option does not apply to this The country/system has contributed new new knowledge on ILSA. dimension. dimension. knowledge on ILSA. ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA If any, country/system-specific results Country/system-specific results and Country/system-specific results and Country/system-specific results and and information are not disseminated in information are disseminated irregularly information are regularly disseminated in information are regularly and widely the country/system. in the country/system. the country/system. disseminated in the country/system. Products to provide feedback to schools This option does not apply to this Products to provide feedback to schools Products to provide feedback to schools and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are not made available. sometimes made available. systematically made available. There is no media coverage of the ILSA There is limited media coverage of the There is some media coverage of the There is wide media coverage of the ILSA results. ILSA results. ILSA results. results. If any, country/system-specific results Results from the ILSA are used in a Results from the ILSA are used in some Results from the ILSA are used in a and information from the ILSA are not limited way to inform decision making in ways to inform decision making in the variety of ways to inform decision used to inform decision making in the the country/system. country/system. making in the country/system. country/system. It is not clear that decisions based on This option does not apply to this This option does not apply to this Decisions based on the ILSA results have ILSA results have had a positive impact dimension. dimension. had a positive impact on students' on students' achievement levels. achievement levels. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 International Large Scale Assessment (ILSA): Development level rating justifications 1. Vietnam has registered for PISA 2012. 2. The office is newly established with few permanent staff. 3. There are some opportunities to learn about the ILSA. 4. The opportunities to learn about the ILSA are available only to the ILSA team members. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 United Nations Educational, Scientific, and Cultural Acknowledgements Organization (UNESCO) Institute for Statistics. Vietnam This report was prepared by The World Bank SABER- Country Indicator Data. Montreal, QC: UNESCO. Data Student Assessment team in collaboration with Thanh retrieved from http://stats.uis.unesco.org/unesco on Thi Mai, World Bank Senior Education Specialist and March 12, 2013. Task Team Leader for education projects in Vietnam. World Bank. 2013. “Vietnam at a Glance.” Available online: Data retrieved from http:// devdata.worldbank.org/AAG/vnm_aag.pdf on April 11, References 2013. Clarke, M. 2012. “What Matters Most for Student Assessment Systems: A Framework Paper.” ———. 2011. “Vietnam: High Quality Education for All READ/SABER Working Paper Series. Washington, DC: by 2020.” Washington, DC: World Bank. World Bank. World Bank Development Indicators. Vietnam Country United Nations Children’s Fund (UNICEF). “Vietnam Indicator Data. Washington, DC: World Bank. Data Education Overview.” Data retrieved from retrieved from http://databank.worldbank.org/data on http://www.unicef.org/vietnam/girls_education.html March 12, 2013. on April 11, 2013. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41 VIETNAM ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. THE WORLD BANK SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42