79940 Kyrgyz Republic SABER Country Report STUDENT ASSESSMENT 2009 Key Policy Areas for Student Assessment Status 1. Classroom Assessment There is recognition among key stakeholders of the importance of classroom assessment and the urgent need to strengthen existing classroom assessment practices (which are very weak) in order to better support student learning. There also is recognition of the need to introduce systematic mechanisms to ensure the quality of classroom assessment practices. While such mechanisms have been piloted in some regions of the country with the support of donor-funded projects, there is a need to institutionalize them at the system level. 2. Examinations The examination system has been operating on a regular basis under the provisions approved by the Government of the Kyrgyz Republic. As the current examination system is based on Soviet principles, it is in the process of being reformed to better align with the demands of the free-market economy. Efforts to improve the examination system have been welcomed by various stakeholders, including the leadership in charge of the examinations. 3. National Large-Scale Assessment (NLSA) The Kyrgyz Republic has been conducting NLSAs on an irregular basis with the support of donor agencies. These assessments measure performance against national curriculum/learning standards. Various mechanisms are being used to better ensure the quality of these NLSAs, including pretesting items and providing technical training for staff. Policy makers have used NLSA results to modify aspects of the education system, including curriculum and assessment. 4. International Large-Scale Assessment (ILSA) The Kyrgyz Republic has participated in two rounds of PISA (2006 and 2009). In both instances, it met all technical standards required to have its data presented in the main displays of the international report. Within the Kyrgyz Republic, results were presented in a consolidated national report and distributed to key decision makers and schools. The Kyrgyz Republic has not, however, taken concrete steps to participate in any other ILSAs and there is no policy document that addresses participation in future ILSAs. THE WORLD BANK KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Introduction SABER-Student Assessment methodology In 2009, the Kyrgyz Republic joined the Russia Education The SABER-Student Assessment framework is built on Aid for Development (READ) Trust Fund program, the the available evidence base for what an effective goal of which is to help countries improve their capacity assessment system looks like. The framework provides to design, carry out, analyze, and use assessments for guidance on how countries can build more effective improved student learning. As part of the READ Trust student assessment systems. The framework is Fund program, and in order to gain a better structured around two main dimensions of assessment understanding of the strengths and weaknesses of its systems: the types/purposes of assessment activities existing assessment system, the Kyrgyz Republic and the quality of those activities. participated in a formal exercise to benchmark its student assessment system under The World Bank’s Assessment types and purposes Systems Approach for Better Education Results (SABER) program. SABER is an evidence-based program to help Assessment systems tend to be comprised of three countries systematically examine and strengthen the main types of assessment activities, each of which performance of different aspects of their education serves a different purpose and addresses different systems. information needs. These three main types are: classroom assessment, examinations, and large-scale, What is SABER-Student Assessment? system level assessments. Classroom assessment provides real-time information SABER-Student Assessment is a component of the to support ongoing teaching and learning in individual SABER program that focuses specifically on classrooms. Classroom assessments use a variety of benchmarking student assessment policies and systems. formats, including observation, questioning, and paper- The goal of SABER-Student Assessment is to promote and-pencil tests, to evaluate student learning, generally stronger assessment systems that contribute to on a daily basis. improved education quality and learning for all. Examinations provide a basis for selecting or certifying National governments and international agencies are students as they move from one level of the education increasingly recognizing the key role that assessment of system to the next (or into the workforce). All eligible student learning plays in an effective education system. students are tested on an annual basis (or more often if The importance of assessment is linked to its role in: the system allows for repeat testing). Examinations (i) providing information on levels of student cover the main subject areas in the curriculum and learning and achievement in the system; usually involve essays and multiple-choice questions. (ii) monitoring trends in education quality over time; Large-scale, system-level assessments provide feedback (iii) supporting educators and students with real- on the overall performance of the education system at time information to improve teaching and particular grades or age levels. These assessments learning; and typically cover a few subjects on a regular basis (such as (iv) holding stakeholders accountable for results. every 3 to 5 years), are often sample based, and use multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional and organizational structures for designing, carrying The indicators are identified based on a combination of out, or using results from the assessment; the criteria, including: availability of sufficient and stable sources of funding; • professional standards for assessment; and the presence of trained assessment staff. • empirical research on the characteristics of effective assessment systems, including analysis of the System alignment refers to the extent to which the characteristics that differentiate between the assessment is aligned with the rest of the education assessment systems of low- versus high-performing system. This includes the degree of congruence nations; and between assessment activities and system learning • theory—that is, general consensus among goals, standards, curriculum, and pre- and in-service experts that it contributes to effective teacher training. assessment. Assessment quality refers to the psychometric quality of Levels of development the instruments, processes, and procedures for the assessment activity. It covers such issues as design and The World Bank has developed a set of standardized implementation of assessment activities, analysis and questionnaires and rubrics for collecting and evaluating interpretation of student responses to those activities, data on the three assessment types and related quality and the appropriateness of how assessment results are drivers. reported and used. The questionnaires are used to collect data on the Crossing the quality drivers with the different characteristics of the assessment system in a particular assessment types/purposes provides the framework country. The information from the questionnaires is and broad indicator areas shown in Table 1. This then applied to the rubrics in order to judge the framework is a starting point for identifying indicators development level of the country’s assessment system that can be used to review assessment systems and in different areas. plan for their improvement. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the development level of its assessment activities compared to best or recommended practice in each area. For each indicator, the rubric displays four development levels—Latent, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Emerging, Established, and Advanced. These levels are At the primary level, net school enrollment is 86 artificially constructed categories chosen to represent key percent; at the secondary level, it is 79 percent. Net stages on the underlying continuum for each indicator. enrollment for boys and girls is equal at both levels. The Each level is accompanied by a description of what government has a policy of compulsory education for all performance on the indicator looks like at that level. children through grade 9. Grades 10-12 are non- compulsory. The primary completion rate is 98 percent. • Latent is the lowest level of performance; it represents absence of, or deviation from, the Although nearly all students complete basic education, desired attribute. results from the 2006 and 2009 administration of the • Emerging is the next level; it represents partial Program for International Student Assessment (PISA), presence of the attribute. which assesses the competencies of 15-year-olds in • Established represents the acceptable minimum reading, mathematics, and science, demonstrate standard. dramatically low levels of performance of Kyrgyz • Advanced represents the ideal or current best students as compared to the average scores of students practice. from other participating countries. While there were improvements in the Kyrgyz Republic’s scores on PISA A summary of the development levels for each between 2006 and 2009, analysis of the PISA results assessment type is presented in Appendix 3. indicates that a lack of quality in teaching is a major determinant of low results on these assessments. In reality, assessment systems are likely to be at different levels of development in different areas. For Government priorities for education reform include the example, a system may be Established in the area of revision and improvement of the national curriculum, examinations, but Emerging in the area of large-scale, introduction of improved teaching strategies and an system-level assessment, and vice versa. While intuition incentive structure to enhance teacher performance, suggests that it is probably better to be further along in and increased monitoring of school fees. as many areas as possible, the evidence is unclear as to whether it is necessary to be functioning at Detailed information on the Kyrgyz Republic’s student Advanced levels in all areas. Therefore, one might view assessment system was obtained from the country’s the Established level as a desirable minimum outcome to self-diagnosis report that was prepared as part of the achieve in all areas, but only aspire beyond that in those READ Trust Fund program in 2009. At the time of the areas that most contribute to the national vision or review, SABER-Student Assessment questionnaires had priorities for education. In line with these considerations, not been developed. Self-diagnosis information was the ratings generated by the rubrics are not meant to be analyzed using the SABER-Student Assessment rubrics. additive across assessment types (that is, they are not It is important to remember that the SABER-Student meant to be added to create an overall rating for an Assessment rubrics primarily focus on benchmarking a assessment system; they are only meant to produce an country’s policies and arrangements for assessment overall rating for each assessment type). The methodology activities at the system or macro level. Additional data for assigning development levels is summarized in would need to be collected to determine actual, on-the- Appendix 4. ground practices in the Kyrgyz Republic, particularly by teachers and students in schools. The following sections discuss the findings by each assessment type, Education in the Kyrgyz Republic accompanied by suggested policy options. The suggested policy options were determined in The Kyrgyz Republic is a low income country in Central collaboration with key local stakeholders based on the Asia. GDP per capita (current US$, 2009) is $871, with Kyrgyz Republic’s immediate interests and needs. annual growth of approximately 1.7 percent. Detailed, completed rubrics for each assessment type in the Kyrgyz Republic are provided in Appendix 5. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Classroom Assessment Classroom assessment information is not required to be disseminated to key stakeholders. There also are no required uses of classroom assessment to support Level of development student learning. Overall, there is recognition among key stakeholders of In the Kyrgyz Republic, an informal system-level the importance of classroom assessment and the urgent document provides guidelines for classroom need to strengthen existing practices in order to better assessment. The availability of this document is support student learning. restricted. The one formal document in existence, does not apply to the entire system, but only to regions Suggested policy options: where classroom assessment projects have been piloted. 1. Make available system-wide resources for teachers to engage in better-quality classroom There are no system-wide resources for teachers to assessment as well as teacher training engage in classroom assessment activities. Although opportunities on effective classroom some manuals were developed under a World Bank- assessment practices. financed project, and subsequently distributed to all schools, these materials are nonetheless scarce. 2. Formalize, and make widely available, a system- level document that provides evidence-based There is an official curriculum or standards document guidelines for classroom assessment practices. that outlines what students are expected to learn, but the document is not specific, especially in identifying the required levels of student performance. There also are no system-level mechanisms to ensure that teachers develop the necessary skills and expertise in classroom assessment. Classroom assessment practices generally appear to be weak, and there are no systematic mechanisms in place to monitor their quality. While some monitoring has been done in regions where classroom assessment projects have been piloted, there is as yet no monitoring in place at the system level. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Examinations There is a clear understanding of what the examinations measure, and there is comprehensive material (such as questions likely to be on the test) available to prepare Level of development for the exams that is accessible to most students. While there are no up-to-date courses or workshops on the examinations available to teachers, teachers are In the Kyrgyz Republic, the examination system has involved in some examination-related tasks (for been operating on a regular basis under provisions example, administering and grading the tests). approved by the government in the Soviet era. The Ministry of Education and Science annually issues There is some technical documentation on the orders on the regulations for the final state examinations, but it is not in a formal report format. examinations of grade 9 and grade 11 graduates, and There also are limited systematic mechanisms in the university entrance examinations. place—such as item validity, and reliability checks—to better ensure the quality of the examinations. Final examinations are administered by teachers at schools. Essay topics for Kyrgyz, Russian, and Uzbek Inappropriate behavior surrounding the examination languages and literature are prepared by the Kyrgyz process is moderate and includes leakage of the content Academy of Education (KAE) and are distributed to schools of a paper prior to the examination; impersonation; by the Ministry of Education and Science two months copying from other candidates; using unauthorized before the examinations. KAE staff use a lottery system to materials, such as prepared answers and notes; select questions for the math examination from an existing collusion among candidates via mobile phones, passing database of questions (all of which have previously been of papers, or equivalent; and obtaining external made available to the public by KAE). The Center for assistance via, for example, the examination supervisor Educational Assessment and Teaching Methods (CEATM) or mobile phones. There are no mechanisms in place to administers the university entrance examination. monitor the consequences of the examinations. Some opportunities that prepare individuals for work on A small proportion of students (less than 10 percent) these examinations are offered in the country. Some are unable to take the examinations. For example, there KAE and CEATM staff members acquire additional skills are no testing accommodations (e.g., braille tests, in assessment by attending training courses and extended time) that would allow students with seminars under international projects. disabilities to take the school leaving exam. In addition, the cost of the university entrance exam and the While the secondary school leaving examination distance to the testing center is prohibitive for some receives consistent funding from the government, students. funding for the university entrance examination is irregular and provided by donors and through student fees. Funding covers design and peer review of tests, Suggested policy options: translation of test questions (into Kyrgyz or Russian), 1. Enhance the credibility of the examinations by replication of test materials, organization and making publicly available a high-quality technical administration of tests, processing and distribution of report on their design and conduct, and introducing test results, statistical analysis, and submission of systematic quality control mechanisms, such as findings to education authorities. Funding does not internal and external review or audit. cover research and development. 2. Monitor the quality and equity consequences of the While most stakeholders groups support the examinations by funding independent research on examinations, independent attempts have been made their impact or creating a permanent oversight to improve the examinations by some groups (primarily committee. donors). These efforts are generally welcomed by the leadership in charge of the examinations. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 National Large-Scale Assessment While the NLSAs are supposed to measure student performance against curriculum or learning standards, (NLSA) there are no mechanisms in place to ensure that they are actually measuring this. Additionally, what these NLSAs Level of development measure is generally, questioned by stakeholder groups. At the same time, there are some mechanisms in place The Kyrgyz Republic has been conducting NLSAs on an to help ensure the quality of the NLSAs. For example, irregular basis, and those that were conducted were pretesting takes place, as does some preliminary driven by donor agencies. The Monitoring Learning training of interviewers and staff conducting statistical Achievements assessment (MLA) of fourth grade analysis. Quality control of NLSAs is conducted using the students in in mathematics, grammar, and life skills was findings of psychometric analysis. conducted in 2001 and 2005, with UNICEF providing funding for the 2005 exercise. A National Sample Based There is some documentation about the technical Assessment (NSBA) of fourth and eighth grade students aspects of the NLSAs, such as test specifications that in mathematics, natural sciences, and reading provide the basis for the compilation of test questions, comprehension was conducted in 2007 and 2009 using but it is not in a formal report format. World Bank funding. There are no government policy documents regulating or authorizing NLSAs. While some NLSA results have not been disseminated in an effective Ministerial orders have been issued, these are not way. However, NLSA results are used by some policy documents. stakeholder groups in a way that is consistent with the purposes and technical characteristics of the There is irregular funding allocated to NLSAs. The assessments. For example, policy makers have used funding that has been provided (mainly by donors) has NLSA results to modify areas of the education system, covered the design and peer review of assessment including curriculum and student assessment. material, translation of test questions into Kyrgyz or Russian, replication of test materials, organization and Currently, there are no mechanisms in place to monitor administration of the assessments, processing and the consequences of NLSAs in the Kyrgyz Republic. distribution of assessment results, statistical analysis of assessment results, and submission of findings to Suggested policy options: education authorities. Funding has not been sufficient to cover research and development activities. 1. Institutionalize the NLSA program by creating a To date, the NLSA program has been managed by a policy framework, ensuring regular government temporary group of people accountable to the Ministry funding to cover core activities, and creating a of Education and Science. The NGO that conducted the permanent NLSA office/unit that would be NSBA was not appointed by the Government as a responsible for NLSA activities. permanent agency to perform NSBA activities, but instead was contracted by the World Bank to carry out 2. Build capacity to ensure the quality of future NSBA activities. In general, the people who have been NLSA activities by, for example, offering training contracted to carry out NLSA activities in the Kyrgyz opportunities to prepare individuals to work on Republic are not adequately trained to carry out these the NLSA. activities. The Kyrgyz Republic also does not currently offer opportunities that prepare individuals for work on NLSAs. Occasional courses or workshops on NLSA are made available to teachers; however, these are offered primarily through donor support. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 International Large-Scale Assessment contributed new knowledge on ILSA to the global evidence base. (ILSA) There is limited media coverage of ILSA results in the Kyrgyz Republic. Country-specific results and information are disseminated irregularly throughout the Level of development country, and products to provide feedback to schools and educators are only sometimes made available. PISA 2009 results were presented in a consolidated report, The Kyrgyz Republic has participated in PISA 2006 and and distributed to decision makers and schools across PISA 2009. The Kyrgyz Republic has not put in place a the country. policy that addresses participation in ILSAs and has not taken concrete steps to participate in an ILSA in the next five years. Participation in the 2006 and 2009 It is not clear whether and to what extent decisions rounds of PISA was financed under a World Bank based on ILSA results have had a positive impact on project. The financing included funding of core ILSA students' achievement levels in the Kyrgyz Republic. activities. Funding did not cover research and development activities. Suggested policy options: While a national coordinator and a team from the 1. Build capacity in the Kyrgyz Republic to carry Center for Educational Assessment and Teaching out ILSA activities by offering workshops on Methods (CEATM) carried out the ILSA activities, CEATM using international assessment databases, staff lacked the capacity needed to carry out the conducting university courses on the topic of assessment activities effectively. international assessments, and providing funding for attending international workshops Opportunities to learn about ILSA, including training in or training on international assessments. basic assessment techniques, are available only to the Kyrgyz Republic’s ILSA team members. The ILSA team 2. Build policymaker interest in, and support for, was able to attend some of the international workshops participation in future ILSAs by informing them and meetings for the 2006 and 2009 PISA exercises. and other key stakeholders (media, educators) of ILSA results and analysis, and their policy and practical implications. For both the 2006 and 2009 PISA exercises, the Kyrgyz Republic met all of the technical standards required to have its data presented in the main displays of the international report. However the country has not SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education system opportunities instruction grade/age system at to the next (or level(s), and to particular into the monitor trends in grade/age workforce) learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the system where the system basis (such as basis (such as allows for allows for every 3-5 years) every 3-5 years) repeats repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a few confined to one subject areas subject areas subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide institutional wide institutional support and ensure the capacity to support and capacity to support and capacity to support and quality of classroom ensure the quality of ensure the quality of ensure the quality of assessment practices. classroom assessment classroom assessment classroom assessment practices. practices. practices. CLASSROOM ASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and and a need to develop There is institutional institutional capacity and institutional capacity to capacity and some strong mechanisms to run the examination. The limited mechanisms to monitor it. The EXAMINATIONS examination typically is monitor it. The examination is of high of poor quality and is examination is of quality and is perceived perceived as unfair or acceptable quality and is as fair and free from corrupt. perceived as fair for corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA There is a stable NLSA place. NLSA in place and a in place. There is in place and institutional need to develop institutional capacity and capacity and strong institutional capacity to some limited mechanisms to monitor run the NLSA. mechanisms to monitor it. The NLSA is of high NATIONAL (OR SYSTEM- Assessment quality and it. The NLSA is of quality and its LEVEL) LARGE-SCALE impact are weak. moderate quality and its information is ASSESSMENT information is effectively used to disseminated, but not improve education. always used in effective ways. There is no history of Participation in an ILSA There is more or less There is stable participation in an ILSA has been initiated, but stable participation in an participation in an ILSA nor plans to participate there still is need to ILSA. There is and institutional capacity in one. develop institutional institutional capacity to to run the ILSA. The capacity to carry out the carry out the ILSA. The information from the INTERNATIONAL LARGE- ILSA. information from the ILSA is effectively used SCALE ASSESSMENT ILSA is disseminated, to improve education. but not always used in effective ways. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 4: Methodology for Assigning Development Levels 1. The country team or consultant collects information 5. The preliminary development level is validated using about the assessment system in the country. expert judgment in cooperation with the country team and The World Bank Task Team Leader. 2. Based on the collected information, a level of development and score is assigned to each dimension in For scores that allow a margin of discretion (i.e., to the rubrics: choose between two levels of development), a final decision has to be made based on expert judgment. For • Latent = 1 score point example, the aforementioned hypothetical country has • Emerging = 2 score points an ‘Enabling Context’ score of 2.33, corresponding to a • Established = 3 score points preliminary level of development of ‘Emerging or • Advanced = 4 score points Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the 3. The score for each quality driver is computed by country team chooses ‘Emerging’ as the most aggregating the scores for each of its constituent appropriate level. dimensions. For example: 6. Scores for certain key dimensions under ‘Enabling The quality driver, ‘Enabling Context,’ in the case of Context’ (in the case of EXAM, NLSA, and ILSA) and ILSA, has 3 dimensions on which a hypothetical country under ‘System Alignment’ (in the case of CLASS) were receives the following scores: Dimension A = 2 points; set as ceiling scores, i.e., the overall mean score for the Dimension B = 2 points; Dimension C = 3 points. The particular assessment type cannot be greater than the hypothetical country’s overall score for this quality score for these key dimensions. These key variables driver would be: (2+2+3)/3 = 2.33 include formal policy, regular funding, having a permanent assessment unit, and the quality of 4. A preliminary level of development is assigned to assessment practices. each quality driver. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Appendix 5: SABER-Student Assessment Rubrics for the Kyrgyz Republic This appendix provides the completed SABER-Student Assessment rubrics for each type of assessment activity in the Kyrgyz Republic. In each row of the rubric, the relevant selection is indicated by a thick border and an asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection (as indicated by a thick border and an asterisk). The explanation or justification text can be located in the “Development level rating justifications� section at the end of each rubric. If a row includes a superscript, but not a thick border and an asterisk, this means that insufficient information was available to determine the relevant selection in the row. 13 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 KYRGYZ REPUBLIC Classroom Assessment 14 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment There is no system-level document that There is an informal system-level There is a formal system-level document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. 1 assessment. classroom assessment. assessment. * This option does not apply to this This option does not apply to this The availability of the document is The document is widely available. 2 dimension. dimension. restricted. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with system learning goals There are no system-wide resources for There are scarce system-wide resources There are some system-wide resources There are a variety of system-wide 3 teachers for classroom assessment. for teachers for classroom assessment. for teachers for classroom assessment. resources available for teachers for classroom assessment. * There is no official curriculum or There is an official curriculum or There is an official curriculum or There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn or students are expected to learn, but the students are expected to learn and to to what level of performance. level of performance required is not what level of performance. * 4 clear. ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: Having effective human resources to carry out classroom assessment activities There are no system-level mechanisms This option does not apply to this There are some system-level There are a variety of system-level to ensure that teachers develop skills dimension. mechanisms to ensure that teachers mechanisms to ensure that teachers and expertise in classroom assessment. develop skills and expertise in classroom develop skills and expertise in classroom assessment. assessment. * 15 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis, and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are 5 from widespread weaknesses or there is known to be weak. known to be of moderate quality. known to be generally of high quality. no information available on classroom assessment practices. There are no mechanisms to monitor the * There are ad hoc mechanisms to monitor There are limited systematic mechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the 6 practices. practices. assessment practices. quality of classroom assessment * practices. ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required to be disseminated to some key required to be disseminated to all key 7 stakeholders. stakeholders. stakeholders. * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessment to support student learning. classroom assessment to support classroom assessment to support classroom assessment to support student learning. student learning, excluding its use as an student learning, including its use as an * input for external examination results. input for external examination results. 16 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Classroom Assessment: Development level rating justifications 1. The formal document that exists does not apply to the entire system. The existing documents only relate to pilot regions where classroom assessment has been piloted. 2. The availability of the informal system-level document that provides guidelines for classroom assessment activities is restricted. 3. There are no system-wide resources for teachers for classroom assessment Even after the provision of manuals developed under a World-Bank-financed project, which were distributed to all schools, these materials are very scarce. 4. The document is not specific, particularly in identifying the level of required performance. 5. It appears that classroom assessment practices are weak. 6. There is some monitoring done in pilot regions where classroom assessment has been introduced, but there is nothing at the system level. 7. There is no requirement to report the results of classroom assessment on a regular basis to anyone. 17 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 KYRGYZ REPUBLIC Examinations 18 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which assessment activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies No standardized examination has taken The standardized examination has been The examination is a stable program that This option does not apply to this 1 place. operating on an irregular basis. has been operating regularly. dimension * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this 2 authorizes the examination. document that authorizes the authorizes the examination. dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this 3 dimension. the public public. dimension. * This option does not apply to this This option does not apply to this The policy document addresses some The policy document addresses all key 4 dimension. dimension. key aspects of the examination. aspects of the examination. * ENABLING CONTEXT 2: Having strong leadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the the examination or are indifferent to it. examination. examination. examination. * There are no attempts to improve the This option does not apply to this There are independent attempts to There are coordinated attempts to examination by stakeholder groups. dimension. improve the examination by stakeholder improve the examination by stakeholder 5 groups. groups. * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this not welcomed by the leadership in dimension. generally welcomed by the leadership in dimension. charge of the examination charge of the examination. * (CONTINUED) 19 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this 6 examination. the examination. examination. dimension. This option does not apply to this Funding covers some core examination * Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. 7 processing or reporting. processing and reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development. dimension. development. * ENABLING CONTEXT 4: Having strong organizational structures The examination office does not exist or The examination office is newly The examination office is a stable This option does not apply to this 10 is newly established. established. organization. dimension. The examination office is not This option does not apply to this The examination office is accountable to * This option does not apply to this 8 accountable to an external board or dimension. an external board or agency. dimension. agency. * Examination results are not recognized Examination results are recognized by Examination results are recognized by Examination results are recognized by by any certification or selection system. certification or selection system in the one certification or selection system in two or more certification or selection 9 country. * another country. system in another country. The examination office does not have The examination office has some of the The examination office has all of the The examination office has state of the the required facilities to carry out the required facilities to carry out the required facilities to carry out the art facilities to carry out the examination. examination. examination. examination. * (CONTINUED) 20 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment 10 examination, issues are pervasive. effectively, with minimal issues. effectively, with no issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on 11 examination. examination. the examination. * 21 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning examinations with learning goals and opportunities to learn It is not clear what the examination This option does not apply to this There is a clear understanding of what This option does not apply to this measures. dimension. the examination measures. dimension. What the examination measures is This option does not apply to this * What is measured by the examination is This option does not apply to this questioned by some stakeholder groups. dimension. largely accepted by stakeholder groups. dimension. * Material to prepare for the examination There is some material to prepare for the There is comprehensive material to There is comprehensive material to is minimal and it is only accessible to examination that is accessible to some prepare for the examination that is prepare for the examination that is 12 very few students. students. accessible to most students. accessible to all students. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the examination There are no courses or workshops on There are no up-to-date courses or There are up-to-date voluntary courses There are up-to-date compulsory courses examinations available to teachers. workshops on examinations available to or workshops on examinations available or workshops on examinations for 13 teachers. to teachers. teachers. * Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in most 14 19 examination-related tasks. examination-related tasks. examination-related tasks. examination-related tasks. * 22 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the assessment meets quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring quality There is no technical report or other There is some documentation on the There is a comprehensive technical There is a comprehensive, high quality documentation. examination, but it is not in a formal report but with restricted circulation. technical report available to the general 15 report format. public. * There are no mechanisms in place to This option does not apply to this There are limited systematic mechanisms There are varied and systematic ensure the quality of the examination. dimension. in place to ensure the quality of the mechanisms in place to ensure the 16 examination. quality of the examination. * ASSESSMENT QUALITY 2: Ensuring fairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the 17 examination process is high. examination process is moderate. examination process is low. examination process is marginal. * The examination results lack credibility The examination results are credible for The examination results are credible for This option does not apply to this 18 for all stakeholder groups. some stakeholder groups. all stakeholder groups. dimension. * The majority of the students (over 50%) A significant proportion of students A small proportion of students (less than All students can take the examination; may not take the examination because of (10%-50%) may not take the examination 10%) may not take the examination there are no language, gender or other language, gender, or other equivalent because of language, gender, or other because of language, gender, or other equivalent barriers. barriers. equivalent barriers. equivalent barriers. 19 * (CONTINUED) 23 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Using examination information in a fair way Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all proper way by all stakeholder groups. stakeholder groups in a proper way. stakeholder groups in a proper way. stakeholder groups in a proper way. * 20 Student names and results are public. This option does not apply to this Students’ results are confidential. This option does not apply to this dimension. dimension. * ASSESSMENT QUALITY 4: Ensuring positive consequences of the examination There are no options for students who There are very limited options for There are some options for students who There is a variety of options for students do not perform well on the examination, students who do not perform well on the do not perform well on the who do not perform well on the 24 21 or students must leave the education examination. examination. examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination. examination. examination. * 24 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Examinations: Development level rating justifications 1. The MOES issues annual orders based on the regulations on final state examination of graduates and the procedure for transfer of students to the next grade. The examination system has been operating on a regular basis since the Soviet Union. 2. The procedure of student testing is established under the provisions approved by the Government of the Kyrgyz Republic. 3. The public can have access to the policy document. 4. The policy document does not address some key aspects of the examination, such as all administration procedures and examiner responsibilities. 5. Primarily by donors. 6. While the secondary school leaving examination exercise is funded consistently without donor aid, funding for the university entrance examination is irregular and is funded by donors and students. 7. Funding includes design and peer review of tests, translation of test questions (into Kyrgyz or Russian), replication of test materials, organization and administration of tests, processing and distribution of test results, statistical analysis and submission of findings to education authorities. 8. Accountable to the Ministry of Education. 9. Examination results are recognized by the certification or selection system in the Kyrgyz Republic. 10. Staff is not adequately trained to carry out examinations. 11. Some staff acquire skills in the assessment of students by attending training courses and seminars under various international projects. The test is designed at the CEATM office by teachers from schools and universities in Kyrgyzstan, who have attended special training and already have sufficient experience in the development of test questions. 12. Likely test questions are widely available to most students on the market. 13. Training is primarily done on the job. 14. Teachers may administer and grade tests, depending on the subject area. 15. There is some documentation on the examination, but it is not in a formal report format. 25 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 16. There are limited systematic mechanisms, such as determining item design, validity, and reliability, in place to ensure the quality of the examination. 17. Inappropriate behavior includes: leakage of the content of an examination paper or part of a paper prior to the examination; impersonation; copying from other candidates; using unauthorized materials such as prepared answers and notes; collusion among candidates via mobile phones, passing of paper, or equivalent; and obtaining external assistance via the supervisor, mobile phone etc. 18. Some stakeholders question school leaving examination and university entrance examination results. 19. For the university entrance examination, the cost of the examination and the distance to the testing center may be prohibitive. For the school leaving examination, students with disabilities are not provided for. 20. Results are not confidential. 21. There are some options for students who do not perform well on the examination. Students can repeat the grade, retake the examination, and/or may attend preparatory courses in order to prepare to retake the examination. 26 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 KYRGYZ REPUBLIC National (or System-Level) Large-Scale Assessment (NLSA) 27 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which NLSA activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the NLSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for NLSA No NLSA exercise has taken place. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this 1 irregular basis. been operating regularly. dimension. There is no policy document pertaining There is an informal or draft policy * There is a formal policy document that This option does not apply to this 2 to NLSA. document that authorizes the NLSA. authorizes the NLSA. dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. * There is no plan for NLSA activity. This option does not apply to this There is a general understanding that the There is a written NLSA plan for the dimension. NLSA will take place. coming years. * ENABLING CONTEXT 2: Having strong public engagement for NLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the NLSA. the NLSA or are indifferent to it. NLSA. NLSA. * (CONTINUED) 28 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for NLSA There is no funding allocated to the There is irregular funding allocated to There is regular funding allocated to the This option does not apply to this NLSA. the NLSA. NLSA. dimension. This option does not apply to this Funding covers some core NLSA * Funding covers all core NLSA activities: This option does not apply to this dimension. activities: design, administration, analysis design, administration, analysis and dimension. 3 and reporting. reporting. This option does not apply to this Funding does not cover research and * This option does not apply to this Funding covers research and dimension. development activities. dimension. development activities. * ENABLING CONTEXT 4: Having strong organizational structures for NLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this 4 team. group of people. institution or unit. dimension. * This option does not apply to this Political considerations regularly hamper Political considerations sometimes Political considerations never hamper dimension. technical considerations. hamper technical considerations. technical considerations. * This option does not apply to this The NLSA office is not accountable to a The NLSA office is accountable to a This option does not apply to this 5 dimension. clearly recognized body. clearly recognized body. dimension. * (CONTINUED) 29 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources for NLSA There is no staff allocated for running an The NLSA office is inadequately staffed The NLSA office is adequately staffed to The NLSA office is adequately staffed to 6 NLSA. to effectively carry out the assessment. carry out the NLSA effectively, with carry out the NLSA effectively, with no minimal issues. issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities to The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the opportunities to prepare individuals for NLSA. * NLSA. work on the NLSA. 30 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the NLSA is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against This option does not apply to this curriculum or learning standards. dimension. curriculum or learning standards. dimension. * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned What the NLSA measures is largely questioned by stakeholder groups. dimension. by some stakeholder groups. accepted by stakeholder groups. * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSA to ensure that it measures what it dimension. measures what it is supposed to intended to measure. is intended to measure. measure. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the NLSA There are no courses or workshops on There are occasional courses or There are some courses or workshops on There are widely available high quality 7 the NLSA. workshops on the NLSA. the NLSA offered on a regular basis. courses or workshops on the NLSA offered on a regular basis. * 31 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the NLSA meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA No options are offered to include all This option does not apply to this At least one option is offered to include Different options are offered to include groups of students in the NLSA. dimension. all groups of students in the NLSA. all groups of students in the NLSA. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in 8 ensure the quality of the NLSA. dimension. ensure the quality of the NLSA. place to ensure the quality of the NLSA. * There is no technical report or other There is some documentation about the There is a comprehensive technical There is a comprehensive, high quality documentation about the NLSA. technical aspects of the NLSA, but it is report but with restricted circulation. technical report available to the general 9 not in a formal report format. public. * ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA 10 NLSA results are not disseminated. NLSA results are poorly disseminated. NLSA results are disseminated in an This option does not apply to this 17 effective way. dimension. * NLSA information is not used or is used This option does not apply to this NLSA results are used by some NLSA information is used by all in ways inconsistent with the purposes dimension. stakeholder groups in a way that is stakeholder groups in a way that is or the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the 11 There are no mechanisms in place to This option does not apply to this assessment. There are some mechanisms in place to * assessment. There are a variety of mechanisms in monitor the consequences of the NLSA. dimension. monitor the consequences of the NLSA. place to monitor the consequences of the NLSA. * 32 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 National (or System-Level) Large Scale Assessment (NLSA): Development level rating justifications 1. The Kyrgyz Republic has been conducting NLSAs on an irregular basis and those that were conducted were driven by donor agencies (MLA with UNICEF funding, and two rounds of NSBA with WB funding). 2. There are no government policy documents (informal or formal) regulating/authorizing the NLSAs. Both the MLA & NSBA were conducted with donor funding and while some Ministerial orders have been issued, these were not policy documents. 3. Funding includes design and peer review of tests, translation of test questions (into Kyrgyz or Russian), replication of test materials, organization and administration of tests, processing and distribution of test results, statistical analysis and submission of findings to education authorities. 4. The NGO that was conducting the NLSA was contracted by the World Bank. It was not appointed by the Government as a permanent agency for this work. Additionally, there are other institutions that are coming into the picture and may be playing the role of the 'NLSA office'. 5. To the Ministry of Education. 6. Staff is not adequately trained to carry out assessment activities. 7. There are occasional courses or workshops on the NLSA, offered primarily through donor support. 8. There are some mechanisms in place to ensure the quality of the NLSA. Pretesting takes place; there has been some preliminary training of interviewers and staff conducting statistical analysis; testing quality control was conducted using the findings of psychometric analysis. 9. There are specifications that provide the basis for compilation of test questions. 10. Results have not been disseminated in an effective way. 11. Policy makers have used NLSA results to modify areas of the education system, including the areas of curriculum and student assessment. 33 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 KYRGYZ REPUBLIC International Large-Scale Assessment (ILSA) 34 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which ILSA takes place in a country or system and the extent to which that framework is conducive to, or supportive of, ILSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for ILSA The country/system has not participated This option does not apply to this The country/system has participated in The country/system has participated in 1 in an ILSA in the last 10 years. dimension. at least one ILSA in the last 10 years. two or more ILSA in the last 10 years. * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concrete steps to participate in an ILSA in dimension. steps to participate in at least one ILSA in dimension. 2 the next 5 years. the next 5 years. * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this 3 addresses participation in ILSA. document that addresses participation in addresses participation in ILSA. dimension. ILSA. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. dimension. ENABLING CONTEXT 2: Having regular funding for ILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at There is regular funding approved by law, 4 ILSA. donors. discretion. decree or norm. * This option does not apply to this Funding covers some core activities of Funding covers all core activities of the This option does not apply to this 5 dimension. the ILSA. ILSA. dimension. Funding does not cover research and This option does not apply to this This option does not apply to this * Funding covers research and 6 development activities. dimension. dimension. development activities. * (CONTINUED) 35 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective human resources for ILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. 7 activities. activities. activities. * This option does not apply to this The national/system coordinator or The national/system coordinator is fluent This option does not apply to this dimension. other designated team member may not in the language of the assessment. dimension. be fluent in the language of the assessment. * This option does not apply to this The ILSA office is inadequately staffed or The ILSA office is adequately staffed or The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, 8 effectively. with minimal issues. with no issues. * 36 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 SYSTEM ALIGNMENT Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Providing opportunities to learn about ILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this international workshops or meetings. international workshops or meetings. workshops or meetings. dimension. The country/system offers no This option does not apply to this * The country/system offers some The country/system offers a wide range opportunities to learn about ILSA. dimension. opportunities to learn about ILSA. of opportunities to learn about ILSA. * This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition 9 team members only. to the country's/system's ILSA team * members. 37 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 ASSESSMENT QUALITY Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of ILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical The country/system met all technical published. standards to have its data presented standards required to have its data standards required to have its data beneath the main display of the presented in the main displays of the presented in the main displays of the The country/system has not contributed international report or in an annex. This option does not apply to this international report. This option does not apply to this * international report. The country/system has contributed new new knowledge on ILSA. dimension. dimension. knowledge on ILSA. * ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA If any, country/system-specific results Country/system-specific results and Country/system-specific results and Country/system-specific results and and information are not disseminated in information are disseminated irregularly information are regularly disseminated in information are regularly and widely 10 the country/system. in the country/system. the country/system. disseminated in the country/system. Products to provide feedback to schools This option does not apply to this * Products to provide feedback to schools Products to provide feedback to schools and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are 11 not made available. sometimes made available. systematically made available. * There is no media coverage of the ILSA There is limited media coverage of the There is some media coverage of the There is wide media coverage of the ILSA 12 results. ILSA results. ILSA results. results. * If any, country/system-specific results Results from the ILSA are used in a Results from the ILSA are used in some Results from the ILSA are used in a and information from the ILSA are not limited way to inform decision making in ways to inform decision making in the variety of ways to inform decision used to inform decision making in the the country/system. country/system. making in the country/system. 13 country/system. * It is not clear that decisions based on This option does not apply to this This option does not apply to this Decisions based on the ILSA results have ILSA results have had a positive impact dimension. dimension. had a positive impact on students' 14 on students' achievement levels. achievement levels. * 38 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 International Large Scale Assessment (ILSA): Development level rating justifications 1. The Kyrgyz Republic has participated twice in PISA (2006 & 2009). 2. The Kyrgyz Republic has not taken concrete steps to participate in an ILSA in the next five years. 3. Participation in the two rounds of PISA was done through financing under a World Bank project. The Kyrgyz Republic itself, however, has not put in place a policy that addresses participation in ILSAs. 4. Both rounds of PISA were funded by a World Bank project. 5. Funding for the Kyrgyz Republic's participation in PISA was provided by the World Bank and covered all core activities. 6. Funding for the Kyrgyz Republic's participation in PISA was provided by the World Bank, and it did not cover research and development. 7. There was a team to carry out the ILSA activities. 8. Staff working on the ILSA lack capacity to carry out the ILSA effectively. 9. Opportunities to learn about ILSA, including training in the basic testing techniques, are available to the country's/system's ILSA team members only. 10. Results and information were presented in a consolidated report, and distributed across the country in order to reach decision makers and schools. Findings of the survey and sample tasks are publicly available and widely discussed across the country. 11. The consolidated report with results was prepared as a complete paper with attachments. It was distributed across the country in order to reach decision makers and all schools. Findings of the survey and sample tasks were publicly available and widely discussed across the country. 12. There is limited media coverage of the ILSA results in the Kyrgyz Republic. 13. ILSA results have not been used to inform decision making in the Kyrgyz Republic. 14. It is not clear whether and to what extent decisions based on ILSA results have had a positive impact on students' achievement levels. 39 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 Acknowledgements This report was prepared by The World Bank SABER- Student Assessment team in collaboration with Dingyong Hou, World Bank Senior Education Specialist and Task Team Leader for education projects in the Kyrgyz Republic, and Aisuluu Bedelbayeva, World Bank Education Specialist. References Clarke, M. 2012. “What Matters Most for Student United Nations Children’s Fund (UNICEF). 2010. Assessment Systems: A Framework Paper.� “Country profile 2010: Education in Kyrgyzstan.� Data READ/SABER Working Paper Series. Washington, DC: retrieved from http://www.unicef.org/ceecis/ World Bank. Tajikistan_2010.pdf on March 12, 2013. Kyrgyz Republic. 2009. “Self-diagnosis in the Area of United Nations Educational, Scientific, and Cultural Student Assessment in Kyrgyzstan.� Report prepared Organization (UNESCO) - Institute for Statistics. Kyrgyz under the Russia Aid for Development Trust Fund Republic Country Indicator Data. Montreal, QC: program. UNESCO. Data retrieved from http://stats.uis. unesco.org/unesco on March 12, 2013 Organisation for Economic Co-operation and Development (OECD). 2010. “PISA 2009 at a Glance.� World Bank. World Bank Development Indicators. Paris: OECD. Kyrgyz Republic Country Indicator Data. Washington, DC: World Bank. Data retrieved from ———. 2007. “PISA 2006: Science Competencies for http://databank.worldbank.org/data on March 12, Tomorrow's World: Volume 1: Analysis.� Paris: OECD. 2013. 40 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS THE KYRGYZ REPUBLIC ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2009 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. THE WORLD BANK 41 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS