WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 West Bank and Gaza SABER Country Report STUDENT ASSESSMENT 2013 Key Policy Areas for Student Assessment Status 1. Classroom Assessment The Basis of Success, Completion, and Repetition for Grades 1 to 12 is an informal document authorized by the Ministry of Education in 2010-2011 that provides guidelines for classroom assessment. Some system-level mechanisms, such as pre- and in-service teacher training opportunities, are also in place to ensure that teachers develop skills and expertise in classroom assessment. Despite this, classroom assessment practices are generally considered to be weak because they tend to provide little useful feedback to students, and are used more as administrative tools than as a pedagogical resource. Although classroom assessment information is required to be disseminated to a number of key stakeholders, limited systematic mechanisms are in place to monitor the quality of classroom assessment activities. For example, there is no external moderation system that reviews the difficulty of classroom assessment activities. 2. Examinations The Tawjihi is an examination administered to grade 12 students primarily for the purposes of student certification for secondary school completion and student selection to a higher education institution. While the examination has been administered since 1960, most stakeholder groups do not strongly support it, and efforts to improve it are not welcomed by the leadership in charge of the examination. While the examination office—the General Directorate for Measurement, Evaluation, and Examination—employs permanent and full-time staff, staff numbers are insufficient to fully meet the needs of the examination. 3. National Large-Scale Assessment (NLSA) Since 2008, the National Assessment has been administered every two years, assessing representative samples of students in grades 4 and 10 in Arabic, Mathematics, and the Sciences. While the Ministry of Education's Strategic Plan for Educational Development (2008-2012) refers to the activities of the National Assessment, the Ministry of Education has not yet devised a comprehensive strategy for educational assessment. Although the NLSA office, the Assessment and Evaluation Department of the Ministry of Education, is a permanent unit created for running the National Assessment, it is inadequately staffed to effectively carry out the NLSA activities. 4. International Large-Scale Assessment (ILSA) The West Bank and Gaza has participated in a number of ILSAs in the last 10 years, including TIMSS (2003, 2007, and 2011) and LAMP (2008). In addition, the Ministry of Education has agreed to participate in TIMSS 2015. Funding for ILSA activities is provided by loans or external donors. A national coordinator and team, who have the relevant experience and training, are responsible for carrying out the ILSA. However, the ILSA office is not sufficiently staffed. WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Introduction SABER-Student Assessment methodology The West Bank and Gaza has focused on increasing student learning outcomes by improving the quality of The SABER-Student Assessment framework is built on education in the country. An effective student the available evidence base for what an effective assessment system is an important component of efforts assessment system looks like. The framework provides to improve education quality and learning outcomes as it guidance on how countries can build more effective provides the necessary information to meet student assessment systems. The framework is stakeholders’ decision-making needs. In order to gain a structured around two main dimensions of assessment better understanding of the strengths and weaknesses of systems: the types/purposes of assessment activities and its existing assessment system, the West Bank and Gaza the quality of those activities. decided to benchmark this system using standardized tools developed under the World Bank’s Systems Assessment types and purposes Approach for Better Education Results (SABER) program. SABER is an evidence-based program to help countries Assessment systems tend to be comprised of three main systematically examine and strengthen the performance types of assessment activities, each of which serves a of different aspects of their education systems. different purpose and addresses different information needs. These three main types are: classroom assessment, examinations, and large-scale, system level What is SABER-Student Assessment? assessments. Classroom assessment provides real-time information to SABER-Student Assessment is a component of the SABER support ongoing teaching and learning in individual program that focuses specifically on benchmarking classrooms. Classroom assessments use a variety of student assessment policies and systems. The goal of formats, including observation, questioning, and paper- SABER-Student Assessment is to promote stronger and-pencil tests, to evaluate student learning, generally assessment systems that contribute to improved on a daily basis. education quality and learning for all. Examinations provide a basis for selecting or certifying National governments and international agencies are students as they move from one level of the education increasingly recognizing the key role that assessment of system to the next (or into the workforce). All eligible student learning plays in an effective education system. students are tested on an annual basis (or more often if The importance of assessment is linked to its role in: the system allows for repeat testing). Examinations cover the main subject areas in the curriculum and usually (i) providing information on levels of student involve essays and multiple-choice questions. learning and achievement in the system; (ii) monitoring trends in education quality over time; Large-scale, system-level assessments provide feedback on the overall performance of the education system at (iii) supporting educators and students with real- particular grades or age levels. These assessments time information to improve teaching and typically cover a few subjects on a regular basis (such as learning; and every 3 to 5 years), are often sample-based, and use (iv) holding stakeholders accountable for results. multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities. 2 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or The indicators are identified based on a combination of policy framework for assessment activities; institutional criteria, including: and organizational structures for designing, carrying out, or using results from the assessment; the availability of sufficient and stable sources of funding; and the x professional standards for assessment; presence of trained assessment staff. x empirical research on the characteristics of effective assessment systems, including analysis of the System alignment refers to the extent to which the characteristics that differentiate between the assessment is aligned with the rest of the education assessment systems of low- versus high-performing system. This includes the degree of congruence between nations; and assessment activities and system learning goals, x theory — that is, general consensus among standards, curriculum, and pre- and in-service teacher experts that it contributes to effective training. assessment. Assessment quality refers to the psychometric quality of Levels of development the instruments, processes, and procedures for the assessment activity. It covers such issues as design and implementation of assessment activities, analysis and The World Bank has developed a set of standardized interpretation of student responses to those activities, questionnaires and rubrics for collecting and and the appropriateness of how assessment results are evaluating data on the three assessment types and reported and used. related quality drivers. Crossing the quality drivers with the different The questionnaires are used to collect data on the assessment types/purposes provides the framework and characteristics of the assessment system in a particular broad indicator areas shown in Table 1. This framework country. The information from the questionnaires is then is a starting point for identifying indicators that can be applied to the rubrics in order to judge the development used to review assessment systems and plan for their level of the country’s assessment system in different improvement. areas. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to 3 WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 provide a country with some sense of the development the rubric displays four development levels—Latent, level of its assessment activities compared to best or Emerging, Established, recommended practice in each area. For each indicator, and Advanced. These levels are artificially constructed Education in the West Bank and Gaza categories chosen to represent key stages on the underlying continuum for each indicator. Each level is The West Bank and Gaza are occupied Palestinian accompanied by a description of what performance on territories in the Middle East and North Africa region the indicator looks like at that level. and are administered by the Palestinian Authority. Due in part to lower levels of donor aid, GDP growth x Latent is the lowest level of performance; it declined from an average of 11 percent in 2010 and represents absence of, or deviation from, the 2011 to 6 percent in 2012. desired attribute. General education in the West Bank and Gaza consists x Emerging is the next level; it represents partial of primary education (grades 1-10) and secondary presence of the attribute. education (grades 11-12). Since it was first established x Established represents the acceptable minimum in 1994, the Ministry of Education and Higher standard. Education (MoEHE) has expanded the education x Advanced represents the ideal or current best system through a number of activities, including practice. support for school construction, curriculum development, and recruitment and training for teachers. Despite challenges due to persistent conflict, A summary of the development levels for each net enrolment rates at the primary and secondary level assessment type is presented in Appendix 3. were 87 percent and 81 percent respectively in 2011. In reality, assessment systems are likely to be at In addition, in 2011, 91 percent of children completed different levels of development in different areas. For a full course of primary school. example, a system may be Established in the area of examinations, but Emerging in the area of large- The West Bank and Gaza developed its first five-year scale, system-level assessment, and vice versa. While Education Development Strategic Plan (EDSP) for the intuition suggests that it is probably better to be period 2000-2005. However, in October 2000, the further along in as many areas as possible, the second Intifada or “uprising” started, delaying most evidence is unclear as to whether it is necessary to activities planned under the EDSP. To address the be functioning at Advanced levels in all areas. continuing needs of the education sector and the Therefore, one might view the Established level as a effects of the second Intifada, such as substantial desirable minimum outcome to achieve in all areas, but damage to school infrastructure, the government only aspire beyond that in those areas that most launched the second EDSP for the years 2008-2012. contribute to the national vision or priorities for This plan focused on three key goals—increasing education. In line with these considerations, the access, quality, and management capacity. With ratings generated by the rubrics are not meant to be respect to access, the plan set out to reduce the additive across assessment types (that is, they are not number of double-shift classrooms; in order to address meant to be added to create an overall rating for an quality, it prioritized the revision of teachers’ guides assessment system; they are only meant to produce an and curricula; and with regard to management overall rating for each assessment type). The capacity, the plan targeted improvements in methodology for assigning development levels is computerized databases and financial management summarized in Appendix 4. systems at the MoEHE. 4 WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Detailed information was collected on the West Bank teachers and students in schools. The following and Gaza’s student assessment system using the sections discuss the findings by each assessment type, SABER-Student Assessment questionnaires and accompanied by suggested policy options. The rubrics. It is important to remember that these tools suggested policy options were determined in primarily focus on benchmarking a country’s policies collaboration with key local stakeholders based on the and arrangements for assessment activities at the West Bank and Gaza’s immediate interests and needs. system or macro level. Additional data would need to Detailed, completed rubrics for each assessment type be collected to determine actual, on-the-ground are provided in Appendix 5. practices in the West Bank and Gaza, particularly by 5 WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Classroom Assessment the quality of classroom assessment activities and how to improve them. Level of development Classroom assessment information is required to be disseminated to a number of key stakeholders. Schools and teachers are required to report on an individual student's performance to school districts, parents, and In the West Bank and Gaza, an informal document, the students. However, classroom assessment information is Basis of Success, Completion, and Repetition for Grades 1 not reported centrally to the Ministry of Education. to 12, authorized by the Ministry of Education, provides guidelines for classroom assessment. In addition, there There are adequate required uses of classroom are some resources available to teachers throughout the assessment to support student learning. Specifically, system to help them engage in classroom assessment classroom assessment activities are supposed to be used activities. For example, textbooks and workbooks, as well to diagnose student learning issues, provide feedback to as scoring criteria and rubrics for grading students' work, students on their learning, and inform parents about are available to teachers for their classroom assessment their child's learning. Although there is no official activities. document covering these issues, teachers' evaluation forms completed by education supervisors and principals Some system-level mechanisms are in place to ensure emphasize the importance of adequately using that teachers develop skills and expertise in classroom classroom assessment to support student learning. assessment. Pre-service teacher training on educational Supervisors also ask teachers to provide a detailed evaluation and assessment is available to all students in analysis of their students' examination results in a faculties of education at national universities. In student report card. addition, all teachers in primary and secondary schools benefit from in-service teacher training on educational Suggested policy options evaluation. Teachers also have opportunities to participate in conferences and workshops, and school 1. Develop a strategy that includes a framework for the inspection and teacher supervision includes a design and use of classroom assessment activities in the component on classroom assessment. West Bank and Gaza in order to build momentum around the importance of classroom assessment. Include in the Nonetheless, classroom assessment practices are strategy a plan to build teachers’ capacity to carry out considered to be weak in several key ways. For example, classroom assessment activities. For example, require classroom assessment activities tend to overly rely on that all teachers learn classroom assessment skills as part multiple-choice, selection-type questions and to focus on of their pre-service training and identify opportunities recall of information at the expense of more for in-service teachers to participate in developing or sophisticated cognitive skills. It is also common for scoring items for large-scale assessments or exams. parents to be poorly informed about students' grades 2. Improve the quality of classroom assessment activities and for classroom assessment activities to provide little by introducing system-level mechanisms to monitor useful feedback to students. classroom assessment practices. Such mechanisms could include the use of observers to evaluate whether Limited mechanisms are in place to systematically classroom assessment practices are in line with their monitor the quality of classroom assessment activities. intended purposes and uses. Additionally, introduce an For example, classroom assessment is a required external moderation system that reviews the difficulty of component of a teacher's performance evaluation. classroom assessment activities and the appropriateness However, there is no external moderation system that of scoring criteria. reviews the difficulty of classroom assessment activities, and government funding is not available for research on 6 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations teachers, and teachers are involved in very few examination-related tasks. Level of development There are limited systematic mechanisms in place to ensure the quality of the examination. For example, while there are internal reviews or observers, there are no external reviews or observers, and external The Tawjihi has been administered regularly since its certifications or audits do not take place. introduction in 1960, when the West Bank and Gaza was part of Jordan. There is a formal policy document, There is some dissatisfaction with the examination. Instructions for the General Secondary Education While policy makers and universities support the Examination Certificate, which authorizes the examination (policy makers strongly so), educators, examination. students, parents, media, think tanks, and non governmental organizations (NGOs) do not display Regular funding for the examination is provided through strong support for it. There is an ongoing discussion a government allocation and student fees. Funding about the examination's difficulty level, as there is a covers all core examination activities: design, belief that the examination questions are too administration, data processing, and reporting. challenging. There are also objections to the paper-and- Additionally, funding covers stationery for test scoring pencil format. Recommended alternatives include the and transportation for test developers, test addition of performance tasks from a student portfolio administrators, and scoring teams. However, funding when calculating the mean scores for students. Despite does not cover research and development activities. these concerns, efforts to improve the examination are not welcomed by the leadership in charge of the The General Directorate for Measurement, Evaluation, examination. and Examination, an office within the Ministry of Education, has had primary responsibility for running the Some mechanisms are in place to monitor the examination since the 1960s. Prior to 1967, the Ministry consequences of the examination. For example, field was under the Hashemite Kingdom of Jordan and follow-ups are undertaken by teachers and researchers therefore examination registers before 1967 are kept by in high- or low-performing schools. However, these field the Jordanian Ministry of Education. Since 1968, follow-ups are not organized or based on a clear examination records and all related documents are kept methodology. by the Ministry of Education in the West Bank and Gaza. Suggested policy options The General Directorate for Measurement, Evaluation, and Examination has some of the required facilities to 1. Provide teachers with opportunities to learn about the carry out the examination. For example, there are Tawjihi by making available up-to-date courses or computers for all technical staff, secure storage facilities, workshops on the examination. Include teachers in a and adequate communication tools. Although there is a variety of examination-related tasks, such as selecting or permanent and full-time staff, it is insufficient to meet creating the examination scoring guides and the needs of the examination. administering the examination. A wide range of opportunities that prepare individuals 2. Introduce various systematic mechanisms to ensure for work on the examination are available in the West the quality of the examination, including conducting pilot Bank and Gaza, including university graduate programs or field testing, and incorporating internal and external and courses specifically focused on educational reviews or observers. measurement and evaluation, and internships in the 3. Ensure that the examination results are used in a fair examination office. However, there are no up-to-date way by developing guides on the appropriate and courses or workshops on the examination available to inappropriate uses of examination results and making 7 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 them available to key stakeholder groups, particularly to parents, educators, and the media. The guides should include discussion of the consequences of inappropriate uses of examination results. 4. Introduce a permanent oversight committee, expert review groups, regular focus groups or surveys of key stakeholders to strengthen oversight of the examination and subsequent reviews of its impact. 5. Encourage researchers at higher education institutions to analyze the examination results to inform policy changes in such areas as teacher training. This could be through the provision of funding for research and analysis. 8 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National Large-Scale Assessment (NLSA) and evaluation; as well as internships or short-term employment in the large-scale assessment office. Level of development Most teachers have access to online courses to learn about the NLSA. The Assessment and Evaluation Department also disseminates an introductory leaflet to all directorates of education so that they can distribute them to all schools in the national sample. Since 2008, the National Assessment has been administered every two years as a way to collect system- There are regular, internal and also ad hoc reviews of the level information on the performance of grade 4 and alignment between the assessment instrument and what grade 10 students in Arabic, Mathematics, and Science. it is supposed to measure. Experts from the National The National Assessment began in 1998 and initially Curricula Center, education supervision teams, and focused on Arabic and Mathematics performance in education assessment teams take part in the working grade 6. A representative sample of students from groups that develop the instruments for the NLSA, government, United Nations Relief and Works Agency ensuring that the assessment tools are credible. (UNRWA), and private schools in the West Bank and Gaza take part in the assessment exercise. Although the General findings and results from the NLSA have been Ministry of Education's plans, such as the Strategic Plan published in reports that are made available to the for Educational Development (2008-2012), refer to the general public. Additional reports are made available at National Assessment activities, the Ministry of Education the school level (but not to the general public). has not put in place a comprehensive strategy for Workshops and presentations on general findings and educational assessment. results from the NLSA are also held for key stakeholders. However, detailed results have only been published for The Ministry of Education’s development activities are the 2008 NLSA administration, and not for the 2010 and funded through a joint financing program under the 2012 administrations. Ministry of Finance, with support from donors such as Finland and Ireland. Since 2008, a budget for National Some mechanisms, such as regular focus groups or Assessment activities has been allocated from this joint surveys of key stakeholders, are in place to monitor the financing program. Funding covers some core NLSA consequences of the NLSA in terms of its impact on policy activities, including assessment administration and data dialog, teaching, and learning. analysis, as well as the printing of test papers, correction of tests, data entry, and the printing of reports that document the results. Although assessment design is not Suggested policy options included under this funding, it is carried out free of charge by a specialized team within the Ministry of Education. 1. Develop a formal, system-level document that specifies the purposes and uses of the NLSA and make it The Ministry of Education’s Assessment and Evaluation available to all key stakeholders including parents. Department is a permanent unit created for running the 2. Introduce dedicated and predictable funding for key assessment. While there is a permanent and full-time NLSA activities, including for assessment design and data staff to carry out the NLSA, it is insufficient since staff are reporting. responsible for other evaluation activities as well. There are some opportunities to prepare individuals for work 3. Make available targeted training opportunities (both on the NLSA. Opportunities include university courses, local and international in nature) to central NLSA staff on some of which are considered part of pre-service teacher key NLSA topics; identify representatives in the district training; funding for attending international programs, offices who will collaborate with the central NLSA staff courses, and workshops on educational measurement on NLSA activities. Develop and make regularly available 9 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 to teachers high quality courses or workshops on the NLSA. 4. Enhance the quality of the NLSA by introducing options to include all student groups in the assessment exercise. Options could include providing accommodations or alternative assessments for students with disabilities and developing plans to ensure that the assessment is administered to students in hard-to-reach areas. 5. Develop a plan to effectively use the results of the NLSA to inform decision making in the West Bank and Gaza. For example, curriculum experts from higher education institutions and specialists from the Ministry of Education could be convened in order to make recommendations for national curriculum guidelines based on NLSA results. 10 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large-Scale Assessment (ILSA) published in refereed reports, the first being the IEA initiative entitled Educational Issues in the Middle East and North Africa Region, Outcomes of the IEA Arab Level of development Region Training Seminar Series 2006/2007. Country-specific results and information from the most recent international assessment were disseminated in The West Bank and Gaza has participated in multiple the West Bank and Gaza. For example, copies of the ILSAs in the last 10 years, including the International international report were distributed to key Association for the Evaluation of Educational stakeholders, and results received coverage on the Achievement's (IEA) Trends in International television, radio, and newspapers. Products providing Mathematics and Science Study (TIMSS) in 2003, 2007, feedback on the results to schools, educators, and and 2011. The West Bank and Gaza also participated in Science and Mathematics curricula developers were also the Literacy Assessment and Monitoring Programme made available. (LAMP), developed by the UNESCO Institute for Statistics (UIS), in 2008. The Ministry of Education (MoE) has ILSA results have been used in tracking the impact of agreed to participate in TIMSS in 2015, which was reforms on student achievement levels and informing included in the five-year education development plan for curriculum improvement, teacher training programs, 2008-2012. and other assessment activities in the country. For example, supplemental material on mathematical topics Funding for participation in ILSAs is sourced from loans covered by TIMSS, but not included in the national or external donors and covers all core activities including curriculum, was developed and disseminated to all international participation fees, implementation of the schools. In addition, decisions based on the ILSA results assessment exercise in the country, processing and have had a positive impact on students' achievement analysis of data collected from implementation of the levels, as evidenced by the improvement in Mathematics assessment exercise, and reporting and disseminating scores on the National Assessment between 2010 and the assessment results in the country. 2012. A national coordinator and team are responsible for Suggested policy options carrying out the ILSA activities. The team and the national coordinator have previous experience working 1. Develop and make publicly available a formal policy on international assessments, have attended some document that addresses the participation of the West international meetings related to the international Bank and Gaza in these assessments to institutionalize assessment, and have the necessary training and ILSA activity. experience to carry out the required assessment activities. However, the team is not sufficiently staffed, 2. Introduce regular and predictable government funding as it comprises only seven part-time members who are for ILSA activities. responsible for providing comprehensive follow-up on all activities related to the ILSA. 3. Expand the available opportunities to learn about international assessments by, for example, collaborating with higher education institutions in the country to offer The West Bank and Gaza has met all technical standards courses or workshops on international assessment. required to have its data presented in the main displays of the international reports. It also has contributed to the 4. Convene key stakeholders in order to develop a plan global knowledge base on international assessments by for using ILSA results to inform resource allocation generating new knowledge and making it available decision making in the West Bank and Gaza’s education through publications. For example, research papers were system. 11 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education opportunities instruction grade/age system at system to the level(s), and to particular next (or into the monitor trends grade/age workforce) in learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the where the basis (such as basis (such as system allows system allows every 3-5 years) every 3-5 years) for repeats for repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a confined to one subject areas subject areas few subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques 12 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses 13 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide wide institutional support and ensure the capacity to support and institutional capacity to capacity to support and quality of classroom ensure the quality of support and ensure the ensure the quality of assessment practices. classroom assessment quality of classroom classroom assessment CLASSROOM practices. assessment practices. practices. ASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and a need to develop There is institutional and institutional institutional capacity to capacity and some capacity and strong run the examination. limited mechanisms to mechanisms to monitor EXAMINATIONS The examination monitor it. The it. The examination is of typically is of poor examination is of high quality and is quality and is perceived acceptable quality and is perceived as fair and as unfair or corrupt. perceived as fair for free from corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA in There is a stable NLSA in place. NLSA in place and a place. There is place and institutional need to develop institutional capacity capacity and strong institutional capacity to and some limited mechanisms to monitor NATIONAL (OR SYSTEM- run the NLSA. mechanisms to monitor it. The NLSA is of high LEVEL) LARGE-SCALE Assessment quality and it. The NLSA is of quality and its ASSESSMENT impact are weak. moderate quality and its information is information is effectively used to disseminated, but not improve education. always used in effective ways. There is neither a Participation in an ILSA There is more or less There is stable history of participation has been initiated, but stable participation in an participation in an ILSA in an ILSA nor plans to there still is need to ILSA. There is and institutional participate in one. develop institutional institutional capacity to capacity to run the ILSA. INTERNATIONAL capacity to carry out the carry out the ILSA. The The information from LARGE-SCALE ILSA. information from the the ILSA is effectively ASSESSMENT ILSA is disseminated, but used to improve not always used in education. effective ways. 14 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 4: Methodology for Assigning Development Levels 6. Scores for certain key dimensions under ‘Enabling Context’ (in the case of EXAM, NLSA, and ILSA) and under 1. The country team or consultant collects information ‘System Alignment’ (in the case of CLASS) were set as about the assessment system in the country. ceiling scores, i.e., the overall mean score for the particular assessment type cannot be greater than the score for these key dimensions. These key variables 2. Based on the collected information, a level of include formal policy, regular funding, having a development and score is assigned to each dimension in permanent assessment unit, and the quality of the rubrics: assessment practices. x Latent = 1 score point x Emerging = 2 score points x Established = 3 score points x Advanced = 4 score points 3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions. For example: The quality driver, ‘Enabling Context,’ in the case of ILSA, has 3 dimensions on which a hypothetical country receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypothetical country’s overall score for this quality driver would be: (2+2+3)/3 = 2.33 4. A preliminary level of development is assigned to each quality driver. 5. The preliminary development level is validated using expert judgment in cooperation with the country team and The World Bank Task Team Leader. For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriate level. 15 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 5: SABER-Student Assessment Rubrics for West Bank and Gaza This appendix provides the completed SABER-Student Assessment rubrics for each type of assessment activity in the West Bank and Gaza. In each row of the rubric, the relevant selection is indicated by a thick border and asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection. The related text can be found in the “Development-level rating justifications” section at the end of each rubric. If a row includes a superscript, but not a thick border and asterisk, this means that insufficient information was available to determine the relevant selection in the row. WEST BANK and GAZA Classroom Assessment 16 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment There is no system-level document that There is an informal system-level There is a formal system-level document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. assessment. classroom assessment. 1 assessment. * This option does not apply to this This option does not apply to this The availability of the document is The document is widely available. 2 dimension. dimension. restricted. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with system learning goals There are no system-wide resources for There are scarce system-wide resources There are some system-wide resources There are a variety of system-wide teachers for classroom assessment. for teachers for classroom assessment. for teachers for classroom assessment. 3 resources available for teachers for classroom assessment. * There is no official curriculum or There is an official curriculum or There is an official curriculum or There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn or to students are expected to learn, but the students are expected to learn and to what level of performance. level of performance required is not clear. what level of performance. 4 * ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: Having effective human resources to carry out classroom assessment activities There are no system-level mechanisms to This option does not apply to this There are some system-level mechanisms There are a variety of system-level ensure that teachers develop skills and dimension. to ensure that teachers develop skills and mechanisms to ensure that teachers expertise in classroom assessment. expertise in classroom assessment. 5 develop skills and expertise in classroom assessment. * 17 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis, and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are from widespread weaknesses or there is known to be weak. 6 known to be of moderate quality. known to be generally of high quality. no information available on classroom assessment practices. There are no mechanisms to monitor the * There are ad hoc mechanisms to monitor There are limited systematic mechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the practices. practices. assessment practices. 7 quality of classroom assessment practices. * ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required to be disseminated to some key required to be disseminated to all key stakeholders. stakeholders. stakeholders. 8 * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessment to support student learning. classroom assessment to support student classroom assessment to support student classroom assessment to support student learning. learning, excluding its use as an input for learning, including its use as an input for external examination results. external examination results. 9 * 18 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Classroom Assessment: Development-level rating justifications 1. An informal system-level document, the Basis of Success, Completion, and Repetition for Grades 1 to 12, authorized by the Ministry of Education in 2010-2011, provides guidelines for classroom assessment. 2. The Basis of Success, Completion, and Repetition for Grades 1 to 12 is widely available. At the beginning of every school year, this document is sent to the directorates of education and then disseminated to all private and public schools, as well as United Nations Relief and Work Agency (UNRWA) schools. UNRWA schools are under the supervision of the UNRWA education sector, serving students from refugee camps in the West Bank and Gaza. 3. There are some system-wide resources available to teachers for classroom assessment. The Basis for Success, Completion, and Repetition for Grades 1 to 12 document also outlines what students are expected to learn in different subject areas at different grade/age levels. Textbooks or workbooks, as well as scoring criteria or rubrics for students' work are available to teachers for their classroom assessment activities. However, other resources, such as online assessment resources and item banks or pools with examples of selection/multiple-choice or supply/open-ended questions are not available as resources to teachers. 4. There is an official curriculum or standards document that specifies what students are expected to learn, but the level of performance required is not clear. The document is the general curricula guidelines developed in 1998 and 1999, which was adopted in principle for the elaboration of textbooks. The document defines teaching objectives and includes a simple description of the content and proposed assessment methods for each subject and grade. However, the document has not been released to the public. 5. There are some system-level mechanisms in place to ensure that teachers develop skills and expertise in classroom assessment. Pre-service teacher training on educational evaluation and assessment is available to all students in faculties of education at national universities. In addition, all teachers in primary and secondary schools benefit from in-service teacher training on educational evaluation. Teachers also have opportunities to participate in conferences and workshops, and school inspection or teacher supervision includes a component on classroom assessment. However, not all teacher training programs include a required course on classroom assessment, and there are no online resources on classroom assessment or opportunities to participate in item development for, or scoring of, large-scale assessments or exams. 6. Classroom assessment practices are known to be weak. While it is uncommon to observe errors in the scoring or grading of students' work, or for classroom assessment activities to not be aligned with the pedagogical or curricular framework, it is common for classroom assessment activities to rely mainly on multiple- choice, selection-type questions and to be mainly about recalling information. At the same time, it is common for teachers to not use explicit or a priori criteria for scoring or grading students' work; grade inflation and the uneven application of standards for grading students' work are serious problems. It is common for parents to be poorly informed about students' grades and for classroom assessment activities to provide little useful feedback to students. Classroom assessment is often used as an administrative or control tool rather than as a pedagogical resource. 7. There are limited systematic mechanisms to monitor the quality of classroom assessment activities. Classroom assessment is a required component of a teacher's performance evaluation and of school inspection or teacher supervision. Additionally, national or other system-wide reviews of the quality of 19 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 education include a focus on classroom assessment. However, there is no external moderation system that reviews the difficulty of classroom assessment activities, appropriateness of scoring criteria, etc., and there is no government funding available for research on the quality of classroom assessment activities and how to improve classroom assessment. 8. Classroom assessment information is required to be disseminated to a number of key stakeholders. Schools or teachers are required to report on an individual student's performance to school districts, parents, and students. However, classroom assessment information is not reported centrally to the Ministry of Education. 9. There are adequate required uses of classroom assessment to support student learning, including its use as an input for external examination results. Specifically, classroom assessment activities are used for diagnosing student learning issues, providing feedback to students on their learning, informing parents about their child's learning, planning next steps in instruction, grading students for internal classroom uses, and providing input to an external examination program. Although there is no official document governing these issues, teachers' assessment forms completed by education supervisors and principals emphasize the importance of adequately using classroom assessment to support student learning. Supervisors also ask teachers to provide a detailed analysis of their students' exam results in a specific format known as a student report card. 20 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 WEST BANK and GAZA Examinations 21 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which assessment activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies No standardized examination has taken The standardized examination has been The examination is a stable program that This option does not apply to this place. operating on an irregular basis. has been operating regularly. 1 dimension * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this authorizes the examination. document that authorizes the authorizes the examination. 2 dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public public.3 dimension. * This option does not apply to this This option does not apply to this The policy document addresses some key The policy document addresses all key dimension. dimension. aspects of the examination.4 aspects of the examination. * ENABLING CONTEXT 2: Having strong leadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the the examination or are indifferent to it. examination. 5 examination. examination. There are no attempts to improve the This option does not apply to this * There are independent attempts to There are coordinated attempts to examination by stakeholder groups. dimension. improve the examination by stakeholder improve the examination by stakeholder groups. 6 groups. * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this not welcomed by the leadership in charge dimension. generally welcomed by the leadership in dimension. of the examination. 7 charge of the examination. * 22 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 (CONTINUED) LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this examination. the examination. examination.8 dimension. * This option does not apply to this Funding covers some core examination Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. processing or reporting. processing and reporting.9 * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development. 10 dimension. development. * ENABLING CONTEXT 4: Having strong organizational structures The examination office does not exist or is The examination office is newly The examination office is a stable This option does not apply to this newly established. established. organization. 11 dimension. * The examination office is not accountable This option does not apply to this The examination office is accountable to This option does not apply to this to an external board or agency.12 dimension. an external board or agency. dimension. Examination results are not recognized by * Examination results are recognized by Examination results are recognized by Examination results are recognized by any certification or selection system. certification or selection system in the one certification or selection system in two or more certification or selection country. another country. systems in another country. 13 * The examination office does not have the The examination office has some of the The examination office has all of the The examination office has state-of-the- required facilities to carry out the required facilities to carry out the required facilities to carry out the art facilities to carry out the examination. examination. examination. 14 examination. * (CONTINUED) 23 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment examination, issues are pervasive. effectively, with minimal issues.15 effectively, with no issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on examination. examination. the examination. 16 * 24 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning examinations with learning goals and opportunities to learn It is not clear what the examination This option does not apply to this There is a clear understanding of what the This option does not apply to this measures. dimension. examination measures. 17 dimension. What the examination measures is This option does not apply to this What is measured by the examination is * This option does not apply to this questioned by some stakeholder groups. dimension. largely accepted by stakeholder groups. dimension. 18 * Material to prepare for the examination is There is some material to prepare for the There is comprehensive material to There is comprehensive material to minimal and it is only accessible to very examination that is accessible to some prepare for the examination that is prepare for the examination that is few students. students. accessible to most students. 19 accessible to all students. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the examination There are no courses or workshops on There are no up-to-date courses or There are up-to-date voluntary courses or There are up-to-date compulsory courses examinations available to teachers. workshops on examinations available to workshops on examinations available to or workshops on examinations for teachers. 20 teachers. teachers. * Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in most examination-related tasks. examination-related tasks. 21 examination-related tasks. examination-related tasks. * 25 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the assessment meets quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring quality There is no technical report or other There is some documentation on the There is a comprehensive technical report There is a comprehensive, high quality documentation. examination, but it is not in a formal but with restricted circulation. technical report available to the general report format. 22 public. * There are no mechanisms in place to This option does not apply to this There are limited systematic mechanisms There are varied and systematic ensure the quality of the examination. dimension. in place to ensure the quality of the mechanisms in place to ensure the quality examination. 23 of the examination. * ASSESSMENT QUALITY 2: Ensuring fairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the examination process is high. examination process is moderate. 24 examination process is low. examination process is marginal. The examination results lack credibility * The examination results are credible for The examination results are credible for This option does not apply to this for all stakeholder groups. some stakeholder groups. all stakeholder groups.25 dimension. * The majority of the students (over 50 A significant proportion of students (10 A small proportion of students (less than All students can take the examination; percent) may not take the examination percent-50 percent) may not take the 10 percent) may not take the examination there are no language, gender or other because of language, gender, or other examination because of language, because of language, gender, or other equivalent barriers. 26 equivalent barriers. gender, or other equivalent barriers. equivalent barriers. * (CONTINUED) 26 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Using examination information in a fair way Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all proper way by all stakeholder groups. stakeholder groups in a proper way. stakeholder groups in a proper way. 27 stakeholder groups in a proper way. * Student names and results are public. This option does not apply to this Students’ results are confidential. 28 This option does not apply to this dimension. dimension. * ASSESSMENT QUALITY 4: Ensuring positive consequences of the examination There are no options for students who do There are very limited options for There are some options for students who There is a variety of options for students not perform well on the examination, or students who do not perform well on the do not perform well on the examination. who do not perform well on the 29 students must leave the education examination. examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination. examination. 30 examination. * 27 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations: Development-level rating justifications 1. The examination is a stable program that has been operating regularly since it was begun in 1960, when the West Bank and Gaza were part of Jordan. 2. There is a formal policy document, Instructions for the General Secondary Education Examination Certificate, authorized by the Ministry of Education via the General Examination Committee yearly. 3. The policy document is available to and easily accessible by the public. 4. The policy document addresses some key aspects of the examination, including a description of the purpose of the examination, authorized uses of results, an outline of procedures for special/disadvantaged students, specifications for who can sit for the examination, and an identification of rules about preparation. However, the policy document does not outline governance, distribution of power and responsibilities among key entities, or state funding sources. Although this policy document does not outline procedures to investigate and address security breaches, cheating, or other forms of inappropriate behavior, or explain alignment with curricula and standards and the format of the examination questions, other policy documents contain content relevant to these issues. 5. Most stakeholder groups do not strongly support the examination. While policy makers and universities support the examination (policy makers strongly so), educators, students, parents, media, think tanks, and NGOs do not support it. Teacher unions and employers are neutral to the examination program. There is an ongoing discussion about the examination's difficulty level, as there is belief that the examination questions are difficult. There are also objections to the paper-and-pencil format, and recommended alternatives include the addition of performance tasks of a student portfolio when calculating the mean scores for students. 6. There are independent attempts to improve the examination by stakeholder groups. 7. Efforts to improve the examination are not welcomed by the leadership in charge of the examination. 8. There is regular funding allocated by the government and obtained from student fees for the examination. 9. Funding covers all core examination activities: design, administration, data processing and reporting. Additionally, funding covers stationery for test scoring, and transportation for test developers, test administrators, and scoring teams. 10. Funding does not cover research and development. 11. The examination office is a stable organization. The General Directorate for Measurement, Evaluation, and Examination, an office or branch within the Ministry of Education, has had primary responsibility for running the examination since the 1960s. Prior to 1967, the Ministry was under the Hashemite Kingdom 28 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 of Jordan and therefore examination registers before 1967 are kept by the Jordanian MoE. Since 1968, examination records and all related documents are kept by the MoE in the West Bank and Gaza. 12. The examination office is not accountable to an external board or agency. 13. Examination results are recognized by a certification or selection system in the West Bank and Gaza and in other Arab countries. 14. The examination office has some of the required facilities to carry out the examination. There are computers for all technical staff, secure storage facilities, access to adequate computer servers, the ability to backup data, and adequate communication tools. However, the examination office lacks a secure building. 15. There is permanent or full-time staff, but it is insufficient to meet the needs of the examination. However, only a few issues have been identified with the performance of the human resources who are responsible for the examination. These issues are poor training of test administrators or unclear instructions and guidelines for administering the examination, as well as weakness in test design. 16. The country offers a wide range of opportunities that prepare for work on the examination. Opportunities include: university graduate programs (master’s degree programs) and university courses specifically focused on educational measurement and evaluation, non-university training courses or workshops on educational measurement and evaluation, funding for attending international programs, courses, or workshops on educational measurement and evaluation, and internships in the examination office. 17. There is a clear understanding that the examination measures the national school curriculum guidelines or standards. 18. What the examination measures is questioned by some stakeholder groups. 19. There is comprehensive material to prepare for the examination that is accessible to most students. The competencies tested by the examination are included in textbooks provided at a low cost by the Ministry. Examples of the types of questions that are on the examination are also available. Additionally, examination instructions, which describe how to calculate mean scores (since subjects carry different weights) are distributed to all schools hosting students that are taking the examination, registered students, and Education offices. These examination instructions are available nine months before the examination. However, a framework document explaining what is measured on the examination and information on how to prepare for the examination are not publicly available. 20. There are no up-to-date courses or workshops on the examination available to teachers. 21. Teachers are involved in very few examination-related tasks. Teachers are involved in selecting or creating examination questions and supervising examination procedures; however, they are not involved in tasks such as administering the examination, scoring the examination, or selecting or creating examination scoring guides. They are also not involved in acting as a judge or in resolving inconsistencies between examination scores and school grades. 29 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 22. There is some documentation on the examination but it is not in a formal report format. 23. There are limited systematic mechanisms in place to ensure the quality of the examination. For example, while there is internal review or observers, there is no external review or observers, and external certification or audit does not take place. Additionally, pilot or field testing or translation verification do not take place. 24. Inappropriate behavior surrounding the examination process is moderate. Issuing forged certificates or altering results information, collusion among candidates, copying from other candidates, and impersonation (when an individual other than the registered candidate takes the examination) are behaviors that occur, although rarely. When these behaviors are detected, students are expelled from the examination. There is a document which clarifies the measures corresponding to each case of reported inappropriate behavior. 25. The examination results are credible for all stakeholder groups. 26. All students can take the examination; there are no language, gender or other equivalent barriers. 27. Most stakeholder groups, including policy makers, teacher unions, students, think tanks and NGOs, universities, and employers, use the examination results in a proper way. However, educators, parents, and the media have used the examination results in an improper way. It is believed that the media uses students' results out of the context of the examinations. Also, it is believed that educators and parents improperly use examination results when ranking schools, evaluating the quality of the education system, or using results as predictors of students’ success in university. 28. Students' results are confidential. Although schools publish students' results, students' names are not disclosed, only their seating number. 29. There are some options for students who do not perform well on the examination. For example, students may retake the examination or repeat a grade; however, options to attend remedial or preparatory courses to prepare to retake the examination or options for students to choose less selective schools, universities, or tracks, are not available. 30. There are some mechanisms in place to monitor the consequences of the examination. For example, field follow-ups are undertaken by teachers and researchers to high or low-performing schools; however, these are not organized or based on a clear methodology. Mechanisms such as a permanent oversight committee, expert review groups, regular focus groups or surveys of key stakeholders, are not in place. 30 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 WEST BANK and GAZA National (or System-Level) Large-Scale Assessment (NLSA) 31 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which NLSA activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the NLSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for NLSA No NLSA exercise has taken place. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this irregular basis. been operating regularly. 1 dimension. * There is no policy document pertaining to There is an informal or draft policy There is a formal policy document that This option does not apply to this NLSA. document that authorizes the NLSA. 2 authorizes the NLSA. dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public. 3 dimension. * There is no plan for NLSA activity. This option does not apply to this There is a general understanding that the There is a written NLSA plan for the dimension. NLSA will take place. 4 coming years. * ENABLING CONTEXT 2: Having strong public engagement for NLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the NLSA. the NLSA or are indifferent to it. NLSA. 5 NLSA. * (CONTINUED) 32 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for NLSA There is no funding allocated to the NLSA. There is irregular funding allocated to the There is regular funding allocated to the This option does not apply to this NLSA. 6 NLSA. dimension. * This option does not apply to this Funding covers some core NLSA activities: Funding covers all core NLSA activities: This option does not apply to this dimension. design, administration, analysis and design, administration, analysis and dimension. reporting. 7 reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development activities. 8 dimension. development activities. * ENABLING CONTEXT 4: Having strong organizational structures for NLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this team. group of people. institution or unit. 9 dimension. This option does not apply to this Political considerations regularly hamper Political considerations sometimes * Political considerations never hamper dimension. technical considerations. hamper technical considerations. technical considerations. 10 * This option does not apply to this The NLSA office is not accountable to a The NLSA office is accountable to a clearly This option does not apply to this dimension. clearly recognized body. recognized body. 11 dimension. * (CONTINUED) 33 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources for NLSA There is no staff allocated for running an The NLSA office is inadequately staffed to The NLSA office is adequately staffed to The NLSA office is adequately staffed to NLSA. effectively carry out the assessment. 12 carry out the NLSA effectively, with carry out the NLSA effectively, with no minimal issues. issues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities to The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the NLSA. opportunities to prepare individuals for 13 NLSA. work on the NLSA. * 34 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the NLSA is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against This option does not apply to this curriculum or learning standards. dimension. curriculum or learning standards. 14 dimension. * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned by What the NLSA measures is largely questioned by stakeholder groups. dimension. some stakeholder groups. 15 accepted by stakeholder groups. * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSA to ensure that it measures what it is dimension. measures what it is supposed to measure. intended to measure. intended to measure. 16 * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the NLSA There are no courses or workshops on the There are occasional courses or There are some courses or workshops on There are widely available high quality NLSA. workshops on the NLSA. 17 the NLSA offered on a regular basis. courses or workshops on the NLSA offered on a regular basis. * 35 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the NLSA meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA No options are offered to include all This option does not apply to this At least one option is offered to include all Different options are offered to include all groups of students in the NLSA. 18 dimension. groups of students in the NLSA. groups of students in the NLSA. There are no mechanisms in place to * This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in ensure the quality of the NLSA. dimension. ensure the quality of the NLSA. place to ensure the quality of the NLSA. 19 There is no technical report or other There is some documentation about the There is a comprehensive technical report There is a comprehensive, high quality * documentation about the NLSA. technical aspects of the NLSA, but it is not but with restricted circulation. technical report available to the general in a formal report format. 20 public. * ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA NLSA results are not disseminated. NLSA results are poorly disseminated. NLSA results are disseminated in an This option does not apply to this effective way. 21 dimension. * NLSA information is not used or is used in This option does not apply to this NLSA results are used by some NLSA information is used by all ways inconsistent with the purposes or dimension. stakeholder groups in a way that is stakeholder groups in a way that is the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the assessment. 22 assessment. There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to * There are a variety of mechanisms in monitor the consequences of the NLSA. dimension. monitor the consequences of the NLSA. 23 place to monitor the consequences of the NLSA. * 36 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National (of System-Level) Large Scale Assessment (NLSA): Development-level rating justifications 1. Since 2008, the national assessment has been administered every two years, covering Arabic, Mathematics, and the Sciences for grade 4 and grade 10 students. The national assessment was first administered in 1998, at first covering Arabic and Mathematics for grade 6. When it was administered in 2000, the national assessment covered Arabic, Mathematics, and Science for grades 4 and 6, and in 2005, it covered Arabic and Mathematics for grade 4 students. A representative sample of students from the West Bank and Gaza in government, UNRWA, and private schools take part in the national assessment exercise. 2. The Ministry of Education's plans, such as the Strategic Plan for Educational Development (2008-2012), refer to the national assessment activities; however, the Ministry of Education has not devised a clear strategy for educational assessment. 3. The policy document, the Strategic Plan for Educational Development (2008-2012), is available to the public at the Ministry of Education's website. There is a common understanding that the assessment will take place, but there is no formally written plan. 4. There is a common understanding that the assessment will take place, but there is no formally written plan. 5. Some stakeholder groups oppose the NLSA. Educators, think tanks, NGOs, and universities support the NLSA, and students and parents oppose it due to issues related to test anxiety. Teacher unions and employers are neutral to it. While policy makers and the media generally support the NLSA, some policy makers have expressed concern over the lack of transparency regarding assessment information, and some media sources, such as newspaper and magazine editorials or columns, have criticized the assessment and its uses. 6. There is irregular funding allocated to the NLSA. The Ministry of Education’s education development activities are financed through a joint financing program under the Ministry of Finance, with support from donors such as Finland, Ireland, and KFW. Since 2008, budgets have been allocated to national assessment activities within this joint financing program. 7. Funding covers some core NLSA activities, including assessment administration and data analysis, as well as the printing of examination papers, correction of tests, data entry, and the printing of reports that document the examination results. Although assessment design is not included under this funding, it is done by a specialized team within the Ministry of Education free of charge. However, funding does not cover data reporting, long- or medium- term planning of program milestones, research and development, or staff training. 8. Funding does not cover research and development activities. 9. The Ministry of Education Assessment and Evaluation Department is a permanent unit created for running the assessment. 10. Political considerations never hamper technical considerations. 37 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 11. The Ministry of Education Assessment and Evaluation Department is accountable to a higher office in the MoE. It is under the direct supervision of the Ministry of Education Assistant Deputy for Planning and Development Affairs. 12. The NLSA office is inadequately staffed to effectively carry out the assessment. While there is permanent or full-time staff to carry out the NLSA, it is insufficient since staff are responsible for other evaluation activities as well. Unlike other departments within the MoE, which have staff in district offices as well as in the headquarters, the Assessment and Evaluation Department has staff only in the headquarters. As a result, it is required to rely on part-time members to manage and follow-up on field activities. Recently, the NLSA faced some difficulties in obtaining the latest data on schools, teachers, and students, which has delayed sampling. 13. The West Bank and Gaza offer some opportunities to prepare individuals for work on the NLSA. University courses, some of which are considered part of pre- service teacher training, funding for attending international programs, courses, or workshops on educational measurement and evaluation, as well as internships or short-term employment in the large-scale assessment office, are all available. However, there are no university graduate programs or non-university training courses or workshops on educational measurement and evaluation. In addition, the quality of courses offered by universities differs by institution. 14. The NLSA measures performance against curriculum or learning standards. 15. Some stakeholder groups question what the NLSA measures. . 16. There are regular internal and ad hoc reviews of the alignment between the assessment instrument and what it is supposed to measure. To ensure that the assessment tools are credible, experts from the National Curricula Center, education supervision teams, and education assessment teams take part in the working groups to develop achievement examinations for the NLSA. 17. Occasional courses, workshops and presentations on the NLSA are offered to teachers. In addition, most teachers have access to courses online and the Assessment and Evaluation Department disseminates an introductory leaflet to all the directorates of education so that they may distribute them to all of the schools in the national sample. However, courses or workshops are not offered on a regular basis, and when courses are offered, most teachers do not have access to live courses or workshops. 18. No options are offered to include all groups of students in the NLSA. Students with special needs are exempted from the national assessment; instead, their evaluation is undertaken by their own schools according to special arrangements. 19. There is a variety of mechanisms in place to ensure the quality of the NLSA. For example, all proctors or administrators are trained according to a protocol, there is a standardized manual for large-scale assessment administrators, a pilot is conducted before the main data collection takes place, all booklets are numbered, there is double data scoring, and scorers are trained to ensure high inter-rater reliability. Additionally, staff from the Assessment and Evaluation Department, as well as a team of education supervisors from the Directorates of Education, conduct visits to some schools participating in the assessment. 38 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 20. There is some documentation about the technical aspects of the assessment but it is not in a formal report format. This documentation exists in a guide on the application of the assessment, which includes the model of the NLSA administration. 21. Reports with general NLSA indicators have been published in the yearly follow-up and assessment reports (which have been made available to the public) and at the school level (which have not been made available to the public). Specifically, these results are disseminated within 12 months of the large-scale assessment being administered and reports with results are made available for all stakeholder groups. The main reports on the results contain information on overall achievement levels and sub-groups, as well as information on trends over time overall and for subgroups. These reports also contain standard errors. Workshops or presentations on the general results are also held for key stakeholders. However, detailed results have been only published for the 2008 NLSA administration. The 2010 and 2012 detailed results have not been published yet as they are currently being used to develop central policies. 22. NLSA results are used by some stakeholder groups in a way that is consistent with the purposes and technical characteristics of the assessment. 23. There are some mechanisms, such as regular focus groups or surveys of key stakeholders, in place to monitor the consequences of the NLSA. However, other mechanisms, such as a permanent oversight committee, expert review groups, themed conferences that provide a forum to discuss research and other data on the consequences of the large-scale assessment, and funding for independent research on the impact of the large-scale assessment are not available. 39 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 WEST BANK and GAZA International Large-Scale Assessment (ILSA) 40 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which ILSA takes place in a country or system and the extent to which that framework is conducive to, or supportive of, ILSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for ILSA The country/system has not participated This option does not apply to this The country/system has participated in at The country/system has participated in in an ILSA in the last 10 years. dimension. least one ILSA in the last 10 years. two or more ILSA in the last 10 years. 1 * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concrete steps to participate in an ILSA in dimension. steps to participate in at least one ILSA in dimension. the next 5 years. the next 5 years. 2 There is no policy document that There is an informal or draft policy There is a formal policy document that * This option does not apply to this addresses participation in ILSA. document that addresses participation in addresses participation in ILSA. dimension. ILSA. 3 This option does not apply to this The policy document is not available to * The policy document is available to the This option does not apply to this dimension. the public. public. 4 dimension. * ENABLING CONTEXT 2: Having regular funding for ILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at There is regular funding approved by law, ILSA. donors. 5 discretion. decree or norm. This option does not apply to this * Funding covers some core activities of the Funding covers all core activities of the This option does not apply to this dimension. ILSA. ILSA.6 dimension. * Funding does not cover research and This option does not apply to this This option does not apply to this Funding covers research and development activities. dimension. dimension. development activities.7 * 41 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 (CONTINUED) LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective human resources for ILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. activities. activities. activities.8 * This option does not apply to this The national/system coordinator or other The national/system coordinator is fluent This option does not apply to this dimension. designated team member may not be in the language of the assessment.9 dimension. fluent in the language of the assessment. * This option does not apply to this The ILSA office is inadequately staffed or The ILSA office is adequately staffed or The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, effectively.10 with minimal issues. with no issues. * 42 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Providing opportunities to learn about ILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this international workshops or meetings. international workshops or meetings. 11 workshops or meetings. dimension. The country/system offers no This option does not apply to this * The country/system offers some The country/system offers a wide range opportunities to learn about ILSA. dimension. opportunities to learn about ILSA. 12 of opportunities to learn about ILSA. * This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition team members only. to the country's/system's ILSA team members. 13 * 43 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of ILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical This option does not apply to this published. standards to have its data presented standards required to have its data dimension. beneath the main display of the presented in the main displays of the international report or in an annex. international report. 14 * The country/system has not contributed This option does not apply to this This option does not apply to this The country/system has contributed new new knowledge on ILSA. dimension. dimension. knowledge on ILSA. 15 * ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA If any, country/system-specific results Country/system-specific results and Country/system-specific results and Country/system-specific results and and information are not disseminated in information are disseminated irregularly information are regularly disseminated in information are regularly and widely the country/system. in the country/system. the country/system. disseminated in the country/system. 16 * Products to provide feedback to schools This option does not apply to this Products to provide feedback to schools Products to provide feedback to schools and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are not made available. sometimes made available. 17 systematically made available. * There is no media coverage of the ILSA There is limited media coverage of the There is some media coverage of the ILSA There is wide media coverage of the ILSA results. ILSA results. 18 results. results. If any, country/system-specific results * Results from the ILSA are used in a limited Results from the ILSA are used in some Results from the ILSA are used in a variety and information from the ILSA are not way to inform decision making in the ways to inform decision making in the of ways to inform decision making in the used to inform decision making in the country/system. country/system.19 country/system. country/system. * It is not clear that decisions based on ILSA This option does not apply to this This option does not apply to this Decisions based on the ILSA results have results have had a positive impact on dimension. dimension. had a positive impact on students' students' achievement levels. achievement levels.20 * 44 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large Scale Assessment (ILSA): Development-level rating justifications 1. The West Bank and Gaza has participated in multiple ILSAs in the last 10 years, including the International Association for the Evaluation of Educational Achievement's (IEA) Trends in Mathematics and Science Study (TIMSS) in 2003, 2007, and 2011. Additionally, the West Bank and Gaza participated in the Literacy Assessment and Monitoring Programme (LAMP), developed by the UNESCO Institute for Statistics (UIS), in 2008. 2. The MoE in the West Bank and Gaza has agreed to participate in TIMSS in 2015. 3. The West Bank and Gaza has an informal policy document that addresses participation in international assessments. TIMSS was included in the five-year education development plan for 2008-2012. 4. The Strategic Plan for Education Development (2008-2012), authorized in 2008 by the MoE, is published on the MoE website, along with the yearly follow-up and assessment report in Arabic and English. The Plan is also disseminated to private and public institutions, as well as directorates of education. 5. Funding for participation in the ILSA is sourced from loans or external donors. In the past three years, a special funding method was adopted by the MoE—the allocation of a special funding basket from a number of donor countries. 6. Funding covers all core activities of the ILSA, including international participation fees, implementation of the assessment exercise in the country, processing and analyzing data collected from implementation of the assessment exercise, reporting and disseminating the assessment results in the country, attendance at international expert meetings for the assessment exercise, and research and development. 7. Funding covers research and development activities. There are development projects within the MoE funded by donors which often cover some activities related to international assessment studies, especially those pertaining to quality assessment or impact of such programs on pilot schools. These are funded independently of joint funding programs, such as those sponsored by the World Bank and USAID. 8. There is a team and a national coordinator to carry out the ILSA activities. 9. The national coordinator is fluent in the language of the assessment. 45 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 10. The team responsible for carrying out the ILSA has previous experience working on international assessments, has attended some international meetings related to the international assessment, and has the necessary training or experience to carry out the required assessment activities. However, the team is not sufficiently staffed, as it is composed of seven part-time members who are responsible for providing comprehensive follow-up of all activities. These team members work under the Assessment and Evaluation Department, which carries out many of the educational assessment activities within the MoE. Additionally, there have been some issues with translation of the assessment instruments. 11. The ILSA team has attended some international workshops or meetings. However, the West Bank and Gaza is represented in many of these meetings by no more than two members due to budget constraints. In some instances, no one can attend such meetings because of security measures or visa delays. 12. The country offers some opportunities to learn about ILSA, including online courses on international assessments. However, other opportunities, including workshops or meetings on using international assessment databases, university courses on the topic of international assessments, and funding for attending international workshops or training on international assessments are not available. 13. Opportunities to learn about ILSA are available to a wide audience, in addition to the country's ILSA team members. For instance, opportunities are also available to university students studying assessment or a related area, and professionals or university staff interested in assessment. Additionally, information related to international assessments is often made available under the supervision of international and local organizations targeting quality improvement. 14. The West Bank and Gaza has met all technical standards required to have its data presented in the main displays of the international report. 15. The West Bank and Gaza has contributed to the global knowledge base on international assessments by generating new knowledge and making it available through publications. For example, research papers were published in refereed reports, the first being the IEA initiative entitled Educational Issues in the Middle East and North Africa Region, Outcomes of the IEA Arab Region Training Seminar Series 2006/2007. Additionally, a special chapter on the West Bank and Gaza was published in the TIMSS report, which can be accessed at www.iea.nl. 16. Country specific results and information from the most recent international assessment were disseminated in the West Bank and Gaza. For example, copies of the international report were distributed to key stakeholders, results were communicated through a press release, results received coverage on the television, radio, or newspapers, and brochures and PowerPoint presentations with the results were made available online or distributed to key stakeholders. Products providing feedback to schools and educators about the results were also made available. Additionally, Science and Mathematics curricula developers were informed about the results. 17. Products to provide feedback to schools and educators about the ILSA results are sometimes made available. 46 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 18. Media coverage of the ILSA results is limited to a few short articles and brief television/radio reports. 19. Results from the ILSA have been used in tracking the impact of reforms on student achievement levels, informing curriculum improvement, teacher training programs, and other assessment activities in the system. However, results have not been used in informing resource allocation. 20. Decisions based on the ILSA results have had a positive impact on students' achievement levels. For instance, supportive material covering mathematical skills encompassed in TIMSS and not included in national curricula was developed and disseminated to all schools. Additionally, translated samples of paragraphs allowed by the IEA were sent to all schools, providing a good example for teachers to use in developing their classroom exams. Also, an improvement in the educational achievement level in Mathematics was noted in the national assessment study between 2010 and 2012, which may be due to steps taken on the basis of TIMSS 2007 results at the national level. 47 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 References Acknowledgements This report, part of a 16-country benchmarking exercise Clarke, M. 2012. “What Matters Most for Student in the Middle East and North Africa and Africa regions, Assessment Systems: A Framework Paper.” READ/SABER was prepared by the World Bank SABER-Student Working Paper Series. Washington, DC: World Bank. Assessment team, in partnership with the Arab League Educational, Cultural, and Scientific Organization Ministry of Education and Higher Education. 2008. (ALECSO) and the United Nations Educational, Scientific, “Education Development Strategic Plan 2008-2012.” and Cultural Organization (UNESCO), which led data- Ramallah: Palestinian National Authority. collection efforts. This effort is part of the Arab Regional Agenda for Improving Education Quality (ARAIEQ), led by United Nations Children’s Fund. 2011. “Child Friendly ALECSO in partnership with the World Bank. It benefited Schools in the Occupied Palestinian Territory: A Case from feedback and review by Ernesto Cuadra, Lead Study.” Occupied Palestinian Territory: UNICEF. Education Specialist, Husein Abdul-Hamid, Senior Education Specialist, and Juan Manuel Moreno, Lead United Nations Educational, Scientific, and Cultural Education Specialist, in the World Bank’s Education Organization – Institute for Statistics. 2011. UIS Statistics Global Practice, as well as comments received during a in Brief. “Education Profile: Palestine.” Montreal: national validation workshop held in the West Bank and UNESCO-UIS. Gaza. World Bank. 2013. “Country Brief: West Bank and Gaza.” Washington, DC: World Bank. Data retrieved from http://web.worldbank.org/WBSITE/EXTERNAL/COUNTRI ES/MENAEXT/WESTBANKGAZAEXTN/0,,menuPK:294374 ~pagePK:141132~piPK:141107~theSitePK:294365,00.ht ml on October 15, 2013. 48 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS WEST BANK AND GAZA ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 www.worldbank.org/education/saber r The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. 49 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS