Ethiopia   SABER Country Report STUDENT ASSESSMENT 2009 100104         Key Policy Areas for Student Assessment Status 1. Classroom Assessment There is no system-level document that provides guidelines for classroom  assessment activities. There is, however, an official curriculum that specifies what students are expected to learn, although the level of performance required is unclear. Regional Education Bureaus receive federal support that may be applied to classroom assessment activities within the context of their curriculum implementation and teacher development activities. The use of classroom assessment to support student learning, while required, is very limited. 2. Examinations The national examinations program is stable and has been operating on a  regular basis. There is a formal, publicly-available policy document that authorizes the examinations. Regular funding, mainly from the registration fee that students must pay in order to take the examination, supports the examinations program. There is a clear understanding of what the examinations measure because they are based on the curriculum materials used to teach students. The examination results are credible for all stakeholder groups, and are used by most of these groups in an appropriate way. 3. National Large-Scale Assessment (NLSA) The NLSA program has been operating on a regular basis with support from  various donors. Although there is no formal document authorizing the NLSA, there is a general understanding that it will take place. While Ethiopia offers some courses to prepare individuals for work on the NLSA, such as those offered at Addis Ababa University, there are no courses on the NLSA geared to teachers. There are no mechanisms in place to monitor the consequences or impact of NLSA activities on education quality or achievement levels. 4. International Large-Scale Assessment (ILSA) Ethiopia has not yet participated in an ILSA, although there is an informal plan  to participate in one in the next few years.      THEWORLDBANK ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Introduction SABERǦStudentAssessmentmethodology   The SABERͲStudent Assessment framework is built on In 2009, Ethiopia joined the Russia Education Aid for the available evidence base for what an effective Development (READ) Trust Fund program, the goal of assessment system looks like. The framework provides which is to help countries improve their capacity to guidance on how countries can build more effective design, carry out, analyze, and use assessments for student assessment systems. The framework is improved student learning. As part of the READ Trust structured around two main dimensions of assessment Fund program, and in order to gain a better systems:thetypes/purposesofassessmentactivitiesand understanding of the strengths and weaknesses of its thequalityofthoseactivities. existing assessment system, Ethiopia participated in a  formal exercise to benchmark this system under The World Bank’s Systems Approach for Better Education Assessmenttypesandpurposes  Results (SABER) program. SABER is an evidenceͲbased Assessmentsystemstendtobecomprisedofthreemain program to help countries systematically examine and types of assessment activities, each of which serves a strengthentheperformanceofdifferentaspectsoftheir different purpose and addresses different information educationsystems. needs. These three main types are: classroom  assessment, examinations, and largeͲscale, system level WhatisSABERͲStudentAssessment? assessments.   ClassroomassessmentprovidesrealͲtimeinformationto SABERͲStudentAssessmentisacomponentoftheSABER support ongoing teaching and learning in individual program that focuses specifically on benchmarking classrooms. Classroom assessments use a variety of student assessment policies and systems. The goal of formats, including observation, questioning, and paperͲ SABERͲStudent Assessment is to promote stronger andͲpencil tests, to evaluate student learning, generally assessment systems that contribute to improved onadailybasis. educationqualityandlearningforall.   Examinations provide a basis for selecting or certifying National governments and international agencies are students as they move from one level of the education increasingly recognizing the key role that assessment of system to the next (or into the workforce). All eligible student learning plays in an effective education system. students are tested on an annual basis (or more often if Theimportanceofassessmentislinkedtoitsrolein: thesystemallowsforrepeattesting).Examinationscover (i) providing information on levels of student the main subject areas in the curriculum and usually learningandachievementinthesystem; involveessaysandmultipleͲchoicequestions. (ii) monitoring trends in education quality over  time; LargeͲscale, systemͲlevel assessments provide feedback (iii) supporting educators and students with realͲ on the overall performance of the education system at time information to improve teaching and particular grades or age levels. These assessments learning;and typicallycoverafewsubjectsonaregularbasis(suchas (iv) holdingstakeholdersaccountableforresults. every 3 to 5 years), are often sample based, and use  multipleͲchoiceandshortͲanswerformats.Theymaybe  nationalorinternationalinscope.   Appendix 1 summarizes the key features of these main  typesofassessmentactivities.       SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  2 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Qualitydriversofanassessmentsystem Table 1: Framework for building an effective  assessmentsystem,withindicatorareas The key considerations when evaluating a student assessment system are the individual and combined qualityofassessmentactivitiesintermsoftheadequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, systemalignment,andassessmentquality.  Enabling context refers to the broader context in which the assessment activity takes place and the extent to whichthatcontextisconduciveto,orsupportiveof,the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional The indicators are identified based on a combination of andorganizationalstructuresfordesigning,carryingout, criteria,including: or using results from the assessment; the availability of x professionalstandardsforassessment; sufficient and stable sources of funding; and the x empiricalresearchonthecharacteristicsofeffective presenceoftrainedassessmentstaff. assessment systems, including  analysis  of  the  characteristics  that  differentiate  between  the System alignment refers to the extent to which the assessment systems of lowͲ versus highͲperforming assessment is aligned with the rest of the education nations;and system.Thisincludesthedegreeofcongruencebetween x theory—that is, general consensus among experts assessment activities and system learning goals, thatitcontributestoeffectiveassessment. standards, curriculum, and preͲ and inͲservice teacher  training. Levelsofdevelopment   Assessmentqualityreferstothepsychometricqualityof The World Bank has developed a set of standardized the instruments, processes, and procedures for the questionnaires and rubrics for collecting and evaluating assessment activity. It covers such issues as design and data on the three assessment types and related quality implementation of assessment activities, analysis and drivers. interpretation of student responses to those activities,  and the appropriateness of how assessment results are The questionnaires are used to collect data on the reportedandused. characteristics of the assessment system in a particular  country.Theinformationfromthequestionnairesisthen Crossing the quality drivers with the different appliedtotherubricsinordertojudgethedevelopment assessmenttypes/purposesprovidestheframeworkand level of the country’s assessment system in different broad indicator areas shown in Table 1. This framework areas. is a starting point for identifying indicators that can be  used to review assessment systems and plan for their The basic structure of the rubrics for evaluating data improvement. collected using the standardized questionnaires is  summarized in Appendix 2.The goal of the rubrics is to  provide a country with some sense of the development  level of its assessment activities compared to best or  recommendedpracticeineacharea.Foreachindicator,the  rubricdisplaysfourdevelopmentlevels—Latent,Emerging,  Established, and Advanced. These levels are artificially  constructed categories chosen to represent key stages on      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  3 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  the underlying continuum for each indicator. Each level is TheEthiopiaEducationandTrainingPolicyandStrategy, accompaniedbyadescriptionofwhatperformanceonthe introducedin1994,createdanewstructureforprimary indicatorlookslikeatthatlevel. andsecondaryschoolingconsistingofan8Ͳyearprimary  education cycle, followed by two years of general x Latent is the lowest level of performance; it secondary education and two years of preparatory representsabsenceof,ordeviationfrom,thedesired secondary education. As a result of this strategy, the attribute. education system has a renewed focus on student x Emerging is the next level; it represents partial learning and problemͲsolving skills, an emphasis on presenceoftheattribute. technical and vocational education and training, and x Established represents the acceptable minimum support for the teaching of primary students in their standard. mother tongue. The strategy also has supported the x Advanced represents the ideal or current best creationofselfͲcontainedclassesingrades1–4(i.e.,one practice. teachertocoverallcoresubjects).   A summary of the development levels for each In recent years, Ethiopia’s education sector has assessmenttypeispresentedinAppendix3. undergone significant expansion. At the same time, it  continues to face challenges, especially in the area of In  reality,  assessment  systems  are  likely  to  be  at quality. In response, the Government of Ethiopia different  levels  of development in different areas. For introducedacomprehensivereformprogramtoimprove example, a system may be Established in the area  of the quality of general education (Grades 1 through 12), examinations,butEmergingintheareaoflargeͲscale, referred to as the General Education Quality systemͲlevel assessment, and vice versa. While intuition Improvement Program (GEQIP). The key objectives of suggeststhatitisprobablybettertobefurtheralonginas GEQIPareto:(i)improvestudentlearningoutcomes;(ii) manyareasaspossible,theevidenceisunclearasto increase primary completion rates; and (iii) increase whetheritisnecessarytobefunctioningatAdvanced secondaryenrolmentrates. levels  in  all  areas. Therefore, one might view the  Established level as a desirable minimum outcome to Detailed information on Ethiopia’s student assessment achieve in all areas, but only aspire beyond that in those system was obtained from the country’s selfͲdiagnosis areas that most contribute to the national vision or reportthatwaspreparedaspartoftheREADTrustFund priorities for education. In line with these considerations, program in 2009. At the time of the review, SABERͲ the ratings generated by the rubrics are not meant to be Student Assessment questionnaires had not been additive across assessment types (that is, they are not developed. SelfͲdiagnosis information was analyzed meant to be added to create an overall rating for an using the SABERͲStudent Assessment rubrics. It is assessment system; they are only meant to produce an important to remember that the SABERͲStudent overallratingforeachassessmenttype).Themethodology Assessment rubrics primarily focus on benchmarking a for assigning development levels is summarized in country’s policies and arrangements for assessment Appendix4. activities at the system or macro level. Additional data  wouldneedtobecollectedtodetermineactual,onͲtheͲ groundpracticesinEthiopia,particularlybyteachersand EducationinEthiopia students in schools. The following sections discuss the  findings by each assessment type, accompanied by Ethiopia is a low income country in East Africa. GDP per suggested policy options. The suggested policy options capita(currentUS$,2009)is$350.8,withannualgrowthof were determined in collaboration with key local approximately 6.5 percent. At the primary level, gross stakeholders based on Ethiopia’s immediate interests schoolenrollmentisaround102percent;atthesecondary and needs. Detailed, completed rubrics for each level, it is around 34 percent. While enrollment is high for assessmenttypeareprovidedinAppendix5. bothgendersinprimaryschool(98percentforgirlsand106 percentforboys),only30percentofgirlsand38percentof boysareenrolledinsecondaryschool.      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  4 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ClassroomAssessment Classroomassessmentinformationisnotrequiredtobe disseminated to key stakeholders. The use of classroom assessment to support student learning, while required, Levelofdevelopment isinfactverylimited.   In Ethiopia, there is no systemͲlevel document that  provides guidelines for classroom assessment activities. Suggestedpolicyoptions: There is, however, an official curriculum document that  specifies what students are expected to learn, although  thelevelofperformancerequiredisnotclear. 1. Build teacher capacity by making available  systemͲwide resources/materials to make it Regional Education Bureaus (REB) receive federal easier for them to engage in classroom support that may be applied to classroom assessment assessment, and by introducing a variety of activities within the context of their curriculum systemͲlevel mechanisms to ensure that implementation and teacher development activities teachers develop the necessary skills and (under the framework of the school improvement expertiseinclassroomassessment. program).   2. Improve the quality of classroom assessment There are no systemͲwide resources for teachers for practicesbyintroducingsystematicmechanisms classroom assessment, and there are no systemͲlevel tomonitortheirquality. mechanisms to ensure that teachers develop skills and  expertiseinclassroomassessment.    In general, classroom assessment practices suffer from widespreadweaknesses,andmonitoringoftheirquality iscarriedoutonlyonanadhocbasis.                            SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  5 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Examinations Comprehensivematerialtopreparefortheexaminations is accessible by all students as the examinations are based on the regular curriculum taught to students in Levelofdevelopment school.   In Ethiopia, students take the Primary School Certificate Limited systematic mechanisms are in place to ensure Examinationattheendofgrade8,theEthiopianGeneral the quality of the examinations. There has not been a Secondary Education Certificate Examination at the end tradition of producing a formal technical report on the ofgrade10,andtheEthiopianHigherEducationEntrance examinations, but some informal documentation is Examinationattheendofgrade12.Theseexaminations available. have been operating on a regular basis. The 1994  Education and Training Policy is a publically available In appropriate behavior surrounding the examinations policydocumentthatauthorizestheseexaminationsand process is moderate and the results are viewed as addressessomeoftheirkeyaspects,includingthegrades credible by all stakeholder groups. All students can take in which they should be given, and the criteria for their the examinations; there are no language, gender, or grading. other equivalent barriers. Student results are  confidential. Examination results are used by most Most stakeholders groups support the examinations. stakeholdergroupsinanappropriateway. Funding for the examinations comes from the  registration fees that examinees are required to pay in Currently, there are no mechanisms in place to monitor ordertotakethem.Fundingisusuallysufficienttocover theconsequencesoftheexaminations. all core examination activities (design, administration,  data processing, and reporting), but not research and  development. Suggestedpolicyoptions:   The examinations office has been operating since 1950, 1. Make available comprehensive, highͲquality and is accountable to the Ministry of Education. It is technical reports on the examinations to the currently undergoing reorganization in accordance with general public, and introduce varied and the Business Process ReͲengineering (BPR) study systematic mechanisms to better ensure the undertaken by the Ministry of Education in 2009.The qualityoftheexaminations. examinationsofficeisadequatelystaffedforcarryingout  core examination activities. It has some, but not all, of 2. Provide teachers with opportunities to learn therequiredfacilitiesandequipment. about the examinations by introducing upͲtoͲ  date compulsory courses or workshops on Someopportunitiesthatprepareindividualsforworkon examinations and by involving teachers in more the examinations are available in the country, including examinationͲrelatedtasks. academic courses on relevant topics offered by   education and psychology departments in universities andcollegesthroughoutthecountry.  WhileteachersareinvolvedinsomeexaminationͲrelated tasks, such as question writing and examination administration, there are no upͲtoͲdate courses or workshopsontheexaminationsavailabletoteachers.          SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  6 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  NationalLargeͲScaleAssessment The NLSA is based on the existing national curriculum and standards. Reviews to ensure that the assessment (NLSA) measures what it is intended to measure in relation to thiscurriculumareconductedonaninformalandadhoc Levelofdevelopment basis.   Efforts are made to include all groups of students in the The NLSA program in Ethiopia has been operating on a NLSA in that the assessment is translated into the regular basis, with an NLSA exercise carried out every languageofinstructionforeachregion. threetofouryears.   There is some documentation about the technical The first, second, and third Ethiopian Baseline National aspects of the NLSA, but it is not in a formal report Learning Assessments (EBNLA) were conducted in the format. 1999/2000, 2003/2004, and 2006/2007 school years,  respectively. The EBNLA assessed grade 4 students in NLSA results have not been widely disseminated to key English,BasicReading,Mathematics,andEnvironmental stakeholders. Few copies of the reports are available. Science, and grade 8 students in English, Mathematics, The2009NLSAresultswere,toacertainextent,usedfor Physics, Chemistry, and Biology. The Ethiopian First curriculumdevelopmentandteachertraining. National Learning Assessment (EFNLA) of students in  grades 10 and 12 was conducted at the end of the Currently, there are no mechanisms in place to monitor 2008/2009schoolyearinEnglish,Mathematics,Physics, the consequences or impact of the NLSA on education Chemistry,andBiology. qualityorachievementlevels.   While there is no policy document pertaining to NLSA  activity in the country, there is a general understanding Suggestedpolicyoptions: thattheexerciseswilltakeplace.   1. Institutionalize the NLSA by creating a publiclyͲ FundingfortheNLSAexerciseshasbeenprovidedonan available policy document authorizing the adͲhoc basis by the Ministry of Education and donors. program, and introducing regular government Funding has tended to cover only basic aspects of the fundingforcarryingoutallcoreNLSAactivities. core assessment activities. At times, there has been  insufficient funding to cover all core activities. Funding 2. Ensure that the NLSA supports student learning tendsnottocoverresearchanddevelopment. byconductingregularinternalreviewstoensure  thatitmeasureswhatitisintendedtomeasure, In 2009, there was no permanent unit for the NLSA and and introducing a variety of mechanisms to an institutional home was still being worked out.  In monitor the consequences and impact of the general,theNLSAofficeisatemporarygroupofpeople. NLSA.  The current team is comprised of four staff, some of whom have no background or training in NLSAs. However, Ethiopia does offer some opportunities to prepare individuals for work on the NLSA, such as assessmentͲrelated courses offered by Addis Ababa University.Therearecurrentlynocoursesorworkshops ontheNLSAavailabletoteachers.Coursesorworkshops associated with previous NLSA exercises were only for policymakersandhighͲleveleducators.         SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  7 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  InternationalLargeͲScaleAssessment(ILSA) Levelofdevelopment  Ethiopia has not participated in an ILSA in the last 10 years, and the country has not taken concrete steps to participate in one in the next 5 years (although apparently,thereareinformalplanstodoso).   Suggestedpolicyoptions:  1. Create an opportunity for highͲlevel discussion among key stakeholders on priority education policyquestionsorproblemsforwhichILSAdata couldbeuseful.  2. Determinetheneedfor,andpossiblenextsteps inrelationto,participatinginanILSAexercise.      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  8 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Appendix1:AssessmentTypesandTheirKeyDifferences  Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform classroom the system at performance of level of the educational instruction particular the education education system opportunities grade/age system at to the next (or level(s), and to particular into the monitor trends in grade/age workforce) learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often where more often on a regular on a regular the system allows where the system basis (such as basis (such as for repeats allows for every 3-5 years) every 3-5 years) repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually Usually essay and Usually essay observation to choice and short multiple choice multiple choice and multiple questioning to answer and short choice paper-and-pencil answer tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a few confined to one subject areas subject areas subjects or two subjects Additional Yes, as part of the Frequently Yes Seldom Seldom information teaching process collected from students? Scoring Usually informal Varies from Usually Varies from Varies from and simple simple to more involves simple to more simple to more statistically statistically statistically statistically sophisticated sophisticated sophisticated sophisticated techniques techniques techniques techniques      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  9 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Appendix2:BasicStructureofRubricsforEvaluatingDataCollectedonaStudentAssessmentSystem  Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses             SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  10 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Appendix3:SummaryoftheDevelopmentLevelsforEachAssessmentType  Assessment Type LATENT EMERGING ESTABLISHED ADVANCED  Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide institutional wide institutional support and ensure the capacity to support and capacity to support and capacity to support and quality of classroom ensure the quality of ensure the quality of ensure the quality of assessment practices. classroom assessment classroom assessment classroom assessment practices. practices. practices. CLASSROOMASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and and a need to develop There is institutional institutional capacity and institutional capacity to capacity and some strong mechanisms to run the examination. The limited mechanisms to monitor it. The EXAMINATIONS examination typically is monitor it. The examination is of high of poor quality and is examination is of quality and is perceived perceived as unfair or acceptable quality and is as fair and free from corrupt. perceived as fair for corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA There is a stable NLSA place. NLSA in place and a in place. There is in place and institutional need to develop institutional capacity and capacity and strong institutional capacity to some limited mechanisms to monitor run the NLSA. mechanisms to monitor it. The NLSA is of high NATIONAL(ORSYSTEMͲ Assessment quality and it. The NLSA is of quality and its LEVEL)LARGEͲSCALE impact are weak. moderate quality and its information is ASSESSMENT information is effectively used to disseminated, but not improve education. always used in effective ways. There is no history of Participation in an ILSA There is more or less There is stable participation in an ILSA has been initiated, but stable participation in an participation in an ILSA nor plans to participate there still is need to ILSA. There is and institutional capacity in one. develop institutional institutional capacity to to run the ILSA. The capacity to carry out the carry out the ILSA. The information from the INTERNATIONALLARGEͲ ILSA. information from the ILSA is effectively used SCALEASSESSMENT ILSA is disseminated, to improve education. but not always used in effective ways.      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  11 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Appendix 4: Methodology for Assigning  DevelopmentLevels   6. Scores for certain key dimensions under ‘Enabling Context’(inthecaseofEXAM,NLSA,andILSA)andunder 1. The country team or consultant collects information ‘System Alignment’ (in the case of CLASS) were set as abouttheassessmentsysteminthecountry. ceiling scores, i.e., the overall mean score for the  particular assessment type cannot be greater than the 2. Based on the collected information, a level of score for these key dimensions. These key variables developmentandscoreisassignedtoeachdimensionin include formal policy, regular funding, having a therubrics: permanent assessment unit, and the quality of  assessmentpractices. x Latent=1scorepoint  x Emerging=2scorepoints x Established=3scorepoints x Advanced=4scorepoints  3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions.Forexample:  Thequalitydriver,‘EnablingContext,’inthecaseofILSA, has 3 dimensions on which a hypothetical country receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypotheticalcountry’soverallscoreforthisqualitydriver wouldbe:(2+2+3)/3=2.33  4.Apreliminarylevelofdevelopmentisassignedtoeach qualitydriver.  5. The preliminary development level is validated using expert judgment in cooperation with the country team andTheWorldBankTaskTeamLeader.   For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriatelevel.           SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS  12 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Appendix5:SABERǦStudentAssessmentRubricsforEthiopia  ThisappendixprovidesthecompletedSABERͲStudentAssessmentrubricsforeachtypeofassessmentactivityinEthiopia.Ineachrowoftherubric,therelevant selection is indicated by a thick border and an asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection(asindicatedbyathickborderandanasterisk).Theexplanationorjustificationtextcanbelocatedinthe“Developmentlevelratingjustifications”section attheendofeachrubric.Ifarowincludesasuperscript,butnotathickborderandanasterisk,thismeansthatinsufficientinformationwasavailabletodetermine therelevantselectionintherow.      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 13 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009         ETHIOPIA ClassroomAssessment   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 14 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ENABLINGCONTEXTANDSYSTEMALIGNMENT Overallpolicyandresourceframeworkwithinwhichclassroomassessmentactivitytakesplaceinacountryorsystem,andthedegreetowhichclassroom assessmentactivityiscoherentwithothercomponentsoftheeducationsystem. LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXTANDSYSTEMALIGNMENT1: Settingclearguidelinesforclassroomassessment There is no systemͲlevel document that There is an informal systemͲlevel There is a formal systemͲlevel document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. assessment.1 classroomassessment. assessment. * This option does not apply to this This option does not apply to this The availability of the document is Thedocumentiswidelyavailable dimension. dimension. restricted.. * ENABLINGCONTEXTANDSYSTEMALIGNMENT2: Aligningclassroomassessmentwithsystemlearninggoals There are no systemͲwide resources for There are scarce systemͲwide resources There are some systemͲwide resources There are a variety of systemͲwide teachersforclassroomassessment. forteachersforclassroomassessment. forteachersforclassroomassessment. resources available for teachers for classroomassessment. * There is no official curriculum or There is an official curriculum or There is an official curriculum or There is an official curriculum or standardsdocument. standards document, but it is not clear standards document that specifies what standards document that specifies what whatstudentsareexpectedtolearnorto students are expected to learn, but the students are expected to learn and to whatlevelofperformance. levelofperformancerequiredisnotclear. whatlevelofperformance. ENABLINGCONTEXTANDSYSTEMALIGNMENT3: * Havingeffectivehumanresourcestocarryoutclassroomassessmentactivities TherearenosystemͲlevelmechanismsto This option does not apply to this TherearesomesystemͲlevelmechanisms There are a variety of systemͲlevel ensure that teachers develop skills and dimension. toensurethatteachersdevelopskillsand mechanisms to ensure that teachers expertiseinclassroomassessment. expertiseinclassroomassessment. develop skills and expertise in classroom assessment. *                      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 15 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   ASSESSMENTQUALITY Qualityofclassroomassessmentdesign,administration,analysis,anduse. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENTQUALITY1: Ensuringthequalityofclassroomassessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are from widespread weaknesses or there is knowntobeweak. knowntobeofmoderatequality. knowntobegenerallyofhighquality. no information available on classroom assessmentpractices. * There arenomechanisms to monitor the Thereareadhocmechanismstomonitor Therearelimitedsystematicmechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the practices. practices. assessmentpractices. quality of classroom assessment practices. * ASSESSMENTQUALITY2: Ensuringeffectiveusesofclassroomassessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required tobe disseminatedto some key required to be disseminated to all key stakeholders. stakeholders. stakeholders. * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessmenttosupportstudentlearning. classroomassessmenttosupportstudent classroomassessmenttosupportstudent classroomassessmenttosupportstudent learning.2 learning, excluding its use as an input for learning, including its use as an input for *externalexaminationresults. externalexaminationresults.    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 16 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ClassroomAssessmentǣ‡˜‡Ž‘’‡–Ž‡˜‡Ž”ƒ–‹‰Œ—•–‹ˆ‹…ƒ–‹‘•   1.TherearenoclassroomassessmentguidelinesatthefederallevelexceptindirectlythroughsupporttotheRegionalEducationBureaus(REBs)forcurriculum implementationandteacherdevelopmentactivitiesundertheframeworkoftheschoolimprovementprogram.  2.Useofclassroomassessmentinformationisrequired,butlimited.                          SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 17 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009          ETHIOPIA Examinations   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 18 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ENABLINGCONTEXT Overallframeworkofpolicies,leadership,organizationalstructures,fiscalandhumanresourcesinwhichassessmentactivitytakesplaceinacountryorsystem andtheextenttowhichthatframeworkisconduciveto,orsupportiveof,theassessmentactivity. LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT1: Settingclearpolicies No standardized examination has taken The standardized examination has been The examination is a stableprogram that This option does not apply to this place. operatingonanirregularbasis. hasbeenoperatingregularly.1 dimension * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this authorizestheexamination. document that authorizes the authorizestheexamination.2 dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. thepublic. public.3 dimension. * This option does not apply to this This option does not apply to this Thepolicydocumentaddressessomekey The policy document addresses all key dimension. dimension. aspectsoftheexamination.4 aspectsoftheexamination. * ENABLINGCONTEXT2: Havingstrongleadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the theexaminationorareindifferenttoit. examination.  examination. examination.  * There are no attempts to improve the This option does not apply to this There are independent attempts to There are coordinated attempts to examinationbystakeholdergroups. dimension.  improve the examination by stakeholder improve the examination by stakeholder  groups. groups. * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this notwelcomedbytheleadershipincharge dimension. generally welcomed by the leadership in dimension. oftheexamination chargeoftheexamination.5 *                   (CONTINUED) SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 19 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT3: Havingregularfunding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this examination. theexamination. examination.6 dimension. * This option does not apply to this Funding covers some core examination Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. processingorreporting. processingandreporting.7 * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and 8 dimension. development.  dimension. development. * ENABLINGCONTEXT4: Havingstrongorganizationalstructures Theexaminationofficedoesnotexistoris The examination office is newly The examination office is a stable This option does not apply to this newlyestablished. established. organization.9 dimension. * Theexaminationofficeisnotaccountable This option does not apply to this The examination office is accountable to This option does not apply to this toanexternalboardoragency. dimension. anexternalboardoragency.10 dimension. * Examinationresultsarenotrecognizedby Examination results are recognized by Examination results are recognized by Examination results are recognized by anycertificationorselectionsystem. certification or selection system in the one certification or selection system in two or more certification or selection country. anothercountry. systeminanothercountry.11 * Theexaminationofficedoesnothavethe The examination office has some of the The examination office has all of the Theexaminationofficehasstateoftheart required facilities to carry out the required facilities to carry out the required facilities to carry out the facilitiestocarryouttheexamination. examination. examination.12 examination.           *        (CONTINUED)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 20 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT5: Havingeffectivehumanresources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment examination,issuesarepervasive. effectively,withminimalissues.13 effectively,withnoissues. * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on examination. examination.14 theexamination. *    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 21 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  SYSTEMALIGNMENT Degreetowhichtheassessmentiscoherentwithothercomponentsoftheeducationsystem. LATENT EMERGING ESTABLISHED ADVANCED SYSTEMALIGNMENT1: Aligningexaminationswithlearninggoalsandopportunitiestolearn It is not clear what the examination This option does not apply to this Thereisaclearunderstandingofwhatthe This option does not apply to this measures. dimension. examinationmeasures. dimension. * What the examination measures is This option does not apply to this What is measured by the examination is This option does not apply to this questionedbysomestakeholdergroups. dimension. largelyacceptedbystakeholdergroups.15 dimension. * Materialtopreparefortheexaminationis Thereissomematerialtoprepareforthe There is comprehensive material to There is comprehensive material to minimal and it is only accessible to very examination that is accessible to some prepare for the examination that is prepare for the examination that is fewstudents. students. accessibletomoststudents. accessibletoallstudents.16 * SYSTEMALIGNMENT2: Providingteacherswithopportunitiestolearnabouttheexamination There are no courses or workshops on There are no upͲtoͲdate courses or ThereareupͲtoͲdatevoluntarycoursesor ThereareupͲtoͲdatecompulsorycourses examinationsavailabletoteachers. workshops on examinations available to workshops on examinations available to or workshops on examinations for teachers. teachers. teachers. * Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in most examinationͲrelatedtasks. examinationͲrelatedtasks. examinationͲrelatedtasks.17 examinationͲrelatedtasks. *    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 22 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ASSESSMENTQUALITY Degreetowhichtheassessmentmeetsqualitystandards,isfair,andisusedinaneffectiveway. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENTQUALITY1: Ensuringquality There is no technical report or other There is some documentation on the Thereisacomprehensivetechnicalreport There is a comprehensive, high quality documentation. examination, but it is not in a formal butwithrestrictedcirculation. technical report available to the general reportformat.18 public. * There are no mechanisms in place to This option does not apply to this Therearelimitedsystematicmechanisms There are varied and systematic ensurethequalityoftheexamination. dimension. in place to ensure the quality of the mechanismsinplacetoensurethequality examination.19 oftheexamination. * ASSESSMENTQUALITY2: Ensuringfairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the examinationprocessishigh. examinationprocessismoderate.20 examinationprocessislow. examinationprocessismarginal. * The examination results lack credibility The examination results are credible for The examination results are credible for This option does not apply to this forallstakeholdergroups. somestakeholdergroups. allstakeholdergroups.21 dimension. * The majority of the students (over 50 Asignificantproportionofstudents(10%Ͳ A small proportion of students (less than All students can take the examination; percent) may not take the examination 50%) may not take the examination 10%) may not take the examination there are no language, gender or other because of language, gender, or other because of language, gender, or other because of language, gender, or other equivalentbarriers.22 equivalentbarriers. equivalentbarriers. equivalentbarriers. *                   (CONTINUED)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 23 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENTQUALITY3: Usingexaminationinformationinafairway Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all properwaybyallstakeholdergroups. stakeholdergroupsinaproperway. stakeholdergroupsinaproperway. stakeholdergroupsinaproperway. * Studentnamesandresultsarepublic. This option does not apply to this Students’resultsareconfidential. This option does not apply to this dimension. dimension. * ASSESSMENTQUALITY4: Ensuringpositiveconsequencesoftheexamination Therearenooptionsforstudentswhodo There are very limited options for Therearesomeoptionsforstudentswho There is a variety of options for students not perform well on the examination, or studentswhodonotperformwellonthe do not perform well on the who do not perform well on the students must leave the education examination. examination.23 examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination.24 examination. examination. *          SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 24 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   Examinationsǣ‡˜‡Ž‘’‡–Ž‡˜‡Ž”ƒ–‹‰Œ—•–‹ˆ‹…ƒ–‹‘•  1,2,3,4.Thegrades8,10and12examinationshavebeenoperatingformanyyears.Theschoolleavingcertificateexaminationhasbeeninplacesince1950. The1994EducationandTrainingPolicyprovidesforexaminationsatthreelevels(Grades8,10,and12). TheaboveͲreferencedpolicydocumentisavailabletothepublic.(Butthereisnospecificexaminationspolicydocument). Theabovedocumentdeterminesthegradesinwhichtheexaminationsshouldbegiven,andthecriteriaforgrading.  5.Theleadershipemploysadditionalpersonnel,investsinmoderntechnologyandworkstomaketheexaminationentitysemiͲautonomous.  6.Everyexamineeisrequiredtopayaregistrationfeeinordertotaketheexamination.Thefundforexaminationscomesfromfeescollectedfromtheexaminees.  7.In2009,fundingforGrades10and12examinationswassufficienttocovercoreactivities.  8.BasedonDr.Gregory's2011draftreportonthestatusofnationalexaminationsactivityinEthiopia,itseemsthattheagencyinchargeoftheGrades10and12 examinationshasnothadsufficientfundsoverthepastfewyearstocarryoutR&Dorupgradeequipmentorexaminationtechniques.  9. The examination office has been in existence since 1950 in one form or another. It is currently undergoing reorganization in accordance with the BPR study undertakenbytheMoEin2009.  10.TheexaminationagencyisaccountabletotheMoE.  11.Thereisrecognitionbymanysystemsinothercountries.  12.BasedonDr.Gregory's2011draftreportonthestatusofnationalexaminationsactivityinEthiopia,itseemsthattheagencyinchargeoftheGrades10and 12examinationsdoesnothavealloftherequiredfacilitiesforcarryingoutsecureandhighͲqualityexaminations,andthatthishasbeenthecaseforseveralyears.  13.Staffingseemstobeadequateforcoreactivities.Butthereareissuesintermsoftheabilitytoattracttherightstaffandretainthem.  14. In 2009, teachers, university lecturers and subject matter experts were involved at various stages of the examinations process. Education and Psychology departmentsinuniversitiesandcollegesthroughoutthecountryoffercoursesthatpreparestudentsforworkinexaminations.  15.Complaintsaboutexamresultsareveryminimal.  16.Theexaminationsarebasedonthecurriculummaterialswithwhichthestudentsaretaught,whichareprovidedbytheschoolstothestudents.  17.Withregardtonationalexaminations,teachersareinvolvedinexamwritingandexamadministrationtasks. SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 25 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   18.BasedonDr.Gregory's2011draftreportonthestatusofnationalexaminationsactivityinEthiopia,itseemsthattherehasnotbeenatraditionofproducing aformaltechnicalreportontheGrades10and12examinations,butthatsomeinformaldocumentationisavailable.  19. Based on Dr. Gregory's 2011 draft report on the status of national examinations activity in Ethiopia, it seems that there has not been a strong tradition of qualitycontrolorassurancemechanismsfortheGrades10and12examinations.  20. There have been allegations of cheating or other inappropriate behavior in relation to the national examinations. But these allegations were not high when seeninrelationtothetotalnumberofexamtakers.  21.Examresultsarecredibleforallgroups.Thereisnoevidenceshowingotherwise.  22.Therearenobarriersforstudents.  23.StudentswhodonotjointhepreparatoryprogrammemayjoinTVETprogrammesortaketheexaminationsagainandimprovetheirgrades.Studentswhofail (scorebelowthepassmark)maytakeshortskillstrainingtojointhelabormarketorretaketheexam.  24.BasedonDr.Gregory's2011draftreportonthestatusofnationalexaminationsactivityinEthiopia,itseemsthattherearenomechanismsinplacetomonitor theconsequencesoftheseexaminations.             SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 26 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009        ETHIOPIA National(orSystemǦLevel)LargeǦScaleAssessment(NLSA)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 27 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ENABLINGCONTEXT Overallframeworkofpolicies,leadership,organizationalstructures,fiscalandhumanresourcesinwhichNLSAactivitytakesplaceinacountryorsystemandthe extenttowhichthatframeworkisconduciveto,orsupportiveof,theNLSAactivity. LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT1: SettingclearpoliciesforNLSA NoNLSAexercisehastakenplace. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this irregularbasis. beenoperatingregularly.1 dimension. * Thereisnopolicydocumentpertainingto There is an informal or draft policy There is a formal policy document that This option does not apply to this NLSA.2 documentthatauthorizestheNLSA. authorizestheNLSA. dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. thepublic.3 public. dimension. * ThereisnoplanforNLSAactivity. This option does not apply to this Thereisageneralunderstandingthatthe There is a written NLSA plan for the dimension. NLSAwilltakeplace.4 comingyears. * ENABLINGCONTEXT2: HavingstrongpublicengagementforNLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the AllstakeholdergroupssupporttheNLSA.5 theNLSAorareindifferenttoit. NLSA. NLSA. *                   (CONTINUED)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 28 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT3: HavingregularfundingforNLSA ThereisnofundingallocatedtotheNLSA. Thereisirregularfundingallocatedtothe There is regular funding allocated to the This option does not apply to this NLSA.6 NLSA. dimension. * This option does not apply to this FundingcoverssomecoreNLSAactivities: Funding covers all core NLSA activities: This option does not apply to this dimension. design, administration, analysis and design, administration, analysis and dimension. reporting.7 reporting. * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. developmentactivities.8 dimension. developmentactivities. * ENABLINGCONTEXT4: HavingstrongorganizationalstructuresforNLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this team.5 groupofpeople.9 institutionorunit. dimension. * This option does not apply to this Political considerations regularly hamper Political considerations sometimes Political considerations never hamper dimension. technicalconsiderations. hampertechnicalconsiderations.10 technicalconsiderations. * This option does not apply to this The NLSA office is not accountable to a TheNLSAofficeisaccountabletoaclearly This option does not apply to this dimension. clearlyrecognizedbody.11 recognizedbody. dimension. *                   (CONTINUED)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 29 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT5: HavingeffectivehumanresourcesforNLSA There is no staff allocated for running an TheNLSAofficeisinadequatelystaffedto The NLSA office is adequately staffed to The NLSA office is adequately staffed to NLSA. effectivelycarryouttheassessment.12 carry out the NLSA effectively, with carry out the NLSA effectively, with no minimalissues. issues. * The country does not offer opportunities This option does not apply to this Thecountryofferssomeopportunitiesto The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the opportunities to prepare individuals for NLSA. NLSA.13 workontheNLSA. *    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 30 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  SYSTEMALIGNMENT DegreetowhichtheNLSAiscoherentwithothercomponentsoftheeducationsystem. LATENT EMERGING ESTABLISHED ADVANCED SYSTEMALIGNMENT1: AligningtheNLSAwithlearninggoals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against Thisoptiondoesnotapplytothisdimension. curriculumorlearningstandards. dimension. curriculumorlearningstandards.14 * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned by What the NLSA measures is largely accepted questionedbystakeholdergroups. dimension. somestakeholdergroups. bystakeholdergroups.15 * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSAtoensurethatitmeasureswhatitis dimension. measureswhatitissupposedtomeasure. intendedtomeasure.16 intendedtomeasure. * SYSTEMALIGNMENT2: ProvidingteacherswithopportunitiestolearnabouttheNLSA Therearenocoursesorworkshopsonthe There are occasional courses or Therearesomecoursesorworkshopson There are widely available high quality NLSA.17 workshopsontheNLSA. theNLSAofferedonaregularbasis. courses or workshops on the NLSA offeredonaregularbasis. *    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 31 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ASSESSMENTQUALITY DegreetowhichtheNLSAmeetstechnicalstandards,isfair,andisusedinaneffectiveway. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENTQUALITY1: EnsuringthequalityoftheNLSA No options are offered to include all This option does not apply to this Atleastoneoptionisofferedtoincludeall Differentoptionsareofferedtoincludeall groupsofstudentsintheNLSA. dimension. groupsofstudentsintheNLSA.18 groupsofstudentsintheNLSA. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in ensurethequalityoftheNLSA. dimension. ensurethequalityoftheNLSA.19 placetoensurethequalityoftheNLSA. * There is no technical report or other There is some documentation about the Thereisacomprehensivetechnicalreport There is a comprehensive, high quality documentationabouttheNLSA. technicalaspectsoftheNLSA,butitisnot butwithrestrictedcirculation.16 technical report available to the general inaformalreportformat.20 public. * ASSESSMENTQUALITY2: EnsuringeffectiveusesoftheNLSA NLSAresultsarenotdisseminated. NLSAresultsarepoorlydisseminated.21 NLSA results are disseminated in an This option does not apply to this effectiveway. dimension. * NLSAinformationisnotusedorisusedin This option does not apply to this NLSA results are used by some NLSA information is used by all ways inconsistent with the purposes or dimension. stakeholder groups in a way that is stakeholder groups in a way that is the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the assessment.22 assessment. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in monitortheconsequencesoftheNLSA.23 dimension. monitortheconsequencesoftheNLSA. placetomonitortheconsequencesofthe NLSA. *                      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 32 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  National(orSystemǦLevel)LargeScaleAssessment(NLSA)ǣ‡˜‡Ž‘’‡–Ž‡˜‡Ž”ƒ–‹‰Œ—•–‹ˆ‹…ƒ–‹‘•  1.In2009,theNLSAprograminEthiopiawasoperatingonaregularbasis.ButfundingforthevariousNLSAexerciseswasbeingsourcedfromdifferentdonors, andtheassessmentsweretakingplaceroughlyevery3to4years.  2.AtthestartofREAD(2009),Ethiopiadidnothaveanykind(formal,informal,draft)ofpolicydocumentonNLSAactivity.  3.Therewasnopolicydocumentavailablein2009.  4.AlthoughtherewasnoformalpolicydocumentunderpinningtheNLSAin2009,therewasageneralunderstandingthattheNLSAwouldtakeplaceevery3to 4years.  5.Basedonourinformation,thereisnooppositiontoNLSA.  6.NLSAactivityispartiallyfundedbytheMOEandpartiallybydonors.Thefundingisstilladhoc.  7.FundinghastendedtocoveronlybasicaspectsofNLSAactivities.Sometimes,therehasbeeninsufficientfundingtocoverallcoreactivities.  8.FundinghasprimarilyfocusedonsupportingtheactualcarryingoutofNLSAactivitiesandnotonR&Dorsecondaryanalysis.  9.In2009, theNLSA teamwas comprisedofasmallnumberof staff(4),someofwhomhadnobackgroundor traininginNLSA. Therewasnopermanentunit, andaninstitutionalhomewasstillbeingworkedout.  10.Noprecedenceofpoliticalconsiderationshamperingtechnicalconsiderations.  11.In2009,theNLSAofficewasnotaccountabletoaclearlyrecognizedbodyasitwasintransition.  12.In2009,theNLSAofficedidnothaveenoughstafftoeffectivelycarryoutNLSAactivities.  13.ThereweresomeassessmentͲrelatedcoursesofferedbyAddisAbabaUniversity  14.NLSAwasalignedwithexistingcurriculumandstandards.  15.MOEandotherstakeholdershaveacceptedtheNLSA  16.In2009,thereweresomeproceduresinplaceforreviewingthealignmentoftheNLSAtestwiththeconstructs/contentitwasintendedtomeasure,butthese procedureswerenotformalizedorstandardized. SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 33 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   17.TheonlycoursesorworkshopsassociatedwithpreviousNLSAexerciseshavebeenforpolicymakersandhighͲleveleducators,andnotforclassroomteachers.  18.TheNLSAistranslatedintothelanguageofinstructionforeachregion.  19. In 2009, there were some procedures in place for reviewing the alignment of the NLSA test with the constructs/content it was intended to measure. This wouldallowustosaythattherewere'somemechanismsinplacetoensurethequalityoftheNLSA.'  20.In2009,noformaltechnicalreportswereavailablefortheNLSA.  21.In2009,NLSAresultswerenotbeingwidelydisseminatedtokeystakeholders.Fewcopiesofthereportwereavailable.  22.In2009,theNLSAresultswere,toacertainextent,usedforcurriculumdevelopmentandteachertraining.  23.In2009,therewerenomechanismsinplacetomonitortheconsequencesoftheNLSA.                SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 34 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009        ETHIOPIA InternationalLargeǦScaleAssessment(ILSA)   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 35 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ENABLINGCONTEXT Overallframeworkofpolicies,leadership,organizationalstructures,fiscalandhumanresourcesinwhichILSAtakesplaceinacountryorsystemandtheextentto whichthatframeworkisconduciveto,orsupportiveof,ILSAactivity. LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT1: SettingclearpoliciesforILSA The country/system has not participated This option does not apply to this Thecountry/systemhasparticipatedinat The country/system has participated in inanILSAinthelast10years.1 dimension. leastoneILSAinthelast10years. twoormoreILSAinthelast10years. * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concretestepstoparticipateinanILSAin dimension. stepstoparticipateinatleastoneILSAin dimension. thenext5years.2 thenext5years. * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this addressesparticipationinILSA. document that addresses participation in addressesparticipationinILSA. dimension. ILSA. This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. thepublic. public. dimension. ENABLINGCONTEXT2: HavingregularfundingforILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at Thereisregularfundingapprovedbylaw, ILSA. donors. discretion. decreeornorm. This option does not apply to this Fundingcoverssomecoreactivitiesofthe Funding covers all core activities of the Funding covers all core activities of the dimension. ILSA. ILSA. ILSA. Funding does not cover research and This option does not apply to this This option does not apply to this Funding covers research and developmentactivities. dimension. dimension. developmentactivities.                   (CONTINUED)  SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 36 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  LATENT EMERGING ESTABLISHED ADVANCED ENABLINGCONTEXT3: HavingeffectivehumanresourcesforILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. activities. activities. activities. This option does not apply to this Thenational/systemcoordinatororother Thenational/systemcoordinatorisfluent This option does not apply to this dimension. designated team member may not be inthelanguageoftheassessment. dimension. fluentinthelanguageoftheassessment. This option does not apply to this The ILSA office is inadequately staffed or The ILSA office is adequately staffed or The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, effectively. withminimalissues. withnoissues.                      SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 37 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  SYSTEMALIGNMENT DegreetowhichtheILSAmeetstechnicalqualitystandards,isfair,andisusedinaneffectiveway. LATENT EMERGING ESTABLISHED ADVANCED SYSTEMALIGNMENT1: ProvidingopportunitiestolearnaboutILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this internationalworkshopsormeetings. internationalworkshopsormeetings. workshopsormeetings. dimension. The country/system offers no This option does not apply to this The country/system offers some The country/system offers a wide range opportunitiestolearnaboutILSA. dimension. opportunitiestolearnaboutILSA. ofopportunitiestolearnaboutILSA. This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition teammembersonly. to the country's/system's ILSA team members.    SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 38 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  ASSESSMENTQUALITY DegreetowhichtheILSAmeetstechnicalqualitystandards,isfair,andisusedinaneffectiveway. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENTQUALITY1: EnsuringthequalityofILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical The country/system met all technical published. standards to have its data presented standards required to have its data standards required to have its data beneath the main display of the presented in the main displays of the presented in the main displays of the internationalreportorinanannex. internationalreport. internationalreport. The country/system has not contributed This option does not apply to this This option does not apply to this The country/system has contributed new newknowledgeonILSA. dimension. dimension. knowledgeonILSA. ASSESSMENTQUALITY2: EnsuringeffectiveusesofILSA If any, country/systemͲspecific results Country/systemͲspecific results and Country/systemͲspecific results and Country/systemͲspecific results and and information are not disseminated in information are disseminated irregularly informationareregularlydisseminatedin information are regularly and widely thecountry/system. inthecountry/system. thecountry/system. disseminatedinthecountry/system. Products to provide feedback to schools This option does not apply to this Products to provide feedback to schools Products to provide feedback to schools and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are notmadeavailable. sometimesmadeavailable. systematicallymadeavailable. There is no media coverage of the ILSA There is limited media coverage of the ThereissomemediacoverageoftheILSA There is wide media coverage of the ILSA results. ILSAresults. results. results. If any, country/systemͲspecific results ResultsfromtheILSAareusedinalimited Results from the ILSA are used in some ResultsfromtheILSAareusedinavariety and information from the ILSA are not way to inform decision making in the ways to inform decision making in the of ways to inform decision making in the used to inform decision making in the country/system. country/system. country/system. country/system. ItisnotclearthatdecisionsbasedonILSA This option does not apply to this This option does not apply to this Decisions based on the ILSA results have results have had a positive impact on dimension. dimension. had a positive impact on students' students'achievementlevels. achievementlevels.     SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 39 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  InternationalLargeScaleAssessment(ILSA)ǣ‡˜‡Ž‘’‡–Ž‡˜‡Ž”ƒ–‹‰Œ—•–‹ˆ‹…ƒ–‹‘•  1.Asof2009(thereferenceyearforthisbaselinerating),EthiopiahadyettoparticipateinanILSAexercise.Nordidithaveconcreteplanstoparticipateinany upcomingexercises.Theoverallratingforthisassessmentarea,therefore,isLatent.  2.ThereisaninformalplanbytheMOEtoparticipateinanILSAinthecomingtwoyears.  EthiopiahasnotparticipatedinaninternationallargeͲscaleassessmentandhasnottakenconcretestepstoparticipateinaninternationallargeͲscale assessmentinthenextfiveyears.Therefore,thereisasufficientamountofinformationtodeterminethedevelopmentleveloftheILSAsysteminEthiopia,and theremainderoftheILSArubricisintentionallyleftblank. SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 40 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009  Netherlands Organisation for International Cooperation Acknowledgements in Higher Education (NUFFIC). 2012. Country Module – This report was prepared by The World Bank SABERͲ Ethiopia.TheHague:NUFFIC. StudentAssessmentteamincollaborationwithRajendra  Dhoj Joshi, World Bank Senior Education Specialist and United Nations Educational, Scientific, and Cultural TaskTeamLeaderforeducationprojectsinEthiopia. Organization (UNESCO) Institute for Statistics. Ethiopia  Country Indicator Data. Montreal, QC: UNESCO. Data retrieved from http://stats.uis.unesco.org/unesco on References March12,2013.  Clarke, M. 2012. “What Matters Most for Student WorldBank.2005.“EducationinEthiopia:Strengthening AssessmentSystems:AFrameworkPaper.”READ/SABER the Foundation for Sustainable Progress.” Washington, WorkingPaperSeries.Washington,DC:WorldBank. DC:WorldBank.   Matters, G., and J. Pitman. 2010. “Review of National ———. World Bank Development Indicators. Ethiopia LearningAssessmentActivityinEthiopia:StatusandNext Country Indicator Data. Washington, DC: World Bank. Steps.” Melbourne: ACER. Report prepared under the Data retrieved Russia Education Aid for Development Trust Fund fromhttp://databank.worldbank.org/dataon March 12, program. 2013.   SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 41 ETHIOPIAۣSTUDENTASSESSMENT  SABERCOUNTRYREPORT|2009   www.worldbank.org/education/saber  The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in thisworkdonotnecessarilyreflecttheviewsofTheWorldBank,itsBoardofExecutiveDirectors,orthegovernmentstheyrepresent.TheWorld Bankdoesnotguaranteetheaccuracyofthedataincludedinthiswork.Theboundaries,colors,denominations,andotherinformationshown onanymapinthisworkdonotimplyanyjudgmentonthepartofTheWorldBankconcerningthelegalstatusofanyterritoryortheendorsement oracceptanceofsuchboundaries. SYSTEMSAPPROACHFORBETTEREDUCATIONRESULTS 42