81091 SABER – SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS STUDENT ASSESSMENT 10 Disseminating and Using Student Assessment Information in Russia Yulia Tyumeneva 10ȱ SABER – SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS STUDENT ASSESSMENT DisseminatingȱandȱUsingȱ StudentȱAssessmentȱ InformationȱinȱRussiaȱ YuliaȱTyumenevaȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ȱ ©ȱ2013ȱTheȱInternationalȱBankȱforȱReconstructionȱandȱDevelopmentȱ/ȱ ȱ TheȱWorldȱBankȱ 1818ȱHȱStreetȱNWȱ WashingtonȱDCȱ20433ȱ Telephone:ȱ202Ȭ473Ȭ1000ȱ Internet:ȱwww.worldbank.orgȱ ȱ 1ȱ2ȱ3ȱ4ȱ 15ȱ14ȱ13ȱ12ȱ ȱ ȱ ThisȱworkȱisȱaȱproductȱofȱtheȱstaffȱofȱTheȱWorldȱBankȱwithȱexternalȱ contributions.ȱTheȱfindings,ȱinterpretations,ȱandȱconclusionsȱexpressedȱinȱ thisȱworkȱdoȱnotȱnecessarilyȱreflectȱtheȱviewsȱofȱTheȱWorldȱBank,ȱitsȱ BoardȱofȱExecutiveȱDirectors,ȱorȱtheȱgovernmentsȱtheyȱrepresent.ȱ TheȱWorldȱBankȱdoesȱnotȱguaranteeȱtheȱaccuracyȱofȱtheȱdataȱincludedȱ inȱthisȱwork.ȱTheȱboundaries,ȱcolors,ȱdenominations,ȱandȱotherȱ informationȱshownȱonȱanyȱmapȱinȱthisȱworkȱdoȱnotȱimplyȱanyȱjudgmentȱ onȱtheȱpartȱofȱTheȱWorldȱBankȱconcerningȱtheȱlegalȱstatusȱofȱanyȱterritoryȱ orȱtheȱendorsementȱorȱacceptanceȱofȱsuchȱboundaries.ȱ ȱ RightsȱandȱPermissionsȱ Theȱmaterialȱinȱthisȱworkȱisȱsubjectȱtoȱcopyright.ȱBecauseȱTheȱWorldȱBankȱ encouragesȱdisseminationȱofȱitsȱknowledge,ȱthisȱworkȱmayȱbeȱ reproduced,ȱinȱwholeȱorȱinȱpart,ȱforȱnoncommercialȱpurposesȱasȱlongȱasȱ fullȱattributionȱtoȱthisȱworkȱisȱgiven.ȱ Anyȱqueriesȱonȱrightsȱandȱlicenses,ȱincludingȱsubsidiaryȱrights,ȱ shouldȱbeȱaddressedȱtoȱtheȱOfficeȱofȱtheȱPublisher,ȱTheȱWorldȱBank,ȱ1818ȱ HȱStreetȱNW,ȱWashington,ȱDCȱ20433,ȱUSA;ȱfax:ȱ202Ȭ522Ȭ2422;ȱeȬmail:ȱ pubrights@worldbank.org.ȱ ȱ Coverȱdesign:ȱPatriciaȱHord.GraphikȱDesign,ȱAlexandria,ȱVAȱ ȱ ȱ ȱ Contentsȱ Abstractȱ...............................................................................................................ȱv AboutȱtheȱAuthorȱ...........................................................................................ȱvii Acknowledgmentsȱ ix ............................................................................................ȱ ExecutiveȱSummaryȱ.........................................................................................ȱ xi Introductionȱ ........................................................................................................ȱ1 TheȱUnifiedȱStateȱExaminationȱ........................................................................ȱ3 InternationalȱLargeȬScaleȱAssessmentsȱ........................................................ȱ11 LessonsȱLearnedȱ..............................................................................................ȱ14 Bibliographyȱ.....................................................................................................ȱ20 ȱ Figuresȱ Figureȱ1.ȱRegionalȱAnalysisȱofȱtheȱ2011ȱUnifiedȱStateȱExaminationȱ Results,ȱChuvashȱRepublicȱ....................................................................ȱ10 ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ iiiȱ ȱ ȱ ȱ ȱ Abstractȱ Thisȱ caseȱ studyȱ examinesȱ howȱ theȱ Russianȱ Federationȱ disseminatesȱ andȱ usesȱinformationȱfromȱitsȱstudentȱassessmentȱsystem,ȱdrawingȱlessonsȱforȱ otherȱ countriesȱ seekingȱ toȱ moreȱ effectivelyȱ useȱ theirȱ ownȱ assessmentȱ data.ȱ Russia’sȱ Unifiedȱ Stateȱ Examinationȱ (USE)ȱ isȱ primarilyȱ usedȱ forȱ studentȱ selectionȱ andȱ certificationȱ purposes,ȱ butȱ aȱ varietyȱ ofȱ otherȱ usesȱ haveȱ beenȱ attachedȱ toȱ it,ȱ includingȱ informingȱ pedagogy,ȱ ensuringȱ accountability,ȱandȱmonitoringȱeducationȱquality.ȱThisȱvarietyȱofȱusesȱhasȱ hadȱ bothȱ positiveȱ andȱ negativeȱ consequencesȱ forȱ theȱ schoolȱ system.ȱ Informationȱ fromȱ internationalȱ largeȬscaleȱ assessmentsȱ (e.g.,ȱ PISA,ȱ TIMSS,ȱ andȱ PIRLS)1ȱ hasȱ beenȱ widelyȱ usedȱ toȱ introduceȱ reformsȱ inȱ theȱ schoolȱ system.ȱ Factorsȱ thatȱ haveȱ affectedȱ theȱ differentialȱ useȱ ofȱ USEȱ andȱ internationalȱ largeȬscaleȱ assessmentȱ dataȱ include,ȱ amongȱ others,ȱ theȱ purposeȱofȱaȱgivenȱassessment,ȱitsȱdesignȱfeaturesȱandȱlevelȱofȱcredibility,ȱ andȱaccessȱtoȱtheȱassessmentȱdatabase.ȱ ȱ ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ PISAȱ–ȱProgrammeȱforȱInternationalȱStudentȱAssessment;ȱTIMSSȱ–ȱTrendsȱinȱInternationalȱ 1 MathematicsȱandȱScienceȱStudy;ȱPIRLSȱ–ȱProgressȱinȱInternationalȱReadingȱLiteracyȱStudy.ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ vȱ ȱ ȱ ȱ AboutȱtheȱAuthorȱ Yuliaȱ Tyumenevaȱ headsȱ theȱ Educationalȱ andȱ Psychologicalȱ Measurementȱ master’sȱ degreeȱ programȱ atȱ theȱ Higherȱ Schoolȱ ofȱ Economics,ȱ aȱ nationalȱ researchȱ universityȱ inȱ Russia,ȱ whereȱ sheȱ isȱ aȱ lecturerȱ andȱ researcher.ȱ Sheȱ isȱ anȱ expertȱ inȱ developmentalȱ psychologyȱ andȱ educationalȱ assessment,ȱ withȱ aȱ deepȱ knowledgeȱ ofȱ nationalȱ andȱ regionalȱassessmentȱsystems.ȱHerȱexperienceȱincludesȱsecondaryȱanalysisȱ ofȱ nationalȱ andȱ internationalȱ largeȬscaleȱ assessmentȱ results,ȱ asȱ wellȱ designingȱ questionnairesȱ andȱ surveysȱ inȱ theȱ fieldȱ ofȱ education.ȱ Tyumenva’sȱ currentȱ researchȱ projectȱ isȱ entitledȱ “Analysesȱ ofȱ Factorsȱ Connectedȱ withȱ Studentȱ Performanceȱ inȱ Internationalȱ Comparisonȱ Research.”ȱ Sheȱ hasȱ aȱ Ph.D.ȱ inȱ psychologyȱ fromȱ theȱ Russianȱ Academyȱ ofȱ Education.ȱ ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ viiȱ ȱ ȱ ȱ ȱ Acknowledgmentsȱ Forȱ theirȱ adviceȱ andȱ valuableȱ comments,ȱ theȱ authorȱ wouldȱ likeȱ toȱ thankȱ Victorȱ Bolotov,ȱ viceȱ presidentȱ ofȱ theȱ Russianȱ Academyȱ ofȱ Education;ȱ Galinaȱ Kovaleva,ȱ head,ȱ Centerȱ forȱ Assessmentȱ ofȱ Educationalȱ Quality,ȱ RussianȱAcademyȱofȱEducation,ȱandȱnationalȱprojectȱmanagerȱforȱTIMSS,ȱ PIRLS,ȱ andȱ PISA;ȱ andȱ Isakȱ Frumin,ȱ academicȱ supervisor,ȱ Graduateȱ Schoolȱ ofȱ Education,ȱ Higherȱ Schoolȱ ofȱ Economics.ȱ Thanksȱ alsoȱ areȱ extendedȱ toȱ Sergeyȱ Bochenkov,ȱ head,ȱ Departmentȱ forȱ Assessmentȱ ofȱ Educationalȱ Quality,ȱ Chuvashȱ Republic,ȱ forȱ makingȱ availableȱ theȱ dataȱ thatȱwereȱusedȱinȱthisȱpaper.ȱ ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ ixȱ ȱ ȱ ȱ ȱ ȱ ExecutiveȱSummaryȱ Theȱpurposeȱofȱthisȱcaseȱstudyȱisȱtwofold:ȱ(1)ȱtoȱpresentȱhowȱtheȱRussianȱ Federationȱ disseminatesȱ andȱ usesȱ studentȱ assessmentȱ information,ȱ andȱ (2)ȱ toȱ drawȱ lessonsȱ forȱ otherȱ countriesȱ aimingȱ toȱ improveȱ theȱ useȱ ofȱ assessmentȱ informationȱ and,ȱ therefore,ȱ theȱ effectivenessȱ ofȱ theirȱ assessmentȱsystems.ȱTheȱpaperȱexaminesȱtwoȱmainȱtypesȱofȱassessments:ȱ theȱ Unifiedȱ Stateȱ Examinationȱ (USE)ȱ ofȱ theȱ Russianȱ Federationȱ andȱ wellȬknownȱinternationalȱlargeȬscaleȱassessments.ȱ ȱ Theȱ USEȱ isȱ theȱ mostȱ importantȱ assessmentȱ programȱ inȱ Russia.ȱ Theȱ highȬstakesȱ nationalȱ examinationȱ isȱ usedȱ primarilyȱ toȱ certifyȱ secondaryȱ schoolȱ studentsȱ andȱ toȱselectȱ studentsȱ intoȱ tertiaryȱ education.ȱ TheȱUSEȱ isȱ alsoȱusedȱforȱaȱvarietyȱofȱotherȱpurposes,ȱincludingȱinformingȱpedagogy;ȱ holdingȱ regions,ȱ schools,ȱ andȱ teachersȱ accountable;ȱ andȱ monitoringȱ educationalȱ quality.ȱ Whileȱ usingȱ theȱ sameȱ instrumentȱ forȱ aȱ varietyȱ ofȱ purposesȱ mayȱ seemȱ efficient,ȱ itȱ alsoȱ canȱ createȱ problems.ȱ Forȱ example,ȱ sinceȱRussiaȱdoesȱnotȱhaveȱaȱnationalȱlargeȬscaleȱassessmentȱprogramȱforȱ systemȱ monitoringȱ andȱ accountabilityȱ purposes,ȱ theȱ USEȱ hasȱ endedȱ upȱ beingȱusedȱtoȱfillȱthisȱgap,ȱevenȱthoughȱitȱisȱnotȱactuallyȱdesignedȱtoȱyieldȱ theȱkindȱofȱinformationȱrequiredȱtoȱcarryȱoutȱsuchȱsystemȱmonitoringȱandȱ accountability.ȱ Internationalȱ largeȬscaleȱ assessmentsȱ alsoȱ playȱ anȱ importantȱ roleȱ inȱ theȱ Russianȱ educationȱ system.ȱ Sinceȱ theȱ fallȱ ofȱ theȱ Sovietȱ Union,ȱ Russiaȱ hasȱ consistentlyȱ participatedȱ inȱ theȱ mainȱ internationalȱ assessmentȱ programs,ȱ mostȱ notably,ȱ PISA,ȱ TIMSS,ȱ andȱ PIRLS.2ȱ Theȱ resultsȱ ofȱ theseȱ assessmentsȱ haveȱ createdȱ awarenessȱ ofȱ theȱ needȱ toȱ reformȱ theȱ schoolȱ systemȱ inȱ Russia.ȱ Accessȱ toȱ theȱ assessmentȱ databasesȱ hasȱ allowedȱ forȱ policyȱ researchȱ andȱ theȱ formulationȱ ofȱ actionȱ plansȱ toȱ introduceȱ suchȱ reforms.ȱ Internationalȱ assessmentsȱ haveȱ thusȱ servedȱ asȱ aȱ referenceȱ forȱ newȱ nationalȱ learningȱ standards,ȱ curricula,ȱ andȱ textbooks.ȱ Thereȱ isȱ aȱ generalȱ consensusȱ thatȱ theȱ informationȱ generatedȱ byȱ theseȱ assessmentsȱ hasȱbeenȱeffectivelyȱusedȱtoȱimproveȱeducationalȱqualityȱinȱtheȱcountry.ȱ Severalȱ factorsȱ accountȱ forȱ theȱ differentialȱ impactȱ ofȱ theȱ USEȱ andȱ internationalȱassessmentsȱonȱtheȱRussianȱeducationȱsystem.ȱTheȱUSEȱwasȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ ȱ PISAȱ–ȱProgrammeȱforȱInternationalȱStudentȱAssessment;ȱTIMSSȱ–ȱTrendsȱinȱInternationalȱ 2 MathematicsȱandȱScienceȱStudy;ȱPIRLSȱ–ȱProgressȱinȱInternationalȱReadingȱLiteracyȱStudy.ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ xiȱ primarilyȱ designedȱ toȱ selectȱ andȱ certifyȱ students,ȱ notȱ toȱ informȱ policy.ȱ Theȱ nationalȱ examinationȱ programȱ stillȱ needsȱ toȱ buildȱ itsȱ credibilityȱ andȱ doȱaȱbetterȱjobȱofȱrefiningȱandȱdisseminatingȱitsȱconceptualȱframeworkȱasȱ wellȱ asȱ grantingȱ broaderȱ accessȱ toȱ assessmentȱ data.ȱ Inȱ contrast,ȱ internationalȱ assessmentsȱ wereȱ designedȱ withȱ theȱ primaryȱ purposeȱ ofȱ informingȱ policy,ȱ theyȱ haveȱ highȱ credibility,ȱ andȱ theirȱ dataȱ areȱ widelyȱ availableȱforȱpolicyȱresearch.ȱ ȱ Countriesȱ aimingȱ toȱ improveȱ theȱ disseminationȱ andȱ useȱ ofȱ theȱ dataȱ generatedȱ byȱ theirȱ assessmentȱ systemsȱ canȱ learnȱ fromȱ theȱ followingȱ lessons.ȱ Theȱ effectiveȱ useȱ ofȱ assessmentȱ informationȱ hasȱ toȱ beȱ carefullyȱ planned;ȱitȱwillȱnotȱhappenȱspontaneously.ȱHavingȱclearȱpurposesȱforȱtheȱ informationȱ isȱ aȱ startingȱ point.ȱ Involvingȱ keyȱ institutionsȱ isȱ requiredȱ toȱ meetȱ thoseȱ purposes.ȱ Disseminatingȱ informationȱ isȱ aȱ required,ȱ butȱ insufficient,ȱ conditionȱ forȱ success.ȱ Theȱ effectiveȱ useȱ ofȱ assessmentȱ dataȱ requiresȱ clearȱ guidelines,ȱ actionȱ plans,ȱ andȱ recommendations.ȱ Ofȱ note,ȱ keyȱ featuresȱ ofȱ anȱ assessmentȱ programȱ affectȱ theȱ useȱ ofȱ theȱ dataȱ itȱ generates.ȱ Theseȱ featuresȱ include,ȱ amongȱ others,ȱ aȱ clearȱ conceptualȱ framework,ȱ reportingȱ studentȱ performanceȱ levelsȱ basedȱ onȱ learningȱ standards,ȱ andȱ attachingȱ stakesȱ toȱ theȱ assessmentȱ results.ȱ Reformersȱ shouldȱ beȱ awareȱ ofȱ theseȱ featuresȱ inȱ orderȱ toȱ maximizeȱ theȱ positiveȱ impactsȱofȱtheȱassessmentȱsystemsȱinȱtheirȱcountries.ȱ ȱ ȱ ȱ xiiȱ YuliaȱTyumenevaȱ DisseminatingȱandȱUsingȱStudentȱ AssessmentȱInformationȱinȱRussiaȱ YuliaȱTyumenevaȱ Introductionȱ Theȱeffectiveȱuseȱofȱstudentȱassessmentȱinformationȱisȱkeyȱtoȱimprovingȱaȱ country’sȱeducationȱsystem.ȱCountriesȱthatȱhaveȱsetȱupȱstableȱassessmentȱ systemsȱ oftenȱ faceȱ manyȱ challengesȱ inȱ effectivelyȱ disseminatingȱ andȱ usingȱ assessmentȱ data.ȱ Russia’sȱ experienceȱ withȱ secondaryȱ schoolȱ examinationsȱ andȱ internationalȱ largeȬscaleȱ assessmentsȱ provideȱ usefulȱ insightsȱ intoȱ bothȱ theȱ challengesȱ andȱ successesȱ ofȱ theȱ effectiveȱ disseminationȱ andȱ useȱ ofȱ assessmentȱ data.ȱ Thisȱ paperȱ canȱ beȱ usefulȱ forȱ otherȱ countriesȱ thatȱ areȱ introducingȱ reformsȱ toȱ improveȱ theȱ impactȱ ofȱ assessmentȱinformationȱonȱeducationalȱqualityȱandȱequity.ȱ Currently,ȱ Russiaȱ administersȱ theȱ annualȱ nationalȱ Unifiedȱ Stateȱ Examinationȱ (USE)ȱ andȱ regularlyȱ participatesȱ inȱ internationalȱ largeȬscaleȱ assessmentsȱ(e.g.,ȱPISA,ȱTIMSS,ȱandȱPIRLS).ȱTheȱpurposesȱofȱtheȱUSEȱandȱ theȱ internationalȱ assessmentsȱ differȱ significantlyȱ andȱ haveȱ ledȱ toȱ theȱ resultsȱ fromȱ theseȱ assessmentsȱ havingȱ varyingȱ degreesȱ ofȱ influenceȱ onȱ theȱRussianȱeducationȱsystem.ȱ Theȱ USEȱ wasȱ introducedȱ inȱ 2009ȱ withȱ aȱ dualȱ purpose:ȱ toȱ certifyȱ secondaryȱ schoolȱ students,ȱ andȱ toȱ selectȱ studentsȱ intoȱ tertiaryȱ education.ȱ Thisȱ isȱ aȱ majorȱ examinationȱ programȱ thatȱ hasȱ aȱ greatȱ impactȱ onȱ studentȱ educationalȱ pathsȱ andȱ careers.ȱ Theȱ USEȱ isȱ mandatory,ȱ takenȱ everyȱ yearȱ byȱ nearlyȱ aȱ millionȱ studentsȱ inȱ 83ȱ subjectȱ areas.ȱ Theȱ examinationȱ isȱ fullyȱ standardizedȱ andȱ centrallyȱ managed,ȱ thusȱ allowingȱ forȱ greaterȱ quality,ȱ equity,ȱ andȱ efficiencyȱ thanȱ institutionalȬlevel,ȱ unstandardizedȱ examinations.ȱ ȱ Today,ȱ USEȱ resultsȱ areȱ usedȱ forȱ aȱ myriadȱ ofȱ purposesȱ beyondȱ studentȱ selectionȱ andȱ certification.ȱ Theseȱ includeȱ informingȱ pedagogy;ȱ holdingȱ regions,ȱ schools,ȱ andȱ teachersȱ accountable;ȱ andȱ monitoringȱ educationalȱ quality.ȱ Theȱ highȬstakesȱ natureȱ ofȱ theȱ USEȱ hasȱ ledȱ toȱ aȱ pushȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 1ȱ forȱ aȱ strongerȱ focusȱ onȱ resultsȱ inȱstudentȱ learning.ȱ Yetȱ itȱ hasȱ alsoȱ ledȱ toȱ aȱ pushȱ forȱ theȱ mechanicalȱ trainingȱ ofȱ studentsȱ onȱ examinationȬtakingȱ techniques.ȱ Participationȱ inȱ internationalȱ assessmentsȱ hasȱ beenȱ aȱ keyȱ featureȱ ofȱ Russianȱ educationalȱ policyȱ sinceȱ theȱ 1990s.ȱ Afterȱ theȱ fallȱ ofȱ theȱ Sovietȱ Union,ȱ Russia,ȱ asȱ aȱ newlyȱ establishedȱ federation,ȱ lookedȱ toȱ revampȱ itsȱ educationȱ systemȱ inȱ orderȱ toȱ successfullyȱ insertȱ itselfȱ intoȱ aȱ globalȱ marketȬorientedȱ economy.ȱ Internationalȱ assessmentsȱ suchȱ asȱ PISA,ȱ TIMSS,ȱ andȱ PIRLSȱ providedȱ theȱ conceptualȱ basisȱ andȱ operationalȱ toolsȱ forȱ reformingȱ nationalȱ learningȱ standards,ȱ curricula,ȱ andȱ textbooks.ȱ Theseȱ reformsȱ wereȱ furtherȱ promptedȱ byȱ Russia’sȱ unsatisfactoryȱ resultsȱ onȱ theȱ internationalȱ assessments,ȱ particularlyȱ inȱ theȱ caseȱ ofȱ secondaryȱ schoolȱstudents.ȱ Thereȱ isȱ aȱ generalȱ consensusȱ thatȱ internationalȱ assessmentsȱ haveȱ greatlyȱ contributedȱ toȱ improvingȱ educationȱ inȱ Russia,ȱ butȱ viewsȱ differȱ regardingȱtheȱcontributionȱofȱtheȱUSE.ȱSeveralȱfactorsȱhaveȱcontributedȱtoȱ thisȱdifferentialȱeffect.ȱFirst,ȱasȱalreadyȱnoted,ȱtheȱprimaryȱpurposeȱofȱtheȱ USEȱ isȱ notȱ toȱ informȱ educationalȱ policy,ȱ whereasȱ thisȱ isȱ theȱ primaryȱ purposeȱ ofȱ internationalȱ assessments.ȱ Consequently,ȱ theȱ examination’sȱ abilityȱtoȱinformȱpolicyȱisȱlimited.ȱSecond,ȱaccessȱtoȱUSEȱdataȱisȱrestricted,ȱ andȱ analysisȱ ofȱ USEȱ results,ȱ scarce.ȱ Inȱ contrast,ȱ accessȱ toȱ internationalȱ assessmentȱ dataȱ isȱ openȱ toȱ theȱ publicȱ andȱ free;ȱ researchersȱ publishȱ analysesȱ thatȱ directlyȱ informȱ policy.ȱ Theseȱ analysesȱ areȱ thenȱ translatedȱ intoȱrecommendationsȱandȱactionȱplans.ȱ ȱ Finally,ȱ useȱ ofȱ USEȱ resultsȱ seemsȱ toȱ beȱ divertingȱ theȱ focusȱ ofȱ manyȱ Russianȱeducatorsȱfromȱdevelopingȱstudents’ȱhigherȬorderȱthinkingȱskillsȱ toȱtrainingȱthemȱforȱtheȱexamination.ȱInternationalȱassessments,ȱhowever,ȱ haveȱ pushedȱ teachersȱ toȱ focusȱ onȱ higherȬorderȱ thinkingȱ skillsȱ byȱ emphasizingȱtheȱapplicationȱofȱknowledgeȱtoȱsolveȱrealȬlifeȱproblems.ȱ Thisȱ paperȱ isȱ organizedȱ asȱ follows.ȱ Theȱ nextȱ sectionȱ discussesȱ theȱ usesȱ ofȱ studentȱ assessmentȱ informationȱ generatedȱ byȱ theȱ USE.ȱ Subsectionsȱ discussȱ issuesȱ surroundingȱ theȱ useȱ ofȱ USEȱ resultsȱ forȱ certifyingȱ andȱ selectingȱ studentsȱ intoȱ tertiaryȱ education,ȱ forȱ informingȱ pedagogy,ȱ forȱ accountabilityȱ purposes,ȱ andȱ forȱ systemȱ monitoring.ȱ Theȱ followingȱ sectionȱ describesȱ theȱ usesȱ ofȱ internationalȱ largeȬscaleȱ assessmentȱ dataȱ toȱ introduceȱ educationalȱ reformsȱ inȱ Russia.ȱ Theȱ lastȱ sectionȱ concludesȱ withȱ lessonsȱ learnedȱ regardingȱ factorsȱ thatȱ impedeȱ orȱ ȱ 2ȱ YuliaȱTyumenevaȱ facilitateȱ theȱ effectiveȱ useȱ ofȱ studentȱ assessmentȱ informationȱ toȱ improveȱ educationalȱquality.ȱ TheȱUnifiedȱStateȱExaminationȱ Theȱ USEȱ isȱ Russia’sȱ mostȱ importantȱ assessmentȱ program.ȱ Itȱ wasȱ introducedȱ atȱ theȱ nationalȱ levelȱ inȱ 2009ȱ toȱ replaceȱ andȱ standardizeȱ aȱ previouslyȱ existingȱ systemȱ ofȱ institutionalȬlevelȱ finalȱ examinations.ȱ Theȱ USEȱ wasȱ oneȱ ofȱ theȱ coreȱ elementsȱ ofȱ aȱ majorȱ reformȱ ofȱ theȱ educationȱ systemȱ inȱ Russia.ȱ Thisȱ wasȱ aȱ veryȱ complexȱ reformȱ thatȱ requiredȱ fightingȱ corruptȱ practicesȱ inȱ universityȱ entranceȱ decisionsȱ andȱ launchingȱ aȱ massiveȱ publicȱ campaignȱ toȱ gainȱ publicȱ support.ȱ Nowadays,ȱ nearlyȱ aȱ millionȱstudentsȱtakeȱtheȱexaminationȱeveryȱyear,ȱallowingȱthemȱtoȱapplyȱ toȱ severalȱ tertiaryȱ educationȱ institutionsȱ throughȱ anȱ automaticȱ andȱ objectiveȱprocessȱ(Bolotovȱetȱal.ȱ2013).ȱ Asȱ alreadyȱ noted,ȱ theȱ USEȱ isȱ usedȱ forȱ aȱ varietyȱ ofȱ purposes,ȱ whichȱ includeȱ secondaryȱ schoolȱ certification;ȱ selectionȱ intoȱ tertiaryȱ education;ȱ informingȱ pedagogy;ȱ holdingȱ regions,ȱschools,ȱ andȱ teachersȱ accountable;ȱ andȱ monitoringȱ educationalȱ quality.ȱ Thisȱ myriadȱ ofȱ usesȱ isȱ inȱ partȱ theȱ consequenceȱ ofȱ Russiaȱ notȱ havingȱ aȱ nationalȱ largeȬscaleȱ assessmentȱ programȱ forȱ monitoringȱ andȱ accountabilityȱ purposes.ȱ Theȱ USEȱ isȱ thereforeȱusedȱtoȱmeetȱfunctionsȱforȱwhichȱitȱwasȱnotȱoriginallyȱdesigned,ȱ withȱbothȱpositiveȱandȱnegativeȱconsequences.ȱ Certifyingȱandȱselectingȱstudentsȱ StudentsȱreceiveȱUSEȱscoresȱbasedȱonȱtheirȱresultsȱinȱmandatoryȱ(Russianȱ languageȱ andȱ mathematics)ȱ andȱ optionalȱ subjectȱ areas.ȱ Theseȱ testȱ scoresȱ canȱbeȱusedȱbyȱstudentsȱtoȱapplyȱtoȱuniversities.ȱTheȱhigherȱtheȱtestȱscore,ȱ theȱhigherȱtheȱchanceȱofȱgettingȱintoȱaȱmoreȱselectiveȱuniversity.ȱStudentsȱ canȱapplyȱforȱupȱtoȱsixȱuniversitiesȱandȱprogramsȱofȱstudy.ȱ ȱ Allȱ universitiesȱ requireȱ studentsȱ toȱ submitȱ testȱ scoresȱ inȱ theȱ mandatoryȱ subjectȱ areasȱ and,ȱ dependingȱ onȱ theȱ programȱ ofȱ study,ȱ optionalȱ subjectȱ areas.ȱ Forȱ example,ȱ onȱ topȱ ofȱ theȱ mandatoryȱ subjectȱ areas,ȱ aȱ medicalȱ programȱ willȱ requireȱ testȱ scoresȱ inȱ biologyȱ andȱ chemistry;ȱ aȱ technicalȱ program,ȱ inȱ physics;ȱ andȱ aȱ programȱ inȱ theȱ humanities,ȱinȱhistoryȱandȱsocialȱstudies.ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 3ȱ Testȱ scoresȱ areȱ alsoȱ usedȱ toȱ classifyȱ studentsȱ intoȱ fourȱ academicȱ performanceȱ levels:ȱ minimal,ȱ low,ȱ medium,ȱ andȱ high.ȱ Aȱ minimalȱ performanceȱlevelȱdoesȱnotȱqualifyȱaȱstudentȱforȱhighȱschoolȱcertification.ȱ CutȬoffȱ pointsȱ thatȱ defineȱ theȱ performanceȱ levelsȱ areȱ establishedȱ eachȱ yearȱbyȱtheȱgovernment,ȱbasedȱonȱthatȱyear’sȱtestȱresults,ȱofficialȱlearningȱ standards,ȱ andȱ expertȱ opinion.ȱ Thisȱ postȱ factoȱ procedureȱ avoidsȱ aȱ situationȱinȱwhichȱtooȱmanyȱstudentsȱfailȱtheȱexamination.ȱUnfortunately,ȱ theȱ criteriaȱ usedȱ toȱ establishȱ theȱ cutȬoffȱ pointsȱ varyȱ fromȱ oneȱ yearȱ toȱ theȱ next,ȱlimitingȱtheȱvalidityȱofȱyearlyȱcomparisons.ȱ ȱ Examinationȱ resultsȱ areȱ producedȱ andȱ recordedȱ byȱ theȱ Federalȱ TestingȱCenterȱofȱtheȱMinistryȱofȱEducation.ȱThisȱcenterȱsendsȱindividualȱ testȱscoresȱtoȱmunicipalitiesȱandȱschools,ȱwhichȱareȱultimatelyȱresponsibleȱ forȱ reportingȱ themȱ toȱ theȱ students.ȱ Testȱ resultsȱ areȱ freeȱ ofȱ chargeȱ andȱ confidential.ȱEachȱstudentȱreceivesȱaȱsecondaryȱschoolȱcertificateȱwithȱtheȱ transcriptȱofȱhisȱorȱherȱtestȱscores.ȱ ȱ Informingȱpedagogyȱ Asȱ aȱ highȬstakesȱ examination,ȱ theȱ USEȱ greatlyȱ shapesȱ teachingȱ andȱ learningȱ inȱ Russianȱ secondaryȱ schoolsȱ andȱ beyond.ȱ USEȱ resultsȱ areȱ usedȱ toȱ createȱ pedagogicalȱ guidelinesȱ toȱ prepareȱ futureȱ studentsȱ forȱ theȱ examinations.ȱ Theseȱ guidelinesȱ haveȱ comeȱ toȱ serveȱ asȱ teachingȱ aidsȱ forȱ teachersȱandȱstudyȱguidesȱforȱstudents.ȱTheȱexaminationsȱareȱexpectedȱtoȱ haveȱ aȱ positiveȱ effectȱ onȱ educationalȱ qualityȱ becauseȱ theyȱ areȱ alignedȱ withȱ nationalȱ learningȱ standards.ȱ Givenȱ theȱ stakesȱ attachedȱ toȱ theȱ examination,ȱ bothȱ teachersȱ andȱ studentsȱ areȱ assumedȱ toȱ beȱ highlyȱ motivatedȱ andȱ focusedȱ onȱ masteringȱ theȱ nationalȱ learningȱ standardsȱ (measuredȱbyȱUSE).ȱ ȱ Bothȱ federalȱ andȱ regionalȱ guidelinesȱ existȱ toȱ prepareȱ teachersȱ andȱ studentsȱ forȱ theȱ USE.ȱ Atȱ theȱ federalȱ level,ȱ theseȱ guidelinesȱ includeȱ examinationȱ contentȱ forȱ eachȱ subjectȱ area,ȱ sampleȱ questions,ȱ andȱ criteriaȱ forȱ scoringȱ openȬendedȱ questions.ȱ Atȱ theȱ regionalȱ level,ȱ guidelinesȱ alertȱ teachersȱ toȱ currentȱ deficienciesȱ inȱ studentȱ readinessȱ forȱ graduationȱ andȱ provideȱ guidanceȱ inȱ overcomingȱ theseȱ challenges.ȱ However,ȱ regionalȱ guidelinesȱoftenȱfocusȱonȱstatisticalȱanalysesȱandȱneglectȱtoȱtranslateȱtheseȱ analysesȱ intoȱ actionȱ plansȱ forȱ improvement.ȱ Consequently,ȱ recommendationsȱ madeȱ atȱ theȱ regionalȱ levelȱ areȱ oftenȱ simplyȱ duplicationsȱofȱfederalȬlevelȱrecommendations.ȱ ȱ 4ȱ YuliaȱTyumenevaȱ Criticsȱ claimȱ thatȱ theȱ USEȱ isȱ havingȱ aȱ detrimentalȱ effectȱ onȱ studentȱ learningȱ andȱ educationalȱ quality.ȱ Examinationȱ preparation—ratherȱ thanȱ teachingȱ andȱ learningȱ highȬorderȱ thinkingȱ skills—hasȱ becomeȱ theȱ keyȱ elementȱ ofȱ schoolȱ actionȱ plans. 3 ȱ Althoughȱ focusingȱ onȱ examinationȱ preparationȱ runsȱ counterȱ toȱ theȱ adviceȱ ofȱ regionalȱ thinkȱ tanks,ȱ thisȱ practiceȱ isȱ facilitatedȱ byȱ theȱ easyȬtoȬuseȱ guidelinesȱ forȱ USEȱ preparation.ȱ Theseȱ guidelinesȱ haveȱ accordinglyȱ becomeȱ aȱ deȱ factoȱ curriculumȱ forȱ teachers,ȱ severelyȱ narrowingȱ theȱ contentȱ andȱ cognitiveȱ skillsȱ taughtȱ inȱ class.ȱAsȱaȱ result,ȱfederalȱ curriculaȱ guidelinesȱ andȱ learningȱ standardsȱ areȱ sidelined.ȱSchoolsȱwithȱtheȱhighestȱpotentialȱriskȱofȱpoorȱfinalȱtestȱresultsȱ (e.g.,ȱruralȱschoolsȱinȱsmallȱtownsȱfarȱfromȱregionalȱcenters)ȱseemȱtoȱhaveȱ theȱmostȱaccentuatedȱfocusȱonȱtheȱuseȱofȱexaminationȱguidelines.4ȱ SchoolsȱwithȱpoorȱUSEȱscoresȱmayȱbeȱsubjectedȱtoȱcloserȱinspection.ȱ Inspectorsȱ gatherȱ schoolȱ observationsȱ andȱ conductȱ interviewsȱ withȱ schoolȱdirectorsȱandȱteachersȱinȱorderȱtoȱproduceȱaȱdiagnosisȱandȱsuggestȱ actionsȱforȱimprovement.ȱForȱinstance,ȱinspectionsȱmayȱshowȱdeficienciesȱ inȱtheȱorganizationȱofȱaȱschoolȱ(e.g.,ȱstudents’ȱfailureȱtoȱattendȱadditionalȱ examinationȱ preparationȱ lessons,ȱ anȱ incompleteȱ curriculum).ȱ Schoolsȱ receiveȱ guidelinesȱ onȱ howȱ toȱ addressȱ theseȱ weaknessesȱ inȱ orderȱ toȱ improveȱ studentȱ learningȱ (andȱ USEȱ results).ȱ Unfortunately,ȱ theseȱ guidelinesȱ oftenȱ missȱ theȱ mark.ȱ Theyȱ tendȱ toȱ narrowlyȱ conceptualizeȱ solutions,ȱ focusingȱ onȱ violationsȱ ofȱ schoolȱ proceduresȱ andȱ rectifyingȱ shortcomingsȱ inȱ theȱ implementationȱ ofȱ aȱ school’sȱ specificȱ activities,ȱ suchȱ asȱimprovingȱlowȱattendanceȱinȱadditionalȱstudentȱpreparationȱclasses.ȱ Someȱ regionalȱ ministriesȱ establishȱ fundingȱ prioritiesȱ basedȱ onȱ USEȱ results.ȱ Forȱ example,ȱ aȱ regionalȱ ministerȱ mayȱ announceȱ theȱ goalȱ ofȱ raisingȱUSEȱscoresȱforȱtheȱregionȱtoȱtheȱnationalȱlevelȱinȱnaturalȱsciences.5ȱ Accordingly,ȱspecialȱfundingȱisȱcreatedȱinȱorderȱtoȱimproveȱteachingȱandȱ learningȱinȱthisȱarea.ȱ Aȱ majorȱ weaknessȱ inȱ theȱ effectiveȱ useȱ ofȱ USEȱ dataȱ isȱ thatȱ teacherȱ trainingȱ programsȱ doȱ notȱ seemȱ toȱ benefitȱ fromȱ it.ȱ Whileȱ theseȱ programsȱ areȱ providedȱ withȱ informationȱ aboutȱ theȱ examinationsȱ (Bolotovȱ etȱ al.ȱ 2013),ȱ theyȱ doȱ notȱ seemȱ toȱ useȱ itȱ toȱ informȱ changesȱ toȱ theirȱ programsȱ ofȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 3ȱ See,ȱforȱexample,ȱMunicipalȱInstituteȱofȱGeneralȱEducationȱofȱtheȱChuvashȱRepublicȱ(n.d.).ȱ ȱ 4ȱ Ibid.ȱ 5ȱ See,ȱ forȱ example,ȱ Ministerȱ ofȱ Educationȱ andȱ Scienceȱ ofȱ theȱ Krasnoyarskȱ Kraiȱ (Territory)ȱ (2010).ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 5ȱ study,ȱ norȱ isȱ itȱ usedȱ asȱ anȱ inputȱ inȱ classȱ discussions.ȱ Thisȱ isȱ aȱ seriousȱ misalignmentȱ thatȱ preventsȱ futureȱ andȱ inȬserviceȱ teachersȱ fromȱ learningȱ aboutȱ examinationȱ results.ȱ Oneȱ wouldȱ expectȱ thatȱ trainingȱ programsȱ wouldȱprovideȱteachersȱwithȱopportunitiesȱtoȱlearnȱaboutȱtheȱcontentȱandȱ skillsȱmeasuredȱbyȱtheȱtests,ȱasȱwellȱasȱtoȱreflectȱonȱhowȱthatȱcontentȱandȱ skillsȱ shouldȱ beȱ taught.ȱ However,ȱ ifȱ USEȱ resultsȱ areȱ usedȱ toȱ informȱ teacherȱtraining,ȱcareȱwillȱneedȱtoȱbeȱtakenȱtoȱpreventȱtheȱmechanicalȱuseȱ ofȱUSEȱguidelinesȱasȱaȱdeȱfactoȱcurriculum.ȱ Holdingȱregions,ȱschools,ȱandȱteachersȱaccountableȱ USEȱ resultsȱ areȱ usedȱ toȱ putȱ pressureȱ onȱ differentȱ levelsȱ ofȱ theȱ schoolȱ systemȱbyȱmeansȱofȱtheȱschoolȱaccreditationȱprocess,ȱschoolȱrankings,ȱtheȱ publicationȱ ofȱ schoolȱ results,ȱ andȱ theȱ distributionȱ ofȱ economicȱ incentivesȱ toȱbothȱteachersȱandȱschools.ȱ ȱ Withȱrespectȱtoȱschoolȱaccreditation,ȱschoolsȱareȱmandatedȱbyȱlawȱtoȱ informȱtheȱfederalȱMinistryȱofȱEducationȱandȱScienceȱofȱstudentȱresultsȱinȱ compulsoryȱ subjectsȱ forȱ theȱ previousȱ threeȱ years.ȱ Inȱ theȱ caseȱ ofȱ persistentlyȱlowȱscores,ȱschoolsȱriskȱbeingȱrefusedȱrenewedȱaccreditation.ȱ WhileȱthisȱscenarioȱhasȱyetȱtoȱplayȱoutȱsinceȱtheȱUSEȱwasȱimplementedȱinȱ 2009,ȱtheȱpossibilityȱremains.ȱ Regionalȱ ministriesȱ ofȱ educationȱ andȱ municipalitiesȱ publishȱ schoolȱ rankingsȱ basedȱ onȱ USEȱ results.ȱ Mostȱ ofȱtheȱ time,ȱ USEȱ scoresȱareȱ theȱ onlyȱ criterionȱ usedȱ toȱ rankȱ theȱ schools,ȱ aȱ practiceȱ thatȱ goesȱ againstȱ theȱ recommendationsȱ ofȱ bothȱ federalȱ agenciesȱ andȱ localȱ thinkȱ tanks.ȱ Measuringȱschoolȱqualityȱbasedȱonȱtheȱproportionȱofȱcertifiedȱstudentsȱisȱ unfairȱtoȱschoolsȱthatȱworkȱinȱdisadvantagedȱcontexts.ȱTheseȱschoolsȱmayȱ beȱ addingȱ aȱ lotȱ ofȱ valueȱ toȱ theirȱ students’ȱ education,ȱ butȱ manyȱ ofȱ theseȱ studentsȱ mayȱ stillȱ notȱ reachȱ theȱ academicȱ levelȱ requiredȱ forȱ certification.ȱ Onȱtheȱotherȱhand,ȱschoolsȱworkingȱinȱsociallyȱadvantagedȱcontextsȱmayȱ notȱ beȱ addingȱ muchȱ valueȱ toȱ theirȱ students’ȱ education,ȱ butȱ mayȱ stillȱ produceȱaȱhighȱproportionȱofȱcertifiedȱstudents.ȱ Schoolsȱ competeȱ toȱ attractȱ moreȱ studentsȱ andȱ fundingȱ byȱ publicizingȱ USEȱ results.ȱ Thus,ȱ theyȱ oftenȱ postȱ onȱ theirȱ websitesȱ reportsȱ thatȱ featureȱ theirȱ averageȱ examinationȱ scores,ȱ theȱ numberȱ ofȱ studentsȱ meetingȱtheȱminimumȱandȱmaximumȱbenchmarks,ȱcomparativeȱanalysesȱ ofȱschoolȱscoresȱwithȱthoseȱofȱotherȱschoolsȱinȱtheȱcityȱorȱdistrict,ȱandȱtheirȱ schoolȱ ranking.ȱ Suchȱ reportsȱ areȱ frequentlyȱ usedȱ toȱ promoteȱ aȱ school’sȱ ȱ 6ȱ YuliaȱTyumenevaȱ performanceȱ inȱ comparisonȱ toȱ otherȱ schools.ȱ Highȱ USEȱ scoresȱ greatlyȱ contributeȱtoȱaȱschool’sȱreputationȱandȱareȱinstrumentalȱinȱattractingȱbothȱ additionalȱfundingȱfromȱtheȱgovernmentȱandȱnewȱstudentsȱtoȱtheȱschool.ȱ ȱ Withȱ respectȱ toȱ economicȱ incentives,ȱ municipalitiesȱ rewardȱ higherȬperformingȱ schoolsȱ basedȱ onȱ theirȱ overallȱ USEȱ scoresȱ fromȱ theȱ previousȱyear.ȱThisȱapproachȱsuffersȱfromȱtwoȱimportantȱweaknesses:ȱ(1)ȱ itȱ doesȱ notȱ takeȱ intoȱ accountȱ changesȱ inȱ USEȱ resultsȱ fromȱ oneȱ yearȱ toȱ another,ȱ andȱ (2)ȱ itȱ doesȱ notȱ takeȱ intoȱ accountȱ theȱ socioeconomicȱ backgroundȱ inȱ whichȱ aȱ schoolȱ operates.ȱ Therefore,ȱ schoolsȱ operatingȱ inȱ sociallyȱ disadvantagedȱ backgroundsȱ tendȱ toȱ beȱ unfairlyȱ penalizedȱ whenȱ theirȱ USEȱ scoresȱ areȱ comparedȱ toȱ thoseȱ ofȱ schoolsȱ operatingȱ inȱ moreȱ sociallyȱadvantagedȱbackgrounds.ȱ Examinationȱ resultsȱ areȱ alsoȱ usedȱ toȱ holdȱ teachersȱ accountableȱ andȱ toȱ distributeȱ bonuses.ȱ Aȱ newȱ nationalȱ wageȱ systemȱ providesȱ teachersȱ withȱincentiveȱbonusesȱbasedȱonȱtheȱperformanceȱofȱtheirȱstudentsȱonȱtheȱ USE.ȱTheȱamountȱofȱtheȱincentiveȱvariesȱamongȱschoolsȱandȱdistricts.ȱThisȱ wageȱ systemȱ isȱ partlyȱ linkedȱ toȱ aȱ nationalȱ teacherȱ evaluationȱ programȱ thatȱlinksȱstudentȱresultsȱtoȱsubjectȱareaȱteachersȱinȱgradeȱ11.ȱExaminationȱ results,ȱmoreover,ȱhaveȱaȱstrongȱimpactȱonȱteachers’ȱreputations;ȱthisȱhasȱ pushedȱ teachersȱ furtherȱ inȱ theȱ directionȱ ofȱ usingȱ USEȱ preparationȱ guidelinesȱasȱaȱdeȱfactoȱcurriculum.ȱ Monitoringȱeducationalȱqualityȱ USEȱ resultsȱ areȱ aȱ keyȱ indicatorȱ ofȱ educationalȱqualityȱ atȱ theȱ nationalȱandȱ regionalȱlevel.ȱFederalȬlevelȱreportsȱexamineȱtheȱextentȱtoȱwhichȱstudentsȱ areȱ meetingȱ curricularȱ standardsȱ inȱ differentȱ subjectȱ areas.ȱ Theȱ Ministryȱ ofȱEducationȱ(throughȱtheȱFederalȱInstituteȱofȱPedagogicalȱMeasurement)ȱ publishesȱ onlineȱ aggregatedȱ informationȱ aboutȱ studentȱ performanceȱ onȱ theȱ examinations.ȱ Theseȱ dataȱincludeȱ averageȱscoresȱinȱ eachȱsubjectȱ area,ȱ theȱ percentageȱ ofȱ studentsȱ meetingȱ differentȱ performanceȱ levelsȱ andȱ passingȱ theȱ examinations,ȱ studentȱ performanceȱ onȱ tasksȱ thatȱ measureȱ differentȱcontentȱandȱskills,ȱandȱitemȱdifficultyȱlevels.ȱYearlyȱcomparisonsȱ areȱ alsoȱ reported,ȱ despiteȱ theȱ factȱ thatȱ theȱ testsȱ areȱ notȱ equatedȱ andȱ theȱ criteriaȱusedȱtoȱdefineȱtheȱperformanceȱlevelsȱmayȱvaryȱfromȱyearȱtoȱyear.ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 7ȱ Overall,ȱ theȱ dataȱ areȱ usedȱ toȱ identifyȱ deficienciesȱ inȱ studentȱ knowledgeȱ andȱprovideȱaȱfoundationȱforȱaction.ȱForȱexample,ȱaȱUSEȱreportȱstated:6ȱ ExamȬtakersȱ inȱ 2009ȱ asȱ aȱ wholeȱ showedȱ lowȱ resultsȱ inȱ solvingȱ geometryȱ problems,ȱ atȱ bothȱ highȱ andȱ lowȱ levelsȱ ofȱ difficulty.ȱ Manyȱ examȬtakersȱ cannotȱ solveȱ geometryȱ problems—notȱ onlyȱ advancedȱ [problems],ȱ butȱ alsoȱ thoseȱ atȱ theȱ basicȱ level.ȱ Theseȱ resultsȱ reflectȱ theȱ situationȱ inȱ schools,ȱ whichȱ wasȱ clearlyȱ unfavorableȱ forȱ theȱ studyȱ ofȱ geometryȱ forȱ manyȱ years,ȱ insofarȱ asȱ theȱ Stateȱ Finalȱ Certificationȱ [theȱ examinationȱinȱplaceȱbeforeȱUSEȱwasȱintroduced]ȱonlyȱtestedȱ10thȱandȱ 11thȱgradeȱalgebraȱandȱdataȱanalysis.ȱ Federalȱ reportsȱ areȱ discussedȱ atȱ conferencesȱ andȱ technicalȱ meetingsȱ ledȱbyȱtheȱMinistryȱofȱEducationȱandȱScienceȱandȱtheȱRussianȱAcademyȱofȱ Education.ȱ Participantsȱ includeȱ specialistsȱ fromȱ federalȱ andȱ regionalȱ governments,ȱasȱwellȱasȱstaffȱfromȱeducationȱcentersȱandȱteacherȱtrainingȱ institutes.ȱ Materialsȱ documentingȱ keyȱ findingsȱ areȱ distributedȱ toȱ participants,ȱwhoȱthenȱdisseminateȱthemȱinȱtheȱregions.ȱ VariedȱstrategiesȱareȱusedȱtoȱdisseminateȱUSEȱresultsȱatȱtheȱregionalȱ level.ȱ Regionalȱ ministriesȱ andȱ departmentsȱ ofȱ educationȱ publishȱ USEȱ resultsȱ onȱ theirȱ websites.ȱ Theyȱ alsoȱ holdȱ pressȱ conferencesȱ attendedȱ byȱ theȱregionalȱmedia.ȱTheseȱpressȱconferencesȱusuallyȱfocusȱonȱtheȱaverageȱ regionalȱUSEȱscoreȱ byȱ subjectȱ area,ȱ theȱ numberȱ ofȱ graduatesȱ whoȱpassedȱ theȱexamination,ȱandȱtheȱnumberȱofȱstudentsȱwhoȱachievedȱtheȱminimumȱ examinationȱ score.ȱ Theseȱ figuresȱ areȱ comparedȱ withȱ nationwideȱ resultsȱ toȱ contrastȱ theȱ regionalȱ performanceȱ withȱ thatȱ ofȱ theȱ restȱ ofȱ theȱ country.ȱ USEȱ resultsȱ andȱ analysisȱ areȱ alsoȱ incorporatedȱ intoȱ regionalȱ annualȱ reportsȱ onȱ theȱ statusȱ ofȱ education.ȱ Reportsȱ ofȱ excessiveȱ numbersȱ ofȱ studentsȱ failingȱ toȱ obtainȱ secondaryȱ schoolȱ certificationȱ raiseȱ concernsȱ aboutȱ theȱ inefficientȱ useȱ ofȱ educationȱ budgets,ȱ asȱ wellȱ asȱ theȱ ineffectivenessȱofȱregionalȱgovernors.ȱ Byȱ trackingȱ changesȱ inȱ studentȱ performanceȱ fromȱ yearȱ toȱ year,ȱ USEȱ scoresȱ areȱ alsoȱ usedȱ toȱ monitorȱ trendsȱ inȱ educationalȱ quality.ȱ Forȱ instance,ȱ changesȱ inȱ theȱ percentageȱ ofȱ studentsȱ whoȱ meetȱ theȱ highȱ performanceȱ levelȱ areȱ usedȱ asȱ anȱ indicatorȱ ofȱ educationalȱ quality.ȱ However,ȱ thisȱ practiceȱ hasȱ beenȱ criticizedȱ becauseȱ USEȱ testsȱ areȱ notȱ comparableȱoverȱtime.ȱItȱisȱnotȱpossibleȱtoȱdiscernȱwhetherȱchangesȱinȱtestȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 6 ȱ SeeȱFederalȱInstituteȱofȱPedagogicalȱMeasurementȱ(2009:ȱ69).ȱ ȱ ȱ 8ȱ YuliaȱTyumenevaȱ scoresȱ areȱ dueȱ toȱ changesȱ inȱ studentȱ performanceȱ orȱ changesȱ inȱ theȱ difficultyȱ levelȱ ofȱ theȱ testsȱ administeredȱ inȱ differentȱ years.ȱ Toȱ equateȱ theȱ tests,ȱ testȱ scoresȱ wouldȱ needȱ toȱ beȱ ratedȱ onȱ theȱ sameȱ scale.ȱ Thisȱ wouldȱ allowȱ forȱ changesȱ inȱ testȱ scoresȱ toȱ beȱ attributedȱ toȱ genuineȱ changesȱ inȱ studentȱ performance.ȱ Asȱ notedȱ above,ȱ comparingȱ resultsȱ fromȱ yearȱ toȱ yearȱ hasȱ alsoȱ beenȱ criticizedȱ dueȱ toȱ variationsȱ inȱ theȱ wayȱ performanceȱ levelsȱareȱdefined.ȱ Finally,ȱ USEȱ resultsȱ areȱ usedȱ toȱ monitorȱ educationalȱ qualityȱ atȱ theȱ regionalȱlevel.ȱRegionalȱcentersȱindependentlyȱanalyzeȱUSEȱresults,ȱoftenȱ producingȱ comprehensiveȱ regionalȱ reportsȱ andȱ analyses.ȱ Forȱ example,ȱ figureȱ 1ȱ showsȱ theȱ differencesȱ inȱ theȱ performanceȱ ofȱ schoolsȱ inȱ differentȱ municipalitiesȱ ofȱ theȱ Chuvashȱ Republic.ȱ Thisȱ typeȱ ofȱ analysisȱ allowsȱ forȱ anȱ understandingȱ ofȱ disparitiesȱ inȱ theȱ qualityȱ ofȱ educationȱ withinȱ regions.ȱ Itȱ alsoȱ providesȱ localȱ governmentsȱ withȱ theȱ informationȱ necessaryȱtoȱtargetȱfundingȱandȱprovideȱadditionalȱsupport.ȱ ȱ Nongovernmentalȱ institutionsȱ alsoȱ analyzeȱ USEȱ resultsȱ toȱ monitorȱ educationalȱqualityȱandȱpushȱforȱreforms.ȱForȱexample,ȱtheȱHigherȱSchoolȱ ofȱ Economicsȱ (throughȱ itsȱ Instituteȱ forȱ Educationalȱ Development)ȱ discussesȱ theȱ resultsȱ ofȱ USEȱ researchȱ atȱ weeklyȱ seminars.ȱ Theseȱ discussionsȱ areȱ oftenȱ takenȱ intoȱ considerationȱ byȱ theȱ Committeeȱ onȱ Educationȱ ofȱ theȱ Publicȱ Chamberȱ which,ȱ inȱ turn,ȱ drivesȱ theȱ adoptionȱ ofȱ initiativesȱ byȱ theȱ Ministryȱ ofȱ Educationȱ andȱ Science.ȱ Unfortunately,ȱ analysisȱofȱstudentȱperformanceȱbyȱindividualȱschoolsȱisȱrare,ȱlargelyȱdueȱ toȱtheȱextremelyȱlimitedȱnumberȱofȱspecialistsȱworkingȱinȱschoolsȱwhoȱareȱ capableȱofȱanalyzingȱsuchȱdata.ȱ Whileȱ theȱ USEȱ providesȱ aȱ richȱ sourceȱ ofȱ informationȱ forȱ monitoringȱ educationalȱ quality,ȱ theȱ examinationsȱ alsoȱ haveȱ importantȱ limitations.ȱ First,ȱUSEȱresultsȱonlyȱprovideȱdataȱonȱstudentȱperformanceȱatȱtheȱendȱofȱ secondaryȱ school,ȱ notȱ theirȱ performanceȱ inȱ earlierȱ grades.ȱ Second,ȱ theȱ examinationȱdifferentiatesȱbetweenȱstudentsȱwhoȱdoȱandȱdoȱnotȱmeetȱtheȱ nationalȱ learningȱ standards.ȱ Itȱ isȱ notȱ aimedȱ atȱ describingȱ whatȱ studentsȱ areȱ ableȱ toȱ doȱ atȱ differentȱ performanceȱ levels,ȱ whichȱ wouldȱ beȱ moreȱ appropriateȱforȱmonitoringȱandȱpedagogicalȱpurposes.ȱSinceȱRussiaȱdoesȱ notȱ haveȱ aȱ nationalȱ largeȬscaleȱ assessmentȱ programȱ forȱ suchȱ purposes,ȱ theȱ USEȱ seemsȱ toȱ beȱ fillingȱ thisȱ gap.ȱ Asȱ notedȱ atȱ theȱ outsetȱ ofȱ thisȱ paper,ȱ thisȱsituationȱhasȱbothȱpositiveȱandȱnegativeȱconsequences.ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 9ȱ  Figureȱ1.ȱRegionalȱAnalysisȱofȱtheȱ2011ȱUnifiedȱStateȱExaminationȱResults,ȱChuvashȱRepublicȱ Differences in School Mean Scores on the USE by Municipality in the Chuvash Republic. 78.0 70.6 Mean score of “the 68.3 67.2 66.8 66.7 best” school 66.1 66.0 66.0 65.2 65.2 65.4 63.9 63.9 63.7 63.0 62.3 62.6 62.0 61.9 Mean score for the 60.7 municipality 60.1 57.3 56.1 55.5 59.8 Mean score of “the 56.3 55.7 worst” school 54.3 53.8 53.6 56.6 53.3 53.4 52.3 52.4 52.2 52.2 52.1 10ȱ 50.8 50.1 49.8 49.9 49.6 49.0 48.1 48.0 47.1 46.1 45.9 44.1 Mean score on 100-point scale, Russian and Mathematics Exams 41.4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Municipality ȱ Source:ȱBochenkovȱ2011.ȱ ȱ Note:ȱTheȱtallerȱtheȱbar,ȱtheȱgreaterȱtheȱdiscrepancyȱbetweenȱtheȱbestȬȱandȱworstȬperformingȱschools.ȱTheȱredȱdotsȱrepresentȱtheȱaverageȱscoreȱ ofȱaȱgivenȱmunicipality.ȱ ȱ ȱ InternationalȱLargeȬScaleȱAssessmentsȱ Afterȱ theȱ collapseȱ ofȱ theȱ Sovietȱ Union,ȱ thereȱ wasȱ aȱ renewedȱ interestȱ inȱ understandingȱ theȱ relativeȱ standingȱ ofȱ Russianȱ studentsȱ comparedȱ toȱ studentsȱ inȱ otherȱ Westernȱ economies.ȱ Wereȱ Russianȱ studentsȱ meetingȱ internationalȱ standards?ȱ Wereȱ theyȱ acquiringȱ theȱ skillsȱ neededȱ toȱ beȱ successfulȱ citizensȱ inȱ aȱ globalȱ marketȱ economy?ȱ Whatȱ wereȱ studentsȱ inȱ otherȱ countriesȱ learning?ȱ Participationȱ inȱ internationalȱ assessmentsȱ allowedȱ Russiaȱ toȱ answerȱ theseȱ questions.ȱ Theȱ assessmentsȱ haveȱ providedȱrichȱinformationȱonȱwhatȱRussianȱstudentsȱknowȱandȱcanȱdoȱinȱ keyȱ areasȱ (e.g.,ȱ reading,ȱ mathematics,ȱ sciences),ȱ comparedȱ toȱ studentsȱ inȱ otherȱ countries.ȱ Theyȱ haveȱ alsoȱ identifiedȱ trendsȱ inȱ studentȱ performanceȱ byȱ comparingȱ theȱ resultsȱ ofȱ differentȱ assessmentȱ rounds.ȱ Finally,ȱ theseȱ assessmentsȱ haveȱ providedȱ Russiaȱ withȱ richȱ contextualȱ informationȱ thatȱ hasȱbeenȱusedȱtoȱinformȱpolicyȱandȱintroduceȱreforms.ȱ Asȱnotedȱearlier,ȱRussiaȱhasȱconsistentlyȱparticipatedȱinȱinternationalȱ largeȬscaleȱ assessmentsȱ inȱ theȱ postȬSovietȱ era.ȱ Sinceȱ theȱ collapseȱ ofȱ theȱ SovietȱUnion,ȱitȱhasȱparticipatedȱinȱ19ȱsuchȱassessments,ȱincludingȱTIMSSȱ inȱ gradesȱ 4,ȱ 8,ȱ andȱ 11ȱ (1995,ȱ 1999,ȱ 2003,ȱ 2007,ȱ 2008,ȱ 2011);ȱ PISA,ȱ whichȱ evaluatesȱ 15ȬyearȬoldȱ studentsȱ (2000,ȱ 2003,ȱ 2006,ȱ 2009);ȱ PIRLS,ȱ whichȱ evaluatesȱtheȱreadingȱliteracyȱskillsȱofȱgradeȱ4ȱstudentsȱ(2001,ȱ2006,ȱ2011);ȱ andȱ otherȱ similarȱ instrumentsȱ (CIVED,ȱ SITES).7ȱ Theseȱ assessmentsȱ areȱ administeredȱ inȱ Russiaȱ byȱ aȱ specialȱ unitȱ ofȱ theȱ Russianȱ Academyȱ ofȱ Educationȱinȱcollaborationȱwithȱregionalȱeducationȱcentersȱ(Bolotovȱet.ȱal.ȱ 2013).ȱ Findingsȱ fromȱ internationalȱ assessmentsȱ areȱ widelyȱ disseminatedȱ andȱ usedȱ toȱ informȱ educationalȱ policyȱ inȱ Russia.ȱ Specifically,ȱ dataȱ areȱ disseminatedȱ online,ȱ atȱ conferencesȱ andȱ seminars,ȱ andȱ withinȱ regionalȱ educationȱ centersȱ andȱ teacherȱ trainingȱ institutes.ȱ Thisȱ practiceȱ generatesȱ extensiveȱ discussionsȱ involvingȱ theȱ Ministryȱ ofȱ Educationȱ atȱ theȱ federalȱ andȱ regionalȱ level.ȱ Theȱ participationȱ ofȱ curricular,ȱ textbook,ȱ andȱ assessmentȱspecialistsȱallowsȱforȱtheȱfindingsȱofȱinternationalȱassessmentsȱ toȱserveȱasȱaȱfeedbackȱmechanismȱforȱtheȱeducationȱsystem.ȱTheȱoutcomeȱ ofȱ theseȱ discussionsȱ areȱ oftenȱ consideredȱ atȱ meetingsȱ ofȱ theȱ Publicȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 7ȱ CIVEDȱ –ȱ Civicȱ Educationȱ Study;ȱ SITESȱ –ȱ Secondȱ Informationȱ Technologyȱ inȱ Educationȱ Study.ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 11ȱ Chamberȱ andȱ thenȱ discussedȱ atȱ theȱ regionalȱ andȱ federalȱ governmentȱ level.ȱ Massȱmediaȱhasȱbeenȱaȱmajorȱengineȱforȱdisseminatingȱinternationalȱ largeȬscaleȱ assessmentsȱ resultsȱ inȱ Russia.ȱ Theȱ generalȱ publicȱ showsȱ greatȱ confidenceȱ andȱ interestȱ inȱ theseȱ assessments.ȱ Inȱ fact,ȱ thereȱ isȱ aȱ growingȱ demandȱ forȱ informationȱ onȱ comparativeȱ education.ȱ Betweenȱ 2009ȱ andȱ 2011,ȱ forȱ example,ȱ moreȱ thanȱ 300ȱ printȱ orȱ electronicȱ articlesȱ discussingȱ PISAȱ resultsȱ wereȱ published,ȱ togetherȱ withȱ interviewsȱ withȱ educationalȱ authorities.ȱ ȱ Asȱ notedȱ earlier,ȱ internationalȱ largeȬscaleȱ assessmentsȱ haveȱ hadȱ significantȱ influenceȱ onȱ educationȱ reformsȱ inȱ Russia.ȱ Thisȱ influenceȱ hasȱ beenȱ largelyȱ triggeredȱ byȱ concernsȱ aboutȱ educationalȱ qualityȱ andȱ theȱ country’sȱ abilityȱ toȱ adaptȱ toȱ aȱ globalȱ marketȱ economy.ȱ Theȱ performanceȱ ofȱRussianȱstudentsȱonȱTIMSSȱandȱPIRLSȱhasȱalwaysȱbeenȱrelativelyȱhighȱ comparedȱ toȱ otherȱ developedȱ countries.ȱ Concernsȱ aboutȱ educationalȱ qualityȱ inȱ theȱ countryȱ firstȱ aroseȱ afterȱ theȱ PISAȱ assessmentsȱ ofȱ 2000ȱ andȱ 2003ȱwhen,ȱforȱtheȱfirstȱtime,ȱRussianȱstudentsȱperformedȱatȱlevelsȱbelowȱ thatȱ ofȱ studentsȱ fromȱ otherȱ developedȱ countries.ȱ Russianȱ secondaryȱ schoolȱ studentsȱ wereȱ losingȱ theȱ relativeȱ advantageȱ theyȱ hadȱ demonstratedȱatȱtheȱprimaryȱlevel.ȱMoreover,ȱresultsȱwereȱnotȱimprovingȱ overȱ time.ȱ Theseȱ findingsȱ triggeredȱ profoundȱ reformsȱ inȱ theȱ educationȱ systemȱ inȱ orderȱ toȱ alignȱ theȱ Russianȱ educationȱ systemȱ (includingȱ curriculumȱ standards,ȱ textbooks,ȱ andȱ examinations)ȱ withȱ internationalȱ teachingȱandȱlearningȱtrends.ȱ Basedȱonȱinternationalȱassessmentȱfindings,ȱtheȱRussianȱAcademyȱofȱ Educationȱhasȱgreatlyȱsupportedȱtheȱintroductionȱofȱeducationȱreformsȱinȱ Russia.ȱ Thisȱ academyȱ playsȱ aȱ tripleȱ roleȱ inȱ theȱ educationȱ system:ȱ (1)ȱ itȱ isȱ inȱchargeȱofȱimplementingȱinternationalȱassessmentsȱinȱtheȱcountry;ȱ(2)ȱitȱ isȱ oneȱ ofȱ theȱ mainȱ developersȱ ofȱ nationalȱ curricularȱ standards;ȱ andȱ (3)ȱ itȱ providesȱ guidelinesȱ andȱ criteriaȱ forȱ textbookȱ development.ȱ Thisȱ tripleȱ roleȱ facilitatesȱ theȱ translationȱ ofȱ internationalȱ assessmentȱ findingsȱ intoȱ educationȱreforms.ȱ Reformsȱ haveȱ alsoȱ beenȱ possibleȱ thanksȱ toȱ policyȱ researchȱ basedȱ onȱ internationalȱ assessmentȱ data.ȱ Threeȱ researchȱ centersȱ inȱ Russiaȱ areȱ involvedȱ inȱ theȱ analysisȱ ofȱ internationalȱ assessments.ȱ Theȱ Russianȱ Academyȱ ofȱ Educationȱ (throughȱ itsȱ Instituteȱ ofȱ Contentȱ andȱ Methodȱ ofȱ ȱ 12ȱ YuliaȱTyumenevaȱ Education) 8 ȱ producesȱ theȱ officialȱ nationalȱ reportȱ forȱ Russia,ȱ whichȱ isȱ publiclyȱ availableȱ onlineȱ andȱ inȱ hardȱ copy.ȱ Secondaryȱ analysesȱ areȱ doneȱ byȱ theȱ Moscowȱ Schoolȱ ofȱ Socialȱ andȱ Economicȱ Sciencesȱ andȱ theȱ Higherȱ Schoolȱ ofȱ Economics.ȱ Theseȱ analysesȱ usuallyȱ focusȱ onȱ factorsȱ associatedȱ withȱ studentȱ performance,ȱ theȱ difficultyȱ levelȱ ofȱ testȱ questionsȱ orȱ problems,ȱandȱtrendsȱinȱstudentȱperformance.ȱ ȱ Internationalȱ assessmentsȱ thusȱ ledȱ toȱ changesȱ inȱ Russianȱ learningȱ standardsȱ andȱ theȱ basicȱ educationȱ curriculum.ȱ Reformsȱ toȱ primaryȱ andȱ secondaryȱ schoolȱ curriculaȱ focusedȱ onȱ newȱ skills.ȱ Atȱ theȱ primaryȱ level,ȱ curricularȱ changesȱ promoteȱ theȱ solvingȱ ofȱ visualȱ geometryȱ problems,ȱ practicalȱworkȱwithȱgeometricȱobjects,ȱworkingȱwithȱtablesȱandȱdiagrams,ȱ understandingȱ theȱ basicȱ principlesȱ ofȱ probabilityȱ theoryȱ andȱ statistics,ȱ aȱ greaterȱfocusȱonȱfractionsȱandȱpercentages,ȱunderstandingȱnumericalȱandȱ symbolicȱsequences,ȱandȱroundingȱandȱestimatingȱcalculationȱresults.ȱ ȱ Atȱ theȱ secondaryȱ level,ȱ theȱ newȱ educationalȱ standardsȱ areȱ basedȱ onȱ PISA.ȱ Moreȱ attentionȱ isȱ nowȱ givenȱ toȱ theȱ practicalȱ applicationȱ ofȱ knowledge,ȱwithȱaȱcompetencyȬbasedȱapproachȱtoȱlearningȱintroducedȱinȱ theȱ classroom.ȱ Curricularȱ changesȱ promotedȱ theȱ developmentȱ ofȱ spatialȱ concepts;ȱ theȱ applicationȱ ofȱ mathematicalȱ knowledgeȱ inȱ realȬlifeȱ situations;ȱ problemȱ solving;ȱ findingȱ intermediateȱ data;ȱ dealingȱ withȱ relationships,ȱ values,ȱ andȱ percentages;ȱ andȱ understandingȱ theȱ basicȱ principlesȱofȱprobabilityȱandȱstatistics.ȱ Textbooksȱ haveȱ beenȱ alignedȱ withȱ theȱ contentȱ andȱ skillsȱ measuredȱ byȱ internationalȱ assessmentsȱ andȱ designedȱ toȱ includeȱ moreȱ realȬlifeȱ examplesȱ andȱ problems.ȱ Theyȱ putȱ greaterȱ emphasis,ȱ forȱ example,ȱ onȱ theȱ applicationȱ ofȱ conceptualȱ modelsȱ toȱ solveȱ practicalȱ problems.ȱ Textbooksȱ areȱ nowȱ alsoȱ requiredȱ toȱ includeȱ interdisciplinaryȱ sectionsȱ thatȱ combineȱ reading,ȱmathematics,ȱandȱscienceȱtopicsȱandȱskills.ȱ Finally,ȱinternationalȱassessmentsȱhaveȱgreatlyȱshapedȱtheȱnewȱUSE,ȱ whichȱ largelyȱ adoptedȱ theȱ PISAȱ approach.ȱ Thisȱ translatedȱ intoȱ aȱ majorȱ emphasisȱ onȱ theȱ useȱ ofȱ studentȱ knowledgeȱ andȱ skillsȱ toȱ solveȱ realȬlifeȱ problemsȱ andȱ theȱ presentationȱ ofȱ problemsȱ thatȱ requireȱ knowledgeȱ andȱ skillsȱinȱdifferentȱsubjectȱareas,ȱamongȱotherȱchanges.ȱWithinȱeachȱsubjectȱ examination,ȱ aboutȱ aȱ quarterȱ ofȱ theȱ problemsȱ assessȱ students’ȱ abilitiesȱ toȱ applyȱ theirȱ knowledgeȱ toȱ newȱ situations.ȱ Inȱ mathematics,ȱ forȱ example,ȱ ȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱȱ 8 ȱ Seeȱtheȱinstitute’sȱwebsiteȱatȱwww.centeroko.ruȱ(accessedȱJulyȱ2013).ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 13ȱ studentsȱ areȱ requiredȱ toȱ independentlyȱ developȱ aȱ modelȱ andȱ methodȱ ofȱ solvingȱ aȱ realȬlifeȱ mathematicsȱ problem.ȱ Studentsȱ areȱ thenȱ requiredȱ toȱ justifyȱandȱshowȱtheȱstepsȱtakenȱtoȱfindȱtheȱmathematicalȱsolution.ȱ ȱ Theȱ USEȱ alsoȱ placesȱ greaterȱ emphasisȱ onȱ communicationȱ skills.ȱ Forȱ example,ȱ studentsȱ areȱ assessedȱ onȱ theirȱ abilityȱ toȱ explainȱ theirȱ pointȱ ofȱ viewȱ basedȱ onȱ availableȱ information,ȱ evaluateȱ writtenȱ statements,ȱ andȱ analyzeȱ theȱ pointȱ ofȱ viewȱ ofȱ aȱ sourceȱ text.ȱ Inȱ aȱ countryȱ thatȱ hadȱ noȱ traditionȱ ofȱ usingȱ multipleȬchoiceȱ questions,ȱ internationalȱ assessmentsȱ wereȱ criticalȱ toȱ validatingȱ andȱ introducingȱ thisȱ newȱ questionȱ formatȱ intoȱ theȱexaminations.ȱ LessonsȱLearnedȱ Russiaȱ hasȱ takenȱ bigȱ stepsȱ towardȱ usingȱ studentȱ assessmentȱ informationȱ asȱ aȱ systematicȱ feedbackȱ mechanismȱ forȱ theȱ educationȱ system.ȱ Nevertheless,ȱ thereȱ areȱ enormousȱ differencesȱ inȱ theȱ wayȱ informationȱ fromȱnationalȱexaminationsȱandȱinternationalȱassessmentsȱareȱused.ȱ ȱ Purposeȱofȱtheȱassessmentȱ Theȱ purposeȱ ofȱ anȱ assessmentȱ significantlyȱ influencesȱ theȱ useȱ ofȱ itsȱ findings.ȱ Theȱ USE,ȱ asȱ noted,ȱ assessesȱ studentȱ performanceȱ forȱ theȱ mainȱ purposesȱ ofȱ secondaryȱ schoolȱ certificationȱ andȱ selectionȱ intoȱ tertiaryȱ education.ȱ Inȱ contrast,ȱ internationalȱ assessmentsȱ areȱ designedȱ toȱ assessȱ studentȱ knowledgeȱ andȱ skillsȱ withȱ theȱ primaryȱ purposeȱ ofȱ informingȱ policy.ȱTheyȱareȱalsoȱdesignedȱtoȱmakeȱinternationalȱcomparisons,ȱwhichȱ drawȱ mediaȱ attentionȱ andȱ putȱ politicalȱ pressureȱ onȱ educationȱ officials,ȱ especiallyȱ inȱ theȱ caseȱ ofȱ poorȱ results.ȱ Itȱ isȱ thereforeȱ notȱ surprisingȱ thatȱ internationalȱ assessmentsȱ haveȱ hadȱ aȱ greaterȱ impactȱ onȱ theȱ reformȱ agendaȱinȱRussia.ȱ Credibilityȱ Theȱcredibilityȱofȱtheȱorganizationsȱleadingȱanȱassessment,ȱasȱwellȱasȱthatȱ ofȱ theȱ assessmentȱ instrumentsȱ andȱ procedures,ȱ isȱ crucialȱ forȱ facilitatingȱ theȱ useȱ ofȱ assessmentȱ information.ȱ Withoutȱ credibility,ȱ thereȱ isȱ noȱ politicalȱwillȱtoȱintroduceȱreformsȱbasedȱonȱassessmentȱfindings.ȱTheȱUSEȱ examinationȱ stillȱ suffersȱ fromȱ lowȱ credibilityȱ amongȱ membersȱ ofȱ theȱ ȱ 14ȱ YuliaȱTyumenevaȱ public,ȱ educationȱ practitioners,ȱ andȱ researchers.ȱ Althoughȱ theȱ federalȱ governmentȱ madeȱ greatȱ effortsȱ toȱ buildȱ trustȱ inȱ theȱ examination,ȱ itȱ stillȱ hasȱ aȱ longȱ wayȱ toȱ go.ȱ Aȱ 2011ȱ surveyȱ showedȱ thatȱ onlyȱ 20ȱ percentȱ ofȱ theȱ countryȱapprovedȱofȱtheȱUSEȱ(PublicȱOpinionȱFoundationȱ2011).ȱ ȱ Buildingȱ credibilityȱ requiresȱ publishingȱ technicalȱ informationȱ aboutȱ theȱ examinationȱ andȱ conductingȱ regularȱ validationȱ studies.ȱ Itȱ alsoȱ requiresȱ fightingȱ corruption.ȱ Unfortunately,ȱ theȱ corruptionȱ thatȱ theȱ USEȱ wasȱ intendedȱ toȱ circumventȱ hasȱ foundȱ itsȱ wayȱ intoȱ theȱ newȱ assessmentȱ system.ȱ Scandalsȱ rangingȱ fromȱ theȱ leakingȱ ofȱ examinationȱ papersȱ aheadȱ ofȱ timeȱ toȱ theȱ falsificationȱ ofȱ resultsȱ haveȱ beenȱ reported.ȱ Theseȱ practicesȱ haveȱaddedȱgreatlyȱtoȱtheȱlackȱofȱpublicȱconfidenceȱinȱtheȱUSE.ȱ ȱ Inȱ contrast,ȱ internationalȱ assessmentsȱ haveȱ highȱ credibilityȱ amongȱ membersȱ ofȱ theȱ public,ȱ educationȱ practitioners,ȱ andȱ researchers.ȱ Thisȱ credibilityȱ isȱ basedȱ onȱ theȱ reputationȱ ofȱ theȱ organizationsȱ leadingȱ theȱ assessments,ȱ asȱ wellȱ asȱ theȱ provenȱ qualityȱ ofȱ theirȱ instrumentsȱ andȱ procedures.ȱTransparencyȱinȱsharingȱtechnicalȱinformation,ȱtogetherȱwithȱ rigorousȱ qualityȱ controls,ȱ hasȱ beenȱ criticalȱ inȱ buildingȱ trustȱ inȱ internationalȱ assessments.ȱ Theȱ differentȱ levelsȱ ofȱ credibilityȱ ofȱ theȱ USEȱ andȱ internationalȱ assessmentsȱ explainȱ toȱ aȱ greatȱ degreeȱ whyȱ theȱ latterȱ haveȱhadȱaȱgreaterȱimpactȱonȱeducationalȱreformsȱinȱRussia.ȱ ȱ Involvementȱofȱkeyȱinstitutionsȱ Theȱ participationȱ ofȱ keyȱ institutionsȱ inȱ assessmentȱ activitiesȱ isȱ criticalȱ toȱ promotingȱ systemicȱ reform.ȱ Theȱ tripleȱ roleȱ playedȱ byȱ theȱ Russianȱ Academyȱ ofȱ Educationȱ (responsibleȱ forȱ implementingȱ internationalȱ assessments,ȱ developingȱ curriculumȱ standards,ȱ andȱ providingȱ textbookȱ guidelines)ȱ hasȱ allowedȱ forȱ theȱ conversionȱ ofȱ internationalȱ assessmentȱ findingsȱintoȱcurriculumȱandȱtextbooksȱchanges.ȱ Fromȱresearchȱfindingsȱtoȱpolicyȱrecommendationsȱ Researchȱ findingsȱ areȱ onlyȱ usefulȱ whenȱ theyȱ areȱ convertedȱ intoȱ policyȱ recommendations.ȱ However,ȱ theȱ Ministryȱ ofȱ Educationȱ atȱ theȱ federalȱ levelȱ doesȱ notȱ haveȱ aȱ teamȱ inȱ placeȱ toȱ analyzeȱ USEȱ dataȱ andȱ produceȱ policyȱreportsȱandȱrecommendations.ȱConsequently,ȱUSEȱfindingsȱhaveȱaȱ limitedȱ impactȱ onȱ educationȱ policy.ȱ Inȱ contrast,ȱ researchȱ findingsȱ fromȱ PISAȱandȱTIMSSȱareȱgenerallyȱaccompaniedȱbyȱpolicyȱrecommendations,ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 15ȱ whichȱ haveȱ beenȱ widelyȱ adoptedȱ byȱ theȱ federalȱ governmentȱ inȱ Russiaȱ andȱhaveȱsignificantlyȱcontributedȱtoȱimprovingȱtheȱeducationȱsystem.ȱ Assessingȱtheȱwholeȱschoolȱcycleȱ Assessmentsȱ thatȱ targetȱ differentȱ gradesȱ inȱ theȱ schoolȱ cycleȱ areȱ moreȱ likelyȱ toȱ beȱ usedȱ forȱ policyȱ decisions.ȱ Theseȱ assessmentsȱ provideȱ aȱ moreȱ completeȱ pictureȱ ofȱ theȱ qualityȱ ofȱ theȱ educationȱ system,ȱ allowingȱ educationȱ policyȱ makersȱ toȱ diagnoseȱ andȱ rectifyȱ challengesȱ atȱ variousȱ stagesȱ ofȱ schooling.ȱ Internationalȱ assessmentsȱ benefitȱ fromȱ theȱ factȱ thatȱ theyȱ targetȱ differentȱ gradesȱ andȱ schoolȱ cycles.ȱ Forȱ example,ȱ PIRLSȱ assessesȱ studentsȱ atȱ gradeȱ 4;ȱ TIMSS,ȱ atȱ gradesȱ 4,ȱ 8,ȱ andȱ 11;ȱ andȱ PISA,ȱ 15ȬyearȬoldsȱ (typically,ȱ gradeȱ 10).ȱ Theȱ USEȱ isȱ limitedȱ toȱ gradeȱ 11,ȱ whenȱ studentsȱfinishȱsecondaryȱeducation.ȱTherefore,ȱthereȱisȱaȱfeelingȱthatȱitȱisȱ alreadyȱ tooȱ lateȱ toȱ tryȱ toȱ introduceȱ improvementsȱ withȱ respectȱ toȱ theseȱ students,ȱ sinceȱ theyȱ haveȱ alreadyȱ left,ȱ orȱ areȱ aboutȱ toȱ leave,ȱ theȱ schoolȱ system.ȱ ȱ Stakesȱattachedȱtoȱanȱassessmentȱ Theȱ usesȱ andȱ impactȱ ofȱ assessmentȱ informationȱ varyȱ widelyȱ dependingȱ onȱ theȱ stakesȱattachedȱ toȱ it.ȱ Theȱ USEȱ isȱaȱ highȬstakesȱ assessmentȱ thatȱ hasȱ directȱ consequencesȱ onȱ studentȱ certificationȱ andȱ selectionȱ intoȱ tertiaryȱ education.ȱ Itȱ alsoȱ hasȱ aȱ strongȱ impactȱ atȱ theȱ schoolȱ level,ȱ withȱ teachersȱ preparingȱ studentsȱ andȱ studentsȱ studyingȱ forȱ theȱ examinations.ȱ Theseȱ seemȱ toȱ beȱ shortȬtermȱ strategiesȱ forȱ takingȱ theȱ examinationȱ atȱ theȱ endȱ ofȱ theȱschoolȱyear.ȱ ȱ Asȱ discussedȱ earlier,ȱ theȱ USEȱ isȱ alsoȱ beingȱ usedȱ asȱ anȱ inputȱ intoȱ schoolȱ accreditationȱ andȱ decisionsȱ onȱ schoolȱ andȱ teacherȱ bonuses.ȱ Theseȱ additionalȱ stakesȱ haveȱ createdȱ undueȱ pressureȱ forȱ superiorȱ results,ȱ producingȱ severalȱ negativeȱ consequences.ȱ Someȱ teachersȱ endȱ upȱ usingȱ theȱ USEȱ frameworkȱ insteadȱ ofȱ theȱ curriculumȱ standards,ȱ whichȱ narrowsȱ theȱ curriculumȱ taughtȱ inȱ class.ȱ Additionally,ȱ theseȱ incentivesȱ haveȱ ledȱ toȱ reportsȱ ofȱ corruptionȱ inȱ theȱ administrationȱ ofȱ theȱ USE,ȱ theȱ leakingȱ ofȱ examinations,ȱandȱtheȱfalsificationȱofȱresults.ȱ Onȱ theȱ otherȱ hand,ȱ internationalȱ assessmentsȱ areȱ lowȬstakesȱ assessmentsȱ thatȱ doȱ notȱ haveȱ anyȱ impactȱ onȱ students,ȱ theirȱ teachers,ȱ orȱ schoolsȱ (althoughȱ theyȱ haveȱ aȱ growingȱ impactȱ onȱ holdingȱ theȱ governmentȱ accountable).ȱ Theseȱ assessmentsȱ have,ȱ however,ȱ hadȱ aȱ ȱ 16ȱ YuliaȱTyumenevaȱ strongȱ influenceȱ onȱ educationalȱ reformȱ inȱ Russia,ȱ leadingȱ toȱ longȬtermȱ strategiesȱforȱimprovingȱeducationalȱquality.ȱ Assessmentȱframeworkȱ Assessmentsȱ shouldȱ haveȱ wellȬarticulatedȱ assessmentȱ frameworksȱ thatȱ justifyȱ theȱ assessmentȱ principlesȱ andȱ approach,ȱ describeȱ theȱ knowledgeȱ andȱ skillsȱ toȱ beȱ tested,ȱ andȱ specifyȱ theȱ typesȱ ofȱ problemsȱ andȱ questionȱ formatȱ thatȱ willȱ beȱ presentedȱ toȱ students.ȱ Theȱ strongȱ frameworkȱ ofȱ internationalȱ assessmentsȱ hasȱ providedȱ aȱ clearȱ roadmapȱ forȱ establishingȱ educationalȱ goalsȱ toȱ guideȱ curricularȱ development,ȱ instruction,ȱ andȱ assessmentȱ inȱ Russia.ȱ Forȱ example,ȱ theȱ PISAȱ approachȱ toȱ assessmentȱ identifiesȱkeyȱobjectivesȱthatȱhaveȱbeenȱadoptedȱintoȱtheȱnewȱeducationalȱ standardsȱ inȱ Russia.ȱ Theseȱ standardsȱ haveȱ sinceȱ beenȱ incorporatedȱ intoȱ theȱ designȱ ofȱ theȱ USE.ȱ Theseȱ changesȱ haveȱ createdȱ greaterȱ alignmentȱ betweenȱ nationalȱ educationȱ standardsȱ andȱ examinations,ȱ onȱ theȱ oneȱ hand,ȱandȱinternationalȱassessments,ȱonȱtheȱother.ȱ Performanceȱlevelsȱ Problematicȱ methodsȱ forȱ determiningȱ studentȱ performanceȱ levelsȱ hamperȱ theȱ useȱ ofȱ thisȱ informationȱ forȱ pedagogicalȱ purposes.ȱ Inȱ Russia,ȱ USEȱ performanceȱ levelsȱ areȱ notȱ definedȱ basedȱ onȱ nationalȱ curriculumȱ standards.ȱ Instead,ȱ theyȱ areȱ definedȱ basedȱ onȱ politicalȱ considerations.ȱ Theȱ methodologyȱ usedȱ byȱ internationalȱ assessmentsȱ toȱ defineȱ performanceȱ levelsȱ seemsȱ moreȱ technicallyȱ soundȱ andȱ fair.ȱ Theseȱ assessmentsȱ typicallyȱ reportȱ whatȱ studentsȱ knowȱ andȱ canȱ doȱ atȱ differentȱ performanceȱ levels,ȱ andȱ theȱ proceduresȱ forȱ definingȱ theseȱ performanceȱ levelsȱisȱmoreȱstableȱoverȱtime.ȱAsȱaȱconsequence,ȱtheȱperformanceȱlevelsȱ usedȱ inȱ internationalȱ assessmentsȱ haveȱ beenȱ aȱ richȱ sourceȱ ofȱ informationȱ forȱRussianȱeducators.ȱ Comparisonsȱofȱassessmentȱresultsȱoverȱtimeȱ Aȱ keyȱ indicatorȱ ofȱ educationalȱ qualityȱ isȱ howȱ farȱ studentsȱ progressȱ fromȱ oneȱ assessmentȱ roundȱ toȱ theȱ next.ȱ Thisȱ requiresȱ comparingȱ studentȱ resultsȱ fromȱ differentȱ assessmentȱ roundsȱ and,ȱ usually,ȱ differentȱ testȱ versions.ȱ Toȱ doȱ thisȱ inȱ aȱ validȱ way,ȱ anȱ assessmentȱ mustȱ meetȱ certainȱ technicalȱ characteristics,ȱ whichȱ usuallyȱ involveȱ puttingȱ questionsȱ fromȱ differentȱ testsȱ onȱ theȱ sameȱ scaleȱ (i.e.,ȱ equating).ȱ Asȱ alreadyȱ noted,ȱ USEȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 17ȱ instrumentsȱareȱnotȱequatedȱfromȱyearȱtoȱyear.ȱTherefore,ȱitȱisȱimpossibleȱ toȱ tellȱ ifȱ changesȱ inȱ examinationȱ resultsȱ areȱ dueȱ toȱ “real”ȱ changesȱ inȱ studentȱperformance,ȱchangesȱinȱtheȱdifficultyȱlevelȱofȱtheȱtest,ȱorȱchangesȱ inȱ theȱ cutȬoffȱ pointsȱ usedȱ toȱ defineȱ performanceȱ levels.ȱ Thisȱ uncertaintyȱ severelyȱ limitsȱ theȱ validityȱ ofȱ USEȱ dataȱ forȱ monitoringȱ trendsȱ andȱ holdingȱ schoolsȱ accountable.ȱ Theȱ Russianȱ federalȱ governmentȱ warnsȱ againstȱ makingȱ conclusionsȱ basedȱ onȱ directȱ yearlyȱ comparisons.ȱ Nevertheless,ȱ regionalȱ ministriesȱ ofȱ educationȱ frequentlyȱ reportȱ yearlyȱ comparisons,ȱ leadingȱ toȱ inappropriateȱ conclusionsȱ aboutȱ changesȱ inȱ studentȱ performance.ȱ Inȱ contrast,ȱ internationalȱ assessmentsȱ useȱ technicallyȱ soundȱ techniquesȱ thatȱ allowȱ forȱ theȱ equatingȱ ofȱ testsȱ fromȱ differentȱyears.ȱThisȱallowsȱforȱvalidȱyearlyȱcomparisonsȱofȱtestȱresults.ȱ Backgroundȱinformationȱ Effectiveȱuseȱofȱassessmentȱdataȱisȱgreatlyȱconditionedȱonȱtheȱavailabilityȱ ofȱstudent,ȱteacher,ȱandȱschoolȱbackgroundȱinformation.ȱThisȱinformationȱ isȱ criticalȱ toȱ understandingȱ theȱ factorsȱ relatedȱ toȱ studentȱ performance,ȱ whetherȱ superiorȱ orȱ inferior.ȱ Internationalȱ assessmentsȱ haveȱ greatlyȱ benefitedȱ fromȱ theȱ collectionȱ ofȱ aȱ vastȱ arrayȱ ofȱ backgroundȱ information.ȱ Forȱ instance,ȱ TIMSSȱ includesȱ aȱ surveyȱ ofȱ students,ȱ theirȱ teachers,ȱ andȱ schoolsȱinȱorderȱtoȱidentifyȱfactorsȱaffectingȱtheȱqualityȱofȱeducation.ȱThisȱ assessmentȱalsoȱcollectsȱdetailedȱinformationȱaboutȱcurriculumȱstandardsȱ andȱtheirȱimplementationȱinȱtheȱclassroom.ȱThisȱinformationȱisȱthenȱusedȱ byȱ theȱ countriesȱ toȱ informȱ decisionȱ makingȱ onȱ educationȱ policy.ȱ Inȱ contrast,ȱ theȱ USEȱ doesȱ notȱ collectȱ backgroundȱ information.ȱ Toȱ date,ȱ theȱ examinationsȱ onlyȱ collectȱ informationȱ onȱ studentȱ genderȱ andȱ schoolȱ location.ȱMeaningfulȱanalysesȱwouldȱrequireȱknowingȱtheȱsocioeconomicȱ statusȱofȱstudents,ȱschools,ȱandȱteachers,ȱamongȱotherȱdataȱpoints.ȱ Accessȱtoȱassessmentȱdatabaseȱ Limitedȱ accessȱ toȱ theȱ USEȱ databaseȱ severelyȱ restrictsȱ theȱ country’sȱ capacityȱ toȱ makeȱ useȱ ofȱ theseȱ richȱ dataȱ toȱ informȱ decisionȱ making.ȱ Inȱ Russia,ȱ onlyȱ theȱ Ministryȱ ofȱ Educationȱ atȱ theȱ federalȱ levelȱ hasȱ fullȱ accessȱ toȱtheseȱdata.ȱForȱreasonsȱofȱconfidentiality,ȱveryȱrestrictedȱinformationȱisȱ circulatedȱbeyondȱthisȱorganization;ȱthisȱinformation,ȱmoreover,ȱdoesȱnotȱ seemȱsufficientȱforȱmeaningfulȱsecondaryȱanalysis.ȱThisȱmeansȱthatȱotherȱ branchesȱ ofȱ government,ȱ independentȱ researchers,ȱ andȱ educationȱ officesȱ ȱ 18ȱ YuliaȱTyumenevaȱ areȱunableȱtoȱuseȱtheȱdataȱtoȱinformȱtheirȱpractices.ȱInȱ2011,ȱhowever,ȱtheȱ Committeeȱ onȱ Educationȱ ofȱ theȱ Publicȱ Chamberȱ initiatedȱ discussionsȱ aboutȱ possibleȱ limitedȱ accessȱ toȱ aȱ depersonalizedȱ USEȱ database.ȱ Byȱ contrast,ȱ openȱ accessȱ toȱ internationalȱ assessmentȱ databasesȱ enablesȱ independentȱ researchersȱ toȱ contributeȱ uniqueȱ analysesȱ toȱ theȱ researchȱ andȱ policyȱ agendaȱ inȱ Russia.ȱ Theseȱ effortsȱ haveȱ complementedȱ theȱ researchȱ doneȱ byȱ theȱ Russianȱ Academyȱ ofȱ Educationȱ andȱ haveȱ compensatedȱ forȱ federalȱ agencies’ȱ limitedȱ resourcesȱ toȱ conductȱ furtherȱ research.ȱ *ȱ*ȱ*ȱ*ȱ*ȱ*ȱ*ȱ*ȱ Countriesȱ aimingȱ toȱ improveȱ theȱ disseminationȱ andȱ useȱ ofȱ studentȱ assessmentȱ informationȱ shouldȱ beȱ awareȱ ofȱ theȱ factorsȱ discussedȱ inȱ thisȱ sectionȱ whenȱ designingȱ theirȱ assessmentȱ systems.ȱ Byȱ usingȱ theȱ rightȱ combinationȱofȱfactorsȱinȱtheȱdesign,ȱtheyȱwillȱbe inȱaȱbetterȱpositionȱtoȱ meetȱtheȱultimateȱpurposeȱofȱaȱstudentȱassessmentȱsystem:ȱtoȱimproveȱ student learningȱandȱeducationȱquality.ȱ ȱ ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 19ȱ Bibliographyȱ Bochenkov,ȱS.,ȱed.ȱ2011.ȱ“AnalysisȱofȱResultsȱofȱUSEȱ2011ȱinȱtheȱChuvashȱ Republic”ȱ (inȱ Russian).ȱ Ministryȱ ofȱ Educationȱ andȱ Youthȱ Policyȱ ofȱ theȱ Chuvashȱ Republic,ȱ Cheboskary.ȱ http://ege21.ru/index/0Ȭ2ȱ (accessedȱJulyȱ2013).ȱCitedȱwithȱpermissionȱofȱtheȱministry.ȱ ———.ȱ 2012.ȱ “Analysisȱ andȱ Interpretationȱ ofȱ USEȱ 2012ȱ Results”ȱ (inȱ Russian).ȱ Managementȱ inȱ Education:ȱ Theoryȱ andȱ Practiceȱ [Russianȱ Academyȱ ofȱ Education]ȱ 3:ȱ 9–34.ȱ http://www.iuorao.ru/ȱ 20120930/216ȬȬȬȬȬȬ2012ȱ ———.ȱ 2013.ȱ “Teacher,ȱ School,ȱ Educationȱ Systemȱ throughȱ theȱ Prismȱ ofȱ USE”ȱ(inȱRussian).ȱProblemsȱofȱModernȱEducationȱ3:ȱ27–47.ȱ Bolotov,ȱ V.,ȱ G.ȱ Kovaleva,ȱ M.ȱ Pinskaya,ȱ andȱ I.ȱ Valdman,ȱ 2013.ȱ “Developingȱ theȱ Enablingȱ Contextȱ forȱ Studentȱ Assessmentȱ inȱ Russia.”ȱ SABERȱ Studentȱ Assessmentȱ Workingȱ Paperȱ 9.ȱ Worldȱ Bank,ȱWashington,ȱDC.ȱ Centerȱ forȱ Qualityȱ Educationȱ Assessmentȱ atȱ INCMȱ RAE.ȱ Variousȱ years.ȱ AnalyticalȱreportsȱonȱPISA,ȱTIMSS,ȱandȱPIRLSȱ(inȱRussian).ȱCenterȱ forȱ Qualityȱ Educationȱ Assessment,ȱ Moscow.ȱ http://www.ȱ centeroko.ru/public.htm.ȱ Clarke,ȱ M.ȱ 2012ȱ “Whatȱ Mattersȱ Mostȱ forȱ Studentȱ Assessmentȱ Systems:ȱ Aȱ Frameworkȱ Paper.”ȱ SABERȱ Studentȱ Assessmentȱ Workingȱ Paperȱ 1.ȱ WorldȱBank,ȱWashington,ȱDC.ȱ Derkachev,ȱ P.,ȱ andȱ I.ȱ Suvorova.ȱ “Unifiedȱ Stateȱ Examinationȱ asȱ aȱ Wayȱ toȱ Assessȱ theȱ Potentialȱ toȱ Obtainȱ Higherȱ Education”ȱ (inȱ Russian).ȱ Inȱ Higherȱ Schoolȱ ofȱ Economics,ȱ “Collectionȱ ofȱ Articlesȱ ofȱ HSEȱ GraduateȱStudents,”ȱ34–64.ȱHigherȱSchoolȱofȱEconomics,ȱMoscow.ȱ ȱ Federalȱ Instituteȱ ofȱ Pedagogicalȱ Measurement.ȱ 2009.ȱ “Resultsȱ ofȱ Unifiedȱ StateȱExaminationȱ(May–June,ȱ2009)”ȱ(inȱRussian).ȱFederalȱInstituteȱ ofȱ Pedagogicalȱ Measurement,ȱ Moscow.ȱ p.ȱ 69.ȱ http://www.fipi.ru/ȱ view/sections/138/docs/478.htmlȱ(accessedȱJulyȱ2013).ȱ ȱ 20ȱ YuliaȱTyumenevaȱ Kovaleva,ȱ G.,ȱ Baranova,ȱ V.,ȱ Koshelenko,ȱ N.,ȱ Krasnianskaya,ȱ K.,ȱ Krasnokutskaya,ȱKrasnovsky,ȱE.,ȱ&ȱLoginova,ȱO.ȱ2012.ȱMainȱResultsȱ ofȱ theȱ Internationalȱ PISAȱ Assessmentȱ 2009;ȱ Analyticȱ Reportȱ (inȱ Russian).ȱMoscow:ȱʂɧXȬPRESS.ȱ Ministerȱ ofȱ Educationȱ andȱ Scienceȱ ofȱ theȱ Krasnoyarskȱ Kraiȱ (Territory).ȱ 2010.ȱ “Implementingȱ theȱ Nationalȱ Educationȱ Initiative—Ourȱ Newȱ School:ȱ Prospectsȱ forȱ theȱ 2010–2011ȱ Academicȱ Year”ȱ (inȱ Russian).ȱ ReportȱpresentedȱatȱtheȱAugustȱ2010ȱTeachers’ȱCouncil,ȱKrasnoyarsk.ȱ KrasnoyarskȱRegionalȱMinistryȱofȱEducationȱandȱScience.ȱ ȱ Municipalȱ Instituteȱ ofȱ Generalȱ Educationȱ ofȱ theȱ Chuvashȱ Republic.ȱ N.d.ȱ “Scheduleȱ forȱ Preparingȱ forȱ theȱ USEȱ inȱ theȱ 2011–2012ȱ Schoolȱ Year/ȱ MOUȱ BichurgaȬBaishevskayaȱ Secondaryȱ School”ȱ (inȱ Russian).ȱ Municipalȱ Instituteȱ ofȱ Generalȱ Education,ȱ Cheboskary,ȱ Chuvashȱ Republic.ȱ http://www.21419s02.edusite.ru/p213aa1.htmlȱ (accessedȱ Julyȱ2013).ȱ Poldin,ȱ O.V.ȱ 2011.ȱ “Predictingȱ Academicȱ Performanceȱ inȱ Collegeȱ Basedȱ onȱUSEȱResults”ȱ(inȱRussian).ȱAppliedȱEconometricsȱ1ȱ(21):ȱ56–69.ȱ ȱ Publicȱ Opinionȱ Foundation.ȱ 2011.ȱ “Unifiedȱ Stateȱ Examination.”ȱ Dominantyȱ 23ȱ (Septemberȱ 6,ȱ 2011).ȱ Publicȱ Opinionȱ Foundation,ȱ Moscow.ȱhttp://bd.fom.ru/pdf/d23ege11.pdf.ȱ Tyumeneva,ȱ Y.,ȱ andȱ Y.ȱ Kuzmina.ȱ 2012.ȱ “Theȱ Effectȱ ofȱ Oneȱ Extraȱ Yearȱ ofȱ Schoolingȱ onȱ PISAȱ Results:ȱ Aȱ Caseȱ ofȱ Countriesȱ withȱ Differentȱ Trackingȱ Systems”ȱ (Decemberȱ 21,ȱ 2012).ȱ Higherȱ Schoolȱ ofȱ EconomicsȱResearchȱPaperȱWPȱBRPȱ08/EDU/2012.ȱAvailableȱonȱtheȱ Socialȱ Scienceȱ Researchȱ Networkȱ (SSRN).ȱ http://ssrn.com/abstract=ȱ 2192462ȱorȱhttp://dx.doi.org/10.2139/ssrn.2192462.ȱ Tyumeneva,ȱ Y.,ȱ andȱ A.ȱ Valdman.ȱ 2013.ȱ “Theȱ Firstȱ Dataȱ fromȱ theȱ Comparativeȱ Analysisȱ ofȱ theȱ Resultsȱ ofȱ theȱ TIMSSȱ 2011ȱ andȱ PISAȱ 2012ȱTests,ȱAdministeredȱtoȱtheȱSameȱSampleȱofȱRussianȱStudents.”ȱ Paperȱ presentedȱ atȱ theȱ internationalȱ conference,ȱ “Russianȱ EducationȱinȱtheȱMirrorȱofȱtheȱInternationalȱComparativeȱStudies,”ȱ Juneȱ 19–20,ȱ 2013,ȱ Russianȱ Academyȱ ofȱ Education,ȱ Moscow.ȱ http://www.centeroko.ru/conf2013/conf2013_eng.htm.ȱ ȱ DisseminatingȱandȱUsingȱStudentȱAssessmentȱInformationȱinȱRussiaȱ 21ȱ Usefulȱwebsitesȱ Centerȱ forȱ Evaluatingȱ theȱ Qualityȱ ofȱ Educationȱ (inȱ Russian):ȱ www.centeroko.ruȱ InternationalȱAssociationȱforȱtheȱEvaluationȱofȱEducationalȱAchievement:ȱ www.iea.nlȱ Organizationȱ forȱ Economicȱ CoȬoperationȱ andȱ Developmentȱ Programmeȱ forȱInternationalȱStudentȱAssessment:ȱwww.oecd.org/pisa/ȱ TIMSSȱandȱPIRLSȱInternationalȱStudyȱCenter:ȱtimss.bc.eduȱ ȱ 22ȱ YuliaȱTyumenevaȱ  ȱ ȱ ȱ ȱ  ȱ ȱ ȱ ȱ List of papers published in the current Working Paper series  &ODUNH0´:KDW0DWWHUV0RVWIRU6WXGHQW$VVHVVPHQW6\VWHPV $)UDPHZRUN3DSHUµ(Available in Russian and Portuguese).  5DPLUH]0´'HYHORSLQJWKH(QDEOLQJ&RQWH[WIRU6WXGHQW $VVHVVPHQWLQ&KLOHµ(Available in Russian).  5DPLUH]0´'LVVHPLQDWLQJDQG8VLQJ6WXGHQW$VVHVVPHQW ,QIRUPDWLRQLQ&KLOHµ(Available in Russian).  /LEHUPDQ-DQG0&ODUNH´5HYLHZRI:RUOG%DQN6XSSRUW IRU6WXGHQW$VVHVVPHQW$FWLYLWLHVLQ&OLHQW&RXQWULHVµ(Available in Russian).  )ORFNWRQ/´7KH'HYHORSPHQWRIWKH6WXGHQW$VVHVVPHQW6\VWHP LQ1HZ=HDODQGµ(Available in Russian).  $OOHQ5´'HYHORSLQJWKH(QDEOLQJ&RQWH[WIRU6FKRRO%DVHG $VVHVVPHQWLQ4XHHQVODQG$XVWUDOLDµ(Available in Russian).  &DVWUR0´'HYHORSLQJWKH(QDEOLQJ&RQWH[WIRU6WXGHQW $VVHVVPHQWLQ%UD]LOµ(Available in Russian).  .DQMHH$DQG6$FDQD´'HYHORSLQJWKH(QDEOLQJ&RQWH[WIRU 6WXGHQW$VVHVVPHQWLQ8JDQGDµ(Available in Russian).  %RORWRY9*.RYDOHYD03LQVND\DDQG,9DOGPDQ´'HYHORSLQJ WKH(QDEOLQJ&RQWH[WIRU6WXGHQW$VVHVVPHQWLQWKH5XVVLDQ)HGHUDWLRQµ 7\XPHQHYD<´'LVVHPLQDWLQJDQG8VLQJ6WXGHQW$VVHVVPHQW ,QIRUPDWLRQLQ5XVVLDµ Forthcoming papers 6HQJKRU2XVPDQH´'LVVHPLQDWLQJDQG8VLQJ6WXGHQW$VVHVVPHQW ,QIRUPDWLRQLQ7KH*DPELDµ This case study examines how the Russian Federation disseminates and uses information from its student assessment system, drawing lessons for other countries seeking to more effectively use their own assessment data. 5XVVLD·V8QLÀHG6WDWH([DPLQDWLRQ 86( LVSULPDULO\XVHGIRUVWXGHQW VHOHFWLRQDQGFHUWLÀFDWLRQSXUSRVHVEXWDYDULHW\RIRWKHUXVHVKDYHEHHQ DWWDFKHGWRLWLQFOXGLQJLQIRUPLQJSHGDJRJ\HQVXULQJDFFRXQWDELOLW\DQG PRQLWRULQJHGXFDWLRQTXDOLW\7KLVYDULHW\RIXVHVKDVKDGERWKSRVLWLYH and negative consequences for the school system. Information from LQWHUQDWLRQDOODUJHVFDOHDVVHVVPHQWV HJ3,6$7,066DQG3,5/6 KDV EHHQZLGHO\XVHGWRLQWURGXFHUHIRUPVLQWKHVFKRROV\VWHP)DFWRUVWKDW KDYHDIIHFWHGWKHGLIIHUHQWLDOXVHRI86(DQGLQWHUQDWLRQDOODUJHVFDOH DVVHVVPHQWGDWDLQFOXGHDPRQJRWKHUVWKHSXUSRVHRIDJLYHQDVVHVVPHQW LWVGHVLJQIHDWXUHVDQGOHYHORIFUHGLELOLW\DQGDFFHVVWRWKHDVVHVVPHQW GDWDEDVH Yulia Tyumeneva, Head, Educational and Psychological Measurement MA program, Higher School of Economics, Russia The Russia Education Aid for Development Trust Fund is a collaboration between the Russian Federation and the World Bank that supports the improvement of student learning outcomes in low-income countries through the development of robust student assessment systems. Visit the READ website at www.worldbank.org/readtf for additional information.