DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Democratic Republic of the Congo SABER Country STUDENT ASSESSMENT Report 2014 Key Policy Areas for Student Assessment Status 1. Classroom Assessment In the Democratic Republic of the Congo (DRC), classroom assessment practices by teachers are perceived as being of ‘moderate’ quality. Resources available to support teachers’ classroom assessment activities include textbooks and rubrics for grading students’ work. Several formal mechanisms, such as pre- and in-service teacher training, also are in place to ensure that all teachers develop competencies in classroom assessment. At the same time, there is no national document that provides overall guidelines for classroom assessment practice, and there are limited mechanisms in place to monitor the quality of these practices across the education system. 2. Examinations The Examen d'Etat has been administered in its current form to all students in grade 12 for over 20 years. A formal policy authorizes the examination and provides its framework. Results are used to certify secondary school completion and make decisions about student selection to tertiary education or for technical jobs. Regular funding is provided by the government and through student fees. Technical commissions and teams involved in different aspects of the examination process implement the formal mechanisms that ensure its quality, such as training all administrators according to a standard protocol. At the same time, there are few opportunities available to key stakeholders (e.g., examination staff, educators, other assessment specialists) to learn more about the examination. Another issue is that students’ results are not confidential. 3. National Large-Scale Assessment (NLSA) Since 2010, the Early Grade Reading Assessment (EGRA) has been implemented every two years for the purposes of monitoring education quality at the country level and supporting schools and teachers. EGRA is administered to a regionally representative random sample of students in grades 2, 4, and 6. A formal policy document authorizes the assessment. Results have influenced the reading curriculum at the primary level and supported the introduction of an in-service teacher training module focused on reading. However, to date, funding for EGRA has solely been provided by external donors. In addition, there are limited opportunities available to key stakeholders in the country to learn more about EGRA, including its results. 4. International Large-Scale Assessment (ILSA) The DRC completed its participation in PASEC in 2010, led by a team in the Ministry of Education’s Directorate for Planning. There is no country-level policy document addressing the country’s participation in PASEC or other ILSAs, and funds for the country’s involvement in PASEC have primarily been provided by external donors. The country also offers minimal opportunities to learn about ILSAs. PASEC results and information have been disseminated using a limited number of communication strategies, specifically, through providing copies of the country-level and international reports to key stakeholders in the country. THE WORLD BANK DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Introduction SABER-Student Assessment methodology The Democratic Republic of the Congo (DRC) has focused The SABER-Student Assessment framework is built on on increasing student learning outcomes by improving the available evidence base for what an effective the quality of education in the country. An effective assessment system looks like. The framework provides student assessment system is an important component guidance on how countries can build more effective to improving education quality and learning outcomes as student assessment systems. The framework is it provides the necessary information to meet structured around two main dimensions of assessment stakeholders’ decision-making needs. In order to gain a systems: the types/purposes of assessment activities and better understanding of the strengths and weaknesses of the quality of those activities. its existing assessment system, DRC decided to benchmark this system using standardized tools Assessment types and purposes developed under The World Bank’s Systems Approach for Better Education Results (SABER) program. SABER is Assessment systems tend to be comprised of three main an evidence-based program to help countries types of assessment activities, each of which serves a systematically examine and strengthen the performance different purpose and addresses different information of different aspects of their education systems. needs. These three main types are: classroom assessment, examinations, and large-scale, system level assessments. What is SABER-Student Assessment? Classroom assessment provides real-time information to support ongoing teaching and learning in individual SABER-Student Assessment is a component of the SABER classrooms. Classroom assessments use a variety of program that focuses specifically on benchmarking formats, including observation, questioning, and paper- student assessment policies and systems. The goal of and-pencil tests, to evaluate student learning, generally SABER-Student Assessment is to promote stronger on a daily basis. assessment systems that contribute to improved education quality and learning for all. Examinations provide a basis for selecting or certifying students as they move from one level of the education National governments and international agencies are system to the next (or into the workforce). All eligible increasingly recognizing the key role that assessment of students are tested on an annual basis (or more often if student learning plays in an effective education system. the system allows for repeat testing). Examinations cover The importance of assessment is linked to its role in: the main subject areas in the curriculum and usually involve essays and multiple-choice questions. (i) providing information on levels of student learning and achievement in the system; (ii) monitoring trends in education quality over Large-scale, system-level assessments provide feedback time; on the overall performance of the education system at (iii) supporting educators and students with real- particular grades or age levels. These assessments time information to improve teaching and typically cover a few subjects on a regular basis (such as learning; and every 3 to 5 years), are often sample based, and use (iv) holding stakeholders accountable for results. multiple-choice and short-answer formats. They may be national or international in scope. Appendix 1 summarizes the key features of these main types of assessment activities. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional The indicators are identified based on a combination of and organizational structures for designing, carrying out, criteria, including: or using results from the assessment; the availability of sufficient and stable sources of funding; and the x professional standards for assessment; presence of trained assessment staff. x empirical research on the characteristics of effective assessment systems, including analysis of the characteristics that differentiate between the System alignment refers to the extent to which the assessment systems of low- versus high-performing assessment is aligned with the rest of the education nations; and system. This includes the degree of congruence between x theory — that is, general consensus among experts assessment activities and system learning goals, that it contributes to effective assessment. standards, curriculum, and pre- and in-service teacher training. Levels of development Assessment quality refers to the psychometric quality of the instruments, processes, and procedures for the The World Bank has developed a set of standardized assessment activity. It covers such issues as design and questionnaires and rubrics for collecting and implementation of assessment activities, analysis and evaluating data on the three assessment types and interpretation of student responses to those activities, related quality drivers. and the appropriateness of how assessment results are reported and used. The questionnaires are used to collect data on the characteristics of the assessment system in a particular Crossing the quality drivers with the different country. The information from the questionnaires is then assessment types/purposes provides the framework and applied to the rubrics in order to judge the development broad indicator areas shown in Table 1. This framework level of the country’s assessment system in different is a starting point for identifying indicators that can be areas. used to review assessment systems and plan for their improvement. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the development level of its assessment activities compared to best or SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 recommended practice in each area. For each indicator, Education in the Democratic Republic of the the rubric displays four development levels—Latent, Emerging, Established, and Advanced. These levels are Congo artificially constructed categories chosen to represent key stages on the underlying continuum for each The DRC is a low-income country in sub-Saharan Africa. indicator. Each level is accompanied by a description of GDP per capita (current US$) is $454, and annual growth what performance on the indicator looks like at that was approximately 8.5% in 2013. level. Two types of public schools operate in the country - x Latent is the lowest level of performance; it schools that are directly controlled by the government represents absence of, or deviation from, the desired and schools that are known as government-regulated attribute. schools. The government-regulated schools are funded x Emerging is the next level; it represents partial by the government (the government is responsible for presence of the attribute. key activities, such as paying teachers and other staff, x Established represents the acceptable minimum providing inspectors for pedagogical quality assurance, standard. and certifying students) and are managed by church x Advanced represents the ideal or current best organizations. Due to limited support from the practice. government, the Catholic Church negotiated additional financial assistance from parents that would provide for A summary of the development levels for each teacher salaries. While this support was meant to be assessment type is presented in Appendix 3. temporary, it has continued because the government’s education budget has been insufficient to fully cover In reality, assessment systems are likely to be at teacher salaries. Private schools accredited by the different levels of development in different areas. For government and private schools not accredited by the example, a system may be Established in the area of government also operate in the country. examinations, but Emerging in the area of large-scale, system-level assessment, and vice versa. While intuition The Ministry of Primary, Secondary, and Technical suggests that it is probably better to be further along in Education 1 adopted the Interim Education Plan (2012- as many areas as possible, the evidence is unclear as 2014), the country’s first education sector plan, which to whether it is necessary to be functioning at laid out three objectives: increasing access to schooling, Advanced levels in all areas. Therefore, one might view equity, and retention in school; improving the quality the Established level as a desirable minimum outcome to and relevance of education; and strengthening achieve in all areas, but only aspire beyond that in those governance in the education sector. The government has areas that most contribute to the national vision or worked to operationalize the education sector plan by priorities for education. In line with these considerations, increasing the amount of the government’s budget the ratings generated by the rubrics are not meant to be allocated to education to 9.5% in 2011, and to more than additive across assessment types (that is, they are not 14% in 2013. meant to be added to create an overall rating for an assessment system; they are only meant to produce an overall rating for each assessment type). The Detailed information was collected on the DRC’s student methodology for assigning development levels is assessment system using the SABER-Student Assessment summarized in Appendix 4. questionnaires and rubrics. It is important to remember that these tools primarily focus on benchmarking a country’s policies and arrangements for assessment activities at the system or macro level. Additional data 1 In December 2014, the Ministry of Primary, Secondary, and Technical Education, following the transfer of vocational training to a new ministry, the Education has been renamed Ministry of Primary, Secondary and Civics Ministry of Technical and Vocational Education. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 would need to be collected to determine actual, on-the- ground practices in the DRC, particularly by teachers and students in schools. The following sections discuss the findings by each assessment type, accompanied by suggested policy options. The suggested policy options were determined in collaboration with key local stakeholders based on the DRC’s immediate interests and needs. Detailed, completed rubrics for each assessment type in the DRC are provided in Appendix 5. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Classroom Assessment cycle. For primary school, the relevant examination is the Test National de Fin d'Etude Primaire (TENAFEP), and for secondary school, it is the Examen d'Etat. Level of development In the DRC, schools are required to report on individual student performance to students and parents at pre- designated times during the school year. A summarized In the DRC, there is no national document that provides record of each student's marks is produced and reported overall guidelines for classroom assessment practice. At to the local education authority, and a copy is kept in the the same time, resources are available to teachers to school. support them in carrying out their classroom assessment activities. These include a National Curriculum document A variety of summative information about student that outlines what students are expected to learn at performance is required to be reported in school report different age and grade levels; textbooks and cards, including a student's individual performance in all workbooks; scoring criteria and rubrics for grading subjects and a student's performance relative to other students’ work; and school report cards that provide students. A qualitative assessment of student teachers with information on the maximum marks that performance in everyday class activities, indicated by the students can be awarded in different subject areas. statements, “Excellent,” “Very good,” “Good,” “or Mediocre”, is also required to be reported. There are also mechanisms in place to ensure that teachers develop competencies in classroom assessment. For example, pre- and in-service teacher Suggested policy options: training programs address competencies in classroom assessment, and teachers have opportunities to 1. Issue: There is no national document that provides participate in conferences and workshops on classroom overall guidelines for classroom assessment practice. assessment. Suggested policy option: The unit responsible for Classroom assessment practices are generally primary- and secondary-level curriculum considered to be of ‘moderate’ quality. While they are development in the Ministry of Primary, Secondary, typically aligned with a curricular framework, they tend and Technical Education should develop a formal to be overly focused on student recall of information and document with guidelines on classroom assessment used as administrative tools rather than pedagogical practice and make it available to teachers across the resources. It is difficult to offer a definitive statement on system. For example, in Uganda, the National the quality of classroom assessment practices since there Curriculum Development Centre produces national are limited mechanisms in place to systematically syllabi and other documents that include guidelines monitor their quality across the education system for classroom assessment. A first step in developing (notwithstanding that classroom assessment is supposed such a document in the DRC could be a consultative to be a required component of school inspection and meeting to explore approaches that teachers can use teacher supervision and performance evaluation). to help students achieve the standards laid out in the DRC’s National Curriculum document. There are several required uses of classroom assessment information, including for diagnosing student learning 2. Issue: There are only some resources for classroom issues, providing feedback to students on their learning, assessment activities that are available to teachers informing parents about their child's learning, planning system-wide. next steps in instruction, grading students for internal classroom uses, and providing input for certification and Suggested policy option: Increase the variety and selection. Classroom assessment marks are combined accessibility of the resources available to teachers with grades achieved on the formal certification for their classroom assessment activities. A starting examination that takes place at the end of each school SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 point could be to have the unit responsible for primary- and secondary-level curriculum development in the Ministry of Primary, Secondary, and Technical Education develop item banks with examples of multiple-choice and open-ended questions aligned with the national curriculum and make them widely available to teachers. Also, the DRC could consider developing something similar to Mozambique’s Provinha, which is a grade 3 reading assessment administered and scored by teachers. The Provinha serves as a resource for teachers, as results are intended to be used formatively at the school level to improve teaching practices and student learning outcomes in reading. 3. Issue: There are minimal formal, country-level mechanisms to monitor the quality of classroom assessment practices. Suggested policy option: Increase or strengthen the formal mechanisms in place to monitor the quality of classroom assessment practices. For example, the unit responsible for research and evaluation in the Ministry of Primary, Secondary, and Technical Education could start by commissioning a national review on the quality of education in the country, which includes a component on classroom assessment practices. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Examinations oral exam - is commonly carried out by teachers themselves, although teachers are sometimes asked to support other examination-related tasks as well. Level of development There is comprehensive material available to prepare for the examination, which is made available to most students by their teachers. Such material includes The Examen d'Etat (National Exam) has been examples of the types of questions that are on the administered in its current form to all students in grade examination, information on how to prepare for the 12 for over 20 years. The results are used for certification examination, and a report on past student performance of secondary school completion (in technical or academic on the examination. fields), and student selection to tertiary education or technical jobs. The technical commissions and teams involved in different aspects of the examination process implement A formal policy document authorizes the examination. the formal mechanisms that ensure its quality. Ordonnance loi numéro 67/250 du 5 juin 1967 is the law Specifically, all proctors and administrators are trained that created the Examen d'Etat and provides its according to a standard protocol; there is a standardized framework. manual for examination administrators; a pilot is conducted before the main data collection takes place; Funding for the examination is provided by the all booklets are numbered; there is double scoring of government based on a law (Arrêté Ministériel numéro data; scorers are trained to ensure high inter-rater MINEPSP/CABMIN/0040/2004 du 20 avril 2004), as well reliability; double processing of data takes place; and as through student fees, which represent the majority of external and internal observers are invited to watch key the funding for the examination. Funding covers all core examination procedures. examination activities; specifically, examination design, administration, data analysis, and data reporting. Inappropriate behavior surrounding the examination is reportedly minimal and involves incidents of collusion Inspection General de l'Education (General Inspectorate among candidates via mobile phones, passing of papers, of Education) is the unit in the Ministry of Education that or the equivalent. Students who do not perform well on has been in charge of the examination since 1967. There the examination may retake the examination, take is an adequate number of core staff responsible for the preparatory courses to retake the examination, or repeat examination that is further supported by additional, the grade. An issue of concern is that students’ results on ‘non-core’ personnel, mainly teachers. These additional the examination are not confidential; student names and personnel are selected to be invigilators, or to support results are published online on the Ministry of Education the core staff in the examination centers. website and are accessible by the public. The country offers a few opportunities to learn about the examination. Such opportunities include workshops and Suggested policy options: presentations on student assessment that are made available to members of the pedagogical committee, inspectors, and assessment specialists following the 1. Issue: The Democratic Republic of Congo offers very administration of the examination. Teachers are few opportunities for key stakeholders to acquire provided with opportunities to learn about the general knowledge and skills relevant for carrying out the content and skills measured by the examination (e.g., Examen d’Etat, including in such areas as through teacher training courses on curriculum and examination design, test and item development, and pedagogy) and about other aspects of the examination the development of scoring criteria. process (e.g., how the tests are developed and scored, how results are reported and used). However, only one Suggested policy option: Develop and offer regular examination-related task - acting as a judge during an opportunities to learn about the Examen d'Etat in SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 particular, and student assessment activities in opportunities to learn about the examination, and general. For example, the General Inspectorate of are involved in only some examination-related tasks. Education could collaborate with local universities to develop a course on student assessment, which is Suggested policy option: The General Inspectorate of offered annually, and made available to educators, Education could provide more opportunities for staff working on the examination, university teachers to be involved in a variety of examination- teachers and students, and assessment specialists. related tasks, such as creating examination Other options that the General Inspectorate of questions and scoring guides. This would help Education could consider include allocating funding improve their understanding of the content and skills for staff working on the Examen d’Etat to attend assessed in the examination. international courses or workshops on assessment topics. For example, in 2011, a team of 10-15 staff 4. Issue: Student results are public and not confidential. from Instituto Nacional de Desenvolvimento de Educação (INDE), the Examinations Council of Suggested policy option: Implement policies and Mozambique (CNECE), and local Mozambique procedures to ensure confidentiality of students’ universities traveled to Brazil to participate in short examination results. For example, assign a unique courses on topics such as test design, item analysis, identification number to each student taking the statistical analysis, and proficiency scale examination and report results according to these development. numbers. 2. Issue: There are minimal mechanisms in place to monitor the impact of the examination. The impact of the examination refers to its broader positive or negative consequences for the education system and society; for instance, whether the examination contributes to enhanced quality in primary or secondary education, or puts undue pressure on schools, teachers, and students. Suggested policy option: Begin to put in place some mechanisms to monitor the impact of the examination. For example, allocate funding from the General Inspectorate of Education for expert review groups to regularly review the impact of the examination on schools, teachers, and students and come up with recommendations for improvements. A first step for the DRC could be to learn from other countries’ experience in this area. For example, a team from the Examinations Council of Zambia traveled to Barbados to learn from the Caribbean Examinations Board about topics such as mitigating malpractice in examinations and using social media to distribute preparation materials. 3. Issue: It is important for teachers to learn about the skills and content assessed by the examination in order to ensure adequate exposure and preparation for students. However, DRC teachers have limited SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 National Large-Scale Assessment (NLSA) also have been provided to inspectors from the Ministry of Education responsible for primary, secondary, and professional instruction, as well as to teachers and test Level of development administrators. Only some teachers had opportunities to learn about EGRA results and student performance on the EGRA. Since 2010, the Early Grade Reading Assessment (EGRA) Formal mechanisms have been put in place to ensure the has been implemented every two years for the purposes quality of the EGRA administration and data processing. of monitoring education quality at the country level and For example, all proctors and administrators have been supporting schools and teachers. It also has served as the trained according to a protocol, and a standardized baseline assessment for the Project for the Improvement manual for administrators was developed. In addition, a of Quality Education (PIEQ), which is a USAID-funded pilot was conducted before the main data collection took education intervention in the DRC. EGRA has been place, all booklets were numbered, and scorers were administered to a regionally representative random trained to ensure high inter-rater reliability. sample of students in grades 2, 4, and 6. EGRA was administered to students in the local language The Ministry of Education authorized the assessment in on an as-needed basis (for example, when the student's 2010 with the formal policy document, dominant language was not French). However, MINEPSP/SG/80/384/2010 DU 25 MARS 2010. This accommodations or alternative assessments were not document is available to the public upon request. There provided for students with disabilities; nor were special is a detailed schedule (derived from the list of plans made to ensure that EGRA was administered to deliverables for EGRA work that the Education students in hard-to-reach areas. Development Center, an international consulting organization, has committed to USAID), specifying the Country results and information have been disseminated years of future assessment rounds, grade level(s) of using a select number of communication strategies. For students to be assessed, and subject areas to be covered. example, copies of the DRC's full country report as well as brochures with a summary of these results have been To date, funding for EGRA has been provided solely by made available to key stakeholders, including the donors and has been sufficient to cover most core Ministry of Education, USAID, international NGOs, and activities, including assessment design and some civil society representatives. However, no products administration, data analysis and reporting, planning, and services providing information about EGRA results and staff per diems and transport. have been disseminated to schools. To date, the team working on EGRA has comprised an EGRA results have provided information on the level of adequate number of staff, including international reading in Congolese schools, and supported a discourse experts from the Research Triangle Institute, staff from on the issues of reading acquisition in the country. EGRA the Ministry of Education, and local project staff. results also have informed the new reading curriculum at Technical staff from the Research Triangle Institute the primary level as well as an in-service teacher training trained local staff in target provinces to implement the module that is focused on improving the reading abilities assessment. However, additional local capacity of students. Specifically, due to EGRA results, reading as development for carrying out assessment activities a subject now figures more prominently in in-service needs to be conducted. teacher training programs. In addition, new reading benchmarks have been developed. Reading will also be The DRC offers very few opportunities to learn about promoted from a sub-subject to a main subject in the NLSAs. Only non-university training courses or primary-level curriculum. workshops on student assessment have been made available to the core NLSA team, educators, and assessment specialists. General presentations on EGRA SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Suggested policy options: collaborate with various media outlets in order to ensure wide coverage of results. For example, in 1. Issue: To date, funding for EGRA has come only from 2013, key education stakeholders came together in external loans or donors. Zambia to devise a broad-reaching communications plan that would raise awareness about learning Suggested policy option: Identify in-country sources outcomes across the country and motivate citizens of funding for NLSA activities and ensure regular to take action. Each activity in the communications funding for NLSA activities in the Ministry of plan was designed to reach specific audiences. For Education’s budget. Draft and formalize a policy instance, an easy-to-read brochure with a summary document that authorizes the use of these resources of NLSA findings and recommendations for action for carrying out the NLSA in the DRC. For example, in was created for teachers, school administrators, and 2011, Ethiopia formally allocated funding and policymakers; a 10-minute film about the NLSA designated a budget line item for its national large- results, entitled, “Are Zambia’s Children Learning?”, scale assessment activities. was created for parents and other concerned citizens and aired on broadcast television; and a DVD, 2. Issue: Given that in-country staff have had limited including a teacher discussion guide and case studies opportunities to learn about the NLSA and student on effective teaching strategies, was developed for assessment in general, and international experts use in in-service teacher trainings. have had a major role in carrying out EGRA, there is room for additional capacity building of local and ministry staff to independently carry out the assessment in the future. Opportunities to learn about the NLSA should include aspects of test design, item development, and scoring criteria. Suggested policy option: Put in place policies and supports to ensure sufficient human resource capacity within the DRC for carrying out the NLSA. One option is to allocate funding for staff working on the EGRA in the Ministry of Education to attend international workshops on student assessment in order to enhance their knowledge of assessment- related activities. For example, staff responsible for carrying out NLSA activities at the Examinations Council of Zambia received funding to attend a statistics course at the University of Michigan, which covered topics such as sampling, basic and advanced statistical analysis, and report writing. 3. Issue: NLSA results and information have not been covered by the media. In order to effectively utilize NLSA results for informing policy, results must be disseminated and understood by various stakeholders, including parents, students, educators, and policymakers. Suggested policy option: Develop a communications strategy for disseminating NLSA results and SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 International Large-Scale Assessment (ILSA) PASEC results have been used to inform decision-making in the country, including the Reform Roadmap in the Level of development Ministry of Education. Results also have been used to track the impact of reforms on student achievement levels; and to inform curriculum improvement, teacher training programs, resource allocation, and other The DRC completed its participation in PASEC in 2010 and assessment activities in the system. Moreover, there is plans to participate in PASEC 2018. evidence of a positive impact of the ILSA on education quality, as PASEC results have contributed to the There is no country-level policy document addressing the Education Sector Strategy and the Education Interim DRC's participation in PASEC, and funds for ILSA activities Plan. in the country have primarily been allocated by donors, such as the World Bank and CONFEMEN. Funding for ILSA Suggested policy options: activities has covered implementation of the assessment exercise, data processing and analysis, disseminating and reporting assessment results in the country, attendance 1. Issue: There is no country-level policy document at international meetings, ILSA staff salaries, and addressing participation in PASEC or other ILSAs. research and development activities. Funding has not Suggested policy option: Develop a policy document covered international participation fees. that outlines goals and guidelines for the DRC’s participation in ILSAs. For example, in Uganda, the A team in the Directorate for Planning in the Ministry of Government White Paper on Education (1992), Education is responsible for ILSA activities. The team was recommended review and diversification of the in charge during the 2010 PASEC assessment exercise. assessments used by the National Examinations Board, which underpinned the decision to The DRC offers minimal opportunities to learn about participate in SACMEQ. The DRC could learn from ILSAs. Some workshops and meetings on understanding Uganda’s experience and take similar steps to ILSA and ILSA databases have been organized by the articulate an overarching framework for ILSA in the Directorate of Planning for staff involved in country. implementation of the ILSA and education partners. Funding for attending international workshops or 2. Issue: The DRC offers minimal in-country trainings on ILSA also has been offered in the context of opportunities to learn about ILSAs. official ILSA meetings. Suggested policy option: Make available regular, in- country opportunities to learn about ILSAs. For DRC country results and other information for PASEC example, provide access to online courses on ILSA to have been disseminated using a small number of individuals working on an ILSA in the Directorate of communication strategies. Copies of both the country- Planning, educators, university teachers and level and international reports were distributed to key students, as well as assessment specialists. stakeholders. In addition, the DRC's results were communicated through a press release, and brochures 3. Issue: Country results and information were covered and PowerPoint presentations with the country's results by only some DRC media outlets. In addition, results were made available online and distributed to key and information were disseminated using a small stakeholders. At the same time, PASEC results were number of communication strategies. communicated to only a few schools in the capital city of Suggested policy option: Ensure that ILSA results are Kinshasa. Some DRC media outlets covered PASEC widely disseminated in the country. Develop results. Media coverage was limited to a few small products, such as PowerPoint presentations or articles, and PASEC results were written about and brochures, specifically geared toward schools, which discussed in online blogs or forums. discuss the results of the ILSA. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education system opportunities instruction grade/age system at to the next (or level(s), and to particular into the monitor trends in grade/age workforce) learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the system where the system basis (such as basis (such as allows for allows for every 3-5 years) every 3-5 years) repeats repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a few confined to one subject areas subject areas subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 13 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide institutional wide institutional support and ensure the capacity to support and capacity to support and capacity to support and quality of classroom ensure the quality of ensure the quality of ensure the quality of assessment practices. classroom assessment classroom assessment classroom assessment practices. practices. practices. CLASSROOM ASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and and a need to develop There is institutional institutional capacity and institutional capacity to capacity and some strong mechanisms to run the examination. The limited mechanisms to monitor it. The EXAMINATIONS examination typically is monitor it. The examination is of high of poor quality and is examination is of quality and is perceived perceived as unfair or acceptable quality and is as fair and free from corrupt. perceived as fair for corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA There is a stable NLSA place. NLSA in place and a in place. There is in place and institutional need to develop institutional capacity and capacity and strong institutional capacity to some limited mechanisms to monitor run the NLSA. mechanisms to monitor it. The NLSA is of high NATIONAL (OR SYSTEM- Assessment quality and it. The NLSA is of quality and its LEVEL) LARGE-SCALE impact are weak. moderate quality and its information is ASSESSMENT information is effectively used to disseminated, but not improve education. always used in effective ways. There is no history of Participation in an ILSA There is more or less There is stable participation in an ILSA has been initiated, but stable participation in an participation in an ILSA nor plans to participate there still is need to ILSA. There is and institutional capacity in one. develop institutional institutional capacity to to run the ILSA. The capacity to carry out the carry out the ILSA. The information from the INTERNATIONAL LARGE- ILSA. information from the ILSA is effectively used SCALE ASSESSMENT ILSA is disseminated, to improve education. but not always used in effective ways. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15 DEMOCRATIC REPUBLIC OF THE CONGO ǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Appendix 4: Methodology for Assigning 6. Scores for certain key dimensions under ‘Enabling Development Levels Context’ (in the case of EXAM, NLSA, and ILSA) and under ‘System Alignment’ (in the case of CLASS) were set as ceiling scores, i.e., the overall mean score for the 1. The country team or consultant collects information particular assessment type cannot be greater than the about the assessment system in the country. score for these key dimensions. These key variables include formal policy, regular funding, having a 2. Based on the collected information, a level of permanent assessment unit, and the quality of development and score is assigned to each dimension in assessment practices. the rubrics: x Latent = 1 score point x Emerging = 2 score points x Established = 3 score points x Advanced = 4 score points 3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions. For example: The quality driver, ‘Enabling Context,’ in the case of ILSA, has 3 dimensions on which a hypothetical country receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypothetical country’s overall score for this quality driver would be: (2+2+3)/3 = 2.33 4. A preliminary level of development is assigned to each quality driver. 5. The preliminary development level is validated using expert judgment in cooperation with the country team and The World Bank Task Team Leader. For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriate level. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Appendix 5: SABER-Student Assessment Rubrics for the Democratic Republic of the Congo DEMOCRATIC REPUBLIC of the CONGO Classroom Assessment SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment (Q1) There is no country-level document (Q1-3) There is an informal or draft (Q1-3) There is a formal country-level (Q1-3) There is a formal country-level that provides guidelines for classroom country-level document that provides document that provides guidelines for document that provides guidelines for assessment. 1 guidelines for classroom assessment. classroom assessment, but the classroom assessment, publicly available document is not available online to online to anybody interested. anybody interested. ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with country learning goals (Q4) There are no country-wide (Q4) There are very few country-wide (Q4) There are some country-wide (Q4) There are a variety of country-wide resources for teachers for classroom resources for teachers for classroom resources for teachers for classroom resources for teachers for classroom assessment. assessment. assessment.2 assessment. (Q5) There is no official curriculum or (Q5) There is an official curriculum or (Q5) There is an official curriculum or (Q5) There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn. students are expected to learn, but the students are expected to learn and the desired level of performance is not clear. desired level of performance. 3 ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: Having effective human resources to carry out classroom assessment activities (Q6) There are no formal country-level (Q6) There are very minimal formal (Q6) There are some formal country- (Q6) There are a variety of formal mechanisms to ensure that teachers country-level mechanisms to ensure that level mechanisms to ensure that country-level mechanisms to ensure that develop competencies in classroom teachers develop competencies in teachers develop competencies in teachers develop competencies in assessment. classroom assessment. classroom assessment. 4 classroom assessment. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment (Q7) Classroom assessment practices are (Q7) Classroom assessment practices are (Q7) Classroom assessment practices are (Q7) Classroom assessment practices are very weak, or there is no information known to be weak. known to be of moderate quality. 5 known to be of high quality. available on classroom assessment practices. (Q8) There are no formal country-level (Q8) There are minimal formal country- (Q8) There are some formal country- (Q8) There are varied formal country- mechanisms to monitor the quality of level mechanisms to monitor the quality level mechanisms to monitor the quality level mechanisms to monitor the quality classroom assessment practices. of classroom assessment practices.6 of classroom assessment practices. of classroom assessment practices. ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment (Q10) There are no required uses of (Q10) There are minimal required uses of (Q10) There are varied required uses of (Q10) There are varied required uses of classroom assessment. classroom assessment. classroom assessment. classroom assessment, including its use as an input for selection or certification.7 (Q11) Schools are not required to report (Q11-12) At least some schools are (Q11-12) All schools are required to (Q11-12) All schools are required to information on individual student required to report information on report information on individual student report information on individual student performance. individual student performance. performance to parents. performance to parents and other key stakeholders. 8 (Q11) Information on student (Q13-14) Minimal information on (Q13-14) Some information on student (Q13-14) A variety of information about performance is not required to be student performance is required to be performance is required to be reported student performance is required to be reported. reported. in school report cards. reported in school report cards.9 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Classroom Assessment: Development-level rating justifications 1. In the Democratic Republic of Congo, there is no national document that provides guidelines for classroom assessment. 2. Resources available to teachers for their classroom assessment activities include the National Curriculum document, which outlines what students are expected to learn at different age and grade levels, textbooks and workbooks, scoring criteria and rubrics for grading students work, as well as school report cards which provide teachers with information on the maximum marks that can be awarded to students in different subjects. 3. The National Curriculum document outlines what students are expected to learn at different age and grade levels, but does not specify the desired level of performance. 4. Country-level mechanisms to ensure that teachers develop competencies in classroom assessment include pre- and in-service teacher training that address competencies in classroom assessment, opportunities for teachers to participate in conferences and workshops on classroom assessment, and school inspection and teacher supervision includes a component focused on classroom assessment. 5. Classroom assessment practices are known to be of moderate quality. Although it is rare that they rely mainly on multiple-choice type questions, they are very commonly about recalling information. It is rare that teachers do not use explicit criteria for grading students' work and to observe errors in the grading of students' work. While it is rare that classroom assessment activities in practice are not aligned with a curricular framework, it is common that classroom assessment activities are used as administrative tools rather than as pedagogical resources. 6. Two formal country-level mechanisms monitor the quality of classroom assessment practices. Specifically, classroom assessment is a required component of a teacher's performance evaluation and classroom assessment is a required component of school inspection and teacher supervision. 7. Classroom assessment information is required to be used for diagnosing student learning issues, providing feedback to students on their learning, informing parents about their child's learning, planning the next steps in instruction, grading students for internal classroom uses, and providing input for certification and selection (specifically, classroom assessment marks are combined with the grades achieved on the examination that is organized by the government at the end of a cycle for certification (Test National de Fin d'Etude Primaire (TENAFEP) for primary school and Examen d'Etat for secondary school). 8. Schools are required to report on individual student performance to students and parents at different pre-designated times during the school year. A summarized record of each student's marks are produced and reported to the local education authority with copy kept in the school. 9. A variety of information about student performance is required to be reported in school report cards, including student's individual performance in all subjects and student's performance relative to other students. Student general appreciation of learning ability (a qualitative assessment of student performance in everyday class exercises, marked by qualitative statements Excellent, Very good, Good, Mediocre) is also required to be reported. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 DEMOCRATIC REPUBLIC of the CONGO Examinations SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ENABLING CONTEXT Overall framework of policies, leadership, institutional arrangements, fiscal and human resources in which the assessment activity takes place in a country, and the extent to which that framework is directly conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for the examination (Q1_III-IV) There is no examination. (Q1_III-IV) The examination has been (Q1_III-IV) The examination has been This option does not apply to this operating on an irregular basis. operating regularly. 1 dimension (Q3) There is no examination, or there is (Q3-5) There is an informal or draft (Q3-5) There is a formal policy that (Q3-5) There is a formal policy that no policy pertaining to the examination. policy that authorizes the examination; authorizes the examination, available authorizes the examination, publicly or there is a formal policy that is not upon request or with restricted access. 2 available online to anyone interested. available. (Q6) There is no examination, or the (Q6-7) The examination is at most (Q6-7) The examination is at least (Q6-7) The examination is fully examination is not standardized. partially standardized at the country partially standardized at the country standardized at the country level, and a level, or has minimum procedures in level, with at least some procedures in variety of procedures are in place to place to ensure standardization. place to ensure standardization. ensure standardization.3 ENABLING CONTEXT 2: Having leadership for the examination (Q8-9) There is no examination, or the (Q8-9) The country has weak leadership (Q8-9) The country has strong leadership (Q8-9) The country has strong leadership country does not have leadership for the for the examination. for the examination, from an individual for the examination from both an examination. person or from a stakeholder body. 4 individual person and a permanent stakeholder body. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for the examination (Q10) There is no examination, or there (Q10-11) There is irregular funding for (Q10-11) There is a regular funding for This option does not apply to this is no funding allocated for the the examination, or the funding is not the examination that is allocated by law dimension. examination. allocated by law or regulation. or regulation. 5 (Q12) There is no examination, or there (Q12) The funding for the examination (Q12) The funding for the examination This option does not apply to this is no funding coming from the comes primarily from donors or loans. comes primarily from the government or dimension. government, student fees, or donors. student fees.6 (Q13) There is no examination, or there (Q13) There is funding to cover at least (Q13) There is funding to cover all or (Q13) There is funding to cover all core is no funding to cover activities. some of the core activities. most core activities. 7 activities, plus research and development. ENABLING CONTEXT 4: Having institutional capacity for the examination There is no examination, or there is no (Q14-15) There is a temporary unit, or a (Q14-15) There is a permanent unit with (Q14-15) There is a permanent unit with examination unit. unit with minimum experience, in charge some experience in charge of the vast experience in charge of the of the examination. examination. examination. 8 (Q16) There is no examination, or it is This option does not apply to this (Q16) The examination unit is This option does not apply to this not clear to which body the examination dimension. accountable to a clearly recognized dimension. unit is accountable. body.9 (Q18) There is no examination, or the (Q18) The examination unit has only a (Q18) The examination unit has all of the (Q18) The examination unit has up-to- examination unit does not have facilities few of the required facilities to carry out required facilities to carry out the date versions of all required facilities to to carry out the examination. the examination. 10 examination. carry out the examination. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having human resources for the examination (Q19-20) There is no examination, or (Q19-20) The examination unit has an (Q19-20) The examination unit has an (Q19-20) The examination unit has an there is no staff allocated to the inadequate number of staff to carry out adequate number of staff to carry out adequate number of staff to carry out examination unit. the examination. the examination, with some quality the examination, with no quality problems. 11 problems. (Q21-22) There is no examination, or the (Q21-22) The country offers very few (Q21-22) The country offers some annual (Q21-22) The country offers a wide range country offers no annual opportunities annual opportunities to learn about the opportunities to learn about of annual opportunities to learn about to learn about the examinations. examinations. 12 examinations to the examination staff. examinations. These opportunities are available to a broad audience, including the examination staff. (Q23-24) There is no examination, or (Q23-24) Teachers have at least some (Q23-24) Teachers have at least some (Q23-24) Teachers have opportunities to teachers have no opportunities to learn opportunities to learn about the opportunities to learn about the learn about different aspects of the about the examination, and are not examination, or are involved in at least examination, and are involved in at least examination, and are involved in most involved in examination-related tasks. some examination-related tasks. some examination-related tasks. 13 examination-related tasks. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the examination with learning goals (Q26-27) There is no examination, or the (Q26-27) The examination is weakly (Q26-27) The examination is at least (Q26-27) The examination is fully aligned examination is not aligned with official aligned with official learning goals or sufficiently aligned with official learning with official learning goals or curriculum, learning goals or curriculum. curriculum, or there are no regular goals or curriculum, and there are and regular external reviews take place reviews to ensure alignment. regular reviews of the examination take to ensure alignment. place to ensure alignment.14 (Q28-29) There is no examination, or (Q28-29) The material is accessible to (Q28-29) There is comprehensive (Q28-29) There is comprehensive there the material to prepare for the less than half of the students. material to prepare for the examination material to prepare for the examination examinations is available to a small that is accessible to most students. 15 that is widely accessible to all or almost number of students at most. all students. (Q30) There is no examination, or the (Q30) The examination is minimally (Q30) The examination is sufficiently (Q30) The examination is fully consistent examination is not consistent with other consistent with other assessment consistent with other assessment with other assessment activities. 16 assessment activities. activities. activities. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ASSESSMENT QUALITY Degree to which the assessment meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the examination (Q31) There is no examination, or there (Q31) There are minimal formal (Q31) There are some formal (Q31) There are a variety of formal are no formal mechanisms in place to mechanisms in place to ensure the mechanisms in place to ensure the mechanisms in place to ensure the ensure the quality of the examination. quality of the examination. quality of the examination. quality of the examination. 17 (Q32) There is no examination, or there (Q32) There is some documentation (Q32) There is a comprehensive technical (Q32) There is a comprehensive technical is no documentation about the technical about the technical aspects of the report about the examination available report about the examination publicly aspects of the examination. examination. upon request or with restricted access. 18 available online. ASSESSMENT QUALITY 2: Ensuring fairness (Q33) There is no examination, or the (Q33) A significant proportion of (Q33) A small proportion of students (Q33) All students can take the majority of the students may not take students may not take the examination may not take the examination because of examination; there are no language, the examination because of language, because of language, gender, or other language, gender, or other equivalent gender or other equivalent barriers. 19 gender, or other equivalent barriers. equivalent barriers. barriers. (Q37; Q33) There is no examination, or (Q37; Q33) Student results are not (Q37; Q33) Student results are (Q37; Q33) Student results are student results are not confidential, or confidential, and there is some confidential, and inappropriate behavior confidential, and there is no inappropriate behavior surrounding the inappropriate behavior surrounding the surrounding the examination is low. inappropriate behavior surrounding the examination is high. 20 examination. examination. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Ensuring appropriate uses of the examination (Q34; Q37) There is no examination, or (Q34; Q37) Student results are perceived (Q34; Q37) Student results are (Q34; Q37) Students’ results are student results are not perceived as as credible by at least some perceived as credible by most perceived as credible by most credible, or are not recognized by any stakeholders. stakeholders, and are nationally stakeholders, and are internationally broader certification or selection system. recognized. recognized.21 (Q39) There is no examination, or there (Q39) There are very few options in the (Q39) There are some options in the (Q39) There are a variety of options in are no options in the education system education system for students who do education system for students who do the education system for students who for students who do not perform well on not perform well on the examination. not perform well on the examination. 22 do not perform well on the examination. the examination. (Q40) There is no examination, or there This option does not apply to this (Q40) There are some mechanisms in (Q40) There are a variety of mechanisms are no mechanisms in place to monitor dimension. place to monitor the examination.23 in place to monitor the examination. the examination. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 Examinations: Development-level rating justifications 1. The Examen d'Etat (National Exam) has been administered to all students in grade 12 for over twenty years in its current form. The examination is used for certification of secondary school completion (in technical or academic fields), and selection to tertiary education or technical jobs. 2. Ordonnance loi numéro 67/250 du 5 juin 1967 is a law that created the Examen d'Etat and provides its framework. Loi Cadre de l'Education was promulgated February 2014. Other relevant policy documents include the ministeriel numero MINEPSP/CABMIN/0040/2004 du 20 avril 2004 and the Loi cadre numero 14/004 du 11 fevrier 2014 de l'enseignement national. The policy documents are available upon request at the Journal Officiel de la Republique and at the Ministry of Education in the "Recueil des directives officielles". 3. The examination is fully standardized (assessment design, administration, scoring and reporting are the same for all students taking the examination). A variety of procedures are in place to ensure the exam's standardization, including that the examination papers and tasks are the same or are equivalent for all students, examination administrators are trained to ensure that all students take the examination under the same conditions, quality control monitors are used to ensure the same administration conditions in all locations where the examinations are taken, the same scoring criteria are used to correct the examination papers and tasks, and examination results are reported using the same procedures for all students. 4. The Ministry of Education provides leadership for the examination. The Ministry of Education is a nationally recognized authority in the country. It has pushed for further development of the examination. It has the influence and power to determine the examination agenda in the country, and has obtained funding for examination activities. The Ministry of Education had managed and continues to manage important assessment projects and programs in the country, and key members have participated in international steering or technical committees on assessment. The Ministry of Education has also been able to convene stakeholders in the country to address examination issues. 5. Funding for the examination is provided by the government based on a law (Arrêté Ministériel numéro MINEPSP/CABMIN/0040/2004 du 20 avril 2004). 6. Funding for the examination is also provided by student fees, which comprises the majority of the funding for the examination. 7. Funding covers all core examination activities, specifically examination design (12% of funding), administration (34% of funding), data analysis (9% of funding), and data reporting (3% of funding). Funding also covers planning (2% of funding) and expenses related to staff (such as per diem, training of scorers, and others, excluding staff salaries) (40% of funding). 8. Inspection Général de l'Education (General inspectorate of education) is the unit in the Ministry of Education that has been in charge of the examination since 1967. 9. The Inspection General de l'Education is accountable to the National Committee for the exam (Comité National de l'Examen d'Etat), which is also part of the Ministry of Education. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 10. The examination unit has up-to-date computers and communication tools for all technical staff. It does not have up-to-date computer servers or top level storage facilities and security for the building. 11. There is an adequate number of core staff who are responsible for the examination. The core staff are supported by the personnel of the ministry, mainly teachers, who are selected to be invigilators or to support the staff in the examination centers. Quality problems have been limited to only the poor training of examination administrators. 12. Only courses, workshops, and presentations on student assessment are available to members of the pedagogical committee, inspectors, and some teachers and assessment specialists following the administration of the examination. 13. Teachers are provided with the opportunities to learn about the content and skills measured by the examination (e.g., courses on curriculum and pedagogy) and opportunities to learn about other aspects of the examination (e.g., how tests are developed and scored, how results are reported and used). However, only one examination-related task - acting as a judge during an oral exam - is mainly carried out by teachers. Teachers are sometimes asked to support other examination-related tasks as well. 14. The examination is strongly aligned with the content areas of the national curriculum. The examination is also aligned with the skills areas of the national curriculum and with the pedagogical approaches and activities that are compatible with the national curriculum. Regular internal and ad hoc reviews of the alignment between the examination and what it is intended to measure take place. 15. Material to prepare for the examination includes examples of the types of questions that are on the examination, information on how to prepare for the examination and a report on past student performance on the examination which is made available to most students by their teachers. While schools are to provide the material to students, the comprehensiveness and quality of the material is dependent upon the quality of the resources that the teacher is able to locate. 16. The examination is fully consistent with the classroom assessment, national and international assessment large-scale assessments. 17. The technical commissions and teams who are involved in different aspects of the examination process implement the formal mechanisms that are in place to ensure the quality of the examination. Specifically, all proctors or administrators are trained according to a protocol, there is a standardized manual for examination administrators, a pilot is conducted before the main data collection takes place, all booklets are numbered, there is double scoring of data, scorers are trained to ensure high inter-rater reliability, double processing of data takes place, and external and internal reviewers or observers are invited to observe key examination procedures. 18. The National board of the Exam "Comité National de l'Examen d' Etat" produces a comprehensive technical report (cf. Arreté ministériel numéro MINEPSP/CABMIN/0040/2004 du 20 avril 2004, article 53) that is available only to certain stakeholders (such as the Minister of Education). SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 19. All students can take the examination. 20. Inappropriate behavior surrounding the examination is minimal and involves only collusion among candidates via mobile phones, passing of papers or equivalent. However, student results are public and not confidential. 21. Examination results are perceived as credible by most stakeholder groups, and are recognized by selection and certification systems in the country and abroad. 22. Students who do not perform well on the examination may retake the examination, take preparatory courses to retake the examination, and may repeat the grade. 23. Only a permanent oversight committee is in place to monitor the examination. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 DEMOCRATIC REPUBLIC of the CONGO National (or System-Level) Large-Scale Assessment (NLSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ENABLING CONTEXT Overall framework of policies, leadership, institutional arrangements, fiscal and human resources in which the assessment takes place in a country, and the extent to which that framework is directly conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for the NLSA (Q3_III) No NLSA has taken place in the (Q3_III, IV) The NLSA has been operating (Q3_III, IV) The NLSA has been operating This option does not apply to this country. on an irregular basis. 1 regularly. dimension. (Q5) There was no NLSA, or there was no (Q5-7) There was an informal or draft (Q5-7) There was a formal policy (Q5-7) There was a formal policy policy document pertaining to NLSA. policy document that authorized the document that authorized the NLSA, document that authorized the NLSA that NLSA. available upon request or with restricted is publicly available online to anyone access. 2 interested. (Q8) There was no NLSA, or there was no (Q8-9) There was a common, informal (Q8-9) There was a schedule for future (Q8-9) There was a publicly available assessment schedule for future NLSAs. understanding that there would be an NLSAs, specifying when (year), who official assessment schedule for future NLSA in the future. (grade level) and what (subject areas) NLSAs, specifying when (year), who would be assessed, but it is not publicly (grade level) and what (subject areas) available.3 would be assessed. ENABLING CONTEXT 2: Having leadership for the NLSA (Q10-11) There was no NLSA, or the (Q10-11) The country had weak (Q10-11) The country had leadership for (Q10-11) The country had leadership for country did not have leadership for the leadership for the NLSA. the NLSA from an individual person or the NLSA from both an individual person NLSA. from a stakeholder body. 4 and a permanent stakeholder body. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for the NLSA (Q12) There was no NLSA, or there was (Q12-13) There was funding for the (Q12-13) There was regular funding for This option does not apply to this no funding for the NLSA. NLSA.5 the NLSA that was allocated by law or dimension. regulation. (Q12) There was no NLSA, or there was (Q14) The funding for the NLSA came (Q14) The funding for the NLSA came This option does not apply to this no funding for the NLSA from internal or only or primarily from loans or external primarily or only from the country's dimension. external sources. donors. 6 internal funding sources. (Q12) There was no NLSA, or there was (Q15) There was funding to cover at least (Q15) There was funding to cover all or (Q15) There was funding to cover all core no funding. some minimum core activities of the most core activities of the NLSA. 7 activities, plus research and NLSA. development. ENABLING CONTEXT 4: Having institutional capacity for the NLSA (Q16) There was no NLSA, or there was (Q16-18) There was a temporary or ad (Q16-18) There was a permanent team, (Q16-18) There was a permanent team, no NLSA team. hoc team in charge of the NLSA, or other at least nationally recognized, with at internationally recognized, with vast equivalent. 8 least some experience in NLSA. experience in NLSA. (Q19-20) There was no NLSA, or it is This option does not apply to this (Q19-20) The NLSA unit was accountable This option does not apply to this unclear to which body the NLSA unit was dimension. to a clearly recognized body. 9 dimension. accountable. (Q21) There was no NLSA, or the NLSA (Q21) The NLSA unit had only a few of (Q21) The NLSA unit had all of the (Q21) The NLSA unit had up-to-date unit did not have facilities to carry out the required facilities to carry out the required facilities to carry out the versions of all required facilities to carry the assessment. assessment. assessment. 10 out the assessment. (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having human resources for the NLSA (Q16) There was no NLSA, or there was (Q22-23) The NLSA team had an (Q22-23) The NLSA team had an (Q22-23) The NLSA team had an no NLSA staff. inadequate number of staff to carry out adequate number of staff to carry out adequate number of staff to carry out the NLSA. the NLSA, with some quality problems. the NLSA, without quality problems. 11 (Q24-25) There was no NLSA, or the (Q24-25) The country offered very few (Q24-25) The country offered some (Q24-25) The country offered a wide country did not offer annual annual opportunities to learn about annual opportunities to learn about range of annual opportunities to learn opportunities to learn about NLSA. NLSA. 12 NLSA, albeit only to the NLSA team about NLSA. These opportunities were members. available to a broad audience, including the NLSA team members. (Q26) There was no NLSA, or teachers This option does not apply to this (Q26) Teachers had annual opportunities (Q26) Teachers had annual opportunities did not have annual opportunities to dimension. to learn about the content and skills to learn about different aspects of the learn about the NLSA. 13 measured by the NLSA. NLSA. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals (Q27-28) There was no NLSA, or the (Q27-28) There were some mechanism (Q27-28) The NLSA was sufficiently (Q27-28) The NLSA was fully aligned with country did not have official learning to review the alignment of the NLSA and aligned with official learning goals or official learning goals or curriculum, and goals or curriculum, or the NLSA was not what it was intended to measure. 14 curriculum, and a regular internal review a regular external review took place to aligned with the official learning goals or took place to ensure alignment. ensure alignment. curriculum. (Q29) There was no NLSA, or students (Q29) Students had limited previous (Q29) Students had sufficient previous (Q29) Students had extensive previous had no previous exposure to the type of exposure to the type of content and skills exposure to the type of content and skills exposure to the type of content and skills content and skills measured by the NLSA. measured by the NLSA. measured by the NLSA. 15 measured by the NLSA. (Q30) There was no NLSA, or the NLSA (Q30) The NLSA was minimally consistent (Q30) The NLSA was sufficiently (Q30) The NLSA was fully consistent with was not consistent with other with other assessment activities. consistent with other assessment other assessment activities. assessment activities. activities. 16 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ASSESSMENT QUALITY Degree to which the assessment meets technical standards, is fair and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA (Q31-32) There was no NLSA, or there (Q31-32) There were informal or ad hoc (Q31-32) There were some formal (Q31-32) There were a variety of formal were no mechanisms to include all mechanisms to include all student mechanisms to include all student mechanisms to include all student groups student groups in the NLSA. groups in the NLSA. groups in the NLSA. 17 in the NLSA. (Q31-32) There was no NLSA, or there (Q33) There were very few formal (Q33) There were some formal (Q33) There were a variety of formal were no formal mechanisms in place to mechanisms in place to ensure the mechanisms in place to ensure the mechanisms in place to ensure the ensure the quality of the NLSA. quality of the NLSA. quality of the NLSA. 18 quality of the NLSA. (Q34) There was no NLSA, or there was (Q34) There was some documentation (Q34) There was a comprehensive (Q34) There was a comprehensive no technical documentation about the about the technical aspects of the NLSA. technical report for the NLSA, available technical report for the NLSA, publicly NLSA. upon request or with restricted access. available online. 19 (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA (Q35-37) There was no NLSA, or country (Q35-37) Country results and (Q35-37) Country results and (Q35-37) Country results and information results and information were not information were disseminated using information were disseminated using a were disseminated using a variety of disseminated. some communication strategy, excluding variety of communication strategies, communication strategies, including dissemination to schools. 20 including dissemination to some schools. dissemination to most schools. (Q38-39) There was no NLSA, or NLSA (Q38-39) NLSA results and information (Q38-39) NLSA results and information (Q38-39) NLSA results and information results and information were not had hardly any coverage in the media. were covered by some media outlets. were covered by a wide variety of media. covered by the media.21 (Q40-41) There was no NLSA, or results (Q40-41) Results from the NLSA were (Q40-41) Results from the NLSA were (Q40-41) Results from the NLSA were from the NLSA were not used to inform minimally used to inform decision used in some ways to inform decision used in a variety of ways to inform decision making in the country. making in the country. 22 making in the country. decision making in the country. (Q42) There was no NLSA, or there were This option does not apply to this (Q42) There were some formal (Q42) There were a variety of formal no mechanisms in place to monitor the dimension. mechanisms in place to monitor the mechanisms in place to monitor the NLSA. 23 NLSA. NLSA. (Q43) There was no NLSA, or there is no This option does not apply to this (Q43) There is a general consensus about (Q43) There is evidence of the positive clear evidence or consensus about the dimension. the positive impact of the NLSA on impact of the NLSA on education quality. positive impact of the NLSA on education education quality. quality. 24 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 National (of System-Level) Large Scale Assessment (NLSA): Development-level rating justifications 1. The Early Grade Reading Assessment (EGRA) has been implemented every two years since 2010 for the purposes of monitoring education quality at the country level, and supporting schools and teachers. It has also served as the baseline assessment for the Project for the Improvement of Quality Education (PIEQ) which is a USAID funded education intervention. EGRA has been administered to a regionally representative random sample of students in grades 2, 4, and 6. 2. The Ministry of Education authorized the assessment with the formal policy document, MINEPSP/SG/80/384/2010 DU 25 MARS 2010, in 2010. This policy document is available to the public upon request. 3. Although there was a detailed schedule specifying the year of future assessments, grade level(s) of students to be assessed, and subject areas, it was only made available upon request. This schedule is derived from the deliverables related to implementing EGRA which have been committed by the Education Development Center to USAID. 4. The Democratic Republic of Congo had leadership for the NLSA from the Education Development Center, who lead a consortium composed of itself, Catholic Relief Services, and the Research Triangle Institute. The Education Development Center had influence and power in determining the NLSA agenda, was able to obtain funding for NLSA activities, and was able to convene stakeholder groups in the country to address NLSA issues. However, a large portion of the oversight and support for NLSA activities came from international staff and experts from outside of the country. 5. Funding for EGRA was allocated at the discretion of donors. 6. Funding for EGRA came only from external loans or donors 7. There was funding to cover most core NLSA activities. Funding for EGRA covered assessment design and administration, data analysis and reporting, planning, and staff per diems and transport. 8. The team working on EGRA was composed of international experts from the Research Triangle Institute, staff from the Ministry of Education, and local project staff. 9. The team responsible for carrying out EGRA was responsible to the donor, USAID. (This evaluation is of the team of international experts and not only of country capacity.) 10. The team responsible for carrying out EGRA had up-to-date computers for all technical staff, top-level building security and storage facilities, and up-to-date computer servers and communication tools. (This evaluation is of the team of international experts and not only of country capacity.) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 11. The team responsible for carrying out the NLSA exercise had an adequate number of staff. The technical staff from the Research Triangle Institute trained local staff in target provinces to implement the assessment. There were no quality problems identified with the performance of the team responsible for the carrying out the EGRA. (This evaluation is of the team of international experts and not only of country capacity.) 12. The Democratic Republic of Congo offered very few annual opportunities to learn about NLSA. There were non-university training courses or workshops on student assessment available annually to the NLSA team, educators, and assessment specialists. Presentations on the NLSA were given to inspectors from the Ministry of Education responsible for primary, secondary, and professional instruction, as well as teachers and test administrators. 13. Only some teachers had opportunities to learn about NLSA results and student performance on the NLSA. These opportunities did not address the content and skills measured by the NLSA. 14. EGRA was aligned with the content and skills areas of the national curriculum, as well as with the pedagogical approaches and activities that are compatible with the national curriculum. Some mechanism were in place review the alignment of the NLSA and what it was intended to measure, including an ad-hoc review of the alignment of the EGRA and what it is intended to measure and a pilot was conducted before each assessment round. However, no regular internal and external review of the alignment between EGRA and what it was intended to measure has taken place. 15. Students had sufficient previous exposure to the type of content and skills measured by the NLSA. Textbooks or other learning resources and teachers in their classes covered similar content and skills to those covered by the NLSA. Moreover, students were exposed to more basic content and skills than those covered by the NLSA. 16. The NLSA was consistent with classroom assessment practices, the End of Primary School National Test (TENAFEP), and PASEC. 17. EGRA was administered to students in the local language when student's language was not French on an as-needed basis. Additionally, accommodations or alternative assessments were not provided for students with disabilities, and special plans were not made to ensure that EGRA was administered to students in hard-to-read areas. 18. Formal mechanisms were in place to ensure the quality of the administration and data processing of the NLSA. For example, all proctors or administrators were trained according to a protocol, and there was a standardized manual for large-scale assessment administrators. In addition, a pilot was conducted before the main data collection took place, all booklets were numbered, and scorers were trained to ensure high inter-rater reliability. 19. A comprehensive technical report on the NLSA was made publicly available online. 20. Country results and information were disseminated using some communication strategy. Copies of the Democratic Republic of Congo's country report and brochures or PowerPoint presentations with the country's results were made available to development partners, including the Ministry of Education, USAID, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 international NGOs, and a few civil society representatives. No products and services providing information about the NLSA results were disseminated to schools. 21. NLSA results and information were not covered by the media. 22. Due to EGRA results, reading as a subject now figures more prominently in in-service teacher training programs. In addition, a new reading curriculum at the primary level and reading benchmarks have been developed. Reading also will be promoted from a sub-subject to a main subject in the primary-level curriculum. 23. No formal mechanisms are in place in the Democratic Republic of Congo to monitor the impact, acceptance, and credibility of the NLSA. 24. There is no clear evidence or consensus about the NLSA having a positive impact on education quality. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 DEMOCRATIC REPUBLIC of the CONGO International Large-Scale Assessment (ILSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ENABLING CONTEXT Overall framework of policies, leadership, institutional arrangements, fiscal and human resources in which the assessment takes place in a country, and the extent to which that framework is directly conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for the ILSA (Q1, Q2) The country has not (Q1, Q2) The country has participated in, (Q1, Q2) The country has completed one (Q1, Q2) The country has completed two participated in an ILSA in the last 10 but not completed, an ILSA in the last 10 ILSA in the last 10 years. 1 or more ILSAs in the last 10 years. years. years. (Q3) The country is not currently This option does not apply to this (Q3) The country is currently This option does not apply to this participating in an ILSA or has not taken dimension. participating in an ILSA or has taken dimension. concrete steps to participate in an ILSA in concrete steps to participate in at least the next 5 years. one ILSA in the next 5 years.2 (Q5) There was no country-level policy (Q5,Q6) There was an informal or draft (Q5,Q6) There was a formal country-level (Q5,Q6,Q7) There was a formal country- document that addressed participation country-level policy document that policy document that addressed level policy document that addressed in the ILSA.3 addressed participation in the ILSA. participation in the ILSA that was participation in the ILSA that was publicly available upon request or with restricted available online to anyone interested. access. ENABLING CONTEXT 2: Having sufficient funding for the ILSA (Q8) There was no funding for (Q9) Funding for the ILSA activities was (Q9) Funding for the ILSA activities was This option does not apply to this participation in the ILSA, discretionary or primarily allocated at the discretion of primarily allocated by law or regulation. dimension. otherwise. the country's government or donors. 4 (Q8) There was no funding from loans, (Q10) There was funding only or (Q10) There was funding primarily from (Q10) There was funding only from the external donors, or internal sources. primarily from loans or external donors.5 the country's internal funding sources. country's internal sources. (Q8) There was no funding for core items (Q11) The ILSA funding covered at least (Q11) The ILSA funding covered most (Q11) The ILSA funding covered most or research and development. minimum core items of the ILSA. core items. core items, plus research and development. 6 (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective institutional and human capacity for the ILSA (Q12-16) There was no ILSA unit or team. (Q12-16) There was a unit or team that (Q12-16) There was a recognized unit or (Q12-16) There was an internationally- carried out the ILSA, with some issues. team, with at least some experience in recognized unit or team, with vast international assessments, that carried experience in international assessments out the ILSA in an effective way. that carried out the ILSA in an effective way. 7 (Q17) On average, the facilities of the (Q17) On average, the facilities of the (Q17) Most of the required facilities of (Q17) The facilities of the ILSA unit were ILSA unit were not up-to-date. ILSA unit were minimally up-to-date. the ILSA unit were up-to-date. 8 up to date. The ILSA unit had up-to-date versions of all of the required facilities to carry out the ILSA. (Q18-20) The country offered no (Q18-20) The country offered minimal (Q18-20) The country offered adequate (Q18-20) The country offered adequate opportunities to learn about ILSAs. opportunities to learn about ILSA.9 opportunities to learn about ILSA. opportunities to learn about ILSA to a broad audience, including the ILSA team and educators. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 43 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the ILSA with learning goals for the country (Q21) The ILSA was not aligned with the (Q21) The ILSA was partially aligned with (Q21) The ILSA was sufficiently aligned (Q21) The ILSA was fully aligned with the country's official learning goals, or the the country's official learning goals. with the country's official learning country's official learning goals. country did not have official learning goals.10 goals. (Q22) Students were not previously (Q22) Students had limited previous (Q22) Students had sufficient previous (Q22) Students had extensive previous exposed to the type of content and skills exposure to the type of content and skills exposure to the type of content and skills exposure to the type of content and skills measured by the ILSA. measured by the ILSA. measured by the ILSA. 11 measured by the ILSA. (Q23) The ILSA was not consistent with (Q23) The ILSA was minimally consistent (Q23) The ILSA was generally consistent (Q23) The ILSA was fully consistent with the country's other assessment activities with the country's other assessment with the country's other assessment the country's other assessment activities. activities. activities. .12 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 44 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 ASSESSMENT QUALITY Degree to which the assessment meets technical quality standards, is fair and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the technical quality of the ILSA (Q24) The country did not meet (Q24) The country met sufficient (Q24) The country met all technical This option does not apply to this sufficient technical standards to have its technical standards to have its data standards required to have its data dimension. data presented in the international presented beneath the main display of presented in the main displays of the report or an annex. the international report or in an annex. international report. 13 ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA (Q25-27) Country results and (Q25-27) Country results and (Q25-27) Country results and (Q25-27) Country results and information information were not disseminated in information were disseminated using at information were disseminated using were disseminated using a variety of the country. least one communication strategy. some communication strategies. 14 communication strategies, including dissemination to most schools. (Q28-29) Country results and (Q28-29) Country results and (Q28-29) Country results and (Q28-29) Country results and information information were not covered by media information were covered by one media information were covered by some were covered by a variety of media in the country. outlet in the country. media outlets in the country. 15 outlets in the country. (Q30-31) Results from the ILSA have not (Q30-31) Results from the ILSA have (Q30-31) Results from the ILSA have (Q30-31) Results from the ILSA have been used to inform decision making. been used in a very limited way to been used in some ways to inform been used in a variety of ways to inform inform decision making in the country. decision making in the country. decision making in the country. 16 (Q32) There is no clear evidence or This option does not apply to this (Q32) There is a general consensus about (Q32) There is evidence of the positive consensus about the positive impact of dimension. the positive impact of the ILSA on impact of the ILSA on education quality. 17 the ILSA on education quality. education quality. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 45 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 International Large Scale Assessment (ILSA): Development-level rating justifications 1. The Democratic Republic of Congo participated and completed PASEC in 2010. 2. The Democratic Republic of Congo has plans to participate in the 2018 round of PASEC. 3. There was no country-level policy document addressing the Democratic Republic of Congo's participation in PASEC. 4. Funds for ILSA activities in the Democratic Republic of Congo have been provided by donors. 5. Funding for ILSA activities have been sourced only from external donors, such as the World Bank and CONFEMEN. 6. Funding for ILSA activities in the Democratic Republic of Congo covers implementation of the assessment exercise, data processing and analysis, disseminating and reporting assessment results in the Democratic Republic of Congo, attendance at international meetings, ILSA staff salaries, and research and development activities. Funding does not cover international participation fees. 7. In the Ministry of Education, a team in the Directorate for Planning is responsible for ILSA activities. The team is internationally recognized and was in charge during the 2010 PASEC assessment exercise. 8. The ILSA unit had up-to-date computer servers and communication tools, and top-level building security and storage facilities. However, the ILSA unit did not have up-to-date computers for all technical staff. 9. Some workshops and meetings on understanding ILSA and ILSA databases have been organized by the Directorate of Planning for staff involved in implementation of the ILSA and education partners. In addition, funding for attending international workshops or trainings on ILSA was offered in the context of official ILSA meetings. However, the Democratic Republic of Congo does not offer university courses on the topic of ILSA or online courses on ILSA. The Democratic Republic of Congo also did not offer funding for international workshops or training on ILSA separate from those offered at special meetings. 10. PASEC was sufficiently aligned with the content and skill areas of the country's curriculum. In addition, it was aligned with pedagogical approaches compatible with the curriculum. 11. Textbooks and teachers in their classes covered similar content and skills to those covered by the ILSA. Moreover, students were exposed to more basic content and skills than those covered by the ILSA. 12. The ILSA was generally consistent with the country's classroom assessment practices, examinations, and national-large scale assessment. However, students did have much familiarity with the multiple-choice question format that appeared on the ILSA. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 46 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 13. The Democratic Republic of Congo met all technical standards required to have its data presented in the main displays of the international report. 14. Country results and information were disseminated using some communication strategies. Copies of both the country-level and international reports were distributed to key stakeholders. In addition, the Democratic Republic of Congo's results were communicated through a press release, and brochures and PowerPoint presentations with the country's results were made available online or distributed to key stakeholders. Only schools in the capital city received products and services providing information about ILSA results. 15. Media coverage was limited to a few small articles, and ILSA results were written about or discussed in online blogs or forums. 16. ILSA results have been used to inform decision-making at the country level in the Democratic Republic of Congo. PASEC results have informed the Reform Roadmap in the Ministry of Education. Results have been used in tracking the impact of reforms on student achievement levels, and informing curriculum improvement, teacher training programs, resource allocation, and other assessment activities in the system. 17. There is evidence of a positive impact of the ILSA on education quality, as PASEC results have contributed to the Education Sector Strategy and the Education Interim Plan. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 47 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 References Acknowledgements Clarke, M. 2012. “What Matters Most for Student This report was prepared by The World Bank SABER- Assessment Systems: A Framework Paper.” READ/SABER Student Assessment team in collaboration with Dung- Working Paper Series. Washington, DC: World Bank. Kim Pham, World Bank Senior Operations Officer and Task Team Leader for education projects in the The Global Partnership for Education. 2014. “In the Democratic Republic of Congo, and Didier Niki-Niki, Democratic Republic of Congo, Partners Rebuild the Education Specialist, Democratic Republic of Congo. The Education System Together.” Washington, D.C.: Global team is grateful to the Congolese education staff who Partnership for Education. Retrieved from contributed information for this report. https://www.globalpartnership.org/success- stories/democratic-republic-congo-partners-rebuild- education-system-together on September 1, 2014. The Ministry of Primary, Secondary, and Vocational Education in the Democratic Republic of Congo. 2013. National Survey on the Situation of Out of School Children and Adolescents: Democratic Republic of Congo. Government of the Democratic Republic of Congo: Kinshasa. Retrieved from http://www.unicef.org/education/files/DRC_OOSCI_Full _Report_(En).pdf on September 1, 2014. World Bank. 2014. Democratic Republic of Congo Country Indicator Data. Washington, DC: World Bank. Data retrieved from http://data.worldbank.org/ on September 1, 2014. - - - . 2014. “Democratic Republic of the Congo: Overview.” Washington, DC: World Bank. Retrieved from http://www.worldbank.org/en/country/drc/overview on September 1, 2014. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 48 DEMOCRATIC REPUBLIC OF THE CONGO ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2014 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 49