Republic of Fiji SABER Country Report EDUCATION MANAGEMENT INFORMATION SYSTEMS 2017 Key Policy Areas Status 1. Enabling Environment The Fiji Education Management System (FEMIS) is supported by a policy document that specifies data collection processes, privacy assurances and allocation of responsibilities. FEMIS is supported by high political buy-in and its data ownership remains at the central level with the Ministry of Education, Heritage and Arts (MoEHA). The responsibilities of the data production process are clearly allocated at the school and district level. However, the policy document is not fully comprehensive. For instance, while there are different compliance requirements for different types of data reported, there are limited enforceable mechanisms to ensure timeliness of reporting processes. Data access is highly hierarchical with read only and data editing specifications. The MoEHA restricts data access and thus, potentially limits FEMIS use by some education stakeholders. 2. System Soundness FEMIS has been tailored to Fiji’s unique country context taking Internet connectivity and existing IT resources into account. It has an established EMIS located at the schools, and managed by a FEMIS administrator or teacher on a daily basis. The system’s two primary modules, FESA and LANA, contain individual level information on school demographics, student progression, infrastructure, examination results and some human resource data including teacher qualifications. The Divisional Education Offices, District Education Offices and schools share the data production task. Schools enter information directly into FEMIS at the school level, which is then accessed by district offices and Ministry at the local and national level, respectively. However, there are only a limited set of indicators that are generated using the FEMIS. Most of these indicators, such as the net and gross enrollment rate are calculated manually by the staff, without any automated processes established. 3. Quality Data FEMIS data is aimed to be comprehensive, accurate and timely. Data are collected from basic education levels: primary and secondary education. Currently, efforts are underway to link the early childhood education and technical education data into FEMIS, thereby tracking student progressing from ECE to technical colleges. Data validation mechanisms within the FEMIS ensure that the quality of data submitted is of good quality. However, timeliness of data submission is an issue given different compliance standards for different data types, making enforcement of deadlines difficult. 4. Utilization for Decision Making Data utilization generally remains at the central level to improve the education system business processes such as allocation of school grants and provision of vouchers to students. The government has data utilization strategies in place, but data access is restricted to people outside the Ministry, and thus utilization is limited. Some isolated efforts are made at the school level to use the data in FEMIS to monitor student performance and inform classroom instruction. There needs to be stronger push from the government to promote a data-driven culture in the country. FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Introduction In 2011, the World Bank Group commenced a multiyear program designed to support countries in systematically examining and strengthening the performance of their education systems. Part of the World Bank’s new Education Sector Strategy, this evidence-based initiative, called SABER (Systems Approach for Better Education Results), uses diagnostic tools for examining education systems and their component policy domains against global standards and best practices and in comparison, with the policies and practices of countries around the world. By leveraging this global knowledge, the SABER tools fill a gap in the availability of data and evidence on what matters most to improve the quality of education and achievement of better results. This report discusses the results of applying the SABER–Education Management Information Systems (EMIS) tool in Fiji. The objectives of this report are to examine the system according to key policy areas, identify successes and challenges in the system, and provide recommendations to support the continued advancement of EMIS in Fiji. Approach of SABER-EMIS Information is a key ingredient in an effective education system. SABER–EMIS aims to help countries improve data collection, data and system management, and data use in decision making. SABER-EMIS assesses the effectiveness of a country’s EMIS, with the aim of informing policy dialogue and helping countries better manage education inputs and processes to achieve overall efficiency and strong learning outcomes. A successful EMIS is credible, operational in planning and policy dialogue as well as teaching, and learning. It produces and monitors education statistics within an education system and has a multifaceted structure, comprising the technological and institutional arrangements for collecting, processing, and disseminating data (Abdul-Hamid 2014). It is crucial for tracking changes, ensuring data quality and timely reporting of information, and facilitating the utilization of information in decision-making. Figure 1: SABER-EMIS Policy Goals and Levers The SABER-EMIS assessment methodology is Policy Goals Policy Levers: legal framework, organizational structure built on four key policy areas that are Enabling and institutionalized processes, human resources, essential to EMIS and must be assessed to Environment infrastructural capacity, budget, data-driven culture understand and ultimately strengthen the Policy Levers: data architecture, data coverage, data System system. Each policy goal is defined by a set of Soundness analytics, dynamic system, serviceability policy levers (actions that help governments Policy Levers: methodological soundness, accuracy and reach the policy goal) and indicators Quality Data reliability, integrity, periodicity and timeliness (measuring the extent to which the policy Utilization for Policy Levers: openness to EMIS users, operational use, levers are achieved) (figure 1). Decision Making accessibility, effectiveness in disseminating findings Source: Abdul-Hamid 2014 A strong enabling environment lays the foundation for an effective EMIS. Enabling environment refers to the laws, policies, structure, resources, and culture surrounding an EMIS that make data collection, management, and access possible. In essence, this policy area is the context in which an EMIS exists. This defined scope of an enabling environment builds on lessons learned from studies of education management systems. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 System soundness ensures key processes, Figure 2: SABER Scoring and EMIS Development structures, and integration capabilities in an 4 effective EMIS. Education data are sourced Advanced from different institutions, but all data feed 3 Comprehensive into and make up EMIS. Databases within an Established enabling 2 environment, EMIS are not viewed as separate databases, Enabling Emerging environment, processes, but as part of the whole EMIS. Key aspects of 1 processes, structure, data Basic enabling system soundness include what data are Latent environment, structure, data management, management, utilization, and covered in EMIS and how they come together Limited enabling processes, utilization in place integration in environment, structure, data place, with in the overarching system. processes, management, with some intelligent utilization integration structure, data analytics management, Quality data establishes the mechanisms utilization required to collect, save, produce, and utilize Source: Abdul-Hamid 2014 information in an accurate, secure, and timely manner. Data quality is a multidimensional concept that encompasses more than just the underlying accuracy of the statistics produced. It means that not only are the data accurate, but that the data address specific needs in a timely fashion. Quality data lays the groundwork for utilization. An effective EMIS is utilized in decision making by all users (parents, students, teachers, principals and policy makers) across the education system. An EMIS needs to be used so that measures can be taken to improve educational quality. Accurate information on education sector performance enables the design of more informed policies and programs. It is imperative to understand where decision making occurs, if the capacity to analyze and interpret education data exists, and if specific data are available to inform decisions. Figure 3: Policy Intent, Implementation and Outcomes Cycle, with Examples Using the EMIS data collection instrument, policy levers are scored on a four-level scale  Policies  Utilization (latent, emerging, established, and  Vision & buy-in  Processes advanced) to assess the extent to which both  Standards  Institutionalization  Strategy  Budget policy intent and implementation are documents  Human Resources achieved (figure 2).  Communication Assessing Policy Intent and Intent Implementation Implementation The EMIS assessment examines policy intent and the degree to which intended policies are effectively implemented on the ground (figure 3). Intent refers to the way in which EMIS and its overarching purpose are Outcomes articulated by decision makers and  Teaching & Learning documented in policies and legislation, as  Management well as standards and strategy documents.  Planning & Monitoring Assessing intent alone only reveals only part  Transparency of the picture.  Governance SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 As such, this EMIS assessment also evaluates policy execution. Implementation refers to the degree to which intentions take place during the day-to-day activities of stakeholders (e.g., policy makers, county administrators, principals, teachers, students, etc.). Implementation can be observed through utilization of EMIS by stakeholders, budget allocation, distribution of human resources, availability of professional development activities, communication and dissemination of information, as well as the extent of institutionalization across the system. Once policy intent and implementation are analyzed, the EMIS assessment explores the results of these two key components, with a focus on system effectiveness and efficiency, in addition to teaching and learning, and management and planning. Strong education systems will ultimately use these outcomes to inform the effectiveness of policies and education strategies and make adjustments as necessary, creating the cyclical process illustrated in figure 3. In Fiji, EMIS intent and implementation were assessed through desk research, analysis of system applications and utilization, as well as interviews with a variety of stakeholders (table 1). Methodology Table 1: Measuring EMIS Intent and Implementation in The EMIS assessment methodology consists of Fiji a review of written policies and technical Policy Intent Implementation documents as well as interviews with key • Multiple meetings with • Analysis of data quality stakeholders across the education system to FEMIS team, and AQEP and comprehensiveness ensure proper implementation. team implementing • Examination of FEMIS professional development Research and investigation for the Fiji EMIS • Extensive review of activities relevant policies, national • Review and analysis of assessment took place between February- strategies, standards, and the utilization practices at April 2017. The authors conducted a planning documents all levels of the education comprehensive review of policies, as well as (provided by the EMIS system technical documents and other background unit and AQEP) materials. To further examine policy intent and • Meetings with the implementation, a series of interviews and Ministry of Education, meetings took place with the Ministry of district governments, and Education and Australian Government funded schools Source: Authors Access to Quality Education Program (AQEP) team. Country Overview Republic of Fiji is an upper middle income country located in the South Pacific and consists of more than 300 islands. The estimated population of Fiji in 2015 was 871,986 while the estimated number of children from the age of 5 to 18 years old was 223,192 for the same year. Of this number, 17,245 children were 5 years old (of preschool age), 129,901 children were of primary school age (between 6 - 13 years of age) and 76,046 were in the secondary school age group (between 14-18 years of age) (Ministry of Education, 2015). There are several ethnic groups, indigenous Fijians and Indo-Fijians being the prominent ones. The geographical distribution contributes to the scattered distribution of schools and students, which poses some challenges to the education system. The country is divided into nine education districts and four divisions. Approximately 90 percent of schools have access to the internet. Although only 90 percent of schools have Internet connections, the SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 estimated remaining 10 percent of schools without Internet are typically small primary schools located in rural or remote areas. The school system is governed by the Education Act and the policies of the Ministry of Education, Heritage, and Arts. The system is characterized by a public-private partnership, where approximately 98 percent of the schools are faith-based or community owned, but funded by the government. The rest 2 percent are government schools. The government initiated a Tuition Free Grant in 2014 to ensure that all parents do not have to bear the burden of paying tuition fee for their children attending primary and secondary schools (Ministry of Education 2015). The overall budget for the Fijian Ministry of Education 2015 was $401,649,600 ($USD 193 million) equating to 11.8 percent of the total government budget (MoEHA 2016). The ministry supports more than 210,000 students including Early Childhood Education (ECE) and employs 5,815 Primary, 4,690 Secondary and 100 Special Education staff (FEMIS 2016). The country’s GDP is approximately USD$4.5 billion Table 2: Education Indicators at a Glance and is one of the most developed countries in the Number of Schools 2016 Pacific region. It is richly endowed with forests, minerals ECE 942 and fish. Agriculture is the main source of GDP and Primary 734 employs more than 70 percent of the workforce. Secondary 181 Remittances, particularly for military staff are increasing Number of Students steadily, eclipsing traditional sectors such as sugar and ECE 15,577 garment manufacturing Sugarcane contributed Primary 145,867 significantly to the industrial growth, and sugar exports Secondary 66,827 and tourism add to the foreign exchange reserves. Net Enrollment Rate However, the presence of cyclones and coups have had Primary 100% an adverse impact on investments and have posed a Secondary 83.6% significant economic challenge. Student-Teacher Ratio ECE 14 Primary 25 The education system is structured in a fashion that Secondary 13 the primary education runs for six years, and lower Public Expenditure on secondary and upper secondary runs for four and three Education years, respectively (see table 1 for basic education % total government 13.7% statistics). While education is not compulsory, it is free expenditure for the first eight years. These schools are mostly run by % of GDP 3.8% either by government, religious communities, or the Source: MoEHA, 2017 districts. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Development of the Fiji Education Management Information System (FEMIS) This section describes the different stages of the system development as well as quality assurance within FEMIS. Early assessment of the Fijian education data system determined a need to integrate the ministry’s major databases into a unified, web-based platform. At that time, the ministry had three main databases, holding partially overlapping data. The databases did not communicate with each other. Development of the FEMIS started in 2012 and was tailored to its surrounding environment taking into account the national availability of Internet service, culture, per capita income, and existing ministry IT resources. Development Stage I: Before FEMIS The goal of the system development was to Box 1: Three main databases prior to FEMIS produce an EMIS with one single data source FESA - Fijian Education Staffing Appointments: Used to over four years. This demonstrated a manage all aspects of teacher placements. FESA is web- substantial change from the initial system, based but only accessible on the government network. where three databases made up the data storage- SIMS (School Information System), SIMS - School Information Management System: Used to collect aggregated paper school survey data on FESA (Education Staffing Information System) teachers, students, finances and assets. SIMS was a and LANA (Literacy and Numeracy Assessment Windows based system installed on PCs at district offices System) (box 1). Before FEMIS, all access to all and the Statistics Unit. databases was via computers on the ministry network. Access to all databases via the LANA - Literacy and Numeracy Assessments: Used to Internet was blocked. issue, mark and report on years 4, 6 and 8 national literacy and numeracy assessments. FILNA is web based and accessible only on the government network and Development Stage II: After FEMIS only within the Examination and Assessment Unit (EAU). At time of writing, FEMIS consists of two physical databases, LANA and FESA, the assessment and staffing databases respectively. The information captured by SIMS database was merged and integrated into FESA. Each database has a corresponding web application. The FESA application maintains staffing data in the FESA database. The LANA application maintains assessment data in the LANA database, but schools administer the class lists in the LANA database using FEMIS. The hardware design consists of a web server to serve web pages and reports over the open Internet, connected to a single (large) database server housing the FESA and LANA databases (figure 4). Senior staff approved the prototype for general use in August 2012. Crucially, security was established to allow as much visibility of data as practical while restricting edits and updates. For example, schools can see their staffing records but cannot alter their pay. However, schools can comment that their staffing data are incorrect, allowing districts to generate simple reports and follow up as required. The school- centric approach brought rapid and substantial buy-in from schools. The first year of FEMIS focused on rolling the system out to schools. An attendance module was quickly introduced along with rudimentary modules for tracking school fees, acquittals against the grant, recording of parent information, student health, an online library, and other features designed to engage schools in maintaining the core, ministry required, student data on gender, date of birth, and class. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 The success and affordability of FEMIS is why it now serves as potential model for other countries in the region such as Vanuatu. It is an excellent example of low-cost system development, implementation, and utilization. Much of the web technology was successfully attached to the existing EMIS, saving countless hours of software development time. In Vanuatu, ministry buy-in to similar concepts is strong, and the system is already improving staff and student data quality substantially. The parallel paper system in FEMIS is crucial as a model for countries such as Vanuatu, where Internet connectivity is considerably behind that of Fiji and paper remains the main data collection instrument. Figure 4: Development Stage II: With FEMIS - Using the Internet LANA database Assessment office using LANA Despite the success of FEMIS to date, substantial work remains for the future FEMIS development. This includes: • Further work with senior MoEHA staff in the use of FEMIS as a decision support tool • Analysis of FEMIS data to monitoring teaching and learning • Students and the wider community accessing FEMIS • Integration of ECE, Technical Education and Higher Education to track longitudinal outcomes • Encouraging utilization of the eLearning, Online Library, and Educators Forum areas • Integration of the EMIS with other ministries, primarily the Ministry of Health • Integration with the Births Deaths Marriages database SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Fiji EMIS Results This section presents the main results of EMIS diagnostics described in the previous sections. Results and scores for each policy goal are presented, along with supporting evidence. Policy Area 1: Enabling Environment Established  Fiji’s enabling environment was assessed in the following areas: (1) Legal Framework, (2) Organizational Structure and Institutionalized Processes, (3) Human Resources, (4) Infrastructural Capacity, (5) Budget, and (6) Data-driven Culture. The Ministry of Education, Heritage and Arts (MoEHA) introduced a FEMIS policy in 2014 to Box 2: List of Supporting Policies for EMIS support the functioning of FEMIS operations. The The supporting and relevant policy documents are: purpose of the policy is to provide a framework • The Constitution of the Republic of Fiji – 2013 that will enhance and facilitate effective, efficient • ICT Development Policy and timely data collection from schools and • Information Technology Centre Govnet Policy reporting to the MoEHA and all stakeholders • National Strategic Development Plan (2015-20) throughout Fiji. This policy is also designed to • Education Sector Strategic Development Plan ensure that the collection and entry of data is (2015-18) undertaken with care and concern to continually • Education Act (1978) raise the quality of data held in the Fiji Education • Examination Act • Public Service Act 1999 Management Information System (MoEHA 2016, p.2). The policy document builds on a number of Source:MoEHA 2016. relevant policies and legislations (box 2). It is the first step towards institutionalizing FEMIS in the country. The policy document clearly articulates the Box 3: Reporting Requirements by the MoEHA processes and mandates for successful implementation of the system. The 10-page The MoEHA is required to share education data with the following entities: document includes aspects such as FEMIS • Fiji Bureau of Statistics report processes, validation mechanisms, timelines, • Ministry of Education Annual Report allocation of responsibilities, privacy • MoE Strategic Development Plan (2015-18) assurances, some indicator coverage, • Ministry of Finance Quarterly Report government utilization and disciplinary action • Ministry of Strategic Planning, National in case of wrong doings. It also sets forth Development and Statistics Report reporting requirements for the MoEHA. The • Peoples Charter for Change Peace and Progress – MoEHA has to follow clearly outlined 2008 performance reporting requirements. By • Roadmap for Democracy and Sustainable Socio- institutionalizing reporting requirements, the economic Development (2009-14) MoEHA ensures that the education data is • Implementation Coordinating Office (ICO) Report integrated with other ministerial and • UNESCO Statistical Data on Educational international donor reports (box 3). However, Attainment the document does not specify how often or • UNESCO’s SDG and EFA achievements what type of education data should be shared. • UNESCO Institute of Statistics The next step would be to explicitly and Source: MoEHA 2016. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 publicly state what data can be expected to be shared and whose responsibility within the MoEHA the sharing process will be. The ownership and responsibility of data collection and management rests with MoEHA. Clear data ownership and responsibility are essential to ensure reliability and accuracy of the data. This is why, where practical, FEMIS has a single entity responsible for each data area with numerous entities using that data. At the same time, however, the entity responsible for maintaining an area of data must have significant interest in the accuracy of that data. This is based on the premise that an entity maintaining data that they do not use consistently results in poor quality data. The successful development of EMIS in Fiji was due to the political support from the government as well as the buy-in and involvement of staff at each level of the education system. The Ministry of Education was keen in developing EMIS as a tool to better manage education system business processes performance. To this end, creating awareness among all the different level of stakeholders was key. As a result, during the early days of inception, a concept note was developed and shared across staff at different levels to get their support. In addition, substantial effort was undertaken to train staff on the concept and benefits of developing EMIS, and how the system could be a useful tool to help improve and expedite their current work. At the ministry level, most senior staff demonstrate significant awareness of FEMIS and yet detailed knowledge and an understanding of the system as well as its potential often are not fully leveraged, until the potential of the data is fully understood and exploited by different stakeholders. As EMIS develops, the focus should move from the use of data for monitoring business processes to assessing the performance of the education system. Government should play a leading role in making the data available to other education stakeholders so they can analyze and use the data to improve the quality of education. The ministry has a dedicated staff overlooking the functioning of EMIS. MoEHA supports an IT Unit manager, an officer dedicated to training and support, a data cleaner that phones schools to ensure accurate data, and a senior programmer. The Access to Quality Education Program (AQEP) 1 supports an additional programmer and a data quality analyst. The data quality analyst is responsible for data quality strategies and education. The assessment unit maintains two full time IT staff dedicated to maintaining the assessment component in the EMIS to reduce dependence on third party software by increasing EMIS assessment functionality such as online marking and distribution of exams. Additionally, AQEP is funding temporary positions of an online training materials producer and a procedure writer. As the project nears completion, the ministry will perform training remotely or on line at substantially reduced cost. Continuous training to EMIS and IT staff at the Ministry of Education was another component that promoted the growth of the system. This achieved three simultaneous objectives: (1) the ministry became capable of providing ongoing support of the EMIS, which is a positive step towards its long-term sustainability; (2) the use of ministry staff avoids the involvement of IT support companies with profit driven agendas that typically run in opposition to, and are often destructive of, ministry needs; (c) international organizations and donors are freed from the software development issues and can focus on 1 AQEP is the agency working with the Ministry of Education and DFAT to implement FEMIS in the country. The budget for the project is from the Australian government. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 technical assistance. The Fijian EMIS was rolled out without the assistance of dedicated Technical Advisors (TA) in other areas of the ministry. Despite the advantages of a ministry-focused approach, the delivery of the EMIS could have benefitted substantially with dedicated TA supporting other areas of the ministry. For instance, TA support in the human resource, finance and assessment areas typically place high demands on an EMIS. A key factor that led to the buy-in and support of EMIS was the provision of training in schools to demonstrate the use and importance of student data. Schools that had access to Internet received training on the use of data. This training had substantial logistical hurdles because of the number of remote islands involved. ‘Train the trainer’ model was piloted, but was not successful. Direct sessions required schools to complete the registration of all student records at the time of the training, sometimes taking up to two full days. The training was repeated at the start of the school year and again at the end of the school year. Training at the start of the school year focused on rolling over and setting up new classes. Training at the end of the year focused on ensuring high accuracy of student data. This not only solved the buy in dilemma in obtaining student data, but also allowed working examples of classes and students. Tools were developed specifically to engage and coerce schools into in maintaining their own data. Although, training was designated as EMIS training, much of the training consisted of acquiring and learning basic computer skills. Numerous approaches were used to identify those who were not equipped to work on computers and provide additional training where possible. Over time, training was refined to target those more in need than others. Data quality reports were developed to determine which schools needed training and those that did not, reducing training costs and acknowledging compliant schools. The training generally covered the basics skills that staff would need during the school year such as for instance how to produce certain reports and how to enter requirements related to staff into the system. The training sessions at the central, district and school level contributed resulted in acceptance of FEMIS as a handy tool for monitoring outcomes. Each school has a FEMIS administrator responsible to maintaining and updating records in EMIS. Regular training sessions to the administrators and school principals significantly helped them to understand the tool better, and work more efficiently. The IT helpdesk at the Ministry of Education was a huge success in solving real time questions/ concerns the schools had with regard to information being inputted in the system. At any point, if the administrator or school teacher had a question, they would email the helpdesk and would get an immediate response that helped smoothen the process. Moreover, the training provided to the staff at the Ministry helped them use the system more effectively. Schools supply and use much of the student data in FEMIS in real time. This combination of schools using the data they provide has led to substantial ownership and data quality at the school level. While schools use the student list for attendance, the ministry uses the same high quality student list for assessment, grant calculation and other processes. The FEMIS budget is largely driven by support from the partner agency AQEP, which has implications on the long-term sustainability of the system. The core areas of EMIS (hardware and software) are funded through external sources. The government funds are only dedicated towards staffing and preparation of annual reports. Having a separate budget allocated to the EMIS Unit would be helpful in ensuring SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 sustainability of the system. Funds received from donors should be channeled through the government systems and documented in the EMIS budget. Policy Area 2: System Soundness Established  Fiji’s EMIS soundness was assessed in five critical areas: (1) Data Architecture, (2) Data Coverage, (3) Data Analytics, (4) Dynamic System, and (5) Serviceability. FEMIS consists of two physical databases, LANA and FESA, the assessment and staffing databases respectively. Each database has a corresponding web application. The FESA application maintains staffing data in the FESA database. The LANA application maintains assessment data in the LANA database, but schools administer the class lists in the LANA database using FEMIS. The hardware design consists of a web server to serve web pages and reports over the open Internet, connected to a single (large) database server housing the FESA and LANA databases. Each primary server has a matching redundant twin used for backup in the event of catastrophic hardware failure of a primary server. The redundant pair also function as the Quality Assurance testing area where new features are tried out and refined before installation on the primary live servers. In addition to the above, a copy of the two main live databases is taken each night and stored on the redundant server. This nightly copy is used exclusively for ad hoc querying by statistics officers. This allows statistics officers to build and execute queries that are not in standard reports in the EMIS. Typically, day old data suffices for most data queries received by the statistics officers. This copy does not function as a data warehouse, instead the data is an exact replica of live data. One of the main positive features of the Fijian EMIS is its affordability. Some cost features were particularly important contributors to the system’s affordability (table 4): (i) Structured Query Language (SQL) database software was provided without cost to the ministry through licensing arrangements already in place with federal IT body; (ii) Internet bandwidth has a cost and is also paid by the federal IT body; (iii) Software development costs such as labor, desks and PCs are provided by the ministry; (iv) At time of writing, recruitment is taking place to acquire a national resource for six months specifically to develop online training materials. This will allow the ministry to cease the costly donor funded direct training and independently sustain a more cost effective on line training program in the longer term; (v) No laptops were provided to end users. All schools brought laptops and their own Internet connectivity to all training without exception. Table 3: Cost of System Development Component Tool Approximate Cost $USD Software development Microsoft Visual Studio $600 to $1,500 per developer tools used to build web pages and reports Database technology Microsoft SQL Server Approximately $3,000 for each of two used to store data servers Server software Windows Server 201X Usually less than $1,000 for each of the four servers. Physical Server 2x web servers plus 2x $60,000 for all four servers Hardware database servers End user training Ministry staff and Approximately $100,000 per annum Technical Advisor(s) (exclusive of Technical Advisor costs) Source: Authors. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Each school has a FEMIS administrator who is Figure 5: FEMIS Administrator inputting data into responsible for maintaining FEMIS. Every school is EMIS at a Primary School assigned an administrator whose primary role is to input education data on a real-time basis into FEMIS. Every morning, school teachers provide the administrators with the attendance data which is inputted into FEMIS on a daily basis, which makes it easy to track student absenteeism on a daily basis. In addition, they keep updating records and data as and when it becomes available. Earlier, this work is done by teachers who found it inconvenient as they couldn’t devote time to teaching in classrooms. On the downside, this reduces the interaction of teachers with FEMIS and its potential utilization as they are not entering this information. In terms of convenience and ease of use, one of the principals of a rural primary school commented, “FEMIS helps reduce transaction costs as we don’t have to go to the district offices to submit paper-based census form or any other manual requests from the Ministry”. FEMIS abandons the census approach that collects pre-aggregated student and teacher data for use in Net Enrollment Rate (NER) and Gross Enrollment Rate (GER) style reports can be problematic. This approach can contribute directly to poor data quality. This is because, traditionally, this type of pre- aggregated, census based data collection is used primarily by the ministry to calculate the school grants where inflated student numbers are often submitted to attract additional grant. Without any comprehensive audits in place, this inaccurate data is then re-aggregated into NER and GER style reports, as was the case in Fiji before the FEMIS style of data collection was introduced, substantially reducing the overall national student counts. Without the resources for ongoing individual school audits, the approach in Fiji is not to use the data for the generation of aggregated reports (even though Fiji continues to report for national and SDG compliance) but to expose the granular data to as many consumers as possible while simultaneously using the data to underpin business processes that demand highly accurate data. By actually using the data to run day to day operations (not for the generation of annual census reports), higher quality data is produced on a daily basis. For example, individual student records are used throughout the year repeatedly for; schools to record attendance, finance to acquit the grants, assessment to issue and report on exams. The other obvious drawback with collecting pre-aggregated data is that drilling down to individual student and teacher records is not possible, nor are complex data correlations such as student attendance by gender against toilet conditions. The school heads, District Education Offices and Divisional Education Offices share responsibilities in the data production process. The school heads, head teachers and principals are responsible for data entry of student data. They upload the required information directly into FEMIS and are personally liable that this is done in a timely, complete and correct manner. The school management should submit the data on school finances upon the request of the school head. The District Education Offices are responsible for compiling the education data and ensuring a smooth process in data production throughout the year. The Divisional Education Offices supervise the District Education Offices in their responsibilities (figure 7). While there are guidelines for staff to adhere to these guidelines, limited enforcement mechanisms are in place. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Figure 6: Division of Responsibilities at the Regional and Local Level Divisional •Oversee the work of the District Education Offices Education •Ensure that all policies are complied with Offices •May recommend disciplinary action in case of non-compliance District •Responsible for distributing and collecting school census Education questionnaire forms and attendence sheets Offices •Processing data and forwarding it to the Divsional Education Offices •Resolving staff issues should they arise, e.g. assignment of teachers •Data collection and entry into FEMIS •Accurate and timely submission of data Schools •Reporting of staff movement •Class audits at specified times Source: MoE 2014; MoEHA 2016. Figure 6 illustrates an example of how some school data compliance is reported in FEMIS. The District Education Offices (and others) can generate a report, which provides them with an overview of teachers, classes and students. Currently, the report does not show information on ECE school data as this Figure 7: Summary of School Data at the District Level Source: FEMIS information is not yet been fully integrated into FEMIS. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 13 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 FEMIS has ensured that all schools (government and community-based) are integrated into its system. FEMIS is fully digitalized and generally data entry is done by uploading the data directly into FEMIS or FEMIS extracts data from other systems. However, as approximately 10 percent of schools in Fiji remain without internet connectivity. Most school-level data input pages have paper equivalents allowing non- connected schools to complete and submit data to district offices for entry into the EMIS. FEMIS includes individual-level data on assessments, enrollments, attendance and infrastructure for basic (primary and secondary) education. Currently, FEMIS collects data from primary and secondary schools, but there are plans to integrate early childhood education and technical colleges data. This would allow information on student progression and performance from ECE to technical and higher education, and would have important implications for designing interventions to monitor and improve outcomes. Table 5 shows best practice examples of data, which should ideally be collected in an EMIS including administrative, financial, human resource and learning outcome data. Staffing and personnel data are also integrated into FEMIS. Data from the staffing system FESA are displayed as read only data in FEMIS along with selected payroll/pay slip data from the national pay system, leave balances, professional development history, qualifications and teacher registration data. Teachers may not alter any of these data but can provide feedback to district offices in the form of a comment on the staffing details page in FEMIS. Districts can list the feedback as the “Staffing Problems” report and follow up as required (figure 8). Once an issue is resolved, the district can clear that issue from the problem report. A tally of problems per district is visible on all ministry and district pages. Currently, however, there is less use of this feature at the district level, and mostly the IT unit within the Ministry resolves the issue of errors in reporting. Figure 8: Screenshot of FEMIS School Reported Staffing Problems Source: FEMIS Registration for external exams is directly derived from FEMIS. One of the many achievements of FEMIS is that student registration for external exams and national literacy and numeracy assessments is fully integrated into the system. The Exams and Assessment Unit utilizes the student registry in FEMIS to automatically enroll students for the appropriate external exams and LANA (year 5 and 7 since 2016; previously Year 4, 6 and 8). This ensures additional buy-in from schools as it makes registration of all SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 students in FEMIS a necessity while Table 4: Best Practice on Data Coverage eliminating a previously separate Data Type Best Practice Fiji cumbersome registration process Administrative Demographic Demographic and allowing for the entry of marks data Health Attendance for some exams by schools and the Attendance (enrollment, (enrollment, printing of student assessment repetitions, dropout, repetitions, dropout, reports. This also facilitates the progression, etc.) progression, etc.) printing of some national exam papers at schools, saving on vast Financial data Budget and revenues Budget and revenues resources previously required to Spending Cash Transfers and Cash transfers and Subsidies securely ship exam papers. subsidies Unit cost per student Unit cost per student Despite the comprehensive data Human General demographics General demographics captured into FEMIS, some resource data Salaries Some Professional indicators are still calculated Performance evaluations Development manually. Despite improvements in Professional development the system many indicators are still Learning Classroom assessments National Assessments calculated manually. For instance, outcome data National assessments the net enrollment rate and gross International assessments enrollment rate for primary and secondary education is still calculated manually, despite all the information into FEMIS. This is because the focus currently has been to collect data to improve business processes instead of calculating statistics for understanding the overall performance of the education system. Advancements of the system over time should encourage the development of these indicators. All students are equipped with unique identifiers. For Fijian students, the birth certificate ID along with the student is also recorded; for foreign students, their passport number is recorded. This technically allows for students to be tracked throughout their academic career. It also is easier to track student movements at the beginning, during and at the end of the school year. This also allows the prevention of schools claiming the same student in two schools simultaneously. Students with disability are also integrated into the regular schools under the Special and Inclusive Policy and their special needs are included in the FEMIS. Teachers are uniquely identified through their staffing system ID and also their provident fund ID. Schools are uniquely identified by school registration numbers issued by the Assets Management Unit. ECE centres are gradually receiving their own unique ID numbers a separate from their attached primary schools. This latter process is ongoing. Numerous attempts to link the EMIS to the civil registry have not been successful. The EMIS records height and weight of students for use by the ministry of Health although data sharing has not progressed past initial meetings. While some basic tools exist to analyze the data collected, analysis has been still weak. While basic descriptive analytics and data tabulations are conducted by staff, data analytics are not automated in the current EMIS. Instead, the EMIS Unit extracts data and runs analysis using excel. Moreover, the use of analysis for planning, monitoring outcomes is not happening. Schools still manually collect data points to track student performance. Policy Area 3: Quality Data Established The quality of data captured by Fiji’s EMIS was assessed in four areas: (1) Methodological Soundness; (2) Accuracy and Reliability; (3) Integrity; and (4) Periodicity and Timeliness. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Clear data ownership and responsibility are essential to ensure reliability and accuracy of the data. This is why, where practical, FEMIS has a single entity responsible for each data area with numerous other entities using that data. At the same time, the ministry is responsible for maintaining overall accuracy of the data. This is based on the premise that an entity using its own data will provide higher quality data for use by itself, with other areas of the ministry benefiting by sharing the higher quality data, putting additional pressure on data accuracy. FEMIS has developed a comprehensive mechanism to main security of the system. FEMIS has four broad account types; ministry, district, school and teacher (figure 9). The IT Unit manages only the ministry and district accounts. • Ministry-level access includes general ministry staff, emergency relief workers, engineers, finance, highly restricted Examination and Assessment Unit (EAU) access, etc. Ministry account types can see all schools in all districts with restrictions on what is visible within the school based on the actual account type. For example, engineers cannot see individual student records, emergency relief workers have restricted views of school data but can run disaster management reports on all schools. Ministry level accounts have extremely limited write access (if any) but tend to have broad visibility of data. • District-level access is broken into two different types: (a) District read only users are the same as ministry users in that they can see all data and change nothing except this access type is restricted to viewing only schools in one district. (b) District read write users are also restricted to schools in one district but this user type can change school data sufficiently to act as a proxy for schools that do not have an Internet connection. Policy mandates these users to maintain data for schools without Internet service. Districts are responsible for managing passwords for all schools in their districts. • School-level access allows a school to log on and manage all students in all grades in the school. School level access can also see all finance, facilities and teacher data for teachers currently in the school. Schools are responsible for managing passwords for all teachers attending the school. The staffing system FESA determines the school a teacher is in at any given time. If FEMIS determines any inconsistencies in the FESA staffing information, teacher access is denied until the issue is resolved. Schools log on with a school code. • Teacher-level access allows a teacher to see their own professional development, leave and other staffing information. If the school level logon has assigned this teacher a class, the teacher can manage all students in that class including student attendance, student fees, student health, etc. The teacher can also view assessment results for students in their assigned class(es). Teachers log on using their teacher ID. • Student-level access and parent/community level access are not yet developed. • Public-level access is not currently permitted by the Ministry. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 The school heads are responsible for the accuracy of student, finance and other school data. The head teachers and principals have to follow the guidelines presented by the MoEHA to ensure that the data are current, correct and complete. If the school heads fail to do so or worse even deliberately falsify data, they will be subject to disciplinary penalties. Disciplinary actions include, but are not limited to salary suspension, disciplinary proceedings and criminal charges. Figure 9: Hierarchy of Data Access IT Unit Manages Approx. 600 Ministry and District Passwords Districts Manage Approx. 1000 School Passwords Schools Manage Approx. 12,000 Teacher Passwords Teachers will in the future manage Approx. 200,000 Student passwords FEMIS is equipped with data validation mechanisms that ensure 95 percent accuracy of data. The FEMIS administrator is responsible for updating and uploading accurate student, finance and other school data into FEMIS at the school level. In addition, the FEMIS administrator is responsible for noting incorrect staff movements that exist in the Fijian Education Staffing Appointments system (FESA) such as teachers on leave not reflected in FESA. Moreover, the FEMIS administrator officer reports to key stakeholders (e.g. head teachers and principals). Then, school heads and principals carry out mandatory class edits at specified times. During those class edits, the head teachers or principals have to verify and confirm the student enrollment data for each Year level. They have to check and compare all student data in the following parameters: age, citizenship, ethnicity, new students, repeaters, start /end term and borders. All of this is done electronically in the FEMIS database by simply ticking a box associated with the class for the given term. Audited financial data is submitted by the school management within four weeks of the school Annual General meeting, the school management is expected to submit all audited financial reports. Figure 10 illustrates a screenshot of the class audit function. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Figure 10: Screenshot of Data Audit Function at the School Level Source: FEMIS Training sessions were provided to schools to ensure data entry and accuracy. Training sessions conducted at the end of the year focused on ensuring high accuracy of student data. Data quality reports were developed to determine which schools needed training and those that did not, reducing training costs and acknowledging compliant schools. Reports include duplicate check numbers, duplicate invoice numbers, students in two schools at the same time, missing birth certificate ID, etc. Figure 11 illustrates a screenshot of options for FEMIS data quality reports. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Figure 11: Screenshot of some FEMIS Data Quality Report Options Source: FEMIS A Data Quality Analyst was recruited with the sole task of assuming responsibility for data to day quality of the data across the system. A person for student records verification was also recruited with the sole task of ensuring student data accuracy. This latter role involves numerous phone calls to schools where students have transferred mid-year and one of the schools has not indicated the student has transferred out. Schools also take interest in their own data, which is evident in the following two comments in the forum. Each comment is a school publicly requesting another school to remove a student that has transferred into their school (figure 12). Figure 12: One Screen Shots of the FEMIS Forum Showing Schools Using the Forum to Communicate with Other Schools to Clean Student Data Source: MoEHA SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 There are different timelines provided for data submission for different types of data, but this may not be adhered to. Attendance data are normally allowed three weeks late at any one time, though this is not enforced. Staffing information is continuously updated as teachers move around along with qualification, leave, etc. Student and class registration has three separate term audits needed to receive grant funding. The rest of the year, the student location and other data are just refined more and more accurate until individual student movements are followed up during the assessment process. By clearly stating the reporting requirements, the MoEHA ensures that data are delivered in a timely manner. However, at this point, there are still some issues with compliance of timelines. Policy Area 4: Utilization for Decision Making Emerging The utilization of Fiji’s EMIS was assessed by examining four areas: (1) openness; (2) operational use; (3) accessibility; and (4) effectiveness in disseminating findings. FEMIS is utilized for a variety Table 5: Operational Use, Best Practice of basic reporting, tracking and Use Best Practice Fiji resource allocation functions. In evaluation • School performance Currently, FEMIS data are used • Student performance for national assessment • Growth reports registration and dissemination, • Diagnostic reports grant allocation, attendance • Graduation rates tracking, determination of • Transition rates textbook requirements, • Teacher performance calculation of transport In governance • Policy decisions • Some Planning subsidies, school finances • Accountability • Management including detailed expenditure • Planning against allocation with budget • Management tracking at school level, staff By schools • Academic performance professional development, • Teacher performance teacher qualifications, teacher • Management leave, teacher • Comparison with other salaries/positions/history, schools school type, school By clients • Parents’ access to EMIS infrastructure, teacher subject • Communities’ access to loading, school mapping, EMIS • Use data to make decisions disaster resource tracking, drug • Use data to demand quality usage, an online library and By government • Performance ratios • Infrastructure forum, school library data, • Infrastructure capacity • Teacher salaries school lease/property/legal • Quality/outcome indicators • Some equality data, substantial individual • Spending efficiency and quality student data such as fees, • Teacher salaries indicators health, assessments, family, • Equality indicators subjects and determination of Source: Adapted from Abdul-Hamid 2014. staffing requirements. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 The data is not yet used effectively to support senior level decision making or monitor the implementation and effectiveness of policy, beyond SDG and parliament reporting. However, FEMIS has the potential to be utilized for a broader variety of operational uses. For example, releasing 50,000 exam papers is fairly effortless but the skewed grades by gender where one gender is substantially more absent than another is effectively ignored. Table 6 provides an overview of best practice standards of EMIS utilization. It covers a broad range of utilization strategies and users. For instance, any EMIS could be used for evaluation and governance purposes. The introduction of individual student and teacher records at the school level encouraged schools to consume their own data, driving up the quality of student data. This also improves communication in schools between staff members as it is possible to monitor real time education data and respond to the needs of individual students and teachers. Individual data records also enable government staff in allocation of grants, resources and developing policies that cater to individual student characteristics. Being an integrated system, it is possible to leverage the attendance reporting, national assessment report, financial data, etc. to compare and draw inferences. For example, the outcome data on student’s assessment can be linked to the attendance data and allocation of material resources. This is especially useful while analyzing the impact of inputs on outputs. Schools can draw up their own attendance records (students and teachers), data on facilities and financial data. Figure 16 illustrates some examples of reports that are available through FEMIS for individual schools. Figure 13: Finance, Material Resources and Attendance Module in FEMIS Source: FEMIS Training Videos, Ministry of Education SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 There is a strong culture of data retention for internal consumption, instead of making the data available to external stakeholders. The MoEHA is very protective of education data and utilization. Although data quality may be moderate to high, the ministry has a culture of retaining data for internal consumption rather than sharing data transparently with stakeholders. At present data is used more to drive and manage internal ministry processes than to support decisions and monitor policy. The use of data in the ministry has seen improved management of finances and vast improvements in assessment and other processes. The MoEHA exercises full ownership and access to education data. A fully operational EMIS system, which is used by all stakeholders has to be open, accessible and transparent to the public. Already in the 2014 Policy Document on FEMIS, the MoEHA clearly stated that all education data belongs to the ministry and any access to data outside of the publicly available report can only be granted with the explicit permission of the Permanent Secretary. Such strict policies and limited data dissemination restrict data access for use. Some dissemination initiatives are taken, but there is no formal data Box 4: Overview of Data Published in the Annual Report 2015 dissemination strategy is in place. The The Annual Report 2015 includes the different types of data, e.g.: annual report of 2015 is the latest • Enrollment data (ECE, primary, secondary and partially also available set of education data (box 4), tertiary education), disaggregated by gender and districts which is accessible to the public. These • Teacher information and salaries (for ECE, primary and reports are also disseminated to all secondary on a national and regional level) schools after the report is made publicly • Professional development and workshops available, and also discussed extensively • Expenditure information in education in conferences, and data is • Scholarship data and tuition fees published bi-annually in Fiji times and • Data on disabilities Fiji Sun. However, the FEMIS policy does • Number of schools listed as government, private and non- government schools not outline any dissemination strategy, • Maintenance and facilities upgrades and most of the staff at the Ministry are • Number of newly built government schools not aware of the annual reports or • Data on libraries (distribution, availability, etc.) dissemination mechanism. This is • Development of new school books because the use of FEMIS is mainly • Offences committed at school (by type of offence) restricted to improving business processes. Advancing to a next level of data usage is essential. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Recommendations and Proposed Activities This section presents a set of recommendations and proposed activities based on the assessment of EMIS in Fiji (table 6). Recommendations and activities aim to Table 6: Fiji EMIS Rankings improve the overall EMIS functionality in a Established 1. Enabling Environment sustainable and effective manner, to ensure  better access and use of information for Established decision making, planning, and student 2. System Soundness  learning. Future activities to improve the EMIS Established should be strategically designed such that they 3. Quality Data  incrementally boost dimensions of the EMIS to Emerging a more advanced level, ultimately improving 4. Utilization for Decision Making  overall EMIS functionality in a sustainable and effective manner. The Strengths Weaknesses Opportunities Threats (SWOT) identification (table 7) summarizes key points from the needs assessment and informs recommendations. Table 7: Fiji EMIS SWOT Profile Strengths Weaknesses • Strong political buy-in • Different compliance requirements for different • FEMIS policy institutionalizes operations types of data not adhered to always • Clear allocation of responsibilities • Data access and use remains mostly at the central • Detailed timelines and requirements stipulated level in policy documents • Limited automated processes to calculate key • Affordable system indicators • School-based management system • Statistical handbooks are not published online • Accurate data regularly. The latest publicly available data is • An established system of cost-efficient training published in the 2015 Annual Report workshops • Poor quality of infrastructure data Opportunities Threats • Ongoing development of establishing an EMIS • Dependence on donor funds and technical • Including ECE and technical education data into assistance can potentially threaten FEMIS in the long FEMIS term • Utilization of data for monitoring education • Restricted data access to the public outcomes and improving teaching and learning • Data dissemination strategy not in place, and not • Promoting research and data access to all reflected in FEMIS policy education stakeholders • Utilization of disability disaggregated data to improve the access and quality of education for children with disability • Increasing compliance/ response rate by all schools to 100 percent The EMIS is only as good as its utilization. This emphasizes the need for data to be utilized across the entire education system. By doing so, system-wide efficiencies, teaching and learning practices can be improved. The aim of any EMIS is to provide reliable, accurate and useful data for decision-making processes while also improving learning and teaching through analysis of assessment data. Without being SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 used, data uses its value and data collection seems pointless. Data utilization should occur at the central, regional, school and local level, even including parents, students and the community. The policy document on FEMIS of 2016 could be expanded to include additional aspects such as data dissemination requirements and timelines. FEMIS would benefit from further institutionalization of data utilization and dissemination by including it into the policy document. The data dissemination strategy should include strict timelines so that the data remains timely, accurate and useful for the public. In addition, special attention may be paid to data access standards. The budget for FEMIS should be explicitly stated in the government budget to ensure long term sustainability of the system. Currently, majority of the costs come from the donor budget, and the detailed expenditure on line items are not accounted for. All donor funds should be routed through the government system, and the budget should be clearly defined with a line for each of the following categories: (a) Data collection; (b) Auditing mechanisms; (c) Staffing; (d) Training and professional development; (e) Dissemination of materials (such as printing and publishing); (f) Infrastructure (software, hardware), and (g) Regular maintenance of the system. Table 8: Examples of Data Dissemination Strategies in Chile In order to provide accurate, reliable Assessment Provide pedagogical support to school and useful information, the list of guidelines principals, pedagogical coordinators, and indicators in FEMIS may be teachers expanded. Currently, data collected is School report Provide pedagogical support to school used for improving education system principals, pedagogical coordinators, and business processes. The focus is not teachers on producing indicators, and the National report Inform policy to decision makers and general indicators currently produced follow a public on student performance across grades manual process. Expanding focus to and regions regularly producing and publishing Newspaper Hold schools accountable by publishing data information on education indicators supplement on school grades by subject across regions will enable stakeholders to assess the Parent report Hold schools accountable and involve parents in school performance of the education system. Online bank item Provide pedagogical support to teachers through releasing test questions from all Data openness and access to subject areas and target grades education data could be facilitated. Press kit Inform policy by providing materials to Currently, the MoEHA retains full journalists and regional education offices ownership of the data and only limited Data files Inform policy, provide pedagogical support, access is granted to users outside the and hold schools accountable, depending on ministry. Individual schools can run research topic, to researchers reports in FEMIS based on their own Data analysis tool Inform policy, provide pedagogical support, data, but no comparative data across and hold schools accountable, depending on schools is available. Making data the type of analysis, for researchers and available for additional buy-in from decision makers Geo-referential Google Maps with the geographical location of different stakeholder groups and system schools and their mean scores for parents to facilitates a data-driven culture. hold schools accountable Ministry of Inform policy, provide pedagogical support, Data should be published in a timely Education website and hold schools accountable manner, disaggregated and in an Source: Ramirez 2013. appropriate format. The latest SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 publicly available data is from 2015 in the form of an Annual Report. The data is often on an aggregate basis and sometimes disaggregated by certain characteristics such as gender, type of school, grade or by region. The report provides an overview of the education system, but there is only limited disaggregated information included. To promote the data driven culture in the country, it is essential to make the education data available to public. Active data dissemination strategies encourage data buy-in and utilization by parents and the community. Data dissemination strategies reach from publicly available individual school reports and annual education reports to newspaper articles, radio contributions and television discussions. Fiji should determine the best strategies in its country specific context taking geography, technology and other factors into consideration. Table 8 illustrates an example of data dissemination. Trainings to schools should focus more on utilization of data for improving teaching and learning, management and planning. Currently, the focus of the FEMIS staff is to design training programs to teach FEMIS administrators and principals on how to use the system and input data for compliance purposes. This was extremely important during the initial stages of FEMIS implementation. The time is now right to transition to a next stage of delivering trainings that empower school heads and teachers on how to utilize the system to manage school operations, monitor school, teacher and student performance. Doing this would also help overcome the issue of lack of interest in FEMIS data by teachers, who now turn to FEMIS administrators for their work, and overlook the extremely useful data that is generated from the system to inform classroom instruction. At the school level, individual school report cards may be used to provide feedback to the staff, parents and the wider community. School report cards should be developed that provide information related to structure of education system, teachers, student performance, infrastructure needs, etc. It is important to design the school report cards effectively, so that the schools can utilize the information presented to them. They need to be clear, accurate and useful. By ensuring that every school receives individual feedback, which (aside from highly sensitive data) is accessible to the general public facilitates comparison between different schools. Not only would such a feedback loop improve data access and utilization, but it would also aide schools in the achievement of their planning and development goals. Many countries such as Australia have already implemented such report cards and have also designed an index, which enables parents to compare index scores nationwide (box 8). This is essential for promoting a data driven Box 5: Examples of Feedback Mechanisms in the Form of School Report Cards In Australia, feedback mechanisms have been established to include individual school report cards. Education stakeholders can compare school performance on the “mySchool” website, where the information is publicly available. mySchool calculates an index scores and provides an overview of general school indicators, which allow parents to compare schools from all over the country (ACARA 2016). Similarly, in Chile school performance data is made publicly available to provide feedback to the schools, parents and other stakeholders. The school performance data is all accessible online and parents can use a search option linked to the unique school’s ID number. In addition, Chile publishes a ranking of best performing schools in the country. The highest ranked schools are rewarded with financial benefits (Bruns, Filmer and Patrinos 2011). In Uganda, the Ministry of Education provides feedback to schools and parents in the form of school report cards. These school profiles are based on the data provided by the annual school census. They are distributed to headmasters, parent-teacher associations and elected officials. The school report cards have been well- received and make it possible for headmasters, parents and the community to track the performance of the school (Cameroon 2012). SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 culture as parents need to have enough information to make school choices and have information on education performance across these schools. Unique student, teacher and school identifiers can be helpful in tracking individual performance throughout their academic careers. Fiji already has an effective unique identifier system in place for all local and foreign students by using their national identification and passport number, respectively. This means that FEMIS is equipped with tracking mechanisms for students and teachers. Longitudinal tracking of students, teachers and schools allows for important policy conclusions, which can be utilized directly to improve student learning outcomes. This will also allow for specialized services to be provided for children with disability. Utilization of data at the school level is important. As the frontline of data collection, schools are essential to the overall success of an EMIS. As a result, efforts to ensure that teachers and principals are not only simple inputters of data, but also thoughtful users of those data will strengthen the overarching EMIS. One way to do this is to establish collaborative networks around data utilization. This could take the form of a professional development community where principals and teachers of different schools in any given area come together to think of innovative ways to use data to manage schools, students and classroom learning. A sustainable and well-functioning EMIS requires the knowledge that EMIS is more than an IT system. At the central and local levels, a common trend that was observed during interviews was that EMIS was seen as an IT system used for automation of the previous work that the Ministry was doing. A common problem with EMIS implementation is the belief that it is primarily technology. For EMIS to be successful, a holistic approach should be pursued that takes into account the many factors, people, processes, and technology, that contribute to long-term success and sustainability. Given FEMIS’s successful implementation and buy-in from all the education stakeholder at the local and central level, the time is right for MoEHA to transition from a compliance-focused approach to a learning focused one. The journey from a compliance-focused EMIS to a learning-focused EMIS is complicated, but it is also a game changer. At the core of this transition is a behavior change in the way that data are valued. The shift moves from seeing data as a tool to monitor progress, to understanding that data, as well as the larger EMIS, can be used to promote teaching and learning as well as management and planning. When designing a new system, start with the policy questions that key stakeholders want the system to answer. By answering this question early in the process, the risk of ending up with a fragmented and limited system is reduced. For this, multiple efforts to build consensus across a variety of departments and stakeholders. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Regional Comparison Fiji, Papua New Guinea, Solomon Islands and Samoa The Pacific countries, namely, Fiji, Papua New Guinea, Samoa and the Solomon Islands have working towards developing an effective EMIS that monitors education outcomes in their respective countries. A SABER evaluation was conducted which provides an overview of the current status of the systems in the four policy areas (table 9). Fiji and Papua New Guinea have advanced further than most of their neighbor countries in the establishment of an EMIS, while Samoa and the Solomon Islands lag behind in some areas. However, each EMIS experience can provide others with lessons learnt and additional insights into the establishment of an EMIS. Table 9: Overview of SABER-EMIS Benchmarking Scores Policy Goals Fiji Papua New Samoa Solomon Guinea Islands Enabling Environment Established Emerging Emerging Latent System Soundness Established Emerging Latent Emerging Quality Data Established Established Latent Emerging Utilization for Decision Making Emerging Emerging Latent Latent Note: The comparison is based on the 2015 SABER-EMIS assessment of Papua New Guinea, Samoa and the Solomon Islands. Fiji may serve as a potential role model for these four countries and others in the region. For instance, Vanuatu is already looking towards FEMIS to gain insight on the process to establish a functional EMIS. The strength of FEMIS lies in its design as a low-cost education data system, including development and implementation. Much of the web technology was successfully attached to the existing EMIS saving countless hours of software development time. Similar to Fiji, the Vanuatu EMIS used an existing database and simply added web functionality with additional features and access to schools. The parallel paper- system in FEMIS is crucial as a model for countries in the region, where internet connectivity is considerably behind that of Fiji and paper remains the main data collection instrument. In fact, internet connectivity is crucial for a timely, accurate and well-functioning EMIS. This is why timing of the roll out of a national EMIS should coincide with very early availability of general Internet nationally. If the roll out takes place after the availability of Internet at schools, third party EMIS vendors will sell EMIS software to the larger schools or schools will download free EMIS software. A scenario of multiple systems operating across numerous school sites is an extremely complex architecture to reconcile and/or roll back to a single national data source. In the case of Fiji, schools were adopting technology faster than the ministry and others initially anticipated. In terms of the enabling environment, Fiji, Papua New Guinea, and Samoa all have introduced policies that officially establish and institutionalize EMIS. Papua New Guinea bases the institutionalization of EMIS on the following policies: the National Education Plan (2005–14) and Information and Communication Technology Policy and Universal Basic Education Plan (2010–19). The policies have strengthened infrastructure, standards, and processes, but they do not commit to strong data-driven decision making in practice. In Samoa, the Education Sector Plan (ESP 2012–18) commits to the creation of an integrated EMIS at all levels of the education system, which is supported by other official policies. In the Solomon Islands, the National Education Action Plan (2004–15) has laid the foundation for EMIS, but they do not explicitly support or define EMIS operations. None of the four countries have dedicated EMIS budget lines in the Ministry of Education’s budget, and the EMIS budget often stems from donor organizations, which may harm EMIS sustainability in the long term. Political buy-in and extensive training for EMIS staff fostered the development of a data-driven culture in Fiji. In fact, the development of EMIS policies is often a reflection of the buy-in and support from the senior management that with an EMIS governments can SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 solve problems related to payment of teacher salaries, allocation of grants and monitoring expenditures. Once the ministry is heavily invested in using the system, the data quality improves, and only then will senior staff actually make decisions based on the data because they are finally assured of its reliability. This is why in the initial phase of an EMIS data access and the emphasis of data-driven education decision making remains at the central level. No policies are in place that stress the importance of data access for all education stakeholders. Generally, the public receives access to aggregate education statistics in the form of the annual statistical yearbook. In terms of budget, all of the four countries rely significantly on donors for funding and support on EMIS operations. The Ministry does not have a dedicated EMIS budget and a line item to EMIS operations, which may have consequences for long term sustainability of the system. In terms of system soundness, Fiji has successfully established a low cost, affordable EMIS, that is able to produce a broad range of education statistics related to assessment, human resources, finance and infrastructure. Each school has a school-based information system which is run by a FEMIS administrator, which helps in reducing transaction costs involved with paper-based census forms. This low-cost system is a good model for small island countries, who do not need expensive complicated systems to collect and manage their education data. The system also allows for easy data integration as the web-based platform can consist of different modules which can be easily integrated and analyzed. Fiji has been able to maintain accurate and quality data because of its non-conventional method of abandoning the census approach that collects pre-aggregated student and teacher data for use in NER and GER style reports can be problematic. This approach can contribute directly to poor data quality. Traditionally this type of pre-aggregated, census based data collection is also used by the ministry to calculate the school grants where inflated student numbers are often submitted to attract additional grant. Without detailed student level audits, the inaccurate pre-aggregated data is then re-aggregated into NER and GER style reports implying a valid data set when reports trend normally for years as is the case in Samoa and Solomon Islands. Papua New Guinea has also made significant advancements in the quality of data collected using their system. First, they have developed a comprehensive EMIS Operations Manual, complete with metadata, data fields, and indicators that links to the national education strategy. In addition, the country has developed an extensive data validation mechanism with validation occurring at multiple times of the data flows: (1) at the District Education Office; (2) Provincial Division of Education; and (3) Department of Education. The process is supported by automated validation processes that flag discrepancies. Data utilization and lack of data-driven culture continues to remain a challenge for the four countries. In general, data is mainly (or exclusively) utilized at the central level and here the focus is primarily on managing business processes such as allocation of grants or voucher systems. In Papua New Guinea, there is a considerable gap between EMIS awareness and usage between central and subnational education staff. At the regional level, staff are often unaware of the system’s data usage capabilities. In Samoa, the annual statistics report is disseminated to all schools by the end of the collection cycle, but the report provides only an aggregate overview, which does not empower schools to truly make use of the education data. Similarly, in the Solomon Islands a two-page school feedback report is sent to schools containing a brief analysis of school performance (e.g. enrollment rates, repetition rates and student-teacher ratio), but it is rarely used in practice. FEMIS allows schools to own and produce their own attendance, enrollment and financial reports, but there is no standardized system to compare school SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 performance or track schools towards their goals. Currently, parents, students or the community cannot access education data for informed decision making. Moreover, improvements in data dissemination are needed so that users are made more aware of data and are able to utilize it effectively. After all, any EMIS is only as good as its utilization. Acknowledgments This report was prepared by SABER-EMIS team members Husein Abdul-Hamid (Task Team Leader), Namrata Saraogi and Diana Mayrhofer in collaboration with the AQEP team lead by Dr. Priscilla Puamau. Fred Brooker (CEI International) led a team to complete this report as part of an independent evaluation of FEMIS. The report benefited immensely from the guidance and support of MoeHA staff: Mr. Iowane Tiko (Permanent Secretary for Education), as well as the entire team at MoEHA. The team is also grateful to the AQEP staff: Narain Sharma (MoE Liaison & Quality Education Coordinator), Semiti Tamo (Monitoring & Evaluation Coordinator), Bianca Murray (Social Protection Specialist) and Colin Connelly (IT Specialist). Finally, the team is especially grateful for the insights provided by principals, teachers and administrators of the schools visited in Fiji. Special thanks to colleagues who provided support: Cassia Miranda and Manar El-Iriqsousi. Abbreviations AQEP Access to Quality Education Program DEO District Education Officers FEMIS Fijian Education Management Information System FESA Fijian Education Staffing Appointments GER Gross Enrollment Rate LANA Literacy and Numeracy Assessments MoEHA Ministry of Education, Heritage and Arts NER Net Enrollment Rate SIMS School Information Management System References Abdul-Hamid, Husein. 2014. “SABER EMIS Framework Paper.” World Bank, Washington, DC. Andrabi, Tahir, Jishnu Das, and Asim Khwaja.. 2015. “Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets.” World Bank Group, Washington, DC. https://www.openknowledge.worldbank.org/handle/10986/21670. Ministry of Education. 2014. “Policy on Fiji Education Management Information System”. Accessed December 10, 2016. Ministry of Education, National Heritage, Culture and Arts. (2014). Annual Report 2014 Ministry of Education, National Heritage, Culture and Arts. Suva: Ministry of Education, National Heritage, Culture and Arts. Dr. John Cocke, George Radin, Joel Birnbaum, Frances Allen. (n.d.). RISC Architecture. Retrieved 06 3, 2016, from IBM: http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/risc/ SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 FEMIS. (2016, 06 10). UIS Survey Table A5- A6. Retrieved 03 10, 2016, from FEMIS: http://www.femis.gov.fj/femis/ReportCriteria/Report_70_10_UIS_Survey.aspx Fiji Access to Quality Education Program—Framework for Delivery. (2010, 10). Retrieved 06 03, 2016, from http://dfat.gov.au: http://dfat.gov.au/about-us/publications/Pages/fiji-access-to-quality-education-program- framework-for-delivery.aspx Fonseca, M. (2015, 1 12). The Biggest Technology Failures of 2014. Retrieved 06 03, 2013, from http://www.intelligenthq.com/: http://www.intelligenthq.com/technology/the-biggest-technology-failures-of- 2014/ Kanaracus, C. (2013, 12 30). The worst IT project disasters of 2013. Retrieved 06 03, 2016, from http://www.infoworld.com/: http://www.infoworld.com/article/2609011/applications/the-worst-it-project- disasters-of-2013.html Ministry of Education, Heritage and Arts (MoEHA). 2016. “Policy on Fiji Education Management Information System”. Accessed December 10, 2016. http://www.education.gov.fj/images/2016/FEMIS_Policy_-Reviewed_2016.pdf Ramirez, Maria Jose. 2013. “Disseminating and Using Student Assessment Information in Chile.” Systems Approach for Better Education Results (SABER) student assessment working paper no. 3. World Bank, Washington, DC. http://documents.worldbank.org/curated/en/2012/01/ 16238811/disseminating-using-student-assessment- information-chile. Short Set of Questions on Disability. (2015, 11 06). Retrieved 06 03, 2016, from Centres for Disease Control and Prevention: http://www.cdc.gov/nchs/washington_group/wg_questions.html SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Appendix A: Summary of Policy Lever Benchmarking Policy goal Policy lever Scorea Weight Benchmark Legal framework 2.01 17% Established Organizational structure and 2.67 17 Established institutionalized processes Enabling environment Human resources 2.19 17 Established Infrastructural capacity 2.64 17 Established Budget 2.26 17 Established Data-driven culture 0.00 15 Latent Data architecture 2.67 20 Established Data coverage 2.40 30 Established System soundness Data analytics 0.44 15 Latent Dynamic system 3.28 15 Advanced Serviceability 2.70 20 Established Methodological soundness 2.21 25 Established Accuracy and reliability 2.68 25 Established Quality data Integrity 1.94 25 Emerging Periodicity and timeliness 2.33 25 Established Openness 0.96 15 Latent Operational use 1.20 50 Emerging Utilization in decision making Accessibility 1.50 20 Emerging Effectiveness in disseminating 1.40 15 Emerging findings a. 0–0.99 = Latent; 1–1.9 = Emerging; 2–2.9 = Established; 3–4 = Advanced. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Appendix B: Extended Rubric, Fiji Scores Highlighted in Red Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced The system contains crucial components of a comprehensive The system enabling The system lacks The system contains contains basic The system contains environment, which major components of most components of components of a crucial components of POLICY AREA 1: ENABLING ENVIRONMENT addresses related a comprehensive a comprehensive comprehensive a comprehensive policy elements and enabling enabling enabling enabling environment enables the environment environment environment functioning of an effective and dynamic system Institutionalization of system: EMIS is institutionalized as an integral part of the education system and the government Responsibility: responsibility for collecting, processing, and disseminating education statistics is given to a clearly designated institution Basic components or agency An existing legal of a legal There is an existing Most elements of a Dynamic framework: the legal framework supports A legal framework is framework or legal framework to 1.1 Legal framework legal framework are framework is dynamic and elastic so a fully functioning not in place informal support a fully in place that it can adapt to advancements in EMIS mechanisms are in functioning EMIS technology place Data supply: the legal framework mandates that schools participate in EMIS by providing education data Comprehensive, quality data: the requirement for comprehensive, quality data is clearly specified in the EMIS legal framework SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Data sharing and coordination: the legal framework allows for adequate data sharing and coordination between the Ministry of Education and agencies and/or institutions that require education data Utilization: the legal framework emphasizes data-driven education policy Budget: the education system budget includes a line item for EMIS Confidentiality: the legal framework guarantees that respondents’ data are confidential and used for the sole purpose of statistics The system is The institutional institutionalized The institutional The system is structure of the within the The system is not structure of the institutionalized system is defined government, has specified in policies, system is not within the Organizational within the well-defined and what exists does clearly specified in government, has well- structure and Organizational structure and government, it has 1.2 organizational not have well-defined policies, it has some defined organizational institutionalized institutionalized processes defined processes, and has organizational organizational processes, and has processes organizational several processes; EMIS has processes, and its several functionalities processes, but its functionalities limited functionalities functionalities are beyond statistical functionalities are beyond statistical limited reporting limited reporting Qualified staff Minimum standards Some staff are The majority of staff All staff are qualified operate the system, of qualification are qualified to operate are qualified to to operate the system, Personnel: the core tasks of EMIS are and opportunities not met for the the system, and operate the system, and well-established 1.3 Human resources identified and EMIS is staffed with are available to majority of staff that limited and frequent opportunities are qualified people improve their operate the system opportunities are opportunities are constantly available to performance and and opportunities are available to available to improve improve staff retention not available to improve staff SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced improve their performance and staff performance performance and performance and retention and retention retention Professional development: professional retention training is available for EMIS staff Data collection: tools for data collection are available Database(s): databases exist under the The system has a The system has a well- umbrella of the data warehouse and well-defined The system has an defined infrastructure have both hardware and software infrastructure to infrastructure that to fully perform its means perform data The system lacks a The system has a Infrastructural allows it to perform data collection, 1.4 Data management system: a system is collection, well-defined basic or incomplete capacity some of its functions management, and in place that manages data collection, management, and infrastructure infrastructure in an integral dissemination processing, and reporting dissemination manner functions in an functions in an Data dissemination: data dissemination integral manner integral manner tools are available and maintained by the agency producing education statistics Personnel and professional development: the EMIS budget contains a specific budget for EMIS The system budget personnel and their professional contains the majority The system budget is The system budget is development of required comprehensive, The system suffers The system has a comprehensive, categories to ensure 1.5 Budget Maintenance: the EMIS budget ensuring that the from serious basic or incomplete ensuring that the that most parts of contains a specific budget for system system is sustainable budgetary issues budget system is sustainable the system are maintenance and recurrent costs and efficient and efficient sustainable and efficient Reporting: the EMIS budget contains a specific budget for reporting costs SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Physical infrastructure: the EMIS budget contains a specific budget for physical infrastructure costs Efficient use of resources: processes and procedures are in place to ensure that resources are used efficiently A data-driven culture A data-driven culture exists that prioritizes The system has a prioritizes data as a The system suffers data management and data-driven culture A data-driven culture fundamental because there is not a utilization within and that demonstrates exists that prioritizes element of data-driven culture beyond the education Data-driven a basic appreciation data management Data-driven culture operations and that prioritizes data system, and evidence Culture of data and interest and utilization within decision making, management and of that culture is in developing and beyond the both inside and data utilization in present in daily better data education system outside of the decision making interaction and utilization practices education system decision making at all levels The system has The system has The processes and The processes and basic processes some processes and structure of EMIS structure of the The system lacks and a structure a structure, but they are sound and system are sound and POLICY AREA 2: SYSTEM SOUNDNESS processes and that do not support do not fully support support the support the structure the components of the components of components of an components of an an integrated an integrated integrated system integrated system system system The system's data The system's data structure has most The data The system's data architecture elements of the data The data architecture architecture is well structure does not includes some architecture; is well defined to 2.1 Data architecture Data architecture defined to ensure have a well-defined components; however, it has some ensure full system full system data architecture however, it is deficiencies that functionality functionality incomplete affect the system's functionality SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Administrative data: EMIS contains administrative data The data in the system are The data in the Financial data: EMIS contains financial comprehensive and The data in the The data in the The data in the system system are far from data cover administrative, system include system include most are comprehensive 2.2 Data coverage being comprehensive, financial, human some of the data but not all of the and cover all data Human resources data: EMIS contains and coverage is resources, and areas data areas areas human resources data limited learning outcomes Learning outcomes data: EMIS contains data learning outcomes data Basic tools and processes are Tools and processes available, but the Tools and processes Tools and processes are available to system is not Tools and processes are available; are available to perform data capable of 2.3 Data analytics Data analytics are used to perform however, data perform data analytics analytics at different conducting limited tabulations analytics are not at different levels on a levels on a regular advanced analytical performed regularly regular basis basis steps (e.g., predictive models, projections) Quality assurance measures: the system is dynamic and maintains The system in place The system in place is quality assurance measures The system in place is not easily The system in place is not easily adaptable is elastic and easily adaptable and The system in place elastic and easily Data requirements and considerations: to changes adaptable to allow requires significant is easily adaptable, adaptable to allow for 2.4 Dynamic system mechanisms exist for addressing new /advancements in for changes time and resources but it remains changes/ and emerging data requirements data needs, as no /advancements in to accommodate reasonably complex advancements in data System adaptability: EMIS is elastic and quality assurance data needs changes and/or needs easily adaptable to allow for changes standards are used advancements and/or advancements in data needs SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Validity across data sources: information brought together from different data and/or statistical frameworks in EMIS is placed within Services provided by Services provided by the data warehouse using structural the system are valid the system are valid and consistency measures across data sources, across data sources, integrate integrate Integration of noneducation databases noneducation noneducation into EMIS: data from sources collected databases into EMIS, Serious issues exist Inconsistencies The data are databases into EMIS, by agencies outside EMIS are and archive data at related to data exist related to consistent and valid; and archive data at 2.5 Serviceability integrated into the EMIS data the service of EMIS validity and data validity and however, some the service of EMIS warehouse clients by ensuring consistency consistency concerns still exist clients by ensuring the Archiving data: multiple years of data the relevance, relevance, are archived, including source data, consistency, consistency, metadata, and statistical results usefulness, and usefulness, and Services to EMIS clients: services timeliness of its timeliness of its provided by the system to EMIS clients statistics statistics include ensuring the relevance, consistency, usefulness, and timeliness of its statistics The system has most mechanisms in place The system has the needed to collect, The system has the mechanisms The system has save, and produce required mechanisms required to collect, basic mechanisms timely, high-quality in place to collect, The system lacks save, produce, and to collect, save, information for use save, produce, and mechanisms to utilize information, and produce in decision making; utilize information, collect, save, or POLICY AREA 3: QUALITY DATA which ensures timely, quality however, some which ensures produce timely, high- accuracy, security, information; additional measures accuracy, security, quality information and timely, high- however, its are needed to and timely, high- for decision making quality information accuracy might be ensure accuracy, quality information for use in decision questionable security, and/ or for use in decision making timely information making that can be used for decision making SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Concepts and definitions: data fields, records, concepts, indicators, and metadata are defined and documented in official operations manuals along with other national datasets and The methodological endorsed by the government The methodological The methodological The methodological basis for producing The methodological basis for producing basis for producing Classification: defined education basis for producing educational basis for producing educational statistics educational statistics system classifications are based on educational statistics statistics follows educational statistics from raw data follows most Methodological technical guidelines and manuals does not follow the basics of from raw data follows 3.1 follows required soundness Scope: the scope of education statistics internationally internationally internationally internationally internationally is broader than and not limited to a accepted standards, accepted accepted standards, accepted standards, accepted standards, small number of indicators (e.g., guidelines, or good standards, guidelines, and good guidelines, and good guidelines, and good measurements of enrollment, class practices guidelines, and practices practices practices size, and completion) good practices Basis for recording: data-recording systems follow internationally accepted standards, guidelines, and good practices Source data: available source data provide an adequate basis for compiling statistics Source data and Source data and Source data and statistical statistical techniques Source data and Validation of source data: source data statistical techniques Source data and techniques have follow most required statistical techniques are consistent with the definition, are sound and Accuracy and statistical techniques basic soundness elements to be are sound and 3.2 scope, and classification as well as time reliable, and reliability lack soundness and and reliability, but sound and reliable, reliable, and statistical of recording, reference periods, and statistical outputs reliability statistical outputs but statistical outputs sufficiently valuation of education statistics sufficiently portray do not portray outputs do not portray reality Statistical techniques: statistical reality reality portray reality techniques are used to calculate accurate rates and derived indicators SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Professionalism: EMIS staff exercise their profession with technical Education statistics Education statistics independence and without outside contained within Education statistics contained within the interference that could result in the the system are contained within the Education statistics system are mostly violation of the public trust in EMIS Education statistics guided by limited system are guided by contained within the guided by principles statistics and EMIS itself contained within the principles of all three principles of 3.3 Integrity system are not of integrity (two of system are guided by integrity (one of the integrity: Transparency: statistical policies and guided by principles the three principles principles of integrity three principles of professionalism, practices are transparent of integrity of professionalism, professionalism, transparency, and Ethical standards: policies and practices transparency, and transparency, and ethical standards in education statistics are guided by ethical standards) ethical standards) ethical standards Periodicity: the production of reports and other outputs from the data The system produces The system The system produces The system produces The system produces warehouse occur in accordance with data and statistics produces some most data and Periodicity and data and statistics all data and statistics 3.4 cycles in the education system neither periodically data and statistics statistics periodically timeliness periodically in a periodically and in a Timeliness: final statistics and financial nor in a timely periodically and in a and in a timely timely manner timely manner statistics are both disseminated in a manner timely manner manner timely manner The system is wholly There are no signs The system is used The system is used The system is wholly utilized by different that EMIS is utilized by some education by most education utilized by different users for decision in decision making by stakeholders, but stakeholders but is users for decision POLICY AREA 4: UTILIZATION FOR DECISION MAKING making at different the majority of not for major not fully operational making at different levels of the education policy decision in governmental levels of the education system stakeholders making decision making education system EMIS stakeholders: EMIS primary The system is open The system lacks The system is open The system is open The system is open to stakeholders are identified and use the to education openness to to some education to the majority of all education system in accordance with the legal stakeholders in education stakeholders in education stakeholders in terms 4.1 Openness framework terms of their stakeholders in terms terms of their stakeholders in of their awareness User awareness: current and potential awareness and of their awareness awareness and terms of their and capacity to utilize EMIS users are aware of EMIS and its capacity to utilize and capacity to utilize capacity to utilize awareness and the system outputs the system the system the system SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced User capacity: EMIS users have the capacity to utilize the skills to interpret, manipulate, and system utilize the data produced by the system to ultimately disseminate findings Utilization in evaluation: data produced by EMIS are used to assess the education system Utilization in governance: data produced by EMIS are used for Data produced by governance purposes the system are Data produced by Data produced by Data produced by the Data produced by the Utilization by schools: data produced used in practice by the system are used the system are used system are not used system are used in by EMIS are used by schools some education in practice by the 4.2 Operational use in practice by the in practice by practice by the main stakeholders majority of Utilization by clients: data produced by main education education education education EMIS are used by clients (including stakeholders stakeholders stakeholders stakeholders parents, communities, and other actors) Utilization by government: the system is able to produce summative indicators (derived variables) to monitor education system Understandable data: data are presented in an easily digestible manner Education statistics Education statistics Widely disseminated data: education are presented in an are presented in an statistics are disseminated beyond the understandable understandable Ministry of Education and/or the manner and are The system suffers The system has The system has manner and are education statistics–producing agency 4.3 Accessibility widely disseminated from serious major accessibility minor accessibility widely disseminated to other EMIS stakeholders using clear platforms accessibility issues issues issues using a clear platform Platforms for utilization: platforms are for utilization, for utilization, standardized across EMIS and are complemented by complemented by customizable to user needs user support user support User support: assistance is provided to EMIS users upon request to help them access the data SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Scoring Description of Policy levers Indicators best practices Latent Emerging Established Advanced Dissemination strategy: national A dissemination plan governments have an information has been dissemination strategy in place implemented; Dissemination of Dissemination is The dissemination of Effectiveness in Dissemination is however, room education statistics reasonably education statistics via 4.4 disseminating neither strategic nor exists for Dissemination effectiveness: via EMIS is strategic strategic, but EMIS is strategic and findings effective improvement (for dissemination of EMIS statistics is and effective ineffective effective full effectiveness in effective relation to strategic engagement) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 Appendix C: Structure of the Ministry of Education (2014) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42 FIJI ǀ EMIS SABER COUNTRY REPORT |2017 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative collects data on the policies and institutions of education systems around the world and benchmarks them against practices associated with student learning. SABER aims to give all parties with a stake in educational results—from students, administrators, teachers, and parents to policy makers and business people—an accessible, detailed, objective snapshot of how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of Education Management Information Systems. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 43