Report No. 24337 CRG Monitoring Educational Performance in the Caribbean April 30, 2003 Human Development Department Caribbean Country Management Unit Latin America and the Caribbean Region Document of the World Bank CURRENCY EQUIVALENTS (Exchange Rate Effective: May 2002) Currency Unit=Belize Dollar, BZ$ 1.0 = US$0.5, BZ$2.0 = US$1.0 Currency Unit=Dominican Peso, DO Pesol.0 = US$0.055, DO Peso 17.86 = US$1.0 Currency Unit = Eastem Caribbean Dollar, ECD$1.0 = US$0.37, ECD$2.7 = US$1.0 Currency Unit=Guyana Dollar, GY$1.0 = US$0.005, GY$190.5 = US$1.0 Currency Unit=Jamaican Dollar, JM$ 1.0 = US$0.02, JM$48.2 = US$1.0 Currency Unit=Trinidad and Tobago Dollar, TT$1.0 = 0.16US$, TT$6.15 = US$1.0 FISCAL YEARS Guyana, Dominican Republic, Grenada, St Kitts and Nevis, St Vincent and the Grenadines: January I - December 31; Belize, Jamaica, St Lucia: April I - March 31; Dominica: July I - June 30 Trinidad and Tobago: October 1 - September 30 ABREVIATIONS AND ACRONYMS CE: Coefficient of Efficiency CEE: Common Entrance Exam CREMIS: Caribbean Regional Education Management of Information System CSEC: Caribbean Secondary Education Certificate CXC: Caribbean Examination Council EFA: Education For All EMIS: Education Management Systems GCR: Gross Completion Rate GER: Gross Enrollment Rate GIR: Gross Intake Rate IEA: Intemational Association for the Evaluation of Educational Achievement LLECE: Latin American Laboratory for Assessment of the Quality of Education NER: Net Enrollment Rate OECS: Organization of the Eastern Caribbean Countries OERU: OECS Education Reform Unit PISA: Program for Intemational Student Assessment PRIE: Regional Education Indicators Project TIMSS: Third Intemational Mathematics and Science Study UIS: UNESCO Institute for Statistics UNPD: United Nations Population Division YIPG: Years-Input Per Graduate Vice President: David De Ferranti Country Director: Orsalia Kalantzopoulos Country Sector Leader: William Experton Sector Director: Ana-Maria Arriagada Sector Manager: Marito Garcia Task Team Leader: Emanuela di Gropello TABLE OF CONTENTS Acknowledgements vi Preface vii Executive Summary viii PART I: MAIN REPORT 1 Section I: Introduction I Section II: Defining Operationally Relevant Education Indicators 2 1I.1 Measurement of Educational Coverage and Internal Efficiency 3 A. Aggregated Indicators 3 B. Grade-Specific Indicators based on Transversal Data 4 C. Grade-Specific Indicators based on Longitudinal (or Pseudo-Longitudinal) Data: Cohort Analysis 6 D. Cost-Effectiveness Indicators 10 11.2 Measurement of Quality 11 Section m: Monitoring Education Indicators in the Caribbean Region 15 II. 1 Current Availability of Indicators for Education Diagnostics in the Caribbean 15 A. Availability at the National Level 16 B. Availability at the Regional Level - 17 C. Summary of Current Available Indicators 18 11.2 The New Indicator Database on Ten Caribbean Countries: Description and Analysis 25 A. Syntheses of the Education Sectors by Country 26 B. Comparative Analysis of the Ten Caribbean Countries 32 Section IV: Conclusions and Policy Recommendations 45 PART II: ANNEXES 49 Annex 1: Education Indicators: Definition and Interpretation 51 Monitoring Educational Performance in the Caribbean Annex 2: Country Education Profiles 65 Belize, 2000/2001 65 Dominica, 1999/2000 68 Dominican Republic, 1999/2000 71 Grenada, 2000/2001 74 Guyana, 1999/2000 77 Jamaica, 1999/2000 80 St Kitts and Nevis, 2000/2001 83 St Vincent and the Grenadines,2000/2001 86 Trinidad and Tobago, 1999/2000 89 Annex 3: Comparative Tables on Educational Performance 93 Annex 4: Measuring Education Quality 102 Bibliography 109 Tables Table 1. Characterization of Coverage and Efficiency Education Indicators 5 Table 2. Typology of Cohort Analyses 7 Table 3. Quality Indicators 14 Table 4. National Assessments at the Primary level 16 Table 5. Education Indicators Reported in 10 Caribbean Countries 21 Table A.1. Coverage of School-age Population in Primary Education 93 Table A.2. Coverage of School-age Population in Secondary Education 94 Table A.3. Student Cohort Survival Rates 95 Table A.4. Other Internal Efficiency Indicators - Primary Education 96 Table A.5. Other Internal Efficiency Indicators - Secondary Education 97 Table A.6. Quality: Acquisition of Cognitive Skills in Primary Education 98 Table A.7. Quality: Acquisition of Cognitive Skills in Secondary Education 99 Table A.8. Other Quality Indicators 100 Table A.9. Public Expenditure Indicators in Education 101 Charts Chart 1. Schooling Profile 8 Chart 2. Survival Rates 10 Chart 3. GERs and GCRs in Primary Education for 10 Caribbean Countries 35 Chart 4. GERs and GCRs in Secondary Education for 10 Caribbean Countries 35 Chart 5. Survival Rates for 10 Caribbean Countries 36 Chart 6. Proportion of Out-Of-School Secondary School age Children in 9 Caribbean Countries 36 Chart 7. Repetition Rates in Primary and Secondary for 10 Caribbean Countries 37 Chart 8. Completion Rates in 10 Caribbean Countries 37 iv Table of Contents Chart 9. Spending per Student in Secondary and GNP per capita in 9 Caribbean Countries 40 Chart 10. Public expenditure on Education and CXC Performance in 9 Caribbean Countries 40 Chart 11. Public expenditure on Education and Completion Rate in 10 Caribbean Countries 41 Chart 12. Unit Costs and Pupil-Teacher Ratios in Secondary Education for 9 Caribbean Countries 41 Chart 13. CXC Performance in 9 Caribbean Countries 43 Chart 14. Completion Rate and English CXC Performance in 10 Caribbean Countries 43 Chart 15. Completion Rate and math CXC Performance in 10 Caribbean Countries 44 Chart 16. CXC Performance and class hours in 8 Caribbean Countries 44 Boxes Box 1. The Caribbean Examination Council (CXC): purpose, advantages and disadvantages 19 Box 2. Education in St. Lucia, 2000/2001 27 ACKNOWLEDGME NTS This report was prepared by Emanuela di Gropello under the general supervision of William Experton (Sector Leader, LCSHD). Lianqin Wang (Education Specialist, HDNED) provided invaluable assistance in the design of the database and its preparation for inclusion in the EDStats Website. Natasha Lopez provided effective assistance in puffing together the data- base. The peer reviewer was Alain Mingat (Lead Education Specialist, AFrHD). Aracelly Woodall was the administrative assistant, Karalee Rocker helped with the editing of the report and Annie Go Dizon with the final formatting. Special thanks go to the Ministries of Education of the ten Caribbean countries for their precious help in providing the information necessary to complete the report. The views ex- pressed in the report are solely those of the World Bank but benefited from the comments of the Ministries. Special thanks also go to the UNESCO Institute of Statistics (UIS) for shar- ing data and giving me the opportunity of presenting the recommendations of this report in its regional workshop. Thanks also go to the Caribbean UNESCO office (UNESCO Kingston) and the Latin Ameri- can and the Caribbean UNESCO office (UNESCO Santiago) for sharing information on UNESCO's activities in the region. Finally, I am also grateful to the same UNESCO Kingston and to Mark Ernest (OERU) for involving the Bank in the debate on qualitative education indicators in the Caribbean and to Werner Meier for writing the paper: "Monitoring School Effectiveness: Selecting Appropriate Qualitative Indicators", which was effectively presented in the UIS regional workshop. In the Bank, other than those cited above, thanks are also due to the members of the Carib- bean HD team - Alberto Rodriguez, Kin Bing Wu, Cynthia Hobbs, Angela Demas and Joshua Gallu who provided support in data collection, advice on the country education sys- tems and useful comments prior and/or during the Decision Meeting. vi PREFACE This study represents a first attempt of providing a comprehensive quantification of educa- tional outcomes in the Caribbean region. It is the first step of a process aimed at improving the benchmarking of educational outcomes through the development and monitoring of op- erationally relevant comparable education indicators. This is an increasing priority in the context of the Bank's education projects in the Caribbean and within the current Education For All (EFA) and Millenium Development Goals (MDGs) initiatives, which all require the identification and monitoring of relevant indicators to establish where the countries stand in relation to specific targets and the progress that they are making towards these targets. With the objective of participating actively in the debate on education indicators in the re- gion, the main recommendations of this report (together with the results of a specially com- missioned paper on the selection of qualitative indicators) were presented at the UNESCO Institute for Statistics (UIS) Caribbean Regional Workshop on Education Indicators held in Trinidad and Tobago on May 5-9, 2002 which involved about 58 education planners and statisticians from 22 Caribbean countries. This study also gave rise to the development of a special database on Caribbean Education Indicators included in the EDStats website. Part II of the report includes the country profiles and comparative tables part of this database. vii EXECUTIVE SUMMARY Section I: Introduction Caribbean countries share special development challenges because of their small size, high vulnerability to external events and limited natural resources. Most countries have made significant public investment in education over the last decades, averaging 4-5% of GDP, and considerable progress has been made in making access to primary educa- tion universal. However, despite this progress, theie are still several deficiencies that need to be addressed related to insufficient coverage and inequitable access in second- ary and post-secondary education as well as the overall low quality and internal effi- ciency of the education system. To be capable of effectively undertaking this global effort and to take specific policy actions, it is imperative to have precise knowledge of the deficiencies in question. Despite recent efforts and initiatives, the region's foremost shortfall remains the availability of comparable indicators that can be effectively ased for policy-relevant diagnostics of the education sectors. The main objectives of this study are: (a) to define a set of operationally relevant education indicators, to be used by countries in the Caribbean region; (b) to provide a database of comparable education indicators in Caribbean countries where data is available, namely Belize, Dominica, Dominican Republic, Grenada, Guyana, Jamaica, St Kitts and Nevis, St Lucia, St Vincent and the Grenadines and Trinidad and Tobago and (c) to propose methods on how the common set of indicators can be used for analyses of the education sectors. Section II: Defining Operationally Relevant Education Indicators The first step in undertaking operational diagnostics of the education sectors is to determine a set of operationally relevant outcomes that can be observed and monitored. Once these outcomes have been determined, it will then be possible to relate them to underlying causes, in order to identify the policy related factors that can be modified. I&Such a benchmarking of education outcomes needs to be done in at least three main areas: coverage, efficiency and quality, in order to provide a comprehensive picture of the performance of the education systems. This information is essential for designing any intervention in the education sector, including the recent EFA initiatives. In the coverage and efficiency area, traditional aggregated enrollment ratios (such as the Gross Enrollment and the Net Enrollment Rate) should be complemented with grade-spe- cific indicators calculated on the basis of transversal and longitudinal data. Completion and survival rates appear to be particularly relevant in exploring the causes behind the enroll- ment figures. A system with low coverage associated with low access to primary and/or secondary will not require the same type of policy interxention that a system with low cover- age associated with high drop-outs of the entering cohort during the schooling cycle: sur- vival rates, derived from a cohort analysis, are particularly useful in casting light on this viii Executive Summary issue as well as on the evolution of the coverage and internal efficiency of the education system when combined with the current intake rate. The efficiency analysis should be complemented with a cost-effectiveness analysis in order to determine the cost of outcomes and thus establish alternative options for reaching them. Finally, in the quality area, a variety of standardized tests, both nationally and non- nationally determined, should be used to provide some objective measurement of educational achievement. These outcome measures should be complemented by an analysis of the education systems' characteristics, i.e. intermediate quality measures. Pertinent intermediate quality indicators were identified. Monitoring them would make it possible to design and implement prompt and effective quality-enhancing interventions. Section m: Monitoring Education Indicators in the Caribbean Region 111.1 Current Availability of Indicators for Education Diagnostics in the Caribbean While the availability of education statistics and indicators has been traditionally limited in the Caribbean, there has recently been substantial effort, both at the national and regional level, to improve the collection and use of education statistics, which are necessary for mak- ing policy relevant decisions in the education sectors. Many Caribbean countries are in the process of strengthening their education statistics col- lection and processing capacity, some through the establishment of comprehensive Educa- tion Management Systems (EMIS). On quality outcomes, most of the countries have by now developed national systems for measuring, monitoring and assessing learning outcomes. These systems are centered on the application of standardized tests to provide inputs for policy and decision-making. At the regional level, the major efforts in promoting the collection of education statistics and the construction of indicators have been led by UNESCO under the Education For All (EFA) initiative and the activities of the recently strengthened UNESCO Institute for Statistics (UIS). On quality outcomes, the activities of the Caribbean Examination Council (CXC), gave rise to a Secondary School Certificate which provides regionally comparable information on student's performance. In the broader Latin American framework, there is also the PRIE (Regional Education Indicators Project) initiative, led by UNESCO/OREALC (Santiago). Despite the on-going effort, the quantity and quality of the indicators available still differs markedly across countries and, while there is good coverage of some indicators, some others are generally not provided. Less indicators are generally available at the secondary than at the primary level. Additionally, poor coverage is particularly noticeable in the areas of quality and, to a lesser extent, efficiency. Finally, grade-specific indicators such as the completion and survival rates are generally not provided. ix Monitoring Educational Performance in the Caribbean M.2 The New Indicator Database on Ten Caribbean C'ountries: Description and Analysis A common set of indicators, measuring coverage, efficiency and quality, was constructed for ten Caribbean countries currently covered by on-going Bank's operations. In putting to- gether this set of indicators, we had two main priorities: (a) create a set of updated indicators useful for establishing thorough and balanced diagnostics of the education sectors, relevant to policy decisions; and (b) ensure comparability by applying the same set of indicators to all countries, using the same methodology. More traditional indicators of coverage and effi- ciency were complemented with indicators that are not generally provided at the national or regional level in the Caribbean and specially designed templates were used to produce cov- erage and efficiency indicators comparable across countries. Two main outputs were produced: (a) country syntheses along the three main areas of cover- age, efficiency and quality and (b) cross-country comparisons of each of the indicators. Country Syntheses: Taking the country of St Lucia as an example, the report shows how the indicators reported in the country profiles (which include, among others, indicators on ex- penditure, enrollment and completion rates, survival rates and test scores) can be combined and analyzed to provide a useful and operationally relevant synthesis of the education sec- tors. Cross-Country Comparison: The report also shows how the comparative database can be used to highlight common characteristics and issues of the ten Caribbean countries involved, after discussing some comparability limitations which arise from differences in the education systems (both in terms of the length of the normal schooling cycles and in terms of the educational options after the end of the normal primary cycle) and the different coverage of some of the received information. The comparison of the ten Caribbean countries revealed several significant and useful points on the state of education in the region, as summarized below: Primary level Coverage: Coverage at the primary level does not seem to be an issue anymore and completion is not generally a problem either, with the exception of three countries. Secondary Level Coverage: In contrast to the primary level, coverage and completion are an issue in the secondary sub-cycle for most of the Caribbean countries included in the report, to various degrees. For some it is a question of access, for others the number of drop-outs during the secondary cycle is the most serious problem. There is still a significant proportion of secondary school age children out of school in half of the ten countries under analysis. Internal Efficiency: For most countries, the internal efficiency is low as illustrated by the low survival rates to the last grade of the secondary cycle and relatively high repetition rates at the secondary level. On average, less than one child out of two entering first grade of primary education today is expected to make it through to the last grade of secondary. Cost-Effectiveness: Almost all countries spend substantially on education, as indicated by a x Executive Summary public education expenditure-GDP ratio of 6% or more. The relationship between educational outcomes and public expenditure in education, while generally positive, is by no means tight. Wide variations exist in spending efficiency across countries, as shown by the fact that similar levels of spending are associated with a wide range of educational outcomes Educational Achievement: Performance on the CXC, which is used as the secondary cycle final exam in nine out of the ten countries included in the report, varies significantly across countries, with passing rates consistently lower in math than in English. The results are worse for the proportion of students passing at least 5 CXC, including English and math, the minimum requirement for access to tertiary education. Section IV: Conclusions and Policy Recommendations Several recommendations as to how to further build on the existing work of Ministries of Education and international organizations in the collection of information and the construction and use of indicators for education diagnostics in the Caribbean can be made and are summarized below: * Caribbean countries should measure the coverage and efficiency of the en- tire education cycle (primary plus secondary). The current quantification ef- fort is stronger at the primary level than at the secondary level, while the main education issues reside in the coverage of the secondary level and the efficiency of the overall education cycle. * Caribbean countries should start producing and using grade-specific indica- tors such as the completion and the survival rates. The report has shown the usefulness of these indicators in a coverage/efficiency diagnostic of the Carib- bean education systems. * Harmonization of education statistics and indicators should continue. The variation in the quality and quantity of indicators available per country makes it difficult to undertake thorough cross-country comparisons of the coverage/effi- ciency of the education systems. * Caribbean countries should improve the collection and use of expenditure data, by reporting and monitoring in a systematic way unit costs per education level and expenditure allocation ratios. The lack of reliable expenditure data pre- vents thorough cost-effectiveness analyses, while the existence of wide varia- tions in spending efficiency across the countries makes these analyses essential. * On the quality side, Caribbean countries should participate in international exams (such as the TIMSS, the PISA and the LLECE). The lack of directly comparable exams at the primary level makes it difficult to make any meaningful comparison of learning outcomes at that level across countries and to provide xi Monitoring Educational Performance in the Caribbean some objective benchmarking on the quality of primary education of each coun- try. At the secondary level, even if the existence of the CXC provides regional comparison of outcomes, participating in other international exams would still provide some international benchmarking of the regional performance. • The proportion of students passing at least. 5 CXC, including English and math, should start to be measured and monitored since this is the minimum requirement for access to tertiary education and performance is particularly un- satisfactory when measured using this indicator. * Finally, Caribbean countries should start producing and monitoring quali- tative indicators, such as school level measures of adequacy of financial, hu- man and physical resources and classroom level measures of quality of instruc- tional methods and techniques. Little information is available on possible deter- minants of the educational achievement performance in the region hampering the design and implementation of quality-enhancing interventions which would be essential given the generally low educational achievement performance of the region. xii SECTION 1: INTRODUCTION Caribbean countries share special development challenges because of their small size, high vulnerability to external events and limited natural resources. Most countries have made significant public investment in education over the last decades, averaging 4-5% of GDP, and considerable progress has been made in making access to primary education universal. However, despite this progress, there are still several deficiencies that need to be addressed. These are largely related to: (a) insufficient coverage and inequitable access in secondary and post-secondary education; (b) low quality of primary and secondary education; and (c) low internal efficiency of the education system. Addressing these deficiencies is at the core of current Caribbean education strategies. In particular, the strategy recommendations included in the Caribbean Education Strategy Re- port (endorsed by the Ministries of Education of the region in October 2000) include: (a) narrowing the knowledge gap; (b) making the school the center of focus of the education system; (c) reducing inequities in the school system; (d) strengthening regional collabora- tion; and (e) enhancing educational financing and management. To be capable of effectively undertaking this global effort and taking specific policy actions, it is, however, imperative to have precise knowledge of the deficiencies in question. Despite recent efforts and initiatives, the region's foremost shortfall remains the availability of com- parable indicators that can be effectively used for policy-relevant diagnostics of the educa- tion sectors. It is in this context that it was decided to begin quantifying educational outcomes at the primary and secondary levels' in the Caribbean region using a set of operationally relevant comparable indicators. This initiative gave rise to this policy note which cannot quite be considered a full piece of sector work but is the first step of a process aimed at improving the benchmarking of educational outcomes in the Caribbean through the development and moni- toring of indicators. This work is all the more important within the framework of the current Education For All (EFA) and Millenium Development Goals (MDGs) initiatives which re- quire the identification and monitoring of relevant indicators to establish where the countries stand in relation to the target of universal primary education. The main objectives of this study are: (a) to define a set of operationally relevant education indicators, to be used by countries in the Caribbean region; (b) to provide a database of comparable education indicators in Caribbean countries where data is available, namely Belize, Dominica, Dominican Republic, Grenada, Guyana, Jamaica, St Kitts and Nevis, St Lucia, I Tertiary education will be treated in a new upcoming sector work. Monitoring Educational Performance in the Caribbean St Vincent and the Grenadines and Trinidad and Tobago and (c) to propose methods on how the common set of indicators can be used for analyses of the education sectors. One of the benefits of this report, from the Bank's perspective, is its contribution to the current debate on establishing operationally relevant indicators and diagnostics in the educa- tion sector2. Another benefit is the benchmarking of educational outcomes of the Caribbean countries covered by current Bank's operations, thus helping the preparation of interven- tions, at the country and regional levels. The database will also provide internationally com- parable indicators available for general consultation within the Bank through its inclusion in the EdStats Website. From the Caribbean's perspective, it is expected that the database, together with the recom- mendations on the selection, measurement and monitoring of policy-relevant education indi- cators, will help the Caribbean countries improve their diagnostics of the education sectors and facilitate policy interventions. Additionally, by providing regionally comparable indica- tors, the database will enable each country to situate its performance relative to other coun- tries in the region and, hopefully, foster regional initiatives and strategies in the education area. The report is divided into four main sections and four annexes. Section II reviews educa- tional indicators in coverage, efficiency and quality relevant to operational diagnostics of the education sectors. Building on these methodologies and indicators, Section m reviews the Caribbean status in producing education indicators for policy-relevant diagnostics and dem- onstrates how a new database can be used to depict where the education sectors stand. The last section presents a set of conclusions and policy recommendations, followed by several annexes. Annex 1 defines and outlines the formulas for many education indicators. Annex 2 presents education profiles of the selected countries and Annex 3 comparative tables on educational performance. Finally, Annex 4 presents the summary of a paper commissioned through this sector work on the selection of appropriate qualitative indicators. SECTION II: DEFINING OPERATIONALLY RELEVANT EDUCATION INDICATORS The first step in undertaking operational diagnostics of the education sectors is to determine a set of operationally relevant outcomes that can be observed and monitored. Once these outcomes have been determined, it will then be possible to relate them to underlying causes, in order to identify the policy related factors that can be modified. Three main dimensions of educational performance should at least be part of each education sector diagnostic: (a) educational coverage, measured by enrollment and completion rates; 2 Debate all the more lively now with the new Education For All initiative launched by the Education Network. 2 Part I: Main Report (b) internal efficiency, measured by student cohort survival rates and other indicators and (c) educational quality, measured by the acquisition of cognitive skills and a variety of intermediate quality indicators. Two other important dimensions are equity and external efficiency. Equity of the education system measures the distribution among socio-economic groups, gender, etc, of educational coverage and quality and external efficiency measures the link between education and labor market outcomes. This report focuses on the three first dimensions of a comprehensive education diagnostic. 11.1 Measurement of Educational Coverage and Internal Efficiency3 A. Aggregated Indicators There are several ways of measuring educational coverage and participation distinguished by their level of aggregation. Traditional aggregated measures of educational coverage include Gross and Net Enrollment Rates (GER and NER) and School-Life Expectancy. The two first indicators compare the amount of children enrolled in a specific education cycle with the school-age population corresponding to the same education cycle (see Annex 1 for the exact formulas, data requirements and limitations). All children enrolled, whatever their age, can be included (GER), or only the children of the official age group for that cycle (NER). The GER is widely used to show the general level of participation in a given level of education. Its comparison with the NER indicates the extent of over-aged and under-aged enrollment. A GER higher than 100% indicates that a country could accommodate all of its expected school-age group but might not be doing so (if the NER is lower than 100%) because of a large proportion of late entrants and/or grade repetition which freeze places for pupils from this expected group. A comparison between the GER and the NER can, in this sense, be quite useful but a more rigorous analysis of internal efficiency of the system through the construction of more elaborated indicators based on cohort analysis (see below) is necessary to detect the precise causes of this gap. Additionally, the NER is often difficult to interpret', suggesting that it might be more convenient to stick to the GER. On the basis of age-specific enrollment and population data, it is also possible to calculate the proportion of primary and secondary school age out-of-school children (called the Out- Of-School Population indicator), by comparing the enrolled population of a certain age group with the same population age group. This complementary indicator is useful in capturing the proportion of a certain school-age group not in school, whatever the level of education. A NER lower than 100% would provide a measure of a certain age group not enrolled at the I To be read with Annex 1 which reports the definition and interpretation of the main education indicators presented in the section. I For instance, the NER presents the disadvantage that the target of 100% NER in primary or secondary school is an unrealistic goal because it would require that every single child enters primary school at exactly the official schooling age, proceeds with the cycle with zero repetition and stays in school with no disruptions resulting in a 100% on-time completion rate. This is a very unrealistic target, in particular in developing countries where exist strong constraints to on-time enrollment. 3 Monitoring Educational Performance in the Caribbean specified level of education, but some of the children of this age group might in fact be enrolled at another level of education, making this measure not truly indicative of the percentage of students not enrolled. The Out-Of-School Population rate makes it possible to capture this situation similar to how age-specific enrollment ratios would capture it for each age group. These types of ratios are also used for calculating School-Life Expectancy, which measures the number of years of education that a child can expect to achieve. This indicator is obtained by summing up the age-specific enrollment ratios for a reference age-range. This first set of aggregated indicators (GER, NER and Out-Of-School Population) provides a quick snapshot of the coverage situation, but fails, however, to provide enough precise infornation for an accurate diagnostic. For instance, the use of only GER or NER rates by schooling cycle can be misleading because they refer to averages across all grades in a schooling cycle, obscuring the nature of the problem. In other words, two countries could have similar net enrollment rates in secondary education, but two completely different situations: in one country, this NER could be the product of low entry rates in the secondary cycle combined with high survival rates to the end of the cycle, while in the other country, the same NER could be the product of high entry rates in the same cycle combined with low survival rates to the end of it. The different causes of the situations would require different policy interventions, which would be undetectable if only the enrollment rates were used. It is therefore recommended to complement traditional aggregated measures with grade-specific indicators constructed on the basis of grade (and age) specific data. These data can be either "transversal", i.e. include enrollment figures per grade (and age) gathered in a single year over several student cohorts, or "longitudinal"5, i.e. include enrollment figures per grade (and age) for the same student cohort (see Table 1). B. Grade-Specific Indicators based on Transversal Data Particularly relevant to the diagnostic is a grade-specific profile for the first and last grades of primary education, and the first and last grades of secondary education, since these are crucial points in the education cycle. As illustrated in Table 1, commonly used measures of coverage, constructed on the basis of transversal data, for the first grade of primary include the Gross and Net Intake Rates, which capture the amount of new entrants in proportion to the official population of the theoretical age. At the end of both the primary and secondary cycle, another useful measurement of educational performance is either the Gross or Net Completion Rate (GCR or NCR), which captures the number of children who successfully complete the primary cycle in proportion to the population of the theoretical completion age for primary or secondary. Finally, for a picture of grade one of secondary, either a Transition Rate6, calculated as the proportion of enrollees of the last grade of primary to those in the first grade of secondary, or a Gross or Net Entry Rate, measured as the number of children who enter the secondary cycle as a proportion of the population with the theoretical age for starting the cycle, would give an accurate account of the schooling situation. 5 Or "pseudo-longitudinal" as explained below. 6 Which, however, requires a longitudinal (or pseudo-longitudinal) data. 4 raru l: ainm eport Table 1: Characterization of Coverage and Efficiency Education Indicators Aggregated indicators (generally based on age-specific transversal data) GER (by education level) NER (by education level) Out-Of-School Population (by primary and/or secondary school-age) School-Life Expectancy Grade-specific indicators (based on grade and age-specific transversal data) Gross or Net Completion Rate (generally for last grade of primary and/or secondary) Gross or Net Intake Rate (generally for first grade of primary and/or secondary) Percentage of Repeaters (per grade and education cycle) Grade-specific indicators (based on grade-specific longitudinal, or pseudo-longitudinal, data) Repetition Rates (per grade and education cycle) Transition Rates (generaly between the primary and secondary cycle) Survival Rates (generally to the last grade of primary and/or secondary education, but the pattern per grade is generally also reported) Aggregated indicators (based on grade-specific longitudinal, or pseudo-longitudinal, data) Years-Input per Graduate (by education cycle or overall) Coefficient of Efficiency (generally by education cycle or overall) A Gross Completion Rate (GCR) (also defined and interpreted in Annex 1) will be by itself a very useful piece of information since it will reveal, in a simple way, the share of all children who complete the education cycle (whatever their age), which tends to be lower than the average enrollment rate due to a higher drop-out rate in the higher grades of the cycle. The GCR is also the indicator adopted by the World Bank for tracking progress of the "Education For All" goals as well as the "Millennium Development Goal" of achieving universal primary completion by the year 20157. The combination of the Gross Enrollment Rate (GER) and the Gross Completion Rate (GCR) will moreover be more indicative than the GER alone. A GER higher than a GCR (as it is generally the case) would for instance suggest a situation where the gross enrollment rates decrease through the grades (or, at least, in the final grade) due to an increased number of drop-outs, while a similar GER and GCR would suggest a situation where gross enrollment rates are more uniform across the grades. Charts 3 and 4 in Section III present the GERs and GCRs for ten Caribbean countries in primary and secondary8. The combination of the GERs and GCRs, if more indicative than 7See The World Bank/HD Network (2002): "Achieving Education forAll by 2015: Simulation results for 47 Low Income countries" for a description and analysis of the EFA goals. I Where the numerator of the GCRs has been proxied by the number of enrollees in the last grade of primary and secondary education, whatever their age, minus the repeaters in these grades. Monitoring Educational Performance in the Caribbean the either GER or NER alone, are not however always easy to interpret. In particular, the GCR may be lower than the GER because of increasing entry rates into the cycle due to a decrease in the school age population for an equivalen.t amount of places and/or additional places for equivalent populations. The problem is that xransversal grade-specific enrollment rates, and, as a consequence, any aggregated measure of enrollment rates, cover different student cohorts. In a transversal schooling profile, the data is collected over a single year, and, thus compare the graduation rates of one set of students with the entry rates of students several years younger. Given this framework, a comparison between the GCR and the first grade Apparent Intake Rate would provide us with the needed information on drop-out rates only if the entry rate across different student cohorts was stable in time (which is unlikely) and the structure of repetition rates across cohorts was precisely known in order to interpret correctly the Gross Completion Rate. If we assume that trends in entry mirror trends in educalional attainment, a way of assessing the evolution of entry rates and enrollments, and improving our diagnostic of the sector would be simply to have a look at the trends in educational attainment of different population- age groups' (household survey data provides this type of information). We could then compare the evolution in attainment over several age groups with the gap between entry and graduation rates to obtain a proxy on the magnitude of drop-out rates. However, a better way of completing an adequate quantitative diagnostic of the education sector is to undertake a cohort analysis, on the basis of longitudinal or pseudo-longitudinal data, which makes it possible to track down the changes that affect a determinate student cohort in time. Cohort analysis allows an assessment of the internal efficiency of the system as well as the detection of the causes of the enrollment and completion performance. C Grade-Specific Indicators based on Longitudinal (or Pseudo-Longitudinal) Data: Cohort Analysis There are several ways for conducting a cohort analysis. The two main ones are the "true" (or "longitudinal") and "reconstructed" (or "pseudo-longitudinal") cohort methods, summarized in Table 2 below, with their respective advantages and disadvantages. The many data requirements of the true cohort method generally make a reconstructed cohort preferable'0. The second method requires a minimum of data on enrollment by grade for two consecutive years and on repeaters by grade from the first to the second year. Several variants then exist on how to produce the pertinent schooling profiles". Following the methodology applied by 9 This is for instance suggested by the OECD (2000): "Investing in Education: analysis of the 1999 World Education Indicators" when it compares entry and graduation rates for the secondary and tertiary education cycle across several countries. '° Even if both would be relevant to be able to explain both the current and "expected" enrollment and comple- tion performance (as explained below). 11 See, for instance, Mingat, Rakotomalala and Tan (2001): "Rapport d'Etat d'un Systeme Educatif National (RESEN)", which suggests a straightforward way of calculating "retention" rates among schooling years, and, then, in combination with the entry rate, obtain pertinent schooling profiles, on the basis of two years con- secutive enrollment of non-repeaters. Also, see Klein and Ribeiro (1 991) on a revised methodology for assess- ing student flows in cases of high repetition and drop-out rates. 6 Part k Main Report UNESCO from this basic information, three main flow rates can be estimated: promotion, repetition and drop-out. Apart from being indicative in themselves, for analysis of patterns and measurement of internal efficiency, the flow rates can also be used in a reconstructed student cohort flow, and under a set of simplifying assumptions (see Table 2 and Annex 1), to derive the Survival Rates. Survival rates measure the percentage of a cohort of students enrolled in the first year of a given education cycle in a given school-year who are expected to reach each successive grade. They are calculated from an hypothetical starting cohort of 1,000 students, tracked down through the grades. These survival rates help us to describe the internal efficiency of the system and, combined with infornation on the current intake rate, determrine schooling profiles by student cohort which will provide useful insights on the explanation of the enrollment and completion performance, as illustrated in the graph below. The graph below shows a situation where, in the primary cycle, the current GER is 103% and, in the secondary cycle, 75% (with GCRs of respectively 100% and 65%). The added schooling profile (obtained by multiplying the survival rate by the gross intake rate) shows that once we take out the impact of different entry rates by focusing on a single cohort, there is, in fact, in the system, low drop-outs in the primary cycle and very significant ones in the secondary cycle. As a consequence, the student cohort enrolled in the last grade of primary Tab[ e 2. of Cohort Analyses Objectives Data requirements Main advantages/disadvantages 'True" cohort Assesses the progress of Extensive. A good school- Very reliable (if school records are reliable) analysis a given school cohort records system is necessary but provides infornation on a past cohort not (based on through the education to be able to retrace the taking into account any possible recent 'longitudinal" cycle tracking the cohort flow of pupils through the progress data) in time grades in past years "Reconstructed" Assesses the progress of Much less extensive. Data Is simple and provides a measure of the cohort analysis a given school cohort on enrollment by grade in "expected" survival rate of a cohort given the (based on through the education two consecutive years and current internal efficiency parameters of the "pseudo- cycle using a simplified repeaters by grade from the model. longitudinal" methodology which first to the second years are data) makes it possible to sufficient to produce three However, it relies on numerous simplifying produce a reconstructed main flow rates: promotion, assumptions. The most important being: (hypothetical) flow repetition and drop-outs, -in the most basic model, it is assumed that diagram of a cohort which are then used in a there are no additional new entrants in any of reconstructed pupil-cohort the subsequent years during the life-time of flow to derive survival rates the cohort, other than the original cohort of and other indicators of 1000 pupils; internal efficiency. -at any given grade, the same rates of However, it is easy to have repetition, promotion, and drop-out apply, the information regardless of whether a pupil has reached that miss-reported grade directly or after one or more repetitions Additionally, calculations are very sensitive to the two reported years, and thus, also to possible mistakes in reporting the information concerning these two years 7 Monitoring Educatz*a a' Performance in the Canbbearn Chart 1: Schooling Profile 120 _ _ _ _ _ _ _ 100 _ _- o CL 80 __ _ CL -o 4 0 20 0 o1 60 4-5 6 7_ _ e 9 1- 11 12 Grades E- Ir,takc. Rate Survival Rate -GER primary- GER secondlary e GCR pnmary * GCR secondarly education r-epresents 97% of the population of official school-age for the last grade of primary, while the one finally reaching graduation represents ornly 60% of the population of official school-age oF graduation from the secondary cycle. The profile also produces an average GER of I C1I c i n primary and 68 % itn secondary. At this poi,nl, somne care needs to be taken in the interpretation of these results. This exercise would be falIlly i ndicative of what happened over- the past few years and, as such, explain the current enrollment and completion results only if it had been carried out on the basis of the "true" cobort method, which makes it possible to reconstnmct exactly the schooling life of the student currently completing primary or secondary education. The reconstructed cohort method really measures "expected" sur-vival rates of a student cofhort based on the current repetition and drop-out rates, which we then applied to the current intake rate in primary to produce a schooling profile. In this setting, the current enrollment and completion rates would be the same that the average enrollment rates and completion rates produced by the profile only if the entry rates were stable in time, as well as the repetition and drop-out rates (i.e. with the sarne current level of internal efficiency). In the example above, the discrepancy in the secondary co)mpletion rates rnight have two possible explanations. The current entry rate is lower tharJ the- one that applied twelve years ago for the bulk of the students completing now secondary edulcation", and/or the cuiTent repetition and dirop-out rates by grade are higher than the past ones suggesting a decreasing efficiency of the school system. 12 That could happen because past entry rate were particularly high due to a lot ofS late entrants or a lower school age populat-n. 8 Part 1: Main Report In any case, the comparison between the schooling profile and the current enrollment and completion rates is fairly pertinent and useful. In particular, the profile illustrates the implications of the current level of wastage on "expected" completion and enrollment rates'3 which can then be usefully compared with the current completion and enrollment rates. Under the assumption that entry rates have been fairly stable in time, we will get an idea of the evolution of the internal efficiency of the education system and its implications on the future rate of participation. Otherwise, in cases where repetition and drop-outs have been fairly stable across student cohorts, we can measure the implications of the changing entry rate on the future rate of participation. In Section III.2, we illustrate these points measuring and comparing the current and expected completion rates for ten Caribbean countries. This exercise is all the more useful in view of the fact that the completion rate currently suggested by UNESCO as the EFA target is constructed by dividing the number of graduates from the reconstructed flow model by the number of students at the beginning of the cohort and multiplying this number by the gross intake rate'4. The methodology is very similar to the one that we actually applied for measuring the expected completion rates in the Caribbean. A mere analysis of the current internal efficiency of the system simply requires showing the survival rate as illustrated in Chart 2 below (taking the same example than above), possibly presenting it with and without repeaters to illustrate clearly the magnitude of the repetition phenomenon. A more precise diagnostic of the wastage produced by repetition and drop-outs can also be made constructing two well known internal efficiency indicators which measure, respectively, the average number of pupil-years'5 per successful completer taking into account the pupil-years spent at school by both graduates and drop-outs (the so-called: Years-Input Per Graduate (YIPG)) and the relation between these total pupil-years and the ones that would be needed to complete the cycle in a case of no repetition and drop-out (the so-called: Coefficient of Efficiency (CE)) (see Annex 1). Defining wastage by YIPG and CE constitutes a rather crude view as regards the benefits that a pupil may draw from the education received before dropping out and of the advantages a pupil may derive from spending an extra year in the grade. Nevertheless, it is still very important for educational planners and policy-makers to know the pattern and magnitude of these two events. In a perfectly efficient system, the YIPG would be equivalent to the theoretical number of schooling years necessary for completing the education cycle and the 13It is however necessary to keep in mind two sorts of factors that suggest caution in the interpretation of the "expected" completion and enrollment rates: (a) first, as mentioned in Table 2, the "reconstructed cohort model" is based on simplifying assumptions like the fact that, at any grade, the same rates of repetition, promotion and drop-out apply regardless of whether a pupil has entered the grade directly or after one or more repetitions (hypothesis of homogeneous behaviour). This can lead to under-estimated survival rates in case of high repeti- tion and drop-out rates; (b) secondly, as mentioned in Table 2 as well, calculations are very sensitive to the two reported years, and thus, also to possible mistakes in reporting the information concerning these two years. 14 See, UIS (2002): "Measuring Completion". 5 One pupil spending one year in a grade is said to have used one pupil-year. 9 Moreitoring Educational Perfonnance in the Caribbean Chart 2: Survival Rates 120 . 100 = so ~0M 0 40 20 - 0 , 1 2 3 4 5 6 7 8 9 10 11 12 Grdes -SuMval rate wfth repetition -- Survival rate wMout repetiton CE would be equal to one. In any other case, it can be said that the resources have not been used as efficiently as possible. In particular, a CE of, for instance, 0.6, would indicate that the cohort used 1.4 times the input required in a system without repetition and drop-out and, as a consequence, the system is inefficient and participation of the expected population age- group might be lower than desired. The costs of graduating 100% of children through 5 years of schooling with no repetition are much lower than the costs to society of graduating 100% of children through 5 years of schooling with, on average, 3 years of repetition. D. Cost-Effectiveness Indicators In order to complete an analysis of the efficiency of an education system the outcome indi- cators (both of coverage and quality) should be complemented with various education ex- penditure data. Specifically measures of: - education expenditure as a % of GDP; - expenditure per student; - expenditure per graduate; - expenditure allocation ratios. The relationship between expenditure per student and GNP per capita can provide some measure of the relative magnitude of unit costs in proportion to country's wealth, providing 10 Part I: Main Report some indication on the priority given to education and highlighting possible efficiency issues. Only relating expenditure, as a percentage of GDP or per student, to schooling outcomes, will however make it possible to estimate, even if only roughly, the cost-effectiveness of an education system. This will mostly be done by comparing a country with others that spend similar amounts in education and determining where schooling outcomes stand comparatively. If cost-effectiveness appears to be an issue, possible underlying causes of poor performance have to be identified. This will generally require the analysis of at least four aspects: (a) repetition rates and years input per graduate (getting back to the analysis developed above), with, if possible, estimates of the expenditure per graduate which are highly sensitive to the amount of repetition; (b) pupil-teacher ratios, the most important determinant of spending in education, and which are not generally associated with higher educational outcomes; (c) average teacher salaries, as the other most important determinant of expenditure; and (d) allocation of expenditures, such as the relative share of investment and especially non-salary recurrent expenditures (i.e. spending on operation and maintenance, supplies and teaching materials, etc) in total education expenditure, with the assumption that low performance can result from an inadequate input mix. To exemplify this cost-effectiveness analysis, Graphs 9, 10 and 11 of Section III.216 illustrate where the different Caribbean countries stand in terms of units costs per GNP per capita and in terms of the relationship between two types of schooling outcomes (gross completion rates and test scores in secondary education) and public expenditure in education as a percentage of GDP17. A few insights on the causes of inefficiency are also outlined. 11.2 Measurement of Quality A diagnostic would be incomplete without an assessment of the quality of the education system. This is because being enrolled in school does not automatically imply that the necessary skills and knowledge are being provided. There are cases of countries with close to full completion rates in primary or secondary, but low student learning outcomes. Learning outcomes are typically be measured through standardized assessments of student learning implemented at the end of the schooling grades. Most countries now have some form of national standardized assessment given at the end of a schooling cycle which measures literacy and numeracy skills. In cases where these do not exist, results of simple school leaving examinations can be used as proxies, but the probable lack of comparability among schools would complicate a meaningful interpretation of the results. Finally, the participation 36 Following a format similar to the one used by The World Bank/HD Network (2001): "Poverty Reduction Strategy Paper (PRSP) Sourcebook: Education" and Mingat and Jee-Peng Tan (1998): "The Mechanics of Progress in Education". " Cost-effectiveness comparisons within this setting are meaningful only if we assume that the current expen- diture as a % of GDP is consistent with the past proportions spent by the countries. Monitoring Educational Performance in the Caribbean in regional assessments or, even, international assessments, such as the TIMSS (the Third International Mathematics and Science Study implemented by the International Association for the Evaluation of Educational Achievement (1EA)'8) and the PISA (the recently OECD launched Program for International Student Assessment)'9, would also provide a country with some measure of learning outcomes, and allow comparisons with other countries, providing some objective benchmarking of the country's performance. It is, ultimately, advisable to measure learning outcomes through both national and non-national exams. Table 3 below summarizes these outcome indicators. In any case, a thorough diagnostic of the quality of an eduration system requires going beyond outcome indicators like standardized assessments, and examining the characteristics of education systems which tend to be strong determinants of learning. There are at least three main reasons for introducing intermediate quality measures: firstly, outcome measures like exam results might not always be available or may even be controversial20, which makes them more difficult to use than the access/efficiency outcome indicators; secondly, analyzing intermediate quality measures will help to pinpoint the quality issues and design adequate policy interventions; and thirdly, these same intermediate quality measures will also be relevant in completing the diagnostic of the educational coverage/efficiency side since these results may in fact be related to quality factors (e.g. low completion rates and high repetition rates might be caused by poor teaching). Analyzing the determinants of learning outcomes (or school effectiveness) is a difficult task which has been described in a variety of literature approaches and studies in both developed and developing countries. In the last decade, as the work of educational planners has moved from increasing school enrollment to improving quality, a burgeoning literature on school effectiveness developed. The fundamental design of school-effectiveness research centers on the association of hypothetical effectiveness-enhancing conditions with measures of output, usually calculated in terms of student achievement. This makes it possible to detect inputs (such as some measures of teachers' characteristics, expenditure per pupil, availability of instructional materials, etc) and processes (such as educational leadership, teachers and students' attitudes and motivation, pedagogical approaches, length of instructional time, etc) which in theory are significantly related to educational achievement. A careful review of the main existing strands of school effectiveness research (as undertaken by Meier (2002) in a paper specially commnissioned through this sector work and whose Executive Summary is included in Annex 421) suggests that this research can be brought together in an integrated model of school effectiveness which casts light on the main determinants of learning outcomes (incorporating at the same time outcomes indicators, such 18 The test is designed to assess achievement levels in mathematics and science at Grades 4 and 8. 19 The test is designed to measure reading, mathematical and scientific hteracy attained by 15-years-olds. 20 No test instrument is value neutral. Tests might be poorly designed, not measuring adequately the existing skills and knowledge. 21 The paper, entitled: "Monitoring School Effectiveness: selecting appropriate qualitative indicators", is available in full version in the project's files. 12 Part L. Main Report completion and repetition rates). Figure 1 below illustrates a three-tiered model of school effectiveness composed of the context, schools and classroom level factors, which postulates a number of causal relationships between the factors as well as between the factors and outcomes (analyzed with some depth in the same Meier (2002)). Figure 1: School Effectiveness Model Context Level MOE Management * Community *Family Background School Level Inputs Process Outputs * Financial *Educational Leadership - *-Student * Human *Monitoring and Evaluation *achievementl * Material attainment Classroom Level *Teacher Background & Style *Classroom Management & Quality Instruction *Student Characteristics The most pertinent qualitative and quantitative factors identified on the basis of the literature review and this conceptual framework22 are summarized in Table 3 below, categon7ed by ttle level to which they refer (community, school or classroom23) and grouped by clusters (curriculum quality, monitoring and evaluation capacity, etc). It will be difficult in practice to construct and monitor all these indicators,24 but if one or two per cluster at each level is selected and tracked in time, together with the available outcome indicators, much light will be cast on the evolution of the quality of the education system. In sum, the recommended methodology for arriving at an accurate diagnostic of the edu- cational sector includes the following: * Thorough diagnostics of the education sectors must start with a good benchmarking of education outcomes. This benchmarking exercise needs to be done, at least, in the three main areas of coverage, efficiency and quality to provide a comprehensive picture of the performance of the education systems; 22 Presented fully in Annex 4. 23 The paper also presents other useful context variables which, however, are not reported here since we are focusing on the supply side. 24 And, as it has been pointed out by a few countries, this will be made all the more difficult by the subjective nature of some of them. It is, however, very difficult to fully avoid elements of subjectivity in qualitative indicators. 13 Monitoring Educational Performance in the Caribbean Table 3: Quality indicators Outcome indicators Performance(a) in national census-based standardized student assessments Performance(a) in national sample-based standardized student assessments Performance(a) in school leaving examinations Performance(a) in regional standardized student assessments Performance(a) in international standardized student assessments Input and process indicators Community level: Curriculum quality: length of instructional time and quantity of time allocated by subjects; presence of measurement and testing elements; etc Monitoring and evaluation capacity: presence and implementation of a performance appraisal policy; presence and quality of an education information management system (EMIS)25; etc School level: Adequacy offinancial, human, physical and material resources: Adequacy of financial resources : per-pupil expenditure Adequacy of human resources: percentage of trained teachers; level of teacher educational attainment Adequacy of physical and material resources: extent to which schools are adequately equipped; presence of a fully functioning school library Educational Leadership Effective use of monitoring and evaluation: teacher attendance rate; teacher punctuality rate; frequency of classroom monitoring; etc Classroom level: Teacher's interactive style: high expectations for academic achievement; prevention of disruptive behaviours, etc. Quality of instructional methods and techniques: available classroom time for learning activities (time on task); available instructional materials, tools or media to enhance learning; frequent student assessment and testing, etc Notes: (a) Measured by passing rates or mean grade. Source: Input and Process indicators are derived from Meier (2002): " Monitoring school effectiveness: selecting appropriate qualitative indicators". * In the coverage and efficiency area, traditional aggregated enrollment ratios should be complemented with grade-specific indicators calculated on the basis of transversal and longitudinal data. This makes it possible to 25 It has been pointed out that in order to be able to use this indicator, standards for an EMIS would have to be established as a prerequisite. 26 Some concern was expressed that the notion of "adequacy" referred to financial resources might be difficult to assess. It is clear that there is no ex-ante "ideal" amount of resources and that assessing if the amount of resources is, satisfactory will require a careful and thorough analysis of the economic situation of the country and of the main challenges faced by its education sector. 14 Part I: Main Report provide a much more accurate picture of what is going on, showing, for instance, what lies behind two similar enrollment rates. Completion and survival rates appear to be particularly relevant in exploring the causes behind the enrollment figures. In particular, through the pattern of drop-outs and repetition that they imply, survival rates provide invaluable information on the current level of internal efficiency of the education system. Additionally, the schooling profiles derived from the survival rates make it possible to assess the "expected" completion and enrollment performance of the system given the current internal efficiency and entry rates, which, beyond being a useful information per-se, compared with the current enrollment and completion rates also provide useful information on the evolution of the internal efficiency and/or the entry rate; * The efficiency analysis should be complemented with a cost-effectiveness analysis, in order to deternine the cost of outcomes and thus establish alternative options for reaching them; * Finally, in the quality area, a variety of standardized tests, both nationally and non-nationally determined, should be used to provide some objective measurement of educational achievement. These outcome measures should be complemented by an analysis of intermediate factors likely to be determining these results (which could also explain some coverage/efficiency results). Pertinent intermediate quality indicators were identified such as school level measures of educational leadership and adequacy of financial and human resources and classroom level measures of quality of instructional methods and techniques. Monitoring these and other indicators would make it possible to design and implement prompt and effective quality-enhancing interventions. SECTION III: MONITORING EDUCATION INDICATORS IN THE CARIBBEAN REGION In this section, we will: (a) review where Caribbean countries stand in terms of the availability of the indicators defined above and (b) describe the new indicator database that was gathered for the ten Caribbean countries which are the focus of this report. Methods will be proposed on how to use the indicators for succinct diagnostics of the education systems. 111.1 Current Availability of Indicators for Education Diagnostics in the Caribbean While the availability of education statistics and indicators has been traditionally limited in the Caribbean, there has recently been substantial effort, both at the national and regional level, to improve the collection and use of education statistics. 15 Monitonng Educational Performance In the Caribbean A. Availabili ty at the National Level At the national level, many countries are in the process of strengthening their education statistics, collection, and processing capacity, some even through the establishment of comprehensi ve Education Management Systems (EMIS), like St Lucia and St Kitts and Nevis, under the World Bank Secondary Education Developmert Project. More or less comprehensive Education Statistical Digests are also being published In most of the countries. (On qua]lity outcomes, we should also highlight that most of the Caribbean countries have developed national systems for measuring, monitoring and assessing learning outcomes The main objective of these systems is to deliver information on academic achievement, uLsing standardli zed tests, to provide inputs for policy and decision-making. Table 4 below reports the existing national assessments by grade in primary education in each of the ten Caribbean countries foc us of the report. While most c:ountries have exams for entering the secondary cycle, care should be taken in interpreting l he results of these exams since in most cases they are used to select the students wvho will have access to secondary on the basis of the available places at that level, meaning that they will not in general provide an objective evaluation of numeracy and literacy skills. Table 4: National Assessments at the Primary level Countries: Exams: Belize Entry exam to secondary: BNSE (Belize National Selection Examination) Dominica Grade 2: National Assessment Entry exam to secondary: CEE (a) Dominican Republic Grade 8_: Pruebas Nacionales Grenada Grade 4: Minimun Competency Test ________ ___ Entry exam to secondary: CEE (a) Guyana Entry exam to secondary: SSEE (Secondary School Entrance __________ Examination) Jamaica Grade 1: Diagnostic Test Grade 4: Literacy Test Grade 6: GSAT (Grade 6 Achievement Test) St Vincent and the Entry Exam to Secondary: CEE (a) Grenadines _ St Kitts and Nevis Grades 3, 4, 5, 6: Test of Standards St Lucia Grade 2 and 4: Minimum Standards Examination _____ Entry to Secondary: CEE (a) T'rinidad and Tobago Entry to secondary: SEA (Secondary Entrance Assessment) Notes: (a) CEE-Common Entrance Exam. Source: Countries' Education Statistical Digests and information directly required to the countries' Ministries of Education. 16 Part 1: Main Report For the purpose of assessing these skills, the standardized tests introduced by several countries at grades 6 or lower constitute much better diagnostic tools of actual student learning. These tools are all the more important at the primary level because of the policy of automatic promotion followed in the lower grades of the cycle by several countries. This could lead to cases of high rates of completion of the cycle but unsatisfactory learning outcomes. B. Availability at the Regional Level At the regional level, UNESCO has led a major effort in the Caribbean by promoting the collection of education statistics and the construction of indicators under the Education For All (EFA) initiative and the activities of the recently strengthened UNESCO Institute for Statistics (UIS). The Caribbean Examination Council (CXC) has led numerous activities on quality outcomes. In the broader Latin American framework, the Regional Education Indicators Project (PRIE) initiative, led by UNESCO/OREALC (Santiago), has done much work on indicators. Education For All (EFA) reports have been prepared for most Caribbean countries across the years, culminating in the particularly extensive regional 2000 EFA assessment. This assessment gave rise to regional reports on the "State of Education in the Caribbean in the 1990's" and on the "Way Forward", including 19 country reports and 27 monographs. The main objective of the UIS in the Caribbean has been to strengthen the collection and reporting of comparative education statistics and indicators through the distribution of ad- hoc data collection questionnaires and special technical assistance, in the form of specialized regional workshops to help fill them out. The collected information is then used to construct a series of comparable education indicators which are included, together with indicators on Latin America and other continents, in electronic databases. These comparable indicators will be included in a Caribbean regional report (which will be the first for the Caribbean area)27. The UIS also participates in the above-mentioned PRIE initiative whose primary objectives are to: (a) construct a set of comparable education indicators for the Americas; (b) strengthen national indicator systems and develop a technical cooperation program; and (c) publish the indicators fostering their use in the adoption of education policies. The initiative led to the selection of 25 indicators grouped in several categories (context, resources, performance and social impact) to be ideally constructed in 42 countries, including 22 Caribbean countries. Finally, the main source of data on quality outcomes comes from the Caribbean Examination Council (CXC) which introduced in 1972 a Secondary Education Certificate (CSEC, more well-known as CXC), currently administered in 15 Caribbean countries, which provides regionally comparable information on the performance of students at the secondary level. 27 As was agreed in the UIS seminar of Trinidad and Tobago. 17 Monimoring Educational Peiformance in the Caribbean Within the CXC, candidates are tested on a variety of subjects, and are required in general to pass at least 5 subjects (at the General Proficiency level), including Math and English, in order to be eligible to pursue tertiary education. The importance and peculiarities of this institution in the Caribbean context makes it worth reviewing, albeit briefly, its purposes, advantages and disadvantages, (see Box 1). C. Summary of Current Available Indicators In Table 5, we summarize which of the indicators reviewed in Section I appear to be available from the national and regional sources mentioned above for the ten Caribbean countries included in this report. Despite the on-going effort, the quantity and quality of the indicators available still differs markedly across countries and, while there is good coverage of some indicators, others are generally not provided. Less indicators are generally available at the secondary than at the primary level. Additionally, poor coverage is particularly noticeable in the areas of quality and, to a lesser extent, efficiency. Finally, grade-specific indicators such as the completion and survival rates are generally not provided. The heterogeneity in available indicators can be clearly seen from a review of the national Education Statistical Digests which, often, do not present indicators but simply some basic enrollment and teacher figures, while in the best case scenario, present detailed information on enrollment (by age and grade), teachers, expenditures and most of the illustrated coverage and efficiency indicators. The heterogeneity also appears in the EFA 2000 Assessments country reports. Harmonizing the quality of the information and indicators among countries and providing for the construction of a larger variety of indicators across all countries, especially related to efficiency and quality, will require further effort. Some promising initiatives, along these lines include: (a) at the regional level, the launch by UNESCO, in strict collaboration with the UIS, of the Caribbean Regional Education Management of Information System (CREMIS), with the primary objective of developing an electronic system to facilitate the management and exchange of educational information regionally and internationally; and (b) at the level of the Eastern Caribbean Countries (OECS), the promotion of a list of common quantitative and qualitative indicators, following the impulse given by the OERU (OECS Education Reform Unit) for a higher level of harmonization of available indicators which should hopefully guide the countries in future data collection efforts. The Caribbean countries have not yet participated in any international assessments, such as the TIMSS or the recent PISA, with the exception of Trinidad and Tobago in the IEA Reading Literacy Test Study conducted in 1990/91. They have also not been involved, with the exception of the Dominican Republic and Cuba, in the Latin American Laboratory for Assessment of the Quality of Education (LLECE), coordinated by UNESCO (Santiago), which, in 1997, developed the First International Comparative Study of Language, Mathematics and Associated Factors for Children in the third and fourth grades of primary education. A supplementary advantage of participating in assessments like the PISA and the LLECE is that participation in these exams also comes with an analysis of the determinants of the exam performance, which can help guide changes in the current education systems. In particular, PISA, apart 18 Part I: Main Report Box 1: The Caribbean Examination Council (CXC): Purpose, advantages and disadvantages Purposes of the Council and main characteristics of the Secondary Education Certificate The Council was established in 1972, as a fully managed Caribbean operation financed mainly by subventions from member territories, with the main purpose of replacing the examinations of overseas Boards with local secondary school examinations of more relevance to the needs of the Caribbean. Based on extensive consultations with the participating countries, it developed syllabuses and examinations for a Caribbean Secondary Education Certificate (CSEC) under three separate schemes-Basic Proficiency, General Proficiency and Technical Proficiency, which is now taken in 15 Caribbean countries. These three schemes are intended primarily for candidates who have completed five years of secondary education. The General Proficiency syllabus and examination is defined as a full equivalent of a "0" level, while the Basic Proficiency one offers a certification for students who cannot make it for a 0 level, making cognitive demands on candidates different from the ones of the General Proficiency. The Technical Proficiency scheme was introduced in response to the demand for secondary school graduates who have a higher degree of practical skills in certain technical subjects. These three schemes had the purpose of providing greater flexibility for students in defining a course of study at the upper secondary level that was responsive to their interests and abilities. The Council, in 1998, also established a post-secondary examination, the Caribbean Advanced Proficiency Examination (CAPE). The CSEC uses a wide range of measurement techniques to assess the performance of candidates. Two modes of assessment are used: a final written examination and internal school-based assessment. In the typical CSEC examination, the multiple-choice paper accounts for about 30% of the total score, while essays or problem-solving papers are used to measure those objectives that they are better suited to measure. The CSEC certificate reports information on a student's overall performance as well as his profile of performance for each subject take. It also presents within-subject profiles which show how a student performed on various dimensions of the subject. Candidates are tested, following the above mentioned format, on a variety of subjects, having in general to pass at least 5 subjects (at the General Proficiency level), including Math and English, to be considered eligible for application to a tertiary institution. The CSEC has been attracting more and more students, resulting in approx. 116,000 candidate entries and 430,000 subject entries in 1998, by offering more and more subjects. It started by offering 5 subjects and, by 1998, was offering 36 subjects in the General and Technical Proficiency schemes clustered around Agriculture, Business Education, Home Economics, Industrial Arts, Information Technology, Modem Languages, Science and Mathematics, Visual Arts, etc. Main advantages of the Secondary Education Certificate (or CSEC) i)The establishment of a regional exam and the common syllabuses have contributed to the acquisition of common knowledge and skills relevant to Caribbean development, promoting the process of regional integration. ii) The CSEC is an established exam. General Proficiency Grades I to m (which also correspond to the pass rates) are widely accepted as satisfactory prerequisites for admission to universities within the Caribbean and some other countries like the US, Canada and the UK and for employment by all Caribbean government and private employers. iii) The CXC reports, each year, entries and grades (ranked from I to VI) obtained for each of the subjects by country, making it possible to track down how CSEC results evolve in time per country and how countries perform relative to one another. Results are fully comparable. This is particularly important for establishing comparable diagnostics of the quality of the education sectors across countries, assuming that the CSEC provides a good measure of learning outcomes. 19 MoiuZtonng Educational Performance in the Caribbean Box 1-continue iv)Teachers are highly involved in the CSEC, through a strong representation in the Subject Panels ithat develop the CSEC syllabuses and participation in the internal assessment component. v) The combination of an internal assessment with the external examination has the advantage of offsetting the unreliability associated with a particular sample of items administered on a particular day (since it is conducted ovi-r several months) and providing information about skills and abilities that cannot be easily and adequately tested externally. Main disadvantages of the Secondary Education Certificate (or CSEC) i)The increase in number of subjects offered from the initial 5 to 36, where subjects like home economics, metals, office procedures, typewriting and woods can be sat at the General Proficiency level together with English, math and integrated sciences could lead to: (a) an actual impoverishment of the curriculum bringing students who should get prepared for good professional opportunities to study subjects which should be studied in technical and vocational education28; and (b) an excessive diversification of subjects taught which could contribute to low cost-effectiveness of the education system through small class sizes and low pupil-teacher ratios. ii) The typical CSEC candidate is expected to offer a combination of subjects under two or three of the above mentioned schemes (General Proficiency, Basis Proficiency and Technical Proficiency). However, experience to date with the examinations suggests that this mix of proficiencies is not as widespread as was originally expected and students have a strong incentive to try to sit the General Proficiency exam whereas the Basic Proficiency is seen at most as a second best which offers very little professional opportunities (entries for Basic Proficiency have in fact been declining from approx. 44,000 in 1989 to 27,000 in 1998). iii)The CSEC runs the risk of promoting an "inward-looking" culture which brings the Caribbean countries to be overly focused on their own specificities with little exposure to the outside world. This is also confirmed by the lack of participation in international exams. iv)The combination of external exams and internal assessments imposes strong common standards in assessing coursework in the last two years of secondary education which reinforce the already fully homogenized programs and syllabuses offered in these last two years by all the secondary schools of the participating Caribbean countries. This might deprive schools of some necessary pedagogical autonomy. from providing a broad assessment of comparative learning outcomes, provides insights on several types of factors that contribute to the development of knowledge and skills, including a detailed analysis of the characteristics of schools and of the teaching-learning process at the classroom level in each country, producing a careful diagnostic of the quality of the education systems, which can then be effective in guiding policy decisions and resource allocation. In conclusion, participating in these exams could be another way for the Caribbean- countries to increase the amount of information on the quality of their education systems. 28 Not all Carnbbean Governments agree on this point, as it was pointed out that "offering technical subjects in the CSEC also makes it possible to afford two advantages: (1) firstly, technical subjects are not seen merely for non-achie vers, but also top students can opt for technical areas and take these subjects at Technical Profi- ciency level; (2) secondly, some secondary school students who are not very academically inclined are also able to take these subjects at the Technical Proficiency level. 20 tr~~~~~~~~~~~~~~~~~~~~~~~~~~t X 0 - 0 o fi° i Euc g c o Eo^i t 4 M tf) & & C~~~~~~~~~~~~C C~ ci~~~~~~~~~~~i 21 10 # 'SX~~~~~~~~~~~' | . _ _ _ _ _ _ _ _ _a__ _ _ _ 6_a__ _g 0~~~~~~~ cj) m~ . W 0 Li 22 .~~~~~ -~~~o 22 2 t.~0 ,t85}-[ 0c0 ,M 000~~~~~~~~~~~~~~~~~~~~~1 40~~~~~~~~~~~~ oo~~~~~~~~ _ -§ 8 $ t | go | id i P 8 | |~~ri~ - -~~~~~~~~~~~~~- 4~~ -N - d8 {i 23 ,o 0 4- - 0 0 > ~~~~~~~~~~~~~~0 .fl .0 ~~~~~0 0 0 0 Cd~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~C o V 0 0 0 0~ ~~~ 0~~~~~~~~~~~ - 0~~~~~>-0 - ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ c o '2.) ~ ~ ~ ~ ~ ~ ~ ~ ,0~~ El) Ca W 0 S~~~~~. I0 _ C> 0 tj ~~~~~~0 0- o ~ ~ ~ ~ ~ ~ oa 0~~~~~~~~~~ 0 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ) t3 ~~~~~~~~~00 v nt3 00 .0 'O Q t3~~~~~ 4.0 L4~~ o ~ ~ ~ ~ ~ *OUO~~~~t 0~~~~~~~~~~~~~~V C.) 0) C, C, F-~~~~~~~~~~~~~~~~~~~~~~~. U- - rz 24 Part L: Main Report 111.2 The New Indicator Database on Ten Caribbean Countries: Description and Analysis Building on the methodological considerations developed in Section I, we have constructed a set of education indicators measuring coverage, efficiency and quality for ten large Bank- related Caribbean countries. In putting together this set of indicators, we had two main priorities: (a) create a set of updated indicators useful for establishing thorough and balanced diagnostics of the education sectors, relevant to policy decisions and (b) ensure comparability by applying the same set of indicators to all countries, using the same methodology. Among the benefits of this database, it is hoped that the benchmarking of educational out- comes will be useful in assisting the Bank prepare its interventions at the country and re- gional levels. The database will also provide internationally comparable indicators available for general consultation within the Bank through its inclusion in the EdStats Website. Fi- nally, at the Caribbean level, it is expected that the database, together with the recommenda- tions on the selection, measurement and monitoring of policy-relevant education indicators, will help the Caribbean countries improve their diagnostics of the education sectors and facilitate policy interventions. Additionally, by providing regionally comparable indicators, the database will enable each country to situate its performance relative to other countries in the region and, hopefully, foster regional initiatives and strategies in the education area. The creation of an updated set of indicators (point (a) above) led us, in a pioneering attempt, to complement more traditional indicators of coverage and efficiency with indicators that are not generally provided at the national or regional level in the Caribbean (see Table 5). In the area of quality, we would have liked to include a much larger variety of indicators measuring the quality of the schooling system but were limited by the lack of information available to construct these types of indicators in the Caribbean. The creation of comparable indicators and methodologies (point (b) above) led us to use specially designed templates to produce coverage and efficiency indicators comparable across countries. More precisely, we used a modified version of the access and efficiency UNESCO templates used to process the information for the Education For All (EFA) reports. The templates have been modified to adapt them to the information requirements of the analysis, by: (a) extending the analysis beyond the primary cycle to include the secondary cycle; (b) reflecting the education sector structure of the Caribbean countries (notably, the duration of the primary and secondary cycle of the countries), modifying some of the indicators accordingly; and (c) introducing a few new indicators not generally calculated. The data collection effort was undertaken to a large extent through four main channels: (a) gathering of the existing Ministries of Education's statistics published in official Education Statistical Digests; (b) close collaboration with the UIS to share the available collected quantitative information on a few countries; (c) when necessary, direct contact with the countries' Ministries of Education to complete the information requirements; and (d) gathering of the CSEC results from the CXC yearly reports. 25 Monitoring Educational Performance in the Caribbean As far as the coverage and efficiency indicators are concerned, most of the latest education statistical digests of the ten analyzed countries report enrollment per age and grade for the most recent year. This helped us in the construction of our indicators. However, past enrollments are usually not reported and there is at times variation in the enrollment format among educational statistical digests of different years, making it difficult to compare the figures and requiring unexpected efforts in constructing the survival rates. Additionally, school age population data, generally not reported in the digests, are problematic to obtain. Different sources can be used to get these data (the United Nations Population Division (UNPD), The World Bank, the countries themselves) but they often provide slightly different estimates making the choice difficult. Ultimately, we decided to stick to the UNPD, using country's data when the UNPD, or World Bank, did not report any information. This happened in the case of Dominica, Grenada and St Kitts and Nevis. The impossibility of comparing the population data for these countries with the ones produced by international sources suggests some caution in interpreting the coverage indicators. Given this, the indicators that we gathered were used to produce two main outputs: (a) country syntheses along the three main areas of coverage, efficiency and quality; and (b) cross-country comparisons of each of the indicators. The country syntheses are reported in Annex 2, while the indicator database, under the form of comparative tables, is reported in Annex 3. A. Syntheses of the Education Sectors by Country For each of the ten Caribbean countries under analysis. we have gathered a set of coverage, efficiency and quality indicators which, combined with further analysis of supply and demand constraints, should help provide thorough diagnostics of the education sectors and dictate adequate interventions. The indicators represent a sub-set of the ones defined in Section I. While in the case of the coverage and efficiency indicators, this is generally due to a process of prioritization which led us to focus on a group of key indicators, in the case of the quality indicators, we were limited by the existing information to the selection of a sub-optimal set. In relation to Section I, we have added a disaggregation by gender of most of the indicators presented given the existence of a strong gender issue in the entire Caribbean area. In Box 2, we take the example of St Lucia to show how these indicators can be combined to provide a useful and operationally relevant synthesis of the education sectors. We also provide some methods on how these indicators can be used to cast light on a number of issues. The graphs and indicators presented can cast some light on some of the main education issues faced by St Lucia. In particular, they can help us to determine: (a) if there is an issue of access and/or survival in primary and/or secondary education; (b) if efficiency is a problem and if so, how it relates to drop-out and repetition rates; (c) how efficiency seems to be evolving in time; and (d) if quality appears to be an issue. The following points can be made: 26 Part I: Main Report BOX 2: Education in St. Lucia, 2000/2001 A: Education System Structure Age 4 17_ 18 Grade__ Enrollment* |3619 3651 |3556| 3607| 57 3607138741 244912551 2655< 270238| _ *Enrollments only correspond to Grade, Primary Level Secondary Level Senior Primary B: Public Expenditure on Education Total Education Expenditure as a % of GDP 6.6 Recurrent Education Expenditure as a % of Total Education Expenditure 70 Public Expenditure per Student (US$), primary 572 Public Expenditure per Student (US$), secondary 824 Note: Expenditure data reflect the actual spending (not budget). C: Coverage of the System Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade K (GIR) 108 103 112 Net Enrollment Rate (NER) 98 97 99 64 [82] 73 [86] 56 [79] Gross Enrollment Rate (GER) 106 105 108 77 [95] 85 [100] 68 [91] Gross Completion Rate (GCR) 106 104 109 72 77 66 Expected Completion Rate (GIR*Survival Rate) 103 96 108 67 66 67 Out-of-school Population (°h) 0 0 0 [11] [9] [13] _ _ Not Applicable 1 Include Senior Primary (grades 7- 9) 120 20 0@0n Total Ferle l\lble Total FerrAle lle R*rEUy Secondary * Net Enrollnunt Rate (NER) l Gross EnrolLrrent Pate (GEl) M Gross QorrpleUon Rate (GCFO M Expected Completion Rate (GIR'Survival Rate) 27 Monitoring Educational Performance in the Caribbean BOX 2-continue: Education in St. I-ucia, 2000/2001 D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 95 93 96 Survival to Form 1 (Fl) 68 76 59 Survival to Form 5 (F5) 62 64 60 Prima Secondary Total Female Male Total Female Male Repetition Rate 2.2 1.8 2.6 0.2 0.2 0.2 Years Input per Graduate 9.6 8.8 10.7 16.5 16.3 16.7 Student-teacher Ratios 24 19 Survival Rates 120 - .0 2 10010 8 _80 (D 0 o ---With Repeaters a -e-*-Without Repeaters 2 40 CL ~ K Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 ES ID ~~~~Schooling Profile >100 a: 80 E70- Ca 2 60- K Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 28 Part 1: Main Report BOX 2-continue: Education in St. Lucia, 2000/2001j E. Education Quality National Mean Scores of Primary Exams* Math 26.2 English 36.8 Pass Rate of Secondary CXC** (%/6) - Math 51 English 58 % students passing at least 5 CXC (**) 27 * Minimum Standards Examinations tested at Grade 4. CXC: Caribbean Examination Council Including English and Math Examination Results 70 60 50 40 30 20 10 0 Wth English Math English Mean Scores of Rinary Exarm Pass Rate of Secondary CXC Primary Secondary % Trained Teachers* 72 | 57 Class Hours per Year 1000 1100 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: Only Government and Government Aided schools. Source: Ministry of Education (2001,2000): "Education Statistical Digest"; Ministry of Education (ad-hoc information requests); CXC (2001); National Statistical Office (latest population projections). 29 Monitoring Educational Performance in the Caribbean * Primary Coverage: An inspection of the Gross Intake Rate together with the GER and the Gross Completion Rate29 at the primary level suggests that neither access nor completion seems to be an issue at the primary level in St Lucia. This conclusion is reinforced by looking at the survival rate to Grade 6 and the related schooling profile which show low drop-out rates at the primary level. * Secondary Coverage: In contrast, a comparison between the GER (or NER) at the primary and secondary level and between the GER and GCR at the secondary level suggests that there might be an issue of access to the secondary cycle with, additionally, some completion issue. This conclusion is again reinforced by looking at the survival rate to Form 1 and to Form 530 and the related schooling profile which show high drop-out rates (or low transition) between the primary and the secondary level and some drop-out during the secondary cycle, mostly concentrated during the last two years. A comparison between the GER and the NER at the secondary level also shows that there would be some room for expansion if repetition was eliminated in primary and secondary, but it would not, however, be possible to accommodate all the secondary-school age population, only an extra fraction of it. * Out-Of-School Children: As a consequence of access and completion in the formal primary and secondary sub-cycles, but also of enrollment in "alternative" school systems31, all primary-school age children are in school, and approximately 90% of the secondary school-age children, indicating an overall substantial coverage of the system. This coverage supports keeping children and youth out of the streets. * Internal Efficiency: The magnitude and pattern of drop-out and repetition rates, shown by the graphed survival rate with and without repeaters, indicate an overall low intemal efficiency of the education system related mostly to the large number of drop-outs (or low transition) between primary and secondary (less than two-thirds of the children entering first grade today are expected to reach Form 5). Low efficiency also gets reflected in the Years Input per Graduate32 which indicates that it takes, on average, more than 16 student years to produce a secondary graduate student in St Lucia. 29 Proxied by the number of students enrolled in the last year of primary or secondary minus the repeaters as a proportion of the children of official age of graduation for primary or secondary. 30 The secondary education cycle of St Lucia (and of other several Caribbean countries) is divided into 5 grades called Forms (Form 1, 2, 3, 4 and 5). 31 The Senior Primary sector present in St Lucia as well as in other Caribbean countries, which, as for all the other countries, we did not include in the primary or secondary enrollment ratios not corresponding to any of these two levels of study, but we considered in our calculation of out-of-school children (see part B of this section for some clarification on this) . 32 Whose magnitude is in fact underestimated by the fact that, in the absence of data on graduates, we basically ended up calculating the number of years necessary to produce an enrollee in the last grade of secondary. 30 Part 1: Main Report * Evolution of Internal Efficiency: The comparison between the "expected" secondary completion rate33, also shown in the schooling profile, and the current completion rate (they appear very similar34) suggests that, assuming that the entry rate to primary education remained stable over the years, also the efficiency parameters (repetition and drop-outs over the whole education cycle) of the system remained basically unchanged35. * Gender Disparities: An inspection of the coverage and efficiency indicators by gender shows that the enrollment and completion rates in primary are higher for males than for females, while the reverse occurs in secondary. This appears to be associated with higher access and lower drop-out rates for males in primary and much higher access (though also higher drop-out rates) for females in secondary. * Educational Achievement in Primary: On the quality side, the situation at the primary level appears worrisome with very low mean performance in the 4k" grade primary exam, suggesting that high completion at that level is not accompanied by satisfactory learning outcomes. The reported intermediate quality indicators, which appear to be satisfactory, indicate that a further analysis of the determinants of educational achievement at the primary level is necessary. * Educational Achievement in Secondary: Also on the quality side, the situation at the secondary level looks more positive, even if borderline in mathematics, judging from the results of the CXC/CSEC (indicated, from now onwards, by the simpler and more familiar acronym, CXC). The proportion of students passing the minimum required of 5 CXC, including English and Math, however is quite low36, indicating low access to tertiary education. Again, the reported intermediate indicators offer only a snapshot of some possible determinants of the performance which needs to be deepened through further analysis. * Cost-Effectiveness: Finally, expenditure levels both in terms of GDP and per student are fairly high in St Lucia and pupil-teacher ratios fairly low (in comparison to other developing countries) suggesting that, given low primary educational achievement 33Calculated multiplying the gross intake rate per the survival rate to the last grade of secondary in the absence of figures on graduates (this really measures completion to the last grade of primary or secondary but is still fairly close to the UIS proposed indicator). 34Assuming a possible imprecision of the cohort-analysis, we can assume a 0-5% points difference between the two completion rates not to be significant. In particular, the reconstructed cohort analysis can overestimate drop-outs in the Caribbean due to significant migration abroad not captured by the model (students migrating abroad are assimilated with drop-outs). 35 If, however, at it appears to be the case in St Lucia, access to secondary education improved in time, that also indicates that reduced drop-outs between primary and secondary were compensated by increased drop-outs within the two education sub-cycles. 36 And appears to be even lower if we express this indicator in function of the students enrolled in first grade of primary. If we do that, that would give us thatjust above 15% of the students starting first grade passes 5 CXC, including English and Math. 31 Monitoring Educational Performance in the Caribbean and insufficient access to secondary education, there might be an allocation and cost- effectiveness issue in the use of resources (a comparative analysis with the other Caribbean countries would be necessary to confirm this issue). What will be the policy implications of these findings? Each of the issues will require a different intervention, which could go from the provision of new school places in case of low access to secondary education to quality-enhancing interventions to improve educa- tional achievement in cases of low quality education and high drop-outs. In order to identify the best "policy levers", it will, however, be necessary to complete the diagnos- tic by a more detailed public expenditure review (including an analysis of the allocation of expenditure across different inputs37), a detailed assessment of supply (availability of school places and teachers) and demand (individual and household level factors) con- straints38 and a further set of indicators capturing the quality of the education provided (quality of instructional methods and techniques, adequacy of human resources, etc). B) Comparative Analysis of the Ten Caribbean Countries The indicators that we gathered also make it possible to compare the selected ten Carib- bean countries since, as mentioned above, we paid particular attention to have most of them constructed according to a similar methodology and to have the same set of indica- tors per country. The comparative tables that were obtained on the basis of this gathering and methodological effort are reported in Annex 3. On the basis of these tables, comple- mented with an assessment of supply and demand constraints and additional quality mea- sures (as in the St. Lucia example), it would be possible to undertake a thorough analysis of coverage, efficiency and quality aspects of the Caribbean education systems which could lead to a number of policy interventions in the area. Since such a study is outside the scope of this report, we will simply show how some of these indicators can be pre- sented and analyzed to highlight characteristics and issues of the Caribbean education sectors, after a few considerations on comparability. Some Caveats on Comparability Even if good, comparability is not perfect and some caution needs to be taken in interpreting the results. Two sets of issues need to be taken into account when comparing the different indicators. A first set of issues are related to the fact that education systems differ somewhat across the analyzed countries, both in terms of the length of the normal schooling cycles and in terms of 37 A functional (among different expenditure programs) and economic (among salaries, recurrent non-salary, etc) classification of expenditure should be provided. " A comprehensive treatment of supply and demand constraints and the best methodological approaches to identify and analyze them is provided in Mingat, Rakotomala and Jee-Peng Tan (2001): " Rapportd'Etatd'un Systeme Educatif National (RESEN)"; Mingat and Suchaut (2000): " Les Systemes educatifs africains"; and The World Bank/HD Network (2001): "Poverty Reduction Strategy Paper (PRSP) Sourcebook: Education". 32 Part I: Main Report the educational options after the end of the normal primary cycle. Concerning the length of the schooling cycle, there is a dominant pattern of a 7 year cycle in primary and 5 year cycle in secondary, but two countries have an 8 year cycle in primary and only 4 in secondary (Dominican Republic and Belize) and two countries have only a 6 year cycle in primary (Guyana and Jamaica). These last two countries also have a shorter overall education cycles since, adding a 5 year cycle in secondary, their complete education cycle is only 11 years, instead of 12. Regarding the structure of the education cycles, several countries, including all the OECS ones (with the exception of St Kitts and Nevis) and Trinidad and Tobago, have a post-primary program of 2 or 3 years after the end of the normal primary cycle that can hardly be assimilated with secondary education given the curriculum offered and the prospects that finishing this cycle offers. Only a small fraction of the students finishing the post-primary cycle can expect to go back to the normal secondary cycle and when they do, have to start from a lower level. The countries themselves do not usually assimilate this cycle with the normal secondary one and the grades differ in name from the formal secondary cycle grades (they are indicated as grades 7, 8 and 9 in contrast to the terminology adopted for formal lower secondary education which is Forms 1, 2 and 3). For all of these reasons and since it would be very misleading for education projects and programs aimed at improving the coverage of secondary education, we have decided not to include the post-primary students in the indicators calculated at the secondary level in these countries39 (however, we show in the countries' profiles and in the comparative table on secondary enrollment ratios (Table A.2), how the secondary enrollment figures would change including these students). These students are included however in the indicator on secondary school age out-of-school children which needs to take into account all children in school, whatever their level. While this seems to be a sensible approach and should improve comparability among countries, it also opens the results to some controversy, since we enter into the difficult ground of having to assess what can be said to be a secondary level education and what cannot. In Guyana and Jamaica, for instance, they have formal fully-fledged secondary programs which co-exist with weaker secondary programs offered in different types of schools. Since we had to draw a line anyway, based on a careful analysis of the secondary programs offered in the different countries, we decided that, for the purposes of comparison, the post-primary education was not equivalent to secondary education. A second set of issues is related to the different coverage of some of the received information. In particular, enrollment data do not cover the private sector schools in some of the countries. While this is not an issue at the secondary level where the participation of the private sector is marginal in all countries, this can be an issue in a few countries at the primary level (even if, for the countries not reporting information, it is estimated that private sector enrollment should not represent more than 5% of overall enrollment). Additionally, some care needs to be taken in the interpretation of the public expenditure data, and, in particular, of the expenditure per student in primary and secondary education, due to possible differences in the actual 39 And they are not included in the indicators calculated at the primary level neither because that would lead to artificially high primary gross enrollment rates including students that are not really in primary anymore (even if they are not quite in secondary neither). 33 Monitoring Educational Performance in the Caribbean coverage of these data related to possible different definitions of the education sector, recurrent/ capital and primary /secondary expenses across the countries (differences that can only be controlled for through the adoption of fully similar methodologies to calculate expenditures40). Coverage and Efficiency The graphs below are based on information included in Tables A. 1, A.2, A.3, A.4 and A.541 in Annex 3 and show a variety of indicators in the coverage/efficiency and quality areas for Belize, Dominica, Dominican Republic, Grenada, Guyana, Jamaica, St Lucia, St Kitts and Nevis, St Vincent and the Grenadines and Trinidad ancl Tobago. The following points can be made on the basis of the above graphs: * Primary Level Coverage: Coverage at the plimary level does not seem to be an issue anymore in the analyzed Caribbean counbies and completion is not generally a problem either, with the exception of three countries (Belize, Dominican Republic, and St. Vincent and the Grenadines). This picture is confirmed looking at the survival rate to the last grade of primary in all countries: the Survival Rate appears to be lower than 80%, indicating significant drop-outs at the primary level, only in Belize, Dominican Republic and St Vincent and the Grenadines42. * Secondary Level Coverage: In contrast to the primary level, coverage and completion are an issue in the secondary sub-cycle of most of the analyzed Caribbean countries (even if to various intensities). Low access to secondary and, above all, high drop- outs during that same cycle are confirmed to be a problem by looking at the survival rates to the first and to the last grade of secondary. Survival rates drop significantly between the last grade of primary and the first grade of secondary in several countries and even more significantly between the first and last grades of secondary in an even broader range of countries. Given this, the graphs also indicate that the issues faced by the countries under analysis differ, and if for some countries access to secondary education seems to be the most serious issue (this is notably the case of Dominica, St Lucia and Grenada), for some others, the number of drop-outs in secondary cycle is the most serious problem (like in Belize, Guyana, Jamaica, and St. Kitts and Nevis). To illustrate the considerations of Section I, it is also interesting to note that we can have countries with similar GERs but very different completion rates due to the relative importance of these two issues. For example, a country with relatively low access to secondary education and a generally low drop-out rate during that cycle, like St Lucia, ends up having a similar GER but a higher cornpletion rate than a country with 40 As the UIS is attempting to promote through special questionnaires and technical assistance modules. 41 Other useful graphs could have been reported, the purpose here is only to provide a synthetic analysis of the indicators that we gathered. 42 However, care needs to be taken in interpreting the results at the primary level for Belize and Dominican Republic since, as mentioned above, these two countries have a 8 year cycle in primary, longer than the aver- age cycle. 34 Part I: Main Report Chart 3: GERs and GCRs in Primary Education for 10 Caribbean Countries 120 100 _LL_ 80 0 60 I ICR Belze Dominica DR Grenada Guyana Jamaica StKitts Slucia SVG TT Countries Chart 4: GERs and GCRs in Secondary Education for 10 Caribbean Countries 120 100 [N G~ER .R60 L j L 2 40 * _ [|| -- 20 0 Belize Dominica DR Grenada Guyana Jamaica StKitts StLucaa SVG TT Counties 35 Monitoring Educational 1'Performance in the Caribbean Chart 5: Survival Rates for 10 Caribbean Countries 120 100 80 oo- _A-_ _ Belize Dominica DR Grenada Guyana Jamaica StKCtts StLucia SVG Tr Countries * SR to last grade of pnmary * SR to first grade of secondary E SR to last grade of secondary Chart 6: Proportion of out-of-school secondary school age children in 9 Caribbean countries 35 - 30 Belrze Dominica DR GrenaJa Jamaica StKitts StLucia SVG r el 20 Countries 36 Part I: Main Report Chart 7: Repetition Rates In Primary and Secondary for 10 Caribbean Countries 16 14 12 10 6 4 2 0 Belize Dominica DR Grenada Guyana Jamaica StKitts StLucia SVG Tr Countries Chart 8: Completion Rates in 10 Caribbean Countries 90 80 B 70 f60 OR / 0 / CD 0 4 - la 0. 30 - 20- 10- 0- 0 10 20 30 40 50 60 70 80 90 GCR In secondakry 37 Monitoring Educational Performance in the Caribbean relatively high access to secondary education but with very low survival rate up to the last grade of the cycle, such as Guyana. I'he insight such an analysis provides confirms the necessity of complementing the traditional aggregated measures with indicators measuring completion and survival. * Out-Of-School Children: There is still a significant proportion of secondary school age children out of school in half of the ten countries under analysis, particularly in Belize and Dominican Republic. The proportion of out-of-school children is lower than expected in Dominica, Grenada, St. Lucia, and St. Vincent and the Grenadines because, as already mentioned, these countries have an extensive post-primary system which, if it cannot be considered equivalent to a secondary level, nonetheless keeps children in school. * Internal Efficiency: The internal efficiency of most education systems in the Caribbean countries under analysis is low as illustrated by the low survival rates to the last grade of the secondary cycle and relatively high repetition rates at the secondary level43. On average, less than one child out of two entering first grade of primary education today is expected to make it through to the last grade of secondary. The combined effect of drop-out and repetition rates on the number of years necessary to produce a secondary graduate is well illustrated by the high YIPGs4 shown by most countries (see Table A.5). As indicated before, the level of internal efficiency varies across countries with a few countries having very low survival rates to the last grade of secondary, such as Belize, Dominican Republic, Guyana, and St. Vincent and the Grenadines, where about one-third or slightly more of the children starting today are expected to reach the last grade of secondary45, and others, like St. Lucia, Grenada, Jamaica, and St. Kitts and Nevis, where about two-thirds of the children make it to the end of secondary. * Evolution of Internal Efficiency and Completion: The countries with the lower survival rates tend to also have lower current gross completion rates and lower "expected" completion rates46 (with the exception of Dominican Republic because of its currently very high gross intake rate), and vice-versa, showing little expected change in the relative position of the countries in terms of completion and, most 43This gets in fact better illustrated in the individual countries' syntheses which report graphically the survival rates with and without repeaters per country. I Whose magnitude is in fact underestimated by the fact that, in the absence of data on graduates, we basically ended up calculating the number of years necessary to produce an enrollee in the last grade of secondary. We should also point out that a limitation of the YIPG indicator is that it is quite sensitive to the total number of years of the schooling cycle. 45 And which also correspond to countries with high repetition rates in secondary, with the exception of Dominican Republic. I Calculated multiplying the gross intake rate per the survival rate to the last grade of primary or secondary in the absence of figures on graduates (this really measures completion to the last grade of primary or secondary but is still fairly close to the UIS proposed indicator). 38 Part 1: Main Report likely, the internal efficiency parameters of their education systems47. A quick look at the evolution within each country shows two cases (St. Kitts and Nevis, and Trinidad and Tobago) where the "expected" completion rate in secondary is significantly lower than the current completion rate. As explained in Section I, before concluding that the internal efficiency of the systems is getting worse48, we should make sure that the intake rates have been stable in time, which could not be the case in St. Kitts and Nevis and Trinidad and Tobago which have a current gross intake rate of only about 95%. Overall, without further evidence it is difficult to come to more precise conclusions on this aspect. Cost-Effectiveness Graphs 9, 10 and 11 below, constructed on the basis of the information presented in Tables A.2, A.7 and A.9, provide a snapshot of the cost-effectiveness dimension, while Graph 12, constructed on the basis of the information presented in Tables A.5 and A.9, provides a possible explanation for spending inefficiencies. The graphs suggest that: * Public Expenditure on Education: All Caribbean countries, with the exception of Dominican Republic, spend substantially on education, as indicated by a public education expenditure-GDP ratio of 6% or more and by the fact that, on average, 15% of the GNP per capita is spent per student in secondary. While the situation is quite uniform in terms of public education expenditure in proportion of GDP, it fluctuates more in terms of the secondary spending per student in proportion of GNP per capita: countries with very similar GNP per capita invest quite different amounts in secondary education in proportion of their GNP (comparing, for instance, Belize and Dominica with Jamaica and Grenada). * Public Expenditure and Outcomes: The relationship between educational outcomes and public expenditure on education as a proportion of GDP, while generally positive, is by no means tight. Wide variations exist in spending efficiency across countries, as shown by the fact that similar levels of spending are associated with a wide range of educational outcomes. The relationship is somewhat closer between spending per student in secondary education in proportion of their GNP per capita and academic achievement as exemplified by CXC results, while the relationship to outcomes disappears completely with the completion rate. * Causes of Spending Inefficiency: Some degree of spending inefficiency is likely to be related to low pupil-teachers ratios since low pupil-teacher ratios are closely associated with higher unit costs (as shown in Graph 12), without generally having a significant relationship with educational achievement. The lack of data on teachers' salary for most of the countries prevents us from analyzing this other determinant of 47 And also some consistency in our calculations and between the different methodological approaches to calculate completion. 48 Which might be happening in some countries following recent modifications in the "automatic promotion" polcy. 39 Monitoring Educational Perfornance in the Caribbean Chart 9: Spending per Student in Secondary Education and GNP per capita in 9 Caribbean Countries 20 *BE 18 ..DO O 16 SVG XL 14SL *SK 14 JA 12 gc 10 * GR * TT Q 10 (U~~~~~~~~~~~~~~~~~~~~~T 10 DR 0. c6 C74 2 0 0 1,0C 2,00D 3,000 4,000 5,000 6,000 7,000 GNP per caprt (US$) Chart 10: Public Expenditure on Education and CXC Performance in 9 Caribbean Countries 90- __ DO 80 SK* 70 *BE - F *.VG 6 0 a--R 50 (U ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ G -30 cn 20 10 0 0 1 2 3 4 5 6 7 8 9 Education Expenditure/GDP (%) 40 Part I: Main Report Chart 11: Public Expenditure on Education and Completion Rate in 10 Caribbean Countries 90 GR. 80 70 SK JA o 60 'S DO 50 L DR SVG r 40 I ~~~~~~~~~~~~~~~E *GU E 30 8 20 10 0 a 1 2 3 4 5 6 7 6 9 Education Expenditure/GDP (%/6) Chart 12: Unit Costs and Pupil-Teacher Ratios in Secondary Education for 9 Caribbean Countries 30 DR 25 ~~~~~~~~~~~~SL 20 2 JA BE .5 0 0 200 400 600 B00 1000 1200 Unit costs (US$) 41 Monitoring Educational Performance in the Caribbean spending. The available evidence on St. Kitts and Nevis, Dominica, and St. Vincent and the Grenadines49 indicates that personnel costs (as a combination of the existing pupil-teacher ratios and salaries) tend to represent too much of the total recurrent costs (on average 90% for the three countries) leaving little room for teaching materials and supplies which, in general, are closely related to outcomes. Quality The remaining graphs illustrate the quality issues and are constructed on the basis of Tables A.2, A.7, and A.8. The comparative analysis was limited to secondary education since the exam outcomes are not comparable across countries at the primary level. In contrast, CXC results are fully comparable50. The graphs suggest the following points on quality: * Educational Achievement: Performance on the CXC, used in nine out of the ten analyzed countries, varies significantly across countries, with passing rates consistently lower in Math than in English. The results would be worse if we were to consider the proportion of students passing at least 5 CXC subjects, including English and Math, the minimum requirement for access to tertiary education51. Unfortunately we could only gather data on this proportion for St. Lucia, St. Kitts and Nevis and Grenada (reported in Table A.4), which all point to unsatisfactory performance. * Educational Achievement and Coverage: If there is a generally positive relationship between secondary completion rates and English CXC performance, the relationship is much less true for Math CXC performance, confirming the need to look at both coverage and quality when making diagnostics of the education systems. * Determinants of Educational Achievement: A generally positive, even if weak, relationship appears to exist between class hours and the CXC results52. More comparative evidence on intermediate quality indicators is needed in order to analyze the determining factors of learning outcomes in the Caribbean. In sum, the common set of indicators that we have established has enabled us to make some useful points on the state of the education in the region. Some of the main conclu- sions are: * Coverage and completion do not seem to be an issue anymore at the primary level in most of the analyzed Caribbean countries. Not included in the countries' profiles and comparative tables. 50 Dominican Republic, which does not take the CXC, cannot however be compared with the other countries. 51 This is not, however, the case in St Kitts and Nevis, where Mathematics at CXC General Proficiency is highly desirable but not a requirement for access to tertiary education. 52 The relation between CXC performance and proportion of trained teachers is difficult to test for the sus- pected little comparability of this last indicator, whose relative magnitude across countries is dependent on the national definition of a "trained" teacher. 42 Part 1: Main Report Chart 13: CXC Performance in 9 Caribbean Countries 90 60 - FI w 50- - 50 0 ~~~~~~~~~~~~~~~~~~~~~Math CXC e401 A _ | ___ | _ __ | _ U | English CXC 40 4 IL 20- 10 - 0 Belize Dominica DR Grenada Guyana Jamaica StKitts StLucia SVG iT Countries Chart 14: Completion Rate and English CXC Performance in 10 Caribbean Countries 90 10 70 Beliza Dominica DR (non Grenada Guyana Jamaica SiKits StLucia SVG ny CXC) Countnies 43 Monitoring Educational Performance in the Caribbean Chart 15: Completion Rate and Math CXC Performance in 10 Caribbean Countries 90 80 70 60 50 ae _ _ * - |-B~~~~~~~~~~~~~~~~~~Meoondary GCF| Belize Dominica DR~ (non Grenada Guyana Jamaica StKitts StLudia SVG TT CXC) Countries Chart 16: CXC Performance and Class Hours in 8 Caribbean Countries 40 - 830 UU020 70 80 GXC)~ ~ ~ ~ ~~~G Chrt16 XCPefomnc adClass Hours in seodr 40 80 DOGS 70 60 20 0 60 80 00 120 100 10 co~~~~~~~~~~ls orsI eodr C)44 Part I: Main Report * In contrast, coverage and completion are still an issue at the secondary level for most of the countries (access is the major problem in some countries, while drop- outs during the cycle is the major problem in others). * The internal efficiency of most of the education systems is low. * Wide variations exist in spending efficiency across countries. * Educational achievement at the secondary level varies significantly across coun- tries and is very unsatisfactory if assessed in terms of the minimum requirement for access to tertiary education. SECTION IV: CONCLUSIONS AND POLICY RECOMMENDATIONS This report set out to present current methodologies for reaching a diagnostic of the education sector, it outlined the challenges particular to the Caribbean region, such as the lack of comparable information for analysis, and explained how the establishment of a common set of educational indicators would enable an analysis of Caribbean education systems. Several recommendations as to how build on the existing work of Ministries of Education and international organizations in the collection of information and construction and use of indicators for education diagnostics in the Caribbean can be made and are summarized below. * Thorough diagnostics of the education sectors should start with a good benchmarking of education outcomes Benchmarking is becoming more and more critical for justifying and designing interventions in the education sector. Recent EFA initiatives, for example, rely on effective benchmarking and monitoring of education outcomes. Effective benchmarking requires the appropriate selection of a set of indicators and availability of reliable statistical information. This exercise needs to be done in at least three main areas: coverage, efficiency and quality in order to provide a comprehensive picture of the performance of the education systems. =>Substantial effort has been undertaken in quantifying the coverage and efficiency of educational systems in the Caribbean region, as shown by the indicators generally reported in the education statistical digests and the UNESCO-led efforts in the area of educational indicators. However: * Caribbean countries should measure the coverage and efficiency of the entire education cycle (primary plus secondary) The current quantification effort is still much stronger at the primary level than at the secondary one. If it is important to pursue the effort at the primary level in view of the EFA targets, an equivalent effort should be also be made in measuring coverage of the secondary level and efficiency of the overall education cycle, where, as shown by our analysis, the main issues reside; 45 Monitoring Educational Performance in the Caribbean * Caribbean countries should start producing and monitoring grade-specific indicators such as completion and survival rates Still r elatively little use is made of grade-specific indicators, constructed on the basis of transversal or longitudinal data. Traditional aggregated enrollment ratios should be complemented by grade-specific indicators calculated on the basis of transversal and longitudinal data. Completion and survi-val rates are particularly relevant in exploring the nature of the problem behind the enrollment figures. In the Caribbean, the primary and/or secondary completion rates are never measured and the survival rate is rarely reported (going only to grade 5 of primary when measured). The usefulness of these and other indicators in a coverage/efficiency diagnostic of the Caribbean education systems has been illustraled in the case of single country and comparative assessments. The indicators are also essential in establishing where the countries stand in terms of the EFA targets. As a consequence, it is recommended that Caribbean countries start producing and using more of this type of indicator. This recommendation was explicitly made by the World Bank at the UIS Regional Workshop on Education Indicators in the presence of Education Planners and Statisticians of all the Caribbean countries. In the same workshop, the UIS, albeit more cautiously, also initiated a debate around the completion rate and the possible methodologies for calculating it. Producing the new indicators will require the Ministries of Education of the region to collect and report data on enrollment per age and grade for the whole education cycle and sector (including public and private schools) in a consistent way across the years53. Since the improvement of existing indicators and the development of new indicators is a key part of the work of the UIS in the 2002-2007 period, the UIS could then provide technical assistance to the countries to ensure that, not only the data collection process, but also the indicators themselves, are based on sound methodologies and take account of international standards; * Harmonization of education statistics and indicators should continue There remains much variation in the quality and quantity of indicators available per country making it difficult to undertake thorough cross-country comparisons of the coverage/efficiency of the education systems. The efforts undertaken by UNESCO through the UTIS initiatives, the PRIE and, more recently, the CREMIS, are all going towards ensuring higher harmonization of education statistics and indicators. It is essential that this effort be sustained, so that the countries may produce a comparable set of indicators, including both the primary and secondary levels and going beyond the traditional measures of educational coverage and efficiency onto more policy- relevant indicators. These recommendations have been presented by the World Bank at the UIS Regional Workshop on Education Indicators; * Caribbean countries should improve the collection and use of expenditure data There is a lack of reliable expenditure data to undertake thorough cost-effectiveness 5 And to coordinate with the Central Statistical Offices to obtain and report the latest school age population estimates. 46 Part I: Main Report analyses. Cost-effectiveness analyses are essential in the context of the Caribbean region where the existing evidence points to the existence of wide variations in spending efficiency across countries. A cost-effectiveness analysis looks at outcomes relative to expenditure in order to assess how costly it is to reach these outcomes and establish alternative options for reaching them. To this purpose, the countries' Ministries of Education need to improve their collection and use of expenditure data, reporting and monitoring in a systematic way unit costs per education level and expenditure allocation ratios. ==In the area of quality, substantial effort has been made towards improving national assessment systems and introducing national diagnostic tests at the primary level, which are useful in measuring learning outcomes. Additionally, the CXC provides for some regional comparison of learning outcomes at the secondary level which gives some useful benchmarking of the relative position of each of the countries. However: * Caribbean countries should participate in international exams On the outcome side, the lack of directly comparable exams at the primary level makes it difficult to compare learning outcomes in a meaningful way across countries and to provide objective benchmarking on the quality of primary education for each country. Additionally, standardized tests which provide a reliable assessment of knowledge and skills at the primary level do not yet exist in all countries. Within this context, it would be reconmnendable to have the countries participating in existing regional and international exams to be testing knowledge and skills. Participation in the Latin American Laboratory for Assessment of the Quality of Education (LLECE) initiative, for instance, would be very advisable. Moreover, this exam also provides an analysis of the determinants of the exam performance, which can then guide policy makers in designing adequate quality-enhancing interventions. Even if the existence of the CXC provides regional comparisons of outcomes at the secondary level, participating in other international exams at that level would also be advisable to provide some international benchmarking of the regional performance, especially given the growing skepticism towards the Caribbean exam. Participation in the new PISA exam would, as in the LLECE case, provide useful information on both learning outcomes and associated factors. These recommendations have been presented by the World Bank at the UIS Regional Workshop on Education Indicators; * The proportion of students passing at least 5 CXC, including English and math, should be monitored The non-availability of information on the proportion of students passing at least 5 CXC, including English and math, reveals that this indicator is not being used. The Caribbean Council or the countries themselves should consider monitoring the passing rate beyond single subjects, given that passing 5 subjects on the CXC, including English and math, is the minimum requirement for access to tertiary education and performance is particularly unsatisfactory when measured using this indicator. It would 47 Monitoring Educational Performance in the Caribbean also be equally important to present this information by gender considering the generally different performance of males and females in the Caribbean; * Caribbean countries should start producing and monitoring qualitative indicators On the input side, very little information is available on possible determinants of the educational achievement performance in the region. Educational outcomes need to be complemented by an analysis of intermediate factors which, adequately monitored, would make it possible to design and implement prompt and effective quality- enhancing interventions. The shortage of intermediate quality indicators is recognized by all Caribbean countries and the main regional actors. Qualitative indicators that can provide a characterization of schools and teachers and indications on the teaching- learning process at the classroom level would be extremely useful in understanding the causes of the still generally low educational achievement at the primary and secondary level. These causes are in fact all the more important in a context where the quality of the education system is the most serious challenge. These considerations led us to undertake an in-depth analysis on the use of qualitative indicators and school effectiveness studies which led to a further refinement of a list of qualitative indicators initially proposed by the Eastern Caribbean Education Reform Unit (OERU) (see Annex 4). This list was presented and discussed regionally in the UIS Regional Workshop on Education Indicators. There was an overall agreement on these indicators which, hopefully, will lead to the actual measurement and monitoring of some of them at the Caribbean level, to finally produce better diagnostics of the quality of the schooling system in the area. This initiative would be perfectly compatible with the participation in some regional initiatives like the LLECE which would make use of both these and other intermediate quality measures to undertake a comprehensive analysis of learning outcomes determinants. 48 Part H: Annexes Annex 1: Education Indicators: Definition and Interpretation .................................... 51 Annex 2: Country Education Profiles .................................... 65 Annex 3: Comparative Tables on Educational Performance .................................... 92 Annex 4: Measuring Educational Quality .................................... 102 49 Part 11: Annexes Apparent Intake Rate Definition: Total number of new entrants in first grade of primary education, regardless of age, expressed as a percentage of the population at the official primary school-entrance age. Purpose: Apparent Intake Rate indicates the general level of access to primary education. It also indicates the capacity of the education system to provide access to grade 1 for the official school-entrance age population. This indicator is used as a substitute to Net Intake Rate in the absence of data on new entrants by single years of age. Calculation method: Divide the number of new entrants into grade 1, irrespective of age, by the population of official school-entrance age, and multiply the result by 100. Formula: AIR t *100 Pa Where: AIRt = Apparent Intake Rate in school-year t Nt = Number of new entrants in the first grade of primary education, in school-year t P. = Population of official primary school entrance-age a, in school-year t. N.B.: When data on new entrants are not separately reported, they can be derived by subtracting the numbers of repeaters from enrolment in first grade, before calculating the apparent intake rate. Data required: New entrants in the first grade of primary education (or enrolment minus repeaters in the first grade); population of the official primary school-entrance age. Data source: School register, school survey or census for data on new entrants by age. Population census or estimates for primary school-entrance age population. Type of disaggregation: The Apparent Intake Rate is to be disaggregated by gender and geographical location (region, rural/urban). Interpretation: A high Apparent Intake Rate indicates a high degree of access to primary education. As this calculation includes all new entrants to first grade (regardless of age), the Apparent Intake Rate can be more than 100%, due to over- and under-aged children entering primary school for the first time. Quality standards: Data on population used in deriving this indicator should refer strictly to the official school-entrance age. Care should be taken not to include repeaters in grade 1 in the calculation, since this will lead to an inflated Apparent Intake Rate. Limitations: A high Apparent Intake Rate may be the effect of a backlog of over-aged children who have not entered school when they were at the official primary school-entrance age. 51 Montitoring Educational Perfornance in the Caribbean Net Intake Rate Dermition: New entrants in the first grade of primary education who are of the official primary school-entrance age, expressed as a percentage of the population of the same age. Purpose: To show the level of access to primary education of the eligible population of primary school-entrance age. Calculation method: Divide the number of children of official primary school-entrance age who enter the first grade of primary education by the population of the samne age, and multiply the result by 100. Formula: NIR t=Nt 100 tN Pa Where: NaRt = Net Intake Rate in school-year t. N, = Number of children of official primary school-entrance age a who enter the first grade of primary education, in school-year t. P' = Population of official primary school-entrance age a, in school-year t. Data required: New entrants in first grade of primary education by single years of age; population of official primary school-entrance age. Data source: School register, school survey or census for data on new entrants by age. Population census or estimates for school-entrance age population. Type of disaggregation: The Net Intake Rate is to be disaggregated by gender and geographical location (region, rural/urban). Interpretation: A high Net Intake Rate indicates a high degree of access to primary education for the official primary school-entrance age children. For countries which have subscribed to the policy goal of universal primary education, a NIR of 100% will be a necessary condition. Quality standards: Data on both new entrants and population used in deriving this indicator should refer strictly to the official school-entrance age. NIR in principle should not exceed 100%. Limitations: This indicator can be distorted by an incorrect distinction between new entrants and repeaters in the first grade. This can be the case especially for under-aged pupils who may repeat the first grade at the official-entrance age. Transition Rate Derimition: The number of pupils (or students) admitted to the first grade of a higher level of education in a given year, expressed as a percentage of the number of pupils (or students) enrolled in the final grade of the lower level of education in the previous year. Purpose: This indicator conveys information on the degree of access or transition from one cycle or level of education to a higher one. Viewed from the lower cycle or level of education, it is considered as an output indicator, viewed from the higher educational cycle or level, it 52 Part H1: Annexes constitutes an indicator of access. It can also help in assessing the relative selectivity of an education system, which can be due to pedagogical or financial requirements. Calculation method: Divide the number of new entrants in the first grade of the specified higher cycle or level of education by the number of pupils who were enrolled in the final grade of the preceding cycle or level of education in the previous school year, and multiply by 100. Formula: EtT+ - Rt+1 TR h+1 h+l,l h+11 *zoo Where: TR = Transition rate (from cycle or level of education h to h+1 in school year t) Et+l ,= number of pupils enrolled in the first grade at level of education h+1 in +1,1 school-year t+1 Rt+ 1 = number of pupils repeating the first grade at level of education h+1 in school- ltll year t+1 Et f = number of pupils enrolled in final grade f at level education a in school year t. Data required: Enrolment in the final grade of a given cycle or level of education and new entrants to (or enrolment minus repeaters) the first grade of the higher cycle or level of education. Data source: School register, school survey or census. Data disaggregation: Transition rate is to be disaggregated by gender, level of education and geographical location (region, rural/urban). Interpretation: High transition rates indicate a high level of access or transition from one level of education to the next. They also reflect the intake capacity of the next level of education. Inversely, low transition rates can signal problems in the bridging between two cycles or levels of education, due to either deficiencies in the examination system, or inadequate admission capacity in the higher cycle or level of education, or both. Quality standards: This indicator should be based on reliable data on new entrants (or on enrolment and repeaters) especially in the first grade of the higher cycle or level of education. Limitations: This indicator can be distorted by incorrect distinction between new entrants and repeaters, especially in the first grade of the specified higher level of education. Students who interrupted their studies for one or more years after having completed the lower level of education, together with the migrant students could also affect the quality of this indicator. School-Life Expectancy Definition: School life expectancy is defined as the total number of years of schooling which a child of a certain age can expect to receive in the future, assuming that the probability of his or her being enrolled in school at any particular age is equal to the current enrolment ratio for thatage. 53 Monitoring Educational Performance in the Caribbean Purpose: This indicator shows the overall level of development of an educational system in terms of the number of years of education that a child cam expect to achieve. Calculation method: For a child of a certain age a, the school life expectancy is calculated as the sum of the age specific enrolment ratios for the reference age-range a to n, divided by 100. Formula: l =a P., Where: sLrE = School life expectancy at an age a in year t = Enrolment of the population of age i (for i = a, a+1,..., n) in school-year t; n denotes the theoretical upper age-limit of schooling. Pt = Population of age i in school-year t. Data required: Enrolment by age at all levels of education; population of official school-age for all levels of education by single years of age. Or, alternatively, the age specific enrolment ratios for all levels of education. Data source: School register, school survey or census for data on enrolment by age. Population censuses and estimates for school-age population. Type of disaggregation: School life expectancy is to be disaggregated by gender and by geographical location (region, urban/rural). Interpretation: Relatively higher school life expectancy indicates greater probability for children to spend more years in education and higher overall retention within the education system. It must be noted that the expected number of years does not necessarily coincide with the expected number of grades of education completed, because of grade repetition. Quality standards: School life expectancy requires complete and reliable data on enrolment and population by single-years of age corresponding to all levels of education for the entire duration of schooling, including tertiary education. Limitations: Caution is required when school life expectancy is used for inter-country comparison; neither the length of the school-year nor the quality of education is necessarily the same in each country. In addition, as this indicator does not directly take into account the effects of repetition, it is not strictly comparable between countries with automatic promotion and those allowing grade repetition. It should also be noted that, depending on countries, the enrolment data do not account for many types of continuing education and training. For these reasons, this indicator should be interpreted in the light of complementary indicators like repetition rates. Gross Enrolment Rate Definition: Total enrolment in a specific level of education, regardless of age, expressed as a percentage of the official school-age population corresponding to the same level of education in give school-year. 54 Part I1: Annexes Purpose: Gross Enrolment Ratio is widely used to show the general level of participa- tion in a given level of education. It indicates the capacity of the education system to enroll students of a particular age-group. It is used as a substitute indicator to net enrol- ment ratio (NER) when data on enrolment by single years of age are not available. Fur- thermore, it can also be a complementary indicator to NER by indicating the extent of over-aged and under-aged enrolment. Calculation method: Divide the number of pupils ( or students) enrolled in a given level of education regardless of age by the population of the age-group which officially corre- sponds to the given level of education, and multiply the result by 100. Formula: GER= j' *100 Where GERB = Gross Enrolment Ratio at level of education h in school-year t = Enrolment at the level of education h in school-year t pt = Population in age-group a which officially corresponds to the level of educa- tion h in school-year t Example: If the entrance age for primary education is 7 years with a duration of 6 years then a is (7-12) years. Data required: Total enrolment for a given level of education. Population of the age- group corresponding to the specified level. Data source: School register, school survey or census for data on enrolment by level of education. Population censuses for school-age population normally obtained from the Central Statistical Office. Types of disaggregation: This indicator is to be disaggregated by gender, by geographi- cal location ( region, urban/rural) and level of education. Interpretation: A high GER generally indicates a high degree of participation, whether the pupils belong to the official age-group or not. A GER value of 100 percent indicates that a country is, in principle, able to accommodate all of its school-age population, but it does not indicate the proportion already enrolled. The achievement of a GER of 100 percent is therefore a necessary but not sufficient condition for enrolling all eligible chil- dren in school. When the GER exceeds 90 percent for a particular level of education, the aggregate number of places for pupils is approaching the number required for universal access of the official age-group. However, this is a meaningful interpretation only if one can expect the under-aged and over-aged enrolments to decline in the future to free places for pupils from the expected age-group. Quality standards: GER at each level of education should be based on total enrolment in all types of schools and education institutions, including public, private and all other institutions that provide organised educational prograrmmes. 55 Monitoring Educational Performance in the Caribbean Limitations: GER can be over 100% due to the inclusion of over-aged and under-aged pupils/students because of early or late entrants, and grade repetition. In this case, a rigorous interpretation of GER needs additional information to assess the extent of rep- etition, late entrants, etc. Net Enrolment Rate Definition: Enrolment of the official age-group for a given level of education expressed as a percentage of the corresponding population. Purpose: To show the extent of participation in a given level of education of children or youths belonging to the official age-group corresponding to the level of education. Calculation method: Divide the number of pupils ( or students ) enrolled who are of the official age-group for a given level of education by the population for the same age- group and multiply the result by 100. Formula: NERt= 44*IOO Where NEIRJ = Net Enrolment Ratio at level of education h in school-year t Et24 = Enrolment of the population of age-group a at a level of education h in school-year t p^2^ = Population in age-group a which officially corresponds to level of education h in school-year t. Example: If the entrance age for primary education is 7 years with a duration of 6 years then a is (7-12) years. Data required: Enrolment by single years of age for a given level of education. Population of the age-group corresponding to the given level of education. Data source: School register, school survey or census for data on enrolment by age. Popu- lation censuses or estimates for school-age population normally obtained from the Central Statistical Office. Iypes of disaggregation: This indicator is to be disaggregated by gender, by geographical location ( region, urban/rural) and by level of education. Interpretation: A high NER denotes a high degree of participation of the official school-age population. The theoretical maximum value is 100%. Increasing trends can be considered as reflecting improving participation at the specified level of education. When the NER is compared with the GER the difference between the two ratios highlights the incidence of under-aged and over-aged enrolment. If the NER is below 100%, then the complement, i.e. the difference with 100% provides a measure of the proportion of children not enrolled at the specified level of education. However, since some of these children/youth could be enrolled at other levels of education, this difference should in no way be considered as indicating the 56 Part H1: Annexes percentage of students not enrolled. A more precise complementary indicator is the age- specific enrolment ratio (ASER) which shows the participation of the population of a par- ticular age in education. Quality standards: NER at each level of education should be based on total enrolment in all types of schools and education institutions, including public, private and all other institutions that provide organized educational programmes. Limitations: For tertiary education, this indicator is not pertinent because of the difficulties in determining an appropriate age-group due to the wide variations in the duration of programmes at this level of education. As regards primary and secondary education, certain difficulties arise when calculating the NER for countries where the net enrolment ratios should be of the order of 100%. Three cases can be mentioned: 1. when the reference date for entry to primary education does not coincide with the birth- days of all of the cohort eligible to enrol at this level of education. 2. when an important part of the population starts primary school earlier than the prescribed age and consequently finishes earlier as well. 3. when there is an increase in the entrance age to primary education with the durations unchanged. N.B. Although the NER cannot exceed 100%, values up to 105% are shown and in these cases there are inconsistencies in the enrolment and/or population data. Gross Completion Rate Definition: The total number of students completing (or graduating from) the final year of primary or secondary education, regardless of age, expressed as a percentage of the population of the official primary or secondary graduation age. Purpose: this is a simple measure that monitors what share of all children complete an education cycle, and in the case of the primary completion rate tracks progress towards the goal of Education for All which is to achieve universal completion by the year 2015. Calculation method: divide the number of students completing (or graduating from) the final year of primary education by the population at the official primary graduation age, and multiply the result by 100. In fact, in the absence of information on graduates, the completion rate is often proxied by the following formula (here applied to the primary case): GCR (primary) = No. of students in the last primary grade - repeating students * 100 Population of official age group for the last primary grade Data required: Total number of students completing (or graduating from) the final year of primary or secondary education. Population of the primary or secondary graduation age. For the proxy formula: Total number of students enrolled in the last year of primary or secondary education and total number of repeaters in the same years. Data source: School register, school survey or census for data on graduates. Population by age normally can be obtained from the Central Statistical Office in the country or from the Edstats, the World Bank education statistical databases. 57 Monitoring Educational Performance in the Caribbean Types of disaggregation: This indicator can be disaggregated by gender and by geo- graphical location (region, urban/rural). If population and school enrollment data on ethnicity are collected, this can also be calculated. Interpretation: A high primary completion rate indicates a high degree of coverage, whether the pupils belong to the population of official graduation age or not. Care should be taken when comparing cross-country results since the duration of basic education varies from 4 to 9 years among countries. Quality standards: Data on population used in deriving this indicator should refer strictly to the official/typical graduation age. Limitations: The completion rate could be over-estimated due to the inclusion of over- aged and under-aged pupils because of early or late entrants. However, this problem will, in most cases, cancel out overtime. Repetition Rate Definition: Proportion of pupils from a cohort enrolled in a given grade at a given school- year who study in the same grade in the following school-year. Purpose: It measures the phenomenon of pupils repeating a grade, and its effect on the internal efficiency of educational systems. In addition, it is one of the key indicators for analysing and projecting pupil flows from grade to grade within an educational cycle. Calculation method: Divide the number of repeaters in a given grade in school-year t+1 by the number of pupils from the same cohort enrolled in the same grade in the previous school- year t. Formula: t4l rit _ Ri Ei Where: ri' = Repetition Rate at grade i in school-year t R!i1 = number of pupils repeating grade i, in school-year t+1 EB = number of pupils enrolled in grade i, in school-year t. Data required: Enrolment by grade for school-year t and number of repeaters from the same cohort by grade for year t+l. Data source: School register, school survey or census for data on enrolment and repeat- ers by grade. Type of disaggregation: Repetition Rate can be disaggregated by sex, by geographical location ( regions, urban/rural ),by level of education and by type of institution (public/ private). 58 Part 11: Annexes Interpretation: Repetition Rate ideally should approach zero percent; a high Repetition Rate reveals problems in the internal efficiency of the educational system. When com- pared across grades, the patterns can indicate specific grades for which there is higher repetition, hence requiring more in depth study of causes and possible remedies. Quality standard: Like other pupil-flow rates (promotion and drop-out rates), the rep- etition rate is derived by analysing data on enrolment and repeaters by grade for two consecutive years. One should therefore ensure that such data are consistent in terms of coverage over time and across grades (cohort analysis). Special attention should also be paid to minimizing some common errors which may bias these flow-rates, such as: Over- reporting enrolment/repeaters (particularly in grade one); incorrect distinction between new entrants and repeaters; transfers of pupils between grades and schools. Limitations: The level and maximum number of grade repetitions allowed can in some cases be determined by the educational authorities with the aim of coping with limited grade capacity and increasing the internal efficiency and flow of pupils (or students). Care should be taken in interpreting this indicator, especially in comparisons between education systems. Survival Rate By Grade Definition: Percentage of a cohort of pupils (or students ) enrolled in the first grade of a given level or cycle of education in a given school-year who are expected to reach each successive grades. Purpose: Survival rate measures the holding power and internal efficiency of an education system. It illustrates the situation regarding retention of pupils (or students ) from grade to grade in schools, and conversely the magnitude of drop-out by grade. Calculation method: Divide the total number of pupils belonging to a school-cohort who reached each successive grade of the specified level of education by the number of pupils in the school-cohort i.e. those originally enrolled in the first grade of the same level of education, and multiply the result by 100. Formula: m SR4- ' *100 Where: Pg,i - EVll- R,i'+i i = grade (1, 2, 3,...,n) t = year (1, 2, 3, ...,m) g = pupil-cohort. SR". = Survival Rate of pupil-cohort g at grade i for a reference year k E} = Total number of pupils belonging to a cohort g at a reference year k Pg,i = Promotees from E}who would join successive grades i throughout successive years t RI = Number of pupils repeating grade i in school-year t. 59 Monitoring Educational Performance in the Caribbean Data required: Enrolment by grade for two consecutive years ( years t and t+1 ); num- ber of repeaters by grade for year t+1. Data source: School register, school survey or census. Type of disaggregation: Survival Rates can be disaggregated by gender, by geographi- cal location (region, urban/rural) and type of institution (private/public). It can also be disaggregated between survival with and without repetition. Interpretation: Survival Rate approaching 100% indicates a high level of retention and low incidence of drop-out. Survival Rate may vary from grade to grade, giving indica- tions of grades with relatively more or less drop-outs. The distinction between survival rate with and without repetition is necessary to compare the extent of wastage due to drop-out and repetition. Survival rate to grade 5 of primary education is of particular interest since this is commonly considered as pre-requisite for sustainable literacy. Quality standards: Since the calculation is based oa pupil-flow rates, the reliability of the Survival Rate depends on the consistency of data on enrolment and repeaters in term of coverage over time and across grades. Limitations: Given that this indicator is usually estimated using cohort analysis models that are based on a number of assumptions, care should be taken in using of the results in comparisons. Assumptions behind the cohort reconstruction model: The methodology of the reconstructed cohort flow model is based on the fundamental concept that for pupils enrolled in a given grade at a certain year, there could be only three eventualities: (a) some of them will be promoted to the next higher grade in the next school year; (b) others will drop-out of school in the course of the year ; (c) the remaining will repeat the same grade the next school year. Based on calculated flow-rates, a cohort of 1000 pupils through the educational cycle may be simulated, with a number of important assumptions: • that there will be no additional new entrants in any of the subsequent years during the life-time of the cohort, other than original cohort of 1000 pupils; * that, at any given grade, the same rates of repetition, promotion, and drop-out apply, regardless of whether a pupil has reached that grade directly or after one or more repetitions (hypothesis of homogenous behaviour); * that the number of times any given pupil will be allowed to repeat must be well defined; * that flow rates for all grades remain unchanged as long as members of the cohort are still moving through the cycle. Note: A comparison of the apparent cohort and reconstructed cohort methods shows that neglecting the repetition factor leads to an under-estimation of survivals and an over-estima- tion of drop-outs. Reliability of the enrolment and repeaters data: It is important to note that since data are not generally directly available on promotees and 60 Part ll: Annexes drop-outs, errors in the data available on enrolment and repeaters would affect the estimates derived for these two flows. Three common errors which may distort the flow rates can be described as follows: 1. Over-reporting enrolment/repeaters (particularly in grade 1): which can be deliberately done by school responsibles when there is a financial incentive, for example if the number of teachers paid by the government is related to the number of pupils enrolled. A different type of over-reporting occurs in countries where parents have incentives to register their children at school at the beginning of the school year, but where a large number of those registered do not attend school or only attend for a very brief period. 2. Incorrect distinction between new entrants and repeaters: which leads, other things being equal, to an under-reporting of repeaters in grade one and, like in point 1, to an over-estima- tion of drop-out from this grade. 3. Yearly variation in the coverage of the data: Assume that, for one reason or another, the data available for year t are complete while those for year t+1 are incomplete. Disregarding other types of errors, this implies that the number of promotees and repeaters in t+l will be under-estimated and the number of drop-outs over-estimated. If further the data for school- year t+2 are complete, this will imply that some of the promotees and repeaters that year were not included in the enrolment the previous year, leading to over-estimation of the pro- motion and repetition rates and under-estimation of the drop-out rate, which may be negative in some cases. While the type of error discussed under points 1 and 2 above are expected mainly to affect the flow rates for the first grade of primary education, incomplete data will naturally distort the rates for all grades. All these types of error lead to biases in indicators of internal effi- ciency. Since the drop-out rate is determined as a residual, it often serves as a test for some types of error: particularly, a negative drop-out rate is a sign of errors in the raw data i.e. enrolment and repeaters reported. Coefficient of Efficiency Definition: The ideal (optimal) number of pupil-years required (i.e. in the absence of repetition and drop-out) to produce a number of graduates from a given school-cohort for a cycle or level of education expressed as a percentage of the actual number of pupil-years spent to produce the same number of graduates. Input-Output ratio, which is the reciprocal of the coefficient of efficiency, is often used as an alternative. N.B. One school-year spent in a grade by a pupil is counted as one pupil-year. Purpose: This is a synthetic indicator of the internal efficiency of an educational system. It summarises the consequences of repetition and drop-out on the efficiency of the educational process in producing graduates. Calculation method: Divide the ideal number of pupil-years required to produce a number of graduates from a given school-cohort for the specified level of education, by the actual number of pupil-years spent to produce the same number of graduates, and multiply the result by 100. 61 Monitonng Educational Performance in the Caribbean Formula: n+k 2;Gg,j* CE 1 = , n__ _ _ __ _ __ _ _ 't(n4ic 'tl2l;g ij=1 For more details, see the flow diagram on cohort analysis. Where: CE1 = Coefficient of Efficiency for a pupil-cohort g G n = the number of pupils graduating from cohort g in final grade n after n years of study (without repetition) Ggi j = the number of pupils graduating from cohort g in final grade n after j years of study Dg,j = the number of pupils (of the cohort g) dropping out after j years of study. k denotes the number of repetitions allowed; n the prescribed normal duration of study for a cycle or level of education; g the pupil-cohort; and j the number of years of study. Data required: Number of graduates and drop-outs by length of study. These data can also be derived using the reconstructed cohort model, which requires enrolment by grade for two consecutive years (years t and t+1); number of repeaters by grade for year t+1 and number of graduates for year t. Data source: School register, school survey or census for data on repeaters and enrolment. hype of disaggregation: The Coefficient of Efficiency can be disaggregated by gender, by geographical location (region, urban/rural), and by school type (private/public). Interpretation: A Coefficient of Efficiency approaching 100% indicates a high overall level of internal efficiency and no wastage due to repetition and drop-out. Coefficient of Efficiency of less than 100% signals inefficiency due to grade repetition and drop-out. As the recipro- cal, the optimum input-output ratio is unity i.e. 1, and inefficiency arises from any point which is greater than one. Quality standards: Since the calculation of this indicator is based on pupil-flow rates, its reliability depends on the consistency of data on enrolment and repeaters in term of coverage over time and across grades. Differences in national regulations concerning the number of repetitions allowed constitute an aspect to be taken into account when using this indicator for inter-country comparisons. Limitations: Given that this indicator is usually derived using cohort analysis models that are based on a number of assumptions, and owing to the highly synthetic nature of this indicator, care should be taken in the use of the results in comparing education systems. From a conceptual viewpoint, having most pupils (or students) graduating within the prescribed duration of the cycle is optimal with regard to economic efficiency and resource utilization, but this does not necessarily imply achievement of the expected learning outcomes. Also, according to this calculation method, early drop-outs (i.e. from lower grades) can result in higher internal efficiency than late drop-out (i.e. from higher grades); this means that efficiency from the economic point of view can be in contradiction with educational objectives aiming 62 Part If: Annexes at retaining pupils in schools until higher grades when they would have acquired the desired knowledge and skills. Years-Input Per Graduate Definition: The estimated average number of pupil-years spent by pupils (or students) from a given cohort who graduate from a given cycle or level of education, taking into account the pupil-years wasted due to drop-out and repetition. N.B. One school-year spent in a grade by a pupil is equal to one pupil-year. Purpose: To assess the extent of educational internal efficiency in terms of the estimated average number of years to be invested in producing a graduate. Calculation method: Divide the total number of pupil-years spent by a pupil-cohort (graduates plus drop-outs) in the specified level of education by the sum of successive batch of graduates belonging to the same cohort. Formula: Y zGg,,* + Y_2 Dg,j * z+k E C4, J=1A For more details, see the flow diagram on cohort analysis. Where: YIGg = Years input per graduate (for graduates belonging to cohort g). Graduates from cohort g after j years of study k = number of grade (or duration) in the cycle. k denotes the number of repetitions allowed; n the prescribed normal duration of study for a cycle or level of education; g the pupil-cohort; and j the number of years of study. Data required: Total number of pupil-years spent by the pupil-cohort and the total number of graduates from the same cohort. These data can be derived using cohort reconstructed model, which requires enrolment by grade for two consecutive years (years t and t+1); num- ber of repeaters by grade for year t+ 1 and number of graduates for year t. Data source: School register, school survey or census or records. Type of disaggregation: The years input per graduate can be disaggregated by gender, by geographical location (region, urban/rural) and by type of institution (private/public). Interpretation: The closer the value of this indicator is to the theoretical number of grades (or duration) of the specified education cycle, the higher the internal efficiency and the lesser the negative effects of repetition and drop-out. A high number of pupil-years per graduate as compared to the normal duration, denotes waste of resources and hence inefficiency. Quality standards: Since the calculation of this indicator is based on pupil-flow rates, its reliability depends on the consistency of data on enrolment and repeaters in term of coverage 63 Monitonng Educational Performance in the Caribbean over time and across grades. Differences in national regulations concerning the number of repetitions allowed constitute an aspect to be taken into account when using this indicator for inter-country comparisons. Linitations: From a conceptual viewpoint, having most pupils (or students) graduating within the prescribed duration of the cycle is optimal with regard to economic efficiency and resource utilization, but this does not necessarily imply achievemeni of the expected learning outcomes. Also, according to this calculation method, early drop-outs (i.e. from lower grades) can result in higher internal efficiency than late drop-out (i.e. from higher grades); this means that efficiency from the economic point of view can be in contradiction with educational objectives aiming at retaining pupils in schools until higher grades when they would have acquired the desired knowledge and skills. 64 Part 11: Annexes Country Profile 1: EDUCATION IN BELIZE, 2000/20001 A: Education System Structure Age 4 * E _ 17 18 Grade _ I Enrollment* 8444 7605 748 7220 6815 6696 6077 5111 4555 36 8828 * Enrollments only correspond to Grade * Primary Level Secondary Level B: Public Expenditure on Education Total Education Expenditure as a % of GDP 6.2 Recurrent Education Expenditure as a % of Total Education Expenditure 80 Public Expenditure per Student (US$), primary 559 Public Expenditure per Student (US$), secondary 749 Note: Expenditure data reflect the actual spending (not budget). C: Coverage of the System Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade I (GIR) 102 117 90 _102_117 Net Enrollment Rate (NER) 99 99 98 40 43 38 Gross Enrollment Rate (GER) 113 112 114 54 57 52 Gross Completion Rate (GCR) 82 83 81 37 40 33 Expected Completion Rate (GIR*Survival Rate) 81 94 69 36 48 26 Out-of-school Population_(%) 0 0 0 36 36 36 _ |fMM Not Applicable 120 100 80 40 Total Ferrile MAale Total Feniale WIVe Prirriry Secondary * Net Enrollmrent Fate (!1\S) a Gross Enrollnrent Rate (GEl) U Gross Cornpleton Rate (G0P1) *Expected Cbrrpletton Rate (GlR'Survival Pate) 65 Monitoring Educational Plerformance in the Caribbean D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 79 80 77 Survival to Form 1 (Fl) 67 72 62 Survival to Form 4 (F4) 35 41 29 Prima =econdary Total Female Male Total Female Male Repetition Rate 9.4 7.8 10.9 8 7.5 8.5 Years Input per Graduate 10.1 24.4 22.3 27.4 Student-teacher Ratios 25 15 Survival Rates 110% - t , 90% 8 70%- - 50%- 0 c 0 2 30% - A|With Repeaters 0 o +Without Repeaters OL 10% II G 1 G2 G3 G4 G5 G6 Fl F2 F3 F4 Schooling Profile - 120% 100% 80% 0) 15 ~ IIG 2G 4G5G 1F 3F t co 60% co 20% c~0% IF 66 Part Mi: Annexes E. Education Quality Pass Rate of Primary Exams* Math 35 English 22 Pass Rate of Secondary CXC** (%) Math 45 English 70 % students passing at least 5 CXC_(** .. * Belize Natbonal Selection Exam at Grade 6. CXC: Caribbean Examination Council Including English and Math Examination Results 80 70 60 50 *R40- 30 20 10 0 Math Engish Math English Pass Rate of Rinary ExarTs Pass Rate of Secondary CXC Primary Secondary % Trained Teachers* 57 64 Class Hours per Year 900 1080 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: Only Government and Government Aided schools. Source: Ministry of Education (2002,2001) "Education Statis- tical Digest 2000-2001/1999-2000"; Ministry of Education (ad-hoc information requests); CXC (2001); UNPD (latest population projections). 67 Monitoring Educational Performance in the Caribbean Country Profile 2: EDUCATION IN DOMINICA, 1999/2000 A: Education System Structure Age 4il 1|ll!||||ilil_ 1 1 _ GradeX X1 1 Enrollment* | 72|1769 |1707 |1617 |1572 |1595 |1738 |1517 |1514 1286 1152 81| *Enrollments only correspond to Grade| Primary Level Secondary Level wSenior Primary B: Public Expenditure on Education | Total Education Expenditure as a % of GDP | 6.7 Recurrent Education Expenditure as a % of Total Education Expenditure 79 Public Expenditure per Student (US$), primary | 547 Public Expenditure per Student (US$), secondary 675 Note Expenditure data reflect the actual spending (not budget). C: Coverage of the System Primary | Secondary |_______________________________________ |Total Female Male Total Female Male Gross Intake Rate to Grade K (GIR) T _100 | _94 |_106 __ ; Net Enrollment Rate (NER) 85 82 88 73 [92] 81 [99] 66 [841 Gross Enrollment Rate (GER) 99 96 103 91 [109] 100 [1191 82 [100] Gross Completion Rate (GCR) 103 107 r 99 | 56 65 46 Ex ected Com letion Rate (GIR*Survival Rate) 94 89 96 57 55 58 Out-of-school Population(%) 1 2 |_1 5 | 1 0 | [3] | [0]1 _Ak lW Not Applicable [I Include Senior Pnmary (grades 7-9) 120 100 80- OR60 40- 20- Total Femiale Male Totalerale le PrirnEry Secondary | Net Enrollnsrnt Rate (NER) 1: Gross Eirollnent Rate (GER) o3 Gross Conrpletion Rate (GCR) 13 Expected Conrpleton Rate (GlR*Survival Rate) 68 Part 11: Annexes D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 94 95 91 Survival to Form 1 (Fl) 68 71 65 Survival to Form 5 (F5) 57 1 58 1 55 1 Prima Seconda .________________________ Total Female Male Total Female Male Repetition Rate 2.2 2.1 2.3 11 8 14 Years Input per Graduate 7.4 7.3 7.5 18 17 18 Student-teacher Ratios 20 18 Survival Rates 100%- t: 90% - 0 8 80% - _. 70% - ai 60% - 50% - C o 40% _ ° 30% | With Repeaters 2 3 +Without Repeaters 0. 20%0o K Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 Schooling Profile ,;c 110% - cc g 100% - 90%- 80% , $ 70% co -t 60% co 2n 5o0%*I I K G1 G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 69 Monitoring Educational Performance in the Caribbean E. Education Quality (2001) Pass Rate of Primary Exams* Math _ 2_ 5 English 40 Pass Rate of Secondary CXC** (%) Math 59.2 English 84.1 % students passing at least 5 CXC_(***) Grade 2 National Assessment CXC: Caribbean Examination Council Including English and Math Examination Results 100 80 60 40 20 0- Math English Math English Pass Rate of Primary Exarns Pass Rate of Secondary CXC 1999/2000 Primary Secondary °%Trained Teachers* 61.1 32.6 Class Hours per Year 930 975 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: All schools included. Source: Ministry of Education (2002,2000) "Indicators 2002/2000"; Ministry of Education (ad-hoc information requests); CXC (2001); National Statistical Office-(latest population projections). 70 Part 11: Annexes Country Profile 3: EDUCATION IN DOMINICAN REPUBLIC, 1999/2000 A: Education System Structure Grade Enrollment* = |269592 1 256704 | 26392371 2283687118821371155749 | 1306341| 115627 | 123565 | 97303 1 130317 | 697607 l * Enrollments only correspond to Grade. " Primary Level XSecondary Level B: Public Expenditure on Education Total Education Expenditure as a % of GDP 2.3 Recurrent Education Expenditure as a % of Total Education Expenditure 95.7 Public Expenditure per Student (US$), primary 134 Public Expenditure per Student (US$), secondary 140 Note: Expenditure data reflect the actual spending (not budget). C: Coverage of the System Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade 1 (GIR) 139 131 148 Net Enrollment Rate (NER) 89 89 90 27 31 24 Gross Enrollment Rate (GER) 107 105 109 54 60 48 Gross Completion Rate (GCR) 62 67 56 41 48 35 Expected Completion Rate (GIR*Survival Rate) 83 88 78 58 66 52 Out-of-school Population (%/.) 10 11 10 38 37 39 25>iW> Not Applicable 120 100 60 4 0 40 Total Female Male Total Female Male Primary Secondary E Net Enrollment Rate (NER) O Gross Enrollment Rate (GER) a3 Gross Completion Rate (G CR) a Expected Completion Rate (GIR*Survival Rate) Monitoring Educational Petformance in the Caribbean D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 8 (G8) 60 67 53 Survival to Grade 1 of Secondary (Gl) 59 66 53 Survival to Grade 4 of Secondary (G4 42 50 35 Primary Secondar Total Female Male Total Female Male Repetition Rate 5.0 3.9 6.1 3.2 2.5 4.2 Years Input per Graduate 9 7 9.1 110.5 19.9 17.9 22.4 Student-teacher Ratios 36 28 Survival Rates 110% - 0 0 70% \ C.) 50% - 0 t 30% - With Repeaters 0 0- o -4Without Repeaters ~-10% Gl G2 G3 G4 G5 G6 G7 G8 Gl G2 G3 G4 Schooling Profile ~140% 2 120% * 100% rn 80% 60% CD, GI G2 G3 G4 G5 G6 G7 GB Gi G2 G3 G4 72 Part II: Annexes E. Education Quality (2001) Pass Rate of Primary Exams* Math 41 English 71 Pass Rate of Secondary Exams** (%) Math 25 English 50 * Pruebas Nacionales tested at G8 Pruebas Nacionales tested at G4 of Secondary Exanrnation Results 80 60 e,40 20 0 Math English Math English Pass Rate of Rimary Exams Pass Rate of Secondary Exas Year 1999/2000 Primary Secondary % Trained Teachers* 79 75 Class Hours per Year 1050 1290 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: All schools included. Source: Ministry of Education (2001): "Estadisticas e Indicadores Educativos 1998-1999"; Ministry of Education (ad-hoc information requests); El Nacional (2002); UNPD (latest population projections). 73 Monitoring Educational Performance in the Caribbean Country Profile 4: EDUCATION IN GRENADA, 2000/2001 A: Education System Structure Age 417 1 _ Grade _Enrollment* 1 978 | 2146 7 2283 | 2352 7 2317 |2424 |2474 |2090 |2118 |2085 |1947 |1651| | | *Enrollments only correspond to grade 1567 1414 17 Primary Level Secondary Level Senior Primary B: Public Expenditure on Education Total Education Expenditure as a % of GDP 6.8 Recurrent Education Expenditure as a % of Total Education Expenditure 64 Public Expenditure per Student (USS), primary 378 Public Expenditure per Student (US$), secondary 392 Note: Expendkture data reflect the actual spending (not budget). C: Coverage of the System Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade K (GIR) 120 121 120_____ Net Enrollment Rate (NER) 99 98 99 62 [87] 72 [90] 52 [84] Gross Enrollment Rate (GER) 110 108 113 82 [1081 93 [112] 70 [1031 Gross Completion Rate (GCR) 106 104 107 84 92 76 Expected Completion Rate (GIR*Survival Rate) 108 103 113 79 92 66 Out-of-school Population (%) 0 0 0 lo] I [0] __ _ M Not Applicable [ ] Include Senior Pnmary (Grades 7-9) 120 100 80 oA 60 40 120 - Total Fenrie Male Total Femrole Male Prirrary Secondary * bt Enrollrrent Rate (NER) a Gross Enrollrrent Rate (GEF9 oB Gross QOrrpletion Rate (GCFO * Expected Corrpletion Pate (GlIRSurvival Rate) 74 Part 11: Annexes D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 90 85 94 Survival to Form 1 (Fl) 73 81 62 Survival to Form 5 (F5) 66 76 55 Primary Secondary Total Female| Male Total Female Male Repetition Rate 2.8 1.9 3.6 1.3 0.8 2.0 Years Input per Graduate 7.7 7.6 15.8 13.9 18.5 Student-teacher Ratios 25 23 Survival Rates 110% t 100% - 8 90% Ca 80% (D 70% - 60% - 0 o 50% I= + With Repeaters 0 ° 0 _ +Without Repeaters| CL 30%- K Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 Schooling Profile r130% 120%- 110% - 100% Co90% !R 80% c 70% W 60% C!3 K G1 G2 G3 G4 G5 G6 F1 F2 F3 F4 F5 75 Monitoring Educational Performance in the Caribbean E. Education Quality Pass Rate of Primary Exams* Math 21 English 25_ 5 Pass Rate of Secondary CXC** (%) Math 23.6 English 56_.8 % students passing at least 5 CXC (***) 13 Minimum Competency Test at Grade 4 ** CXC: Caribbean Examnation Council Including English and Math Examination Results 60 50 40 at 30- 20 - 10 - 0 kvftth Engrish NWth English Pass Rate of Rirrary Exarrs Pass Rate of Secondary CXC Primary Secondary Prmry ry I % Trained Teachers 70 31 Class Hours per Year 878 1040 Trained teachers are those who have received the minimum organized teacher training requirements Notes: Only Government and Government Aided schools. Source: Ministry of Education (2002): "Education Statistical Digest"; Ministry of Education (ad-hoc information requests); CXC (2001); National Statistical Office (latest population projections). 76 Part H1: Annexes Country Profile 5: EDUCATION IN GUYANA, 1999/2000 A: Education System Structure Age 4 i_l| l 17 |18| Grade.l l 1ll Enrollment* |+ 1193931 17665| 17895| 18284| 17466| 16504 16822 143031 132761 11709| 6020] * Eniollments only correspord to Grade Primay Level gSecondary Level B: Public Expenditure on Education Total Education Expenditure as a % of GDP 7 3 | Recurrent Education Expenditure as a % of Total Education Expenditure 61 Public Expenditure per Student (US$), primary T Public Expenditure per Student (US$), secondary Note: Expenditure data reflect the actual spending (not budget) C: Coverage of the System Primary | Secondary Total Female Male Total Female Male Gross Intake Rate to Grade 1 (GIR) 103 105 102 1V_W*- | Net Enrollment Rate (NER) 84 81 87 60 Gross Enrollment Rate (GER) 100 99 101 74 78 69 Gross Completion Rate (GCR) 92 | 93 92 34 39 | 29 Expected Completion Rate (GIR*Survival Rate) 87 88 86 35 42 29 Out-of-school Population (%/6) 16 19 13 3 * |_*_| _ l Not Applicable 120 100 860 Total Ferrele Male Total Ferrale Male Primary Secondary | Net Enrollment Rate (NER) a Gross Enrollnment Rate (GER) |3 Gross Corrpletion Rate (GCR) 13 Expected Complebon Rate (GIR*Survival Rate) 77 Monitoring Educational Performance in the Caribbean D. Internal Efficiency Survival Rate Total Female -Male Survival to Grade 6 (G6) 84 84 84 Survival to Form 1 (Fl) 82 83 _ 81 Survival to Form 5 (F5) 34 40 _ 28 Primary Secondary _ Total Female Male Total Female Male Repetitlon Rate 2.4 2.1 2.7 10 9 11 Years input per Graduate 6.9 6.9 6.9 20.8 19.3 22.3 Student-teacher Ratios 27 18 Survival Rates 110% - t 90% = 0 *.a 70% - - 50%- 0 t 30% - -With Repeaters 2 - Without Repeaters 10%/ Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 Schooling Profile G 120% g 100% _ 80%- S 60% i 40% - co Z 20% 0% O Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 78 Part 11: Annexes E. Education Quality (2001) National Mean Scores of Primary Exams* Math | 43 English J 36.4 Pass Rate of Secondary CXC** (%) Math 19.5 English 36.1 % students passing at least 5 CXC * * Secondary School Entrance Examination at Grade 6. CXC: Caribbean Examination Council Including English and Math Examination Results 50 40 30 20 10 0 I Math English N!th English Mean Scores of Priary Exarms Pass Rate of Secondary CXC Year 1999/2000 Primary Secondary % Trained Teachers* 50 54 Class Hours per Year** 965 965 * Trained teachers are those who have received the minimum organized teacher training requirements These hours represent an average of both primary and secondary annual hours. Notes: All schools. Source: Ministry of Education (2002,2000): "Digest of Education Statistics of Guyana 1999-2000/ 1998-1999"; Ministry of Education (ad-hoc informnation requests); UIS (ad-hoc information requests); CXC (2001); UNESCO/OREALC (2001); UNPD (latest population projections). 79 Monitonng Educational ,Performance in the Caribbean Country Profile 6: EDUCATION IN JAMAICA, 1999/2000 A: Education System Structure _ :Age -4 _**E1 _ Grade _1 _ Enrollment* l 600801 55416 56250 62251 l4646t1 51953l 44884 48939 51990 39359 36578 Enrollments only correspond to Grade Primary Level Secondary Level B:: Public Expenditure on Education L Total Education Expenditure as a % of GDP 7.2 Recurrent Education Expenditure as a % of Total Education Expenditure 95 Public Expenditure per Student (US$), primary 313 Public Expenditure per Student (US$), secondary 460 Note Expenditure data reflect the actual spending (not budget). C:: Coverage of the System _________ Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade 1 (GIR) 102 102 103 Net Enrollment Rate (NER) 98 98 98 78 79 77 Gross Enrollment Rate (GER) 102 102 103 86 87 85 Gross Completion Rate (GCR) 95 99 _ 91 68 72 64 Expected Completion Rate (GIR*Survival Rate) 93 96 90 63 68 59 Out-of-school Population (%/o) 1 0 _ 1 18 17 18 _ "Not Applicable 120- 100 80 40- 20 0 ~ Total Ferrale tvaIe Total Fermle Ile Primiary Secondary r. Net Enrollennt Rate ((NER) 03 Gross Enrollrrent Rate (GER) _______ [9Gross Corrpletion Rate (GCR) M Expected Ornpletion Rate (GIR*Survival Rate) 80 Part 11: Annexes D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 91 94 87 Survival to Grade 7 (G7) 91 94 87 Survival to Grade 11 (G1l) 62 67 57 Primary Seconda Total Female Male Total |Female Male Repetition Rate 5.5 3.8 7.1 1.6 1.0 2.2 Years Input per Graduate .8 6.5 7.0 16.0 15.0 17.0 Student-teacher Ratios 30 18 Survival Rates 110% - 0 90%/ CD 70%- 0 c o 50%- :E A With Repeaters CL _ Without Repeaters 0- 30% Gl G2 G3 G4 G5 G6 G7 GB G9 G10 Gil Schooling Profile % 120% - > 110% 2, 100% 90% a aD l 80% - EC .e 70% - CU C- 60%- to 2 50% O Gl G2 G3 G4 G5 G6 G7 G8 G9 G10 G1l 81 Monitoring Educational Performance in the Caribbean E. Education Quality (2001) National Mean Scores of Primary Exams* (2000) Math 49 English 58 Pass Rate of Secondary CXC** (%/) Math 30 English 58 % students passing at least 5 CXC *** _. Grade Six Achievement Test. These are mean scores by subject which are marked out of 100 points CXC: Caribbean Examination Council Including English and Math Examination Results 70 60 - 50 40 30 20 10 0 Meth English Meth English Mean Scores of Prirary Exarrs Pass Rate of Secondary CXC Year 1999/2000 Primary Seconda % Trained Teachers* 81 78 Class Hours per Year** 950 950 Trained teachers are those who have received the minimum organized teacher training requirements These hours represent an average of both primary and secondary annual hours. Notes: All schools. Source: Ministry of Education (2000,1999): "Education Statistics 1999-2000/1998-1999"; Ministry of Education (ad-hoc infornation requests); CXC (2001); UNPD (latest population projections). 82 Part 11: Annexes Country Profile 7: EDUCATION IN ST KITTS AND NEVIS, 2000/2001 A: Education System Structure Age 4 17 18 Grade X = Enrollment* 965 | 931 7 945 | 917 1 979 102-2 1025 | 949 | 947 985 1033 | 657 * Enrollments only correspond to Grade. IPrimary Level Secondary Level B: Public Expenditure on Education Total Education Expenditure as a % of GDP 6.2 Recurrent Education Expenditure as a % of Total Education Expenditure 72 Public Expenditure per Student (US$), primary 861 Public Expenditure per Student (US$), secondary 1076 Note: Expenditure data reflect the actual spending (not budget). C: Coverage of the System Primary _ Secondary Total Female Male Total Female Male Gross Intake Rate to Grade K (GIR) 94 103 87 Net Enrollment Rate (NER) 98 98 98 89 88 91 Gross Enrollment Rate (GER) 104 104 104 100 100 101 Gross Completion Rate (GCR) 110 104 115 67 76 59 Expected Completion Rate (GIR*Survival Rate) 85 91 79 57 68 48 Out-of-school Population (%/.) 0 0 0 9 10 7 now Not Applicable 140 120 Total Fenqale M~abe Total Ferrale N~ele PrinEry Secondary | Net Enrolln ent Rate (NER) C] Gross Enrolln ent Rate (GEFl |oE Gross Cbrrpletion Rate (GCF9 U Expected Conpletin Pete (GIR*Survtval R ate)| 83 Monitoring Educaitonal Plerformance in the Canbbean D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 90 88 91 Survival to Form 1 (Fl) 90 88 91 _.. Survival to Form 5 (F5) 61 66 55 Prima Seconda __________________________ Total Female Male Total Female Male Repetition Rate 3.0 1.6 4.0 5.2 5.8 4.5 Years Input per Graduate 7.6 7.5 7.6 17 16 18 Student-teacher Ratios 19 13 Survival Rates 100% - t 90%- 0 8 80%- co 70% - a) 60%- qO 50%- .o 40%- -i+With Repeaters oL 30%- 2 -3-Without Repeaters a- 20% - I K Gl G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 Schooling Profile C 100% -__. >90% 2 80% _ cc 70%- 60% 2 50%_ o Ca K G 1 G2 G3 G4 G5 G6 F1 F2 F3 F4 F5 84 Part II. Annexes E. Education Quality Pass Rate of Primary Exams* Math 29 English 62.3 Pass Rate of Secondary CXC** (°h) Math 43.2 English 75.4 % students passing at least 5 CXC (***) 26 * Test of Standards at Grade 6 CXC: Caribbean Examination Council Including English and Math Examination Results 80 70 60 50 ~40 30 20 10 0 Nath English Math English Pass Rate of Rinrary Exams Pass Pate of Secondary CXC 2000/2001 Primary Secondary % Trained Teachers* 55.7 29 Class Hours per Year 1404 1482 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: All schools. Source: Ministry of Education (2002,2001): "Statistical Bulletin 2000-2001/1999-2000", Ministry of Education (ad-hoc informnation requests); CXC (2001); National Statistical Office (latest population projections). 85 Monitoring Educatiowal Performance in the Caribbean Country Profile 8: EDUCATION IN ST VINCENT AND THE GRENADINES, 2000/2001 A: Education System Structure Age 4__1 GradeM Enrollment* 2537 2688 2625 2738 2568 2511 2532 1556 1779 1639 1522 1371 *Enrollments only correspond to Grade. Primary Level Secondary Level Senior Primary B: Public Expenditure on Education Total Education Expenditure as a % of GDP 5.8 Recurrent Education Expenditure as a % of Total Education Expenditure 90 Public Expenditure per Student (US$), primary 506 Public Expenditure per Student (US$), secondary 498 Note Expenditure data reflect the actual spending (not budget). C: Coverage of the System ________ Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade K (GIR) 108 105 111 _ll Net Enrollment Rate (NER) 98 96 101 54 [701 63 [75] 45 [65] Gross Enrollment Rate (GER) 111 107 114 67 [83] 77 [90] 57 (761 Gross Completion Rate (GCR) 84 84 85 51 61 41 Expected Completion Rate (GIR*Survival rate) 78 87 64 45 56 30 Out-of-school Population (°h) O O O [1 91 [161 [22 ON= Not Applicable []Include Senior Pnmery (Grades 7-8). 120 100 80 *60 40 Total Fen-ale Male Total Femrale Male Prirrary Secondary 3 Nbt Enrolirnant Rate (NER) 13 Gross EnrollrrLant Rate (GEP) [3 Gross CQrrpletion Rate (GCFO a Expected CorTpletion Rate (GIR*Survival rate) 86 Part 11: Annexes D. Internal Efficiency Survival Rate Total Female Male Survival to Grade 6 (G6) 72 83 58 Survival to Form 1 (Fl) 52 62 37 Survival to Form 5 (F5) 42 53 27 Primary Seconda rvY Total Female Male Total Female Male Repetition Rate 13.5 12.0 15.0 9.5 8.1 11.3 Years Input per Graduate 9.1 9.9 23 20 228 Student-teacher Ratios 21 19 Survival Rates 110% v 100% 0 ~90% 8 80%- < 70%- \o CD 60% - 50%- 0 C 40%- 0 *E 30% - a--- With Repeaters 0 O 20% - + Wthout Repeaters .L 10% K G1 G2 G3 G4 G5 G6 Fl F2 F3 F4 F5 Schooling Profile at 110% ' 100% - 90% - rn 80% 70% - . 60% - c 50%- g 40%- K G1 G2 G3 G4 G5 G6 F1 F2 F3 F4 F5 87 Monitoring Educationail Performance in the Caribbean E. Education Quality Pass Rate of Primary Exams* Math 75 English _ 75 .9 Pass Rate of Secondary CXC** (%/)o) Math 37.5 En lish 63.9 % students passing at least 5 CXC_(***) Common Entrance Examinations at Grade 6 ^^ CXC Canbbean Examination Council Including English and Math Examination Results 80 70 60 50 40 30 20 10 0 Nlth English NVth Eniglish Pass Rate of Primary Exarm Pass Rate of Secondary CXC Primary Secondary % Trained Teachers I 71.8 T 85.9 Class Hours per Year 975 1072 * Trained teachers are those who have received the minimum organized teacher training requirements Notes: Only Government and Government Assisted schools. Source: Ministry of Education (2001): "Directory of Schools and Colleges 2000/2001"; Mimstry of Education (ad-hoc infornation requests); CXC (2001); UTNPD (latest population projections). 88 Part 11: Annexes Country Profile 9: EDUCATION IN TRINIDAD AND TOBAGO, 1999/2000 A: Education System Structure AAe 4 1 1 Grade 1203267I 11 07 Enrollment* | Tt6126723309| 22973 723 99 253308712|078 202747181360|196127 1 *Enrollments only correspond to Grade ig ! 6686 68 Primary Level Secondary Level Senior Primary B: Public Expenditure on Education Total Education Expenditure as a % of GDP 7.7 Recurrent Education Expenditure as a % of Total Education Expenditure 80 Public Expenditure per Student (US$), primary 642 Public Expenditure per Student (US$), secondary 795 Note: Expenditure data reflect the actual spending (not budget) C: Coverage of the System Primary Secondary Total Female Male Total Female Male Gross Intake Rate to Grade K1 (GIR) 96 _97X _ 95 Net Enrollment Rate (NER) 93 95 93 64 [71] 65 62 Gross Enrollment Rate (GER) 104 106 104 71 [80] 73 69 Gross Completion Rate (GCR) 94 94 94 62 63 60 Expected Completion Rate (GIR*Survival Rate) 93 95 93 54 55 52 Out-of-school Population I(%) _ _3 1 3 26 25 _ _27 -4 I-^Ail cRNot Applicable lInclude Senior Pnmary (standards 6 and 7) 120 100 80 40 20 TotalI Female Male Total Female Male Primary Secondary | Net Enrollment Rate (NER) a Gross Enrollment Rate (GER) |EIGross Completon Rate (GCR) M Expected Completion Rate (GlR-Survival Rate) 89 Moniitoring Educational Perfornance in the Caribbean D. Internal Efficiency Survival Rate Total Female Male Survival to Standard 5 (S5) 97 98 98 Survival to Form 1 (Fl) 82 88 78 Survival to Form 5 (F5) 56 57 55 Prima | Secondary Total Female Male Total Femalel Male Repetition Rate 7.8 T7.3 8.2 2.0 2.0 2.2 Years Input per Graduate 7.7 7.6 7.7 18.7 18.5 18.8 Student-teacher Ratios 21 17 Survival Rates 100% t0 90% \ 0 880% - A: 70% - a, 60% - ~-50%- 0 o 40% ° 0%With Repeaters a30%- o --Without Repeaters a0 20%- Kl K2 Si S2 S3 S4 S5 Fl F2 F3 F4 F5 Schooling Profile z 100% a: *V 90% = 80%% % s 70% C)60%- c 50% 2 40% D KK1 K2 S1 S2 S3 S4 S5 F1 F2 F3 F4 F5 90 Part 11: Annexes E. Education Quality (2001) Pass Rate of Primary Exams* - Math 68 English 63 Pass Rate of Secondary CXC** (%) Math 44 English 67.7 % students passing at least 5 CXC (***) .. * Secondary Entrance Assessment at Grade 6 CXC: Caribbean Examination Council Including English and Math Examination Results 80 70 60 50 30- 20- 10 0 Pass Rate of Rirmry Exarm Pass Rate of Secondary CXC Primary Secondary %Trained Teachers*, 1999/2000 76 Class Hours per Year** 1050 * Trained teachers are those who have received the minimum organized teacher training requirements Taken from PRIE report (data of 1998) Notes: All schools. Source: Ministry of Education (ad-hoc informnation requests); UIS (ad-hoc information requests); CXC (2001); UNESCO/OREALC (2001); UNPD (latest population projections). 91 14. V 10 Q C) 4. C-i 00 u 'o Cd Ca cq cn 0 ed 0u cn > oo a% W) (7% 00 L) 4. IJ, 11 'H t 0 Ca r- . 52 C) R r 4. C- CL, oo 00 0, MI tu 0 L) C, > o cq tn C> 0 E U o, C\ C\ 00 a\ > 0 r cn ON cn cn 00 4. ci L4) 0 W) 00 .2 =0 ON W) d) 'Ei C) 5 - n c, r- (ON cn 0 ON rt W) o A -o 0 fd -4 V C\ 00 (ON cr% 00 C ON Cl) 000 0 kn4. 0 -0 -6 CDN N Cr, 00 00 00 r- c tn u CL cl C) lo a, ON CN ON o C\ 00 00 ON C 'OCZ0 0 o z t4 (U CN W) CII, C 00 00 00 00 c) C% 00 00 O\ 00 a\ C\ ON -'O ' 'E 7; 2 c, o 00 (Z) cq Cl) in A bo > -g u 121 0 0 U 4. 0 oN Z C) 4.0 00 00 00 C\ CD cn C14 4. !2 C-4 c, o ON (O (O 0 41.1 0 4 '0' 4. 0 0 o too clq zo o, N N 0 o 00 (ON cr, a, 8 cq cq C14 OU IQ 0 Ca v co Cs 11 ix co ;4 U 0 co 'W2 eQ 2 'c Z 0 E* E* Z A C, 93 __ - F oo z (ON >c 8 ° 8ie o^ ^o gX o_ in o2... 0 X _ so __ _ _ _ _ _ oc W) o 0 4 m C) r- CN 0' - Co CD te) v 0 cs O\ cs > r _ B - - 3o - C 0 z~~~~~~~~~~~~c tn cn \D 8o ol z o~~~~~~~~~~~~c 00 2 Sh tn x _E v 00 S _ t 8 > > > m > X ,~~~%C 0 *0 O _0Cl0\Cl _ st N r > r N o-3 8 9 U,~~~~~~~~~~~~~~~~~~~~~~~~- o! . U &Sao8M8|8E% _ ) -- 0 ~~ 00~~~~0~~tn ~~~ N ~~~~ 00 - Cl o~~~~~~~0 00 Ndr m ~~ w ___a__ c ' 1. X X 1 I w g t en 0 Cl 0;N 00 t- U, - _ _~~~~~~~~~~~~~~~' 0 ~~~~~~~~~~~~~Zt 0 00 t- -~~~ Ir~~~ N00 N N -~~~~~~ Cu -~~~ Cl N ~~~O N ' 0. 00 o 0~~~~~~~~~~~~~ In~~~~~~~qc 000 , -0 ONNsO C 19~ ~~~C N0 I ~ ~ ~ ~ ~ ~ ~ 0 Cl~~~~~; W. o~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~a 0'. 0 '.~~~~~~~o, W ~ -~ cOO'. 0000 N94 W) tn v) 00i t- F M ^ b- W ,n _f) Xo n C14 oo W) otinl oo 00~~~~~~~~~~ in 0t C - c "t 00 '0 e'~ W 0 e 0 000t-%00r - ~ 00 - U > %C oo ^ t X oo o 0% U m - 0 o £ 00 o g _ S o;00 0%0%0 _ c 00 0 _~ In C 0 l 0 0 00 '0 - 00 ON N% 0% 00 a >~~~~~~~~ rU%- 0n 0%0 cI t0- 0t- 0% m .0 -.8 ;t (7-N IOR CWRI t- C.9 0% 0% 0%~~~~~~7 95 0 '0~~~~~~~~~~~~~~~~~4 C2 4 ~ ci iq ..4 C~ 00 1 0 o po N cf) N Cl)~. cO0 C'i oq 0 WI q N W) C4 n t4 o .~~~~~~~~~~~~~~~~~~~~~7 96 i o8~~~~~~~~~~> C.).:, 0 X a u7 0000 oo c_ 000 c O _ O0 C r~- - c-' c1 --r 6 C0 C,- - 4 - - - - - - 0~ *0 U C,0,4: ON (7 C,, 0 00 . N B - C, O exn__ ON _n _) _c _ - O 0 z;i0( C, a mm _ ^ _ mo O - - 0 - _ -- No_ __ -_ 0_. .0 OC? ~ ~ ~ ~ ~ a o 0 cfI \\ C 00 4 o A Y= q C'i c i ~r c 7 eq C)- - -) o…- ___~C C c:),C EnC 0~~~~C CCCDaCa C 0 .. C)ONC) C O 0 N C0j ci (ON~ 9 '0 ci ci ON C cq N ~ crq 0 c U 00~~~l 1 t U -0 0- 0--C -n 0 0000 0 00000 0 a ~~~ci ici c C-- - C-- - 97 . f X - - . S s~~~~~~~~~~~~~~~~~~~~~~~~~Q L t~~~~~ ~~~~~~~OC)t S d J o o _ o c _ ^ X o o E s~ vi ,_2; N t F N ct un s on > \0 S O~~~~~~~Z . _ i_ : Do > s : C ~~~~~~ ~~ ~ ~~~~~~~C.) CS O 1) oO E5t N 0 X tn _C DO C1 S r IlS C1 i; cn tn | o m Cd~~~~~~~~~~~ g8~~~~~ 0= 0~~~~~~~~~~~~~~~~0 ~~~~~~~~~~~~~~or 8 __ _ - X - - - b.~~~~~~~~~~~~~~~~~~~~~~~~~c tn D, .S - c' ci c~ eq e9 o I0 ~~~~~~~4-i ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~4 o -. r- 000\~~~~~ 0 r~9 0~~0 Cu Of LflN''If 0 Cu~~~~~~cqW-I 00 ON Or- oi o ~~~~ -~~~~ n oO-N CD 000 8 ~~~~~~~~~~~ S - - -~~~~~~~~~ 0~~~~~~~~~~~~ Cu 0~~~~~~~~~~~~~~ S~~~~. r- ) 0C 0 0 10, eq ~ 10 B Y O _ 8 S o 8 8 S _ - > 8C C O aifuE r- cn 8 8 n o :~~~~~~4 _n j! t I:cv _C h V ;t~~~~~~) 0i Wc Ct o , r- t O 00oo° ._~~~~~~a Cl O h V 0 7eSOOOe ; 15: D _ _ _ _ _D_\E o cs B e4 8 > w~~~~~~~~o od ON r i U >4 X Ch t r-Xc v7 _ v7 o v7 kt7 t -n r C5 u t a a X a a a a a a~~~~c n X) =~~~~~r ON _ _o _ t 101~~~ Monitoring Educational Performance in the Caribbean MONITORING SCHOOL EFFECTIVENESS: Selecting appropriate qualitative indicators (Meier Werner) This paper was prepared for the UIS Regional Workshop on Education Indicators, Trinidad and Tobago, May 6-9w, 2002, with the purpose of participating in the on-going discus- sion on education indicators in the Caribbean region and supporting the OERU initiative on the selection, measurement and monitoring of qualitative indicators Executive Summary' The complete paper is available in the project's files. 102 Part 11: Annexes While the Caribbean Region has achieved considerable progress in education reform in the last decade of the 20'h Century, rapid changes in the regional economy require further development of the human resource base so as to compete in the global economy of the next century. Continuous improvements in the quality of eclucation are required to continue the transition from a natural resource based economy to a human resource and knowledge-based economy. Effective management of information about the performance of education systems has been identified as a critical weakness and an essential component to future development. With the intention of beginning a consensus building process around a common set of qualitative indicators for the English-speaking Caribbean, the World Bank commissioned this discussion paper for presentation to Education Plarners at the UIS Meeting in Trinidad and Tobago on May 7t, 2002. The paper reviewed the available international research evidence on the relationship between school and classroom-level variables and school effectiveness and proposed a conceptual framework for the organisation of determinant factors and indicators of school effectiveness. School effectiveness research (SER) will refer here to the collective body of knowledge generated by: equal opportunity studies, education production functions studies, school effects research, effective schools research and school improvement programs. What these strands of research have in common is the search for independent variables with the greatest demonstrated effect on student achievement2. Although the construct of 'student achievement' as the dependent variable was operationalised in many different ways, it usually involved a measure of cognitive ability appropriate to the student's level of education using standardised international or national tests. Since there are significant differences in the evolution and findings of SER between developed and developing country research, the paper examines their respective bodies of knowledge separately. The SER evidence from developed countries reveals steady advancements in methodology from regression analysis to the more sophisticated multilevel modelling. Over the years, several researchers have applied a variety of quantitative analysis techniques to large samples of individual SER studies conducted, for the most part, in the USA, Great Britain and The Netherlands. Four key meta-analyses are assembled in table format (Table 2) in order to identify the independent variables significantly correlated with school effectiveness. The first is a synthesis of 134 meta-analyses covering 7,827 individual studies (Fraser 1989). The second is a statistical summary of 91 meta-analyses, in addition to expert opinion and content analysis (VVang, Haertel and Walberg 1993). The third is a statistical meta-analysis summarising results from 103 multilevel studies (Bosker and Witziers 1996). The fourth is a statistical summary of three meta-analyses covering 14-38 studies per variable (Scheerens and Bosker 1997). Based on the analysis of the evidence presented in these meta-analytic studies, the most important proximal indicators of student achievement are those related to: the student's attributes, parental support for education, classroom management, teacher-student interaction, and the quality of the instructional methods and techniques used. Important, but somewhat 2 The term student achievement is preferred here over alternate terms such as student performance and student outcomes that are also commonly used in the literature. 103 Monitoring Educational Performance in the Caribbean more distant indicators are those related to: the socio-economic status of community/family, peer group aspirations, curriculum design, and monitoring and evaluation at the Ministry, school and classroom levels. The SER evidence from developing countries varied considerably in research design, from predominantly production function studies to some recent multilevel analyses. Contrary to the findings in OECD countries, this body of research produced a surprisingly high number of significant correlations for school input and process variables. Four key meta-analyses are assembled in table format (Table 3) in order to identify the independent variables significantly correlated with school effectiveness in developing countries. The first is a review of 26 studies that used a research design (Schiefelbein and Simmons 1981). The second is a review of 72 studies that also used multivariate analysis while controlling for family background (Fuller 1986). The third is a review of 100 input-output studies conducted prior to 1991 that used some form of regression analysis (Hanushek 1995). The fourth is a review of 100 multivariate studies, 46 of which were conducted since 1987, that controlled for family background and defined student achievement in terms of acquiring basic literacy and cognitive skills (Fuller and Clarke 1994). Based on the analysis of the evidence presented in these meta-analytic studies, the most important indicators of student achievement are those related to: the student's health, parents socio-economic status, teacher's background, classroom management and instruction, school management by the principal, as well as numerous school input indicators. Based on these research findings, we can assume that the effect of school inputs on student achievement would diminish as schools approach adequate resource levels. Once this condition has been achieved, then school and classroom process variables appear to have a more significant effect on student achievement. Consequently, education policy-makers, educators and development organisations active in the Caribbean region would be remiss to simply accept the external validity of the conclusions drawn from SER conducted principally in OECD countries, but would be well advised to identify those findings with face validity for their socio-cultural context. Selectively combining the two knowledge bases was critical to the development of a conceptual framework for school effectiveness. To develop an integrated model of school effectiveness it is important to establish a common meaning for the construct itself. The following definition is therefore proposed for the purposes of this paper: School effectiveness is defined as the school's effect measured over time on student achievement and attainment, after controlling for contextual and individual variables. While this definition must be operationalised at the individual student level, the results can be aggregated at the cohort, subject, classroom or school level. Multilevel modelling now allows for measurement at different levels over time. An integrated model of school effectiveness will undoubtedly include several levels of analysis organised into a system model of education. The need for an integrated model of school effectiveness has become increasingly critical as 104 Part IL Annexes the trend towards school-based management and the use of performance indicators continues. The paper proposes a conceptual framework based on the review of the SER literature and the summary of evidence. Figure 3 below presents a three-tiered model of school effectiveness composed of the context, school and classroom level factors. It attempts to replicate reality in that the classroom is nested within the school, which is nested within the context. It postulates a number of causal relationships, the direction of which is signified by the pointed arrows. Based on the research evidence, the determinant factors presented in the conceptual framework are presented below according to their significance for effecting student achievement and attainment: * Student attributes * Family background * Teacher's background * Teacher's interactive style * Effectiveness of classroom management * Quality of instructional methods and techniques * Educational leadership * Effective use of monitoring and evaluation * Adequacy of the school's financial, human, physical and material resources * Community influence * Ministry of Education Management The measurement strategy proposed is one that will favor the use of micro-indicator clusters for each determinant factor, rather than attempting, as so often is the case, to identify the Figure 3. School Effectiveness Model Context Level l* MOE Management * Community *Family Background School Level Inputs Process Outputs * Financial *Educational Leadership _-Student * Human -Monitoring and Evaluation _ achievementt * Materal attainment Classroom Level *Teacher Background & Style -Classroom Management __ & Quality Instruction -Student Characteristics 105 Monitoring Educational Perfornance in the Caribbean 'silver bullet' indicator that explains the largest proportion of variance in student achievement. Each micro-indicator cluster is composed of a comprehensive, yet discrete number of indicators that would be tracked separately. This allows for considerable flexibility in application to different cultural circumstances. In Caribbean countries where particular indicators may be considered more or less relevant, their composite weighting could be easily adjusted. Another advantage of this approach is that indicators of special interest can be analysed separately or in combination to determine their level of correlation with student achievement. However, it should also be noted that developing an indicator system is a trial and error experience that can only be improved after one or more pilot tests of data collection, analysis and appraisal. Some indicators may, after trial use, prove not to be appropriate to the unique cultural circumstances of the region and must then be discarded and replaced if necessary. Nevertheless, the use of micro-indicator clusters appears to be an appropriate approach to the development of an indicator system that will have broad application across a number of Caribbean countries. The list of proposed qualitative indicators, ranked by micro-clusters, is reported below: List of Proposed Qualitative Education Indicators STuDENT ArrIEurEs * Affective behaviour, includes disposition to learn, perseverance, non-disruptive and co-operative behaviour; TEACHER'S INTERACTIVE STYLE The extent to which the teacher: * has high expectations for academic achievement; * encourages positive social interactions; * models appropriate behaviour; * fosters self-esteem and belonging; * provides praise and encouragement; * dissuades disruptive behaviour; * promotes the value of learning; * encourages creativity; * encourages tolerance towards others. EFFECTIVn SS OF CLASSROOM MANAGEMENT The extent to which the teacher: * keeps an orderly classroom environment; * handles routine tasks in a prompt and efficient manner; * has instructional materials ready for use; * minimises distractions and interruptions; * handles behaviour problems with minimal classroom disruption; * monitors absenteeism. 106 Pan 11: Annexes QuALrIY OF INSTRUCTIONAL METHODS AND TECHNIQUES The extent to which the teacher uses: * prior knowledge assessment to tailor instruction to student needs; * curriculum defined learning objectives and related content; * lesson plans to organise instruction; * available classroom time for learning activities, i.e., time on task; * available instructional materials, tools or media to enhance learning; * an appropriate mix of lecturing, demonstration and group discussion; * participatory group and co-operative learning activities; * appropriate questions, e.g., open-closed, range of difficulty, wait time; * homework to increase time on task; * frequent student assessment and testing; * reinforcement; * diagnostic feedback and remediation. EDUCATIONAL LEADERSHP Extent to which the: * Principal provides management leadership; * Principal provides instructional leadership; * Teachers are goal-oriented with high expectations for achievement; * Teachers understand the curriculum requirements; * Teachers work co-operatively together. EFEcTIVi USE OF MONITORING AND EVALUATION * Teacher attendance rate; * Teacher punctuality rate; * Frequency and nature of staff performance appraisals; * Frequency of classroom monitoring; * Presence of standardised curriculum specific school tests; * Extent to which performance information is used for school management. ADEQUACY OF FiNANcIAL, HUmAN, PHYSICAL AND MATERIAL RESOURCES Adequacy of financial resources: * Per pupil expenditure; * Gross student-teacher ratio; Adequacy of human resources: * Percentage of certified (trained) teachers; * Level of teacher educational attainment; * Frequency and duration of teacher in-service training; 107 Monitoring Educational Performance in the Caribbean Adequacy of physical and material resources: * Extent to which schools are adequately equipped; * * Presence of a fully functioning school library; * * Presence of child nutrition and feeding programs. * PARENTAL SUPPORT The extent to which parents: * have higher educational attainment; * exhibit attitudes that support learning; * participate in family activities that support learning; * have high expectations for their children's academic success; * ensure school attendance; * ensure the completion of homework. PERm GROUP INFLUENCE * Level of peer group academic and occupational aspirations; * Absence of peer group substance abuse or criminal activity; CuRRIcuLUM QuALrrY * Level of explicitness and ordering of goals and content; * Degree of structure and clarity of content; * Presence of advance organisers; * Presence of measurement and testing elements; * Quantity of instructional time allocated by subject. MOmrrOuR4G AND EVALUATION CAPACIy * Presence and implementation of a performance appraisal policy; * Presence and implementation of a monitoring and evaluation policy; * Presence and quality of an education information management system. 108 Bibliography Caribbean Elxamination Council (2001): "Statistical Bulletin 2001" Belize-Ministry of Education (2002): "Education Statistical D:.gest 2000-2001" Belize-Ministry of Education (2001): "Education Statistical Digest 1999-2000" Belize-Mini stry of Education (2000): "Education Statistical Digest 1998-1999" Dominica-Ministry of Education, Science and Technology (2002): "Indicators 2002" Dominica-Ministry of Education, Science and Technology (2000): "Indicators 2000" Dominican Republic (2001): "Estadisticas e Indicadores Educativos 1998-1999" Grenada-Ministry of Education (2002): "Education Statistical Digest" Guyana-Ministry of Education (2000): "Digest of Education Statistics of Guyana 1999-2000" Guyana-Ministry of Education (1999): "Digest of Education Statistics of Guyana 1998-1999" Jamaica-Ministry' of Education and Culture (2000): "Education Statistics 1999-2000" Jamaica-Ministry of Education and Culture (1999): "Education Statistics 1998-1999" Klein, R and SC Ribeiro (1991): "O censo educacional e o model de fluxo: o problema de repetencia" - Revista Brasileira de Estadistica Vol.52 (197/198) McMeekin R.W. (1998): "Education Statistics in Latin America and the Caribbean" - Technical Study -- Inter-American Development Bank - Washington D.C. Meier, W (2002): "Monitoring School Effectiveness: selecting appropriate indicators" - Paper prepared for the UIS meeting of Trinidad and Tobago, May 6-9 (mimeo) - Ottawa Mingat A, Rakotomala M and JP Tan (2001): "Rapport d'Etat d'un Systeme Educatif National (RESEN)"- African Region Human Development Unit (mimneo)- The World Bank - Washington D.C. Mingat, A and B Suchaut (2000): "Les Systemes educatifs africains: une analyse economique comparative" - De Boeck Universite - Bruxelles Mingat A and JP Tan (1998): "The Mechanics of Progress in Education" - Policy Research Working Paper N. 2015 - Human Development Department - The World Bank - Washington D.C. OECD (2001): "Knowledge and Skills for life:first results from PISA 2000" - Paris 109 Monitoring Educational Performance in the Caribbean OECD (2000): "Investing in Education: analysis of the 1999 World Education Indicators" - Paris OECD (1992): "The OECD International Education Indicators: a framework for analysis" - Paris Saint Kitts and Nevis-Ministry of Education, Labour and Social Security (2002): "Statistical Bulletin 2000-2001" Saint Kitts and Nevis-Ministry of Education, Labour and Social Security (2001): "Statistical Bulletin 1999-2000" Saint Kitts and Nevis-Ministry of Education, Labour and Social Security (2000): "Statistical Bulletin 1998-1999" Saint Lucia-Ministry of Education (2001): "Education Statistical Digest" Saint Lucia-Ministry of Education (2000): "Education Statistical Digest" Saint Vincent and the Grenadines-Ministry of Education (2001): "Directory of Schools and Colleges 2000/2001" Tan, JP, Soucat, A and A Mingat (2001): "Enhancing development in the HIPC/PRSP context: progress in the Africa Region during 2000" - Africa Region Human Development Working Paper Series - The World Bank - Washington D.C. The World Bank/Human Development Network (2002): "Achieving Education for All by 2015: Simulation results for 47 Low Income countries" (first draft) - Washington D.C. The World Bank/Human Development Network (2001): "Poverty Reduction Strategy Paper (PRSP) Sourcebook: Education" (mimeo) - Washington D.C. The World Bank (1999): "Jamaica: secondary education: improving quality and extending access" - Report N. 19069-JM - Washington D.C. The World Bank (1996): "Trinidad and Tobago. The financing of education" - Report. N.16216-TR - Washington D.C. The World Bank (1993): "Caribbean Region: access, quality and efficiency in education" - A World Bank Country Study - Washington D.C. UNESCO Institute for Statistics (2002): "Measuring Completion" - presentation made at the UIS meeting of Trinidad and Tobago, May 6-9 UNESCO Institute for Statistics (2001): "Statistics and indicators on education, 1998/99: Latin America and the Caribbean, regional report" - Montreal UNESCO Institute for Statistics/EFA Observatory (2001): "Education for All Indicators"-Report on the Expert Group Meeting UNESCO Paris, 25-26 June 2001 UNESCO/OREALC (2001): "Regional Report"- Regional Education Indicators Project (PREE) - Santiago 110 Report No.: 24337 CRG Type: SR