Report No. 39830-CL Chile Institutional Design for an Effective Education Quality Assurance August 31, 2007 Human Development Argentina, Chile, Paraguay and Uruguay Country Management Unit Latin America and the Caribbean Regional Office FOR OFFICIAL USE ONLY Document of the World Bank This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. IA Inspection Authority, United Kingdom IALS International Adult Literacy Survey INE National Institute of Statistics, Chile (Instituto Nacional de Estadistica, Chile) ICA Institute of Chartered Accountants of New Zealand ICT Information and Communications Technology IDR In-DepthReview, Boston I E S Institute for Educational Sciences, United States ILT Instructional Leadership Team, Boston INECSE National Institute for Evaluationand Quality of the Education System, Spain I SI Independent Schools Inspectorate, UnitedKingdom ISSP Individual Student Success Plans, Massachusetts K-12 Kindergarten to grade 12 KED1 Korean Educational Development Institute KERIS Korea Education & Research InformationService KICE Korea Institute of Curriculum and Evaluation LA Local Authority LLECE Latin American Laboratory for the Assessment of Education Quality MCAS Massachusetts Comprehensive Assessment System MIDEUC Interdisciplinar Management of Student's Difficulties, Pontifical Catholic University of Chile (Manejo Interdisciplinario de las Dificultades del Escolar, Pontificia Universidad Catdlica de Chile) MIDEPLAN Ministry of Planning and Cooperation, Chile (Ministerio de Planificacidn y Cooperacidn, Chile) MINEDUC Ministry of Education, Chile (Ministerio de Educacidn, Chile) M O E Ministry of Education, Finlandand New Zealand; Ministry of Education &HumanResourcesDevelopment, Korea; Ministry of Education and Science, Spain MPEA Municipal and ProvincialEducationAuthority, Korea W O E Metropolitan and Municipal Office of Education, Korea NAEP National Assessment of Education Progress, United States NAfW National Assembly for Wales NAGB National Assessment Governing Board, UnitedStates NAO National Audit Office, United Kingdom NCES National Center for Education Statistics, United States NCLB N o Child Left Behind, UnitedStates NEMP National Education Monitoring Project, New Zealand OECD Organisation for Economic Co-operation and Development OF-CMO Basic Objectives and Minimum Obligatory Contents (Objetivos Fundamentales y ContenidosMinimos Obligatorios) P-12 Pre-Kindergarten to grade 12 P E S Progress inInternational Reading Literacy Study PISA Program of International Student Assessment QA Qualification Authority, UnitedKingdom QTS Qualified Teacher Status, UnitedKingdom .. 11 SACGE Education Management Quality Assurance System, Chile (Sistema deAseguramiento de la Calidad de la GestidnEscolar, Chile) SAT Scholastic Aptitude Test, UnitedStates; Scholastic achievement, Korea SBOE State Board of Education, Texas SE Special Education SES Socio-economic status SIMCE Education Quality Measuting System (Sistema de Medicidn de la Calidad de la Educacidn, Chile) SNED Performance EvaluationNational System, Chile (SistemaNacional de Evaluacidn de Desempefio, Chile) SOFOFA ManufacturingPromotion Society, Industrial UnionFederation, Chile (Sociedadde Foment0 Fabril, Federacidn Gremial de la Industria, Chile) ssc State Services Commission, New Zealand; School Site Council, Boston TAKS Texas Assessment of Knowledge & Skills TEA Texas Education Agency THSP Texas HighSchool Project TDA Training and Development Agency for Schools, United Kingdom TIMSS Trends in International Mathematics and Science Study WLB Welsh Language Board WSIP Whole-School Improvement Plan, Boston Vice President: Pamela Cox Country Director: Pedro Alba Sector Manager: Eduardo Velez Bustillo Country Sector Leader: Jesko Hentschel Task Team Leader: Emiliana Vegas ... 111 Acknowledgements T h i s report was prepared by a team led by Emiliana Vegas (Task Team Leader). The team included Joseph Olchefske and Amy Walter from the American Institutes for Research, as well as Erika Molina and Pedro Cerdan Infantes from the World Bank (LCSHE). The report benefited from the overall guidance of Axel van Trotsenburg (Country Director, LCC7C), Eduardo Velez Bustillo (Sector Manager, LCSHE), and Jesko Hentschel (Sector Leader, LC7HD). Peer reviewers for this task included Michael Barber (Consultant, Expert Partner of Global Public Services, McKinsey International), Rod Paige (Consultant, former U.S. Secretary of Education), and JosC Pablo Gdmez- Meza (Sr. Economist, LCSHH). Useful comments at various stages of this study were received from Marta Elena Molares-Halberg (Lead Counsel, LEGLA), Kristian Thorn (Education Specialist, LCSHE), Anthony Gaeta (Country Officer, LCC7), Gwang-Jo Kim (Deputy Minister of Education, Republic of Korea), Sung-Geun Bae (Director for Policy Coordination at the Korean Ministry of Education), and Anja Kurki (American Institutes for Research). The team is especially grateful to Chilean counterparts, whose close collaboration significantly enhanced the report: Yasna Provoste (Minister of Education), Pilar Romaguera (Under-Secretary of Education), Maria O h i a Recart (Under-Secretary of Finance), Pedro Montt Leiva (Director of the Curriculum and EvaluationUnit of Ministry of Education), Jorge Rodriguez (Director of the Studies Division of the Ministry of Finance), Paula Darville (Policy Analyst, Ministry of Finance) and Tania Hernindez (Advisor, Ministry of Finance). The team also acknowledges helpful interviews with Chilean government officials, education analysts, private sector, labor and civil society representatives. For their time and candor, we are thankful to the Education Commission of Chile's Senate, the Education Commission of Chile's Chamber of Deputies the Universidad de Chile, the Universidad Catdlica (MIDEUC, Centro de Estudios), the Universidad Albert0 Hurtado, the Universidad Adolfo Ibhiiez, the Federacidn de Instituciones de Educacidn Particular (FIDE),the Corporacidn Nacionalde Colegios Particulares (CONACEP), the Association of Chilean Municipalidades, Secondary Education Student Leaders, University Student Leaders, the Board of Directors of the National Teachers Union (Colegio de Profesores), Federacidn Gremial de la Industria (SOFOFA), Centro de Estudios Hblicos (CEP), Centro de Estudios para el Desarrollo (CED) and Instituto Libertad y Desarrollo. Last but not least, the team is grateful to Cristiin Cox, whose insights and comments to our analyses helped refine and improve the final report. iv Table of Contents Introduction and executive summary ................................................................................. 1 Background on Chile's Education System ......................................................................... 4 A conceptual framework for analyzing education quality assurance systems and its application to Chi)e's current system ................................................................................ 10 Conceptual framework for the analysis of education quality assurance systems ........10 Application of the conceptual framework to Chile's current education quality assurance system .................................................................................................................. 14 Education quality assurance systems in benchmark systems ......................................... 18 Selection of benchmark education quality assurance systems ....................................... 18 Finland ................................................................................................................................... 19 IV.1.2. N e w Zealand......................................................................................................................... 20 IV.1.3. U.I<.. England, Wales and Northern Ireland................................................................... 21 IV.1.4. U.S.A.:Massachusetts/Boston and Texas/Houston ...................................................... 23 IV.1.5. Republic of Korea................................................................................................................ 24 IV.l.6. Spain....................................................................................................................................... 25 Instructional visions for education provision and quality assurance ............................ 27 Analysis of alternative instructional visions ..................................................................... 28 T h e control-model coordinates analysis ........................................................................... 31 Policy options to strengthen the institutional arrangements for education quality assurance in Chile ................................................................................................................. 35 Implications for the institutional allocation of roles and responsibilities .................... 37 Conclusions and N e x t Steps ............................................................................................... 40 References ........................................................................................................................................... 43 Appendix A: Country reports .......................................................................................................... 47 Appendix B: Institutional functions ............................................................................................... 75 Appendix C: Consultation process ................................................................................................. 79 V I. Introductionandexecutivesummary 1, Beginning in 1980, successive Chilean administrations have increased investment inthe education sector and introduced multiple large-scale education reforms. Among the most important of these reforms are: (i)decentralization of public education service delivery to municipal governments; (ii) a shift to per-student (voucher style) funding for private and municipal schools; (iii) the implementation of performance-based bonuses for teachers; and (iv) the introduction of targeted compensatory programs for under- performing schools. 2. While these reforms have led to significant gains in educational coverage, substantial quality and equity problems remain. For instance, preschool and higher education expansions have yet to reach the lowest income quintiles, and national and international standardized tests show a worrying pattern of stagnation. Striking inequities in the national assessment results persist between municipal schools, which serve students primarily from socioeconomically disadvantaged backgrounds, and private schools, which tend to serve students from more advantaged backgrounds. Furthermore, a worrisome trend exists in terms of Chile's performance on international standardized tests, such as PISA and TIMSS. Students in Chile have been performing well below the average of students from OECD countries, indicating that Chile is confronting a serious challenge of improving education quality in order to better compete in a globalizing world. Indeed, concerns about quality and equity were the main reasons students cited for the nationwidesecondary school student strike that took place inMay and June of 2006. 3. InJuly of 2006, the Government announced a series of policy changes affecting the education sector, including the creation of a new Superintendency of Education. The creation of this new agency implies a reallocation of functions across the Ministry of Education, the Superintendency, and other education institutions. In addition to new education quality assurance functions, the Superintendency of Education will likely assume some of the roles and responsibilities that were previously exercised by the Ministry of Education. A redefinition of functions for quality assurance will inevitably expand beyond the new Superintendency, encompassing the realm of the Ministry of Education as well as all other institutions and individuals that participate inthe provision of education. 4. The main objective of this report is to present the Government of Chile with policy options related to the institutional distribution of roles and responsibilities for effective quality assurance in education. These policy options derive from an analysis of education systems throughout the world that are comparable to Chile's current institutional system but have achieved high levels of academic achievement among a majority of their basic education students. 5. In order to propose options for the design of an institutional setup that may be effective in education quality assurance, the report first presents a conceptual framework for analyzing a country's education quality assurance system. This framework identifies 1 individuals and institutions that participate in the production of education quality as well as key functions that an education system geared towards quality assurance needs to perform. 6. The conceptual framework was then applied to Chile's education system and a group of selected countries. To select a sample of education systems which could provide the most useful information for Chile, the following criteria were employed: (i) the for most part, decentralized education systems such as Chile's; (ii) systems that explicitly aim for the achievement of high performance standards by all students; and (iii) systems that have demonstrated high performance in international assessments of student learning. Using these criteria, seven education systems were selected: Finland, New Zealand, three constituent countries of the United Kingdom (England, Wales, and Northern Ireland), and two districts of the United States of America (Boston, Massachusetts and Houston, Texas). Two additional systems were selected as comparisons that do not meet these criteria in order to have a more comprehensive sample of education systems. These additional two systems are: the Republic of Korea, a centralized education system that is also a high performer education system; and Spain, a medium-performing education system that has a more similar culture to Chile's. 7. In applying the conceptual framework to Chile's current education sector institutions, it became clear that a number of important functions are not explicitly defined. This gap in the definition of roles and responsibilities of participants in the education sector may be a factor impeding the functioning of an effective system of education quality assurance intoday's Chile. 8. The review of the nine selected international education systems revealed the existence of four alternative "instructional visions" for the institutional design of an education system, which we have labeled as follows: (a) Limited State, (b) Quality Contracts, (c) Differentiated Instruction, and (c) Managed Instruction. One can identify countries that succeed in ensuring education quality in each of the three latter visions. In this sense, the decision to adopt a specific instructional vision can be based on social, political, cultural and historical reasons. However, once an instructional vision is adopted, the study suggests that its consistent application may be an important determinant of success ineducation quality assurance. 9. Finally, the distribution of roles and responsibilities for quality assurance across institutions varies depending on the instructional vision that is adopted. Inthis sense, the policy options that the report presents for the Chilean context are based on a particular mix of instructional visions which consider Chile's social, political andhistoricalreality. 10. The remainder of the report is structured as follows. Chapter I1 presents background information on the evolution of Chile's education system since 1980. This information, together with an analysis of the current situation of the education sector, describes the motivation for the study and demonstrates the urgent need to shift the focus ineducationpolicy to quality assurance. 2 11. Chapter I11describes the conceptual framework developed for the analysis of how successful systems carry out education quality assurance. The framework developed identifies education participants, including: students; teachers; principals and school administrators; schools; local governments (districts, municipalities); regional governments (states, provinces); and the national government. Inaddition, the framework lays out eight key functions that any effective education quality assurance system should define: performance standards; performance assessments; performance reporting; impact evaluation of policies and programs; requirements to operate; ensuring adequate and equitable resources; autonomy, intervention, and support; and accountability and consequences. In Chapter 111, this framework i s employed to analyze Chile's current education quality assurance system. 12. Chapter IV applies the framework to the nine education systems selected as comparisons and presents a summary of the quality assurance functions and institutions ineachselectededucation system. 13. Chapter V describes the four alternative instructional visions for quality assurance in education developed as a result of the international review. Each instructional vision implies a different degree of school autonomy relative to government control. At the same time, different visions accommodate varying numbers of instructional models. 14. Finally, Chapter VI presents policy options for the distribution of roles and responsibilities for education quality assurance across individuals and/or institutions, which vary depending on the instructional vision followed. These options take into account Chile's social, political and historical reality. 3 11. Backgroundon Chile's EducationSystem 15. The decentralization process initiated in Chile in the early 1980s transferred the administration of public schools to the municipal governments. The reform also opened the way for private sector participation as a provider of publicly financed education by establishing a voucher-type student-based subsidy. Three types of schools were established: municipal schools, financed by the student-based subsidy granted by the State and runby municipalities; private subsidized schools, financed by the State student- based subsidy and runby the private sector; and private fee-paying schools, financed by fees paid by parents and runby the private sector. 16. While education provision was decentralized to municipalities and private schools, a number of important policy decisions remained within the purview of the national Ministry of Education. These include, among others: determining public (municipal) school teachers' remuneration system and negotiating their contracts; setting operational requirements for schools, teachers, and administrative staffs; setting curricula and student assessment systems; and determining the size of the per-student subsidy. In addition to the Ministry of Education, the Consejo Superior de Educacidn (Superior Education Council) is charged with approving curricula and standards developed by the Ministry. 17. The institutional arrangements for education policy and provision in Chile are summarized in Figure 1. The Ministry of Education, whose maximum authorities (the Minister and Under-Secretary) are appointed by the President of Chile, has offices at the regional and provincial levels. At the regional level, these offices are the Regional Ministerial Secretariats (Secretarias Regionales Ministeriales, or SEREMIs), whose maximum authority (the Regional Ministerial Secretary of Education) is also appointed directly by the President of Chile. At the provincial level, the central Ministry has a Departmental office (Departamentos Provinciales, or DEPROVs) which is responsible of monitoring school service delivery. Public school administration is in the hands of the municipal government, whose maximum authority is the Mayor, an elected public official. Private schools are runby either individual entrepreneurs or corporations. Figure 1:InstitutionalArrangementsinChile's EducationSystem Education Policy Setting, Monitoring, Evaluation: Ministry of Education(Mineduc, Central Office) Regional Ministerial Secretariats of Education ProvincialDepartmentsof Education School administration: Municipalgovernments Private corporations Educationprovisiodservice delivery: Private schools 4 18. To date, the size of the per-student subsidy has been the same for both municipal and subsidized private schools. In addition, the size of the per-student subsidy has also been disassociated from student socioeconomic background, although Congress is currently reviewing a proposal to introduce a preferential subsidy which would provide more resources to students from disadvantaged households. 19. Fee-paying private schools are generally for profit, whereas subsidized private schools can be non-profit or for profit. Non-profit private schools include church schools and those dependent on foundations or private corporations, some of which are linked to sectors of industry. For-profit schools mostly operate like firms, generating returns for their owners. 20. There are two additional important differences between municipal and private schools: (i)student selection and (ii) hiring and compensation. While subsidized teacher private schools can (and regularly do) select their students, municipal schools are required to admit all students interested inenrolling. Regarding teacher contracts, private schools have the authority to directly hire and dismiss teachers and determine their compensation as long as they abide by the Labor Code. In contrast, municipal schools are required to pay teachers according to a national salary scale that is negotiated by the central Ministry of Education with the national teachers unions and their contracts are determined by the Teacher Statute, a more rigid regulation than is the Labor Code. Because municipal school teachers are governed by the Teacher Statute, their dismissal is also much more difficult than that of teachers inprivate schools. 21. Since the return of democracy in 1990, successive governments have increased education investment significantly (see Figure 2), which has resulted in a substantial expansion in the quantity and quality of educational inputs as well as a substantial expansion of the time students spend in schools (Full-Time School reform). At the same time, compensatory programs were introduced aimed at increasing school coverage while expanding school infrastructure and facilities, especially for the most vulnerable students. 5 Figure 2: Evolution of EducationInvestment in Chile, 1990-2004' 890 891 892 893 894 895 896 897 898 899 2000 2001 2002 2003 2004 22. The rise in education investment has translated into significant gains interms of educational coverage in primary and secondary education (see Figure 3). In spite of the increase in enrollments in primary and secondary, enrollment growth in preschool and higher education has been more limited and coverage is well below universal. Additionally, substantial gaps by income in access to preschool and higher education expansions persist (see Figure4 and Figure 5). Figure 3: Enrollment inPrimary and SecondaryEducation, 1990-2004 (percent)2 105 100 95 90 85 80 75 70 - -.-+ 65 - -*+" 60 - ,*..--**--** """..*....*'-''- A.- 55 --. .*.'- '**- -'-..-- 50 4 1 1- - -+- -Net: Primary --e-Gross: Primary - - -+- -Net: Secondary ----c---Gross: SecondaryI Source: Public education expenditure: Financial Resources Department, Budget and Planning Division (MINEDUC); private education expenditure: National Accounts Department's (MINEDUC) calculations based on data from Central Bank of Chile; GDP: Central Bank of Chile. The 2003 figure i s provisional and the *2004 figureDevelopment preliminary. Source: and Studies Department (MINEDUC); National Institute of Statistics CINE) ``Chile hacia el 2050 - Proyecciones de Poblaci6n 1990-2050", August 2005. 6 60 - 49.1 50 - 40 - 1 35.0 1 36.1 30.3 34.0 30 - I 32.4 20 - I 20.4 27.2 10 - 16.9 17.5 I / 0 1 , I II 111 IV V I1 IFigure I 5: HigherEducationEnrollmentRate, by IncomeQuintile (per~ent)~ I 801 73.7 70 - 60 - 46.4 50 - 40 - 32.0 40.2 30 21.2 20 - I 14.5 I 1 121.3 7.8 12.4 lo0 1 4.4 23. While the introduction of the per-student subsidy largely contributed to the expansion in enrollment in primary and secondary schools, the quality of services provided by these schools is an issue of concern. National standardized achievement tests show little improvement over time, even inprimary education where enrollment has been near universal since the early 1970s, and the performance of Chilean students in international tests is well below that of students in OECD countries. For example, while 32 percent of 15-year old students in OECD countries have reading achievement levels in the Programme for International Student Assessment (PISA) that are in the top two levels, only 6 percent of students in Chile do so. Perhaps even more troubling is that while 20 percent of Chilean 15-year olds perform below the first level indicatingthat they cannot master basic reading skills, inOECD countries this figure is 6 percent (Figure 6). 3 Sotrrce:MIDEPLAN,CASENSurveys 1990and2003. 4 Solrrce:MIDEPLAN,CASENSurveys 1990and2003. 7 Figure 6: Percentage of Students in the Different Levels of Performance in PISA 2000 reading, by Country' Percentage of students In the different levels of performance in reading by country 100% 90% 80% 70% 60?6 Level4 Level3 50% 0Level2 j 0 Level 1 40% 1 U-p~ow keyep 30% 20?6 10% 0% 24. The voucher system together with private provision of education was expected to promote competition between schools by attracting and retaining students, creating an "education market" that through competitionwould encourage efficiency and educational quality. 25. The extent to which school choice can raise education quality is a hotly debated issue among education economists worldwide. Proponents of privatization - or, as it is most often referred to in the United States, school choice - advocate in favor of allowing non-governmental groups to provide schooling with the funding partially or totally provided by government. They argue that private schools are more successful in retaining the best of their new teachers and in developing the teaching skills of their existing teachers, and as a result, they tend to generatebetter student outcomes. Some of the reasons include the private schools' greater supervision and mentoring of new teachers, their ability to demand teachers with higher quality education, to attract teachers who exert more effort and independence, and, ultimately, their freedom to dismiss teachers for poor performance (Ballou and Podgursky 1998,Hoxby 2000). Opponents to school choice put forth that, because private schools can select students while public schools cannot, privatization leads to increased sorting by racial/ethnic, socio-economic, and cultural background, leaving the public sector with the difficult task of serving the most disadvantaged children while having less resources than in a fully public (both in terms of finance and provision) education system (see, for example, Fuller and Elmore 1996). 26. The question of whether competitionamong schools can lead to improvements in education quality in Chile is beyond the scope of this study. Previous research on the 5 Source:OECD. 8 relative effectiveness of private and public schools in Chile has yielded varying results. Rodriguez (1988), Aedo and Larraiiaga (1994), Aedo (1997), find that private voucher schools achieve higher levels of student outcomes than do municipal schools. Incontrast, Mizala and Romaguera (2001) and, ina series of papers, McEwan and Carnoy (1999 and 2000) have found that private voucher schools do not perform any differently from municipal schools. However, when distinguishing between non-religious and Catholic voucher schools, McEwan and Carnoy (2000) do find that Catholic voucher schools outperform municipal schools. The main differences among the studies include the samples used, the variables included in the analyses, and how the different sectors are categorized in the analyses. Given the confounding effects of student background, peer effects and other unobservable variables, empirically identifying the impact of competition on student outcomes is methodologically challenging. Thus, research findings remain inconsistent regarding the extent to which competition between private and public schools improves student outcomes and/or raises inequality across different groups of students. 27. However, it is useful to analyze the details of the education market system in Chile to better understand how it may contribute to increase student learning outcomes. Indeed, Chile has made important reforms in the direction of providing school choice to families, and by now these reforms are widely accepted by the majority of Chileans. A concrete achievement of the reform has been to increase coverage and retention. Going back to a system where money does not follow the student, in our view, is unlikely to yield the desired improvements in education quality and equity. In contrast, strengthening the quality assurance functions of each of the participants in the Chilean education system is more likely to produce the improvements in education quality and equity that Chilean society is demanding. 28. Inwhat follows, we present a conceptual framework to analyze education quality assurance systems. We then apply this framework to the current Chilean education system and to selected education systems. Our analysis focuses on the best way to use lessons learned from other countries to buildon what Chile has already achieved. 9 111. A conceptual framework for analyzing education quality assurance systems and its application to Chile's current system 29. Many individuals and institutions work together to generate and support student learning, and they together carry out a set of essential functions to ensure education quality. These educationparticipants and thefunctions they carry out can be synthesized in a conceptual framework that is useful in analyzing how education quality assurance systems operate. The functions identified as essential quality assurance inthe conceptual framework developed for this report build on the English education performance management system, as described inBarber (2004). 111.1. Conceptual framework for the analysis of education quality assurancesystems 30. One can identify a set of participants (individuals and/or institutions) that work together to produce student learning: students, teachers, principals and school administrators, schools, local governments (districts, municipalities), regional governments (states, provinces), and the national government. In addition, an effective education quality assurance system should have well-definedfunctions for each of these participants and have strategies to measure and hold individuals and institutions accountable for how much students learn. Inparticular, we put forth eight key functions that any education system that is geared to ensuring that all students learn should have explicitly defined for each of its participants: (1) performance standards; (2) performance assessments; (3) performance reporting; (4) impact evaluation of policies and programs; (5) requirements to operate; (6)ensuring adequate and equitable resources; (7) autonomy, intervention, and support; and (8) accountability and consequences.6 31. While many of these functions are affected by a country's economic, political, and regulatory framework^,^ the focus of the report is on the extent to which, and how, each of these functions is explicitly defined for all education participants. Further, the functions identified in this conceptual framework are inherent to the education system, and the report posits that unless education participants (from students to national governments) have clearly assigned roles and responsibilities in each of these eight functions, it is unlikely to achieve higheducation quality outcomes. 32. An important stakeholder in the educational process is the family. Research around the world has demonstrated the importantrole that parents and families play inthe learning of their children, as well as the key role that schools play. However, from the The framework for education quality assurance presented inthis report was recently developed by The World Bank for the Government of Chile. We are grateful to Joseph Olchefske and Amy Walter, from the American Institutes for Research, for a productive collaboration in this activity. 7 For example, in Chile and in the Netherlands, national education laws have defined regulatory frameworks that grant private schools substantial freedom to make decisions over performance standards and methods of instruction. This regulatory framework can have direct effects on the extent to which Government can hold private schools accountable for performance. 10 institutional point of view, it is difficult to formally regulate the participation of parents in education. In other words, while certain things, such as performance standards and evaluation mechanisms, can be developed to apply to students, teachers, directors, schools, supporters, and government authorities working in the education sector, it is much more difficult to develop such standards and mechanisms for parents and families. Consequently, although it is important to recognize the vital role that families play in students' learning and in assuring the functioning and quality of educational institutions, this paper does not include recommendations aimed at families as formal participants in the quality assurance system. Table 1 summarizes the eight general functions and participants of an education quality assurance system according to this conceptual framework. Below, we define each of the functions inmore detail. 11 33. Performance standards. For each individual or institutionthat participates in the production of education quality, there should be established targeted performance levels. For example, there should be clearly defined standards for students that lay out what they should know and be able to do at each grade and level of the education system. Similarly, there should be well-defined standards for what teachers at each education level should know and be able to do. 34. Performance evaluation. To be able to assess the extent to which individuals and institutions meet the agreed-upon standards, there must be methodologies inplace. These include standardized methods for objectively measuring what students know and are able to do, as well as appraising the performance of teachers and school leaders. In addition, they include frameworks for analyzing institutional performance, such as the degree to which schools are meeting the learning needs of all students who attend. Such assessments are used to make decisions about levels of autonomy, intervention and support granted to individuals and institutions, as well as accountability and consequences for varying levels of performance. 35. Performance reporting. Just as important as are standards and assessments, an effective education system has established processes for disseminating the outcomes of the performance assessments, including evaluating the frequency, scope, quality, and audiences of information on education quality at all levels. For example, in terms of audiences, individual student assessment information can be made available to students themselves, to their parents or guardians, to their teachers, and to administrators. Teacher assessment information can be made available to school administrators, who can use it to provide differential support to teachers in order to achieve the established performance standards. Similarly, teacher assessment information could be made available to local government officials and parents. School assessment information can be made available to local and regional governments, and local and regional assessment information can be made available to national government authorities. 36. Impact evaluation of policies and programs. An effective quality assurance system must have in place methodologies to regularly evaluate the impact of policies and programs and to incorporate this information into existing and new policies and programs. Some key questions that should be evaluated for most interventions include: (i) Does the program raise student learning or other student-level outcomes (e.g. retention, labor market outcomes)? (ii) How does it improve student outcomes (e.g. process, service delivery)? (iii)At what cost, comparatively speaking, does it achieve the estimated impact? Answering each of these questions implies a different evaluation strategy. It is important to have mechanisms in place to design the impact evaluation of policies and programs before the policies or programs are introduced. Once a policy or program is introduced, it is much more difficult to construct a credible evaluation strategy. 37. Requirements to operate. An education system should have established norms for entry into and operation in the system for each of the individuals and institutions that operate in it. These range from age of entry requirements for students, to professional requirements for teachers, to basic conditions that all schools should meet in order to be allowed to operate. 13 38. Ensuring adequate and equitable resources. Education quality assurance systems should have established management, financing, and administration procedures to achieve the established standards. Although the capacity of an education system to ensure adequate and equitable resources is to a great extent determined by the larger macroeconomic and political context-which affects the total amount of resources devoted to the sector-education participants have scope to make policy decisions that affect resource allocation and distribution. These decisions range from school financing mechanisms-such as per-student subsidies or per-school allocations based on established norms-to the processes whereby resources are channeled to each of the individuals and institutions inthe system. 39. Autonomy, intervention, and support. Besides ensuring adequate resources, it is important to have established instruments that assist individuals and institutions in meetingthe performance standards. These instruments include autonomy insetting policy and managing resources, technical-pedagogic support to teachers and school administrators; as well as facilitating and coordinating with (government and private) support institutions and networks. 40. Accountability and consequences. Finally, an effective education quality assurance system must have in place mechanisms to reward and sanction individuals and institutions for meeting (or not meeting) the agreed-upon requirements and performance standards. Most education systems in Latin America have established direct consequences for students who do not meet the agreed-upon standards; indeed, most systems have secondary school-exit examinations or national university entrance examinations. These examinations have direct consequences for students, who may or may not continue pursuingtheir education goals based on how their performance on such assessments. In contrast, few education systems in the region have established consequences for teachers or schools who fail to meet performance standards. 111.2. Application of the conceptual framework to Chile's current education quality assurance system 41. To illustrate its application for the analysis of education quality assurance systems, we describe the institutional design of Chile's current education system under the lens of this framework. For each cell in Table 1, we address the following two questions: (i) Does the key function of quality assurance exist for this participant of the education system? and (ii) If so, which institutiodindividual is responsible for defining and/or mandating this function? When one or more functions do not exist for a given participant, the cell is left empty. Table 4 in the Appendix summarizes the application of our conceptual framework to Chile's current system of education quality assurance. 42. Performance standards. Primary and secondary school students inChile follow a national curriculum that is defined by the Ministry of Education and sanctioned by the National Education Council. The curriculum, in its present state, only includes compulsory subjects and contents by grade. However, learning standards are currently beingdeveloped for each grade and are expected to be introduced nationwide inthe next school year. 14 43. The standards that apply to schools as institutions in Chile are regulated by Ministry of Education's SACGE (Education Management Quality Assurance System). According to this system schools have the statutory duty to fulfill a number of school management standards. Compliance with compulsory standards is verified via a yearly evaluation that is nationally mandated by Ministry of Education. In addition, schools are also evaluated under Ministry of Education's SNED (National System for Performance Evaluation), a system that assesses school management practices and improvement in SIMCE (Education Quality Measurement System) awarding school level bonuses to the 25 percent best performing schools. SIMCE school averages are widely reported by Ministry of Education. Although currently there is no breakdown in the reporting of results, Ministry of Education plans to publish the report cards of those students benefiting from preferential subsidies. 44. Performance evaluation. Evaluation of student learning against the established curriculum is compulsory on a yearly basis in 4* grade and every two years in grades 8 and 10. The teacher evaluation system in Chile experienced a major reform in 2004-05. The recently approved evaluation system for teachers establishes four required competency levels based on teaching practice and a compulsory yearly evaluation mandated nationally by Ministry of Education. The performance evaluation system for school administrators is similar to that of teachers. Ministry of Education mandates nationwide that principals must satisfy four competency levels based on standards of behavior and actions for which they are evaluated on a yearly basis. 45. Performance reporting. There are no established mechanisms for reporting the outcomes of student performance evaluations and until very recently only school mean test scores were reported to parents and society in general. However, with the recent reforms in teacher performance evaluation, performance reporting mechanisms for teachers and school administrators have been introduced. 46. Impact evaluation of education policies and programs. In Chile, there are no formally established mechanisms for evaluating the impact of education policies and programs. 47. Requirements to operate. The main requirement for students to enter basic education is to comply with the statutory age, whereas remaining in the system is conditional on satisfying statutory competency levels by grade set by each school. Competency, determined by the individual teacher, not only determines the passing of grades but also affects future learning opportunities, since universities administer entrance examinations. Municipalities select teachers for public school posts. In order to be eligible for such posts, teachers must hold a teaching degree and may not have received more than two "below average" ratings in their regular assessments. Municipalities also select principals for public school posts. To access such posts, the only requirement that principals must conform with is a teaching degree. Selection, pay and work conditions for private school principals are determined by each school individually. Finally, to be able to operate, schools must comply with space and health requirements and observe maximum teacherhtudent ratios. In addition, the school sustainer is required to hold a secondary school degree. 15 48. Ensuring adequate and equitableresources. A per-student subsidy is providedto finance the education of students in basic, compulsory grades (1-12) in municipal and private schools. Although in its present form this subsidy is uniform, with no regard to individual student characteristics, Ministry of Education is inthe process of developing a differential subsidy formula that takes into account the student's background. To ensure that all students can access compulsory education, tuition and textbooks are free in publicly financed schools (i.e. municipal schools and subsidized private schools). In addition, about 50 percent of low-income students receive free meals while inschool. 49. Autonomy, intervention, and support. As a support mechanism, principals and provincial supervisors can provide autonomy and support to public school teachers. At present, most of the intervention that takes place i s supervision to ensure that students are meeting the attendance required for schools to receive the full per-student subsidy. Inthe case of school principals, it is school owners and provincial supervisors that have the authority to support or intervene in their work. Public schools that require extra support can access central government compensatory programs, and, some municipalities have stronger capacity than others to support schools. 50. Accountability and consequences. Public school teachers are paid on a common salary scale, yet evaluation results have direct consequences on pay and job stability. Public school principals are paid on a common salary scale and, as inthe case of teachers, evaluation results have direct consequences on pay and job stability. The per-student subsidy is a type of accountability mechanism for schools, as receiving the subsidy is conditional on student attendance. However, there are few accountability measures at the school level based on performance. The new differential subsidy would be accompanied by stronger supervision and consequences for schools based on student performance. 51. The quality assurance structure around the last four participants of the education system (service sustainers, provincial-, regional-, and national-level authorities) is less comprehensive than those for the previous participants (students, teachers, school directors, and schools). At present, no statutory performance standards apply to education sustainers or government levels, and consequently, no mandated evaluations. At the national level however, Ministry of Education is currently developing compulsory standards of supervision. The only performance reporting for government occurs at the national level through the publishing of national SIMCE trends and national results in international assessments (PISA, TIMSS and LLECE). Although there is a national evaluation department theoretically in charge of policy and program impact evaluations, this department is not very active. The finance chain that operates across the education sector starts at the national level, where Ministry of Education submits a budget proposal from the national budget to the Ministry of Finance. Regions receive operational funding from the national government, and inthe same way, provinces receive their budgets from regions. It is finally the municipalities that receive the per-student subsidy to distribute between individual schools. 52. Applying the conceptual framework to Chile's current education system shows that there are various areas where essential quality assurance functions are not explicitly 16 defined. This evidence suggests that a need exists for the strengthening of quality assurance mechanisms inChile's education sector. 53. As the Chilean case described above shows, the conceptual framework developed can be used to analyze, in a somewhat comprehensive way, the institutional design of a given education system with special emphasis on quality assurance mechanisms. The framework is based on the premise that proper quality assurance requires a system that addresses eight key functions. Namely, an education system geared towards quality assurance should, for each participant of the education system, set performance standards, evaluate performance against those standards, report the results of performance evaluations, establish mechanisms for policy and program impact evaluation, define requirements to operate in the system, ensure adequate and equitable resources for participation inthe system, define an autonomy and control structure and put inplace the corresponding support and intervention mechanisms and, finally, define and uphold an accountability and consequence framework relative to the achievement of standards. One of the benefits of the conceptual framework developed here is that it allows for a quick identification of specific areas needing improvement in terms of quality assurance. As evident in the description above, in the Chilean education system, many of the quality assurance functions identified need to be developed or strengthened. In the following chapter we synthesize the institutional structure of nine selected benchmark education systems. * * Chile is not the only country in which these functions are not explicitly defined. If one were to analyze the education quality assurance systems of countries in the Latin America and the Caribbean region, few - if any - countries would have explicit definitions for each of the eight key functions for all participants. However, since Chile is perhaps the lead education reformer in the region and has embarked in a different economic path toward development - marked by high growth rates, sound macroeconomic policies, and real reductions in the poverty rate - and in view of its aspirations to join the OECD, this report focuses on comparing Chile to countries of the reference group of which Chile is aspiring to become a member. 17 IV. Education quality assurance systems in benchmark systems 54. This section applies the conceptual framework developed to study and synthesize the education quality assurance institutional arrangements of the selected benchmark countries. We first describe the criteria employed to select adequate countries for comparison, and then we analyze each benchmark education quality system using the conceptual framework developed inSection 111. IV.1. Selection of benchmark education quality assurance systems 55. The conceptual framework developed in this study posits that an effective education quality assurance system explicitly addresses each of the eight functions described inSection 111.Inorder to understand how these functions are distributed among the different participants of the education system, we first reviewed the institutional frameworks used in other countries with effective education quality assurance systems. To narrow the scope of our analysis, we used three criteria to select countries for comparison. Specifically, we selected countries that meet the following criteria: (i) countries with decentralized education systems; (ii)countries that explicitly aim for the achievement of high performance standards by all students; and (iii)countries that have demonstrated highperformance ininternational assessments of student learning. 56. Based on these criteria, we first selected seven education systems: Finland, New Zealand, three constituent countries of the United Kingdom (England, Wales and Northern Ireland) and two districts in the United States of America (Boston, Massachusetts and Houston, Texas).' To broaden the range of systems for comparing Chile's experience, we also analyzed two education quality assurance systems that do not meet some or all of the criteria above. First, we selected the Republic of Korea, which has a highly centralized education system. The decision to include this country however, stemmed from the desire to include a high performing Asian country in our sample of successful comparator countries. Finally, we chose to include Spain, which is not as high a performer in international assessments of student learning as the rest of the countries included in our study. However, the consultations carried out throughout the study suggested that it was important to include a country that was more culturally similar to Chile inour comparator sample. 57. There are important differences in the income per capita level of Chile and of the countries selected for comparison. These differences, of course, affect Chile's capacity to While the Netherlands has an education system that is similar to Chile's with a large share o f total enrollment in publicly-financed private schools, it does not meet the second selection criteria above, namely, that it has as an explicit goal to have high performance levels by all students. Indeed, currently there is political pressure in the Netherlands to reduce national attainment targets in primary education to provide schools with more autonomy in selecting achievement standards (Dutch Eurydice Unit,Ministry of Education and Culture 2006). 18 invest in education and, consequently, to have strengthen its education quality assurance system (see Figure 7). While our selection of benchmark education systems focuses on learning from their institutional design in order to derive useful lessons for Chile, our analyses recognize that it will take increased investment and time to implement the options we suggest to strengthen education quality assurance inChile. Figure7: Per capita GDP andper-pupilspendingin primaryeducation(inconstant US$ from 2000) inChile and selectedcountries 40,000 I 82 0 Chile ( E l 0 Fed Kingdom UnitedStates GDP per capita 2003 Per-pupilspending in primary2003 58. Table 4 through Table 12 in the Appendix describe, for each of the selected countries (including Chile), the eight quality assurance functions played by participants in the education system. The remainder of this section presents a brief overview of the main functions of the quality assurance institutional framework ineach country. IV.l.l.Finland 59. The education system in Finland is characterized by the vertical allocation of institutional responsibilities, though decentralization trends are evident throughout the system, not just at the local level but deeply ingrained at the school level. At the central administration level, education falls within the purview of two national institutions: the Ministry of Education and the FinnishNational Boardof Education (FNBE).Government authority at the regional level is exercised by the Provincial State Offices. Although these entities are endowed with an Education and Culture Department, education management 19 and administration is not primarily effected at the regional level." Local authorities are responsible for organizing basic education at a local level. The state grants some operating licenses for private schools, but almost all schools providing basic education are maintained by local authorities." Private schools are publicly funded and under public supervision; in other words, they follow the national core curricula and the requirements of the competence-based qualifications confirmedby the FNBE. 60. Regarding the statutory distribution of functions, the Ministry is mainly responsible for preparing educational policy and legislation, while the FNBE, working in close co-operation with the Ministry, is the national agency in charge of education development. The FNBE elaborates and approves national curricula and qualification requirements, carries out evaluations of learning results and offers information and support services. Although ministries direct the central boards in general, they do not intervene in their individual decisions. Thus, the FNBE is comparatively independent within its own field and publicly liable for the legality of its actions. 61. Beyond the national level, the next substantial concentration of education authority is found within municipalities at the local level. Local authorities have the statutory duty to ensure education provision. In addition, they are also responsible for providing student welfare services and ensuring, through direct provision or outsourcing, the delivery of a number of educational services. Municipal governments share the responsibility of financing education with the central government (Ministry of Education). 62. As a result of decentralization, powers concerning the curriculum and its implementation have increasingly been transferred to schools. Education is steered in a more customer-oriented manner in accordance with the objectives set by the educational institutions and their maintaining bodies themselves. Consequently, educational institutions have become differentiated and the options they provide have multiplied, increasing the need for information through evaluation. In this sense, another national agency providing important support to the Ministryof Educationis the FinnishEducation EvaluationCouncil, responsible for planning, coordinating, managing and developing the evaluation of education. IV.1.2. New Zealand 63. The institutional structure of the education system in place inNew Zealand since the reform of 1989 can be described as a heavily decentralized framework. Individual schools have considerable responsibility for their own governance and management, working within the framework of requirements, guidelines and funding set by central government and administered through its agencies. Within this framework, the allocation of roles and responsibilities for quality assurance is distributed between individual schools and national government and agencies, with no significant participation of government entities at the regional or local levels. lo The authority of Provincial State Offices does not extend significantly beyond the monitoring and evaluation of the serviceability of the school network and satisfaction of education demand. In2004, private schools served roughly 3 percent of all compulsory school enrollment. 20 64. At the national level, central government responsibility for education is generally divided between the Ministry of Education and the Education Review Office (ERO). The faculties and responsibilities of the Ministry include setting education policy direction and overseeing the implementation of approved policies, developing curriculum statements, allocating funding and resources to schools, providing and purchasing services for schools and students, collecting and processing education statistics and information, and monitoring the effectiveness of the education system as a whole. ERO is accountable to a Minister separate from that of the Chief Executive Officers of the Ministry of Education. The principal remit of this institution is to evaluate the performance of individual schools and report evaluation results publicly. 65. Also at the national level, a number of agencies operate in an independent manner, accountable to individual governing boards and not reporting directly to any Minister of the Crown. Of particular relevance are the Teachers Council and the New Zealand Qualifications Authority (NZQA). The former is responsible for registering teachers, removing their practicing certificates and approving teacher education programs that can lead to registration. All practicing teachers fall under the remit of this agency, including those in private schools. Meanwhile, the NZQA sets and reviews standards as they relate to qualifications, provides awareness about qualifications quality, oversees the curriculum system and sets all secondary school and many tertiary examinations.l2 66. Administrative authority for most education service provision is devolved away from central government to the educational institutions. For example, schools exercise discretion in the spending of their operational budget, teaching and area allocations. Schools have also earned important autonomy in terms of governance, as exemplified noticeably in the existence of school charters. The school charter is an integral part of school self-management because it reflects the mission, aims, objectives, directions and targets of the board that will give effect to the National Education Guidelines and the board's priorities. Although there is diversity in the forms of institutions through which education is provided, national policies and quality assurance provide continuity and consistency across the system. The school charter itself provides a base against which the school's performance can be assessed. In this sense, private schools (serving roughly 4 percent of all compulsory school enrolment) are also state regulated as their registration i s dependent upon premise, equipment, staffing and curriculum standards. IV.1.3.U.K.:England,Wales andNorthernIreland 67. England, Wales and Northern Ireland are three of the constituent nations of the United Kingdom.l3 Government responsibility for education was radically altered by the UK Government devolution of legislative powers to Scotland, Wales and Northern Ireland in 1999. The Scottish Parliament and Northern Ireland Assembly gained legislative authority in domestic affairs, including the education field.l4While the 12 The remaining national agencies are Special Education Services (SES), Career Services, Education and TrainingSupport Agency (ETSA) and Early ChildhoodDevelopmentUnit (ECDU). l3 Scotland is the fourth and final constituent nation. 14 Due to political turbulence, the Northern Ireland Assembly was suspended in October 2002 and its powers returned to the Government at Westminster. The Secretary of State for Northern Irelandassumed responsibilityfor the directionof the NorthernIrelanddepartments. 21 National Assembly for Wales (NAfW) acquired secondary legislation-making powers, the responsibility for primary legislation in Wales remained with the UK Parliament. These arrangements are clearly evidenced inthe nature of education regulations inWales, which are broadly similar to those inEngland. 68. Education in England, Wales and Northern Ireland is managed and administered at the national and local levels, and not generally affected at the regional level. DfES in England, D f E in Wales and DE in Northern Ireland are the government departments responsible for education at the national level, while at the local level management and administration of education is the responsibility of Local Authorities (LAs) in England and Wales and Education and Library Boards (ELBs) in Northern Ireland. Governing bodies of educational institutions have a high degree of autonomy over the management of their institutions. 69. Central government has powers and responsibility for the total provision of the education services. The Education Departments in England, Wales and Northern Ireland determine national education policy and legislation and plan the direction of the system as a whole. They are also responsible for strategic planning and accordingly, for financial and resource allocation. Education Departments receive important support from two sets of national agencies: qualifications, curriculum and assessment authorities (QAs) and inspection authorities (IAs). QAs (QCA in England, ACCAC in Wales and CCEA in Northern Ireland) are non-departmental public bodies sponsored by and reporting to their respective Education Departments. Their main statutory function is to advise government on matters affecting the school curriculum, pupil assessment and publicly funded qualifications. IAs (Ofsted in England, Estyn in Wales and ET1in Northern Ireland) are non-ministerial government departments responsible for the independent management of the school regulation and inspection systems. Their duties include the inspection of educational services providedby local authorities. 70. Local government and individual institutions implement and administer national and regional policies and also have their own statutory powers and responsibilities. Local authorities are statutorily responsible for ensuring education provision and exercise discretion over, among others, the school funding formula and staffing of schools. The extent of local authority control over the education system however, has diminished in recent years, as public educational institutions themselves are increasingly being granted more administrative and managerial autonomy.l5 Schools have delegated spending discretion on their budgets and hold autonomy over such aspects as the admission policy, teaching methodologies and the school term. A recent publication by the D E S explained the new relationship between local authorities and schools as one where "good schools manage themselves" and local authorities "only intervene in schools" management in inverse proportionto those schools' success. l5 Publicly-funded schools in England and Wales include: i)Community Schools, set by LAs; ii) Foundation Schools, owned by the school trustees or school governing body; and iii)Voluntary Controlled and Voluntary Aided Schools, owned by the school trustees or funding body. InNorthernIreland, publicly- `` funded schools include: i)Controlled Schools, owned by ELBs; Catholic-maintainedSchools, owned by the Catholic Church; iii)Grant-maintained Integrated Schools, owned by school trustees of the board of overnors; and iv) Voluntary Grammar Schools, owned by the school trustees or fundingbody. Department for Education and Employment (DfEE). 2001. "Code of Practice on Local Education Authority - School Relations (DjEE Circular 0027/2001) London. ". 22 71. Maintained schools in England and Wales, and grant-aided schools in Northern Ireland,refer to schools, private or public, which are publicly funded. Inaddition to these schools, private schools also operate in England, Wales and Northern Ireland. Private schools define their own school ethos and are financed by parent fees. Although they are exempt from most regulations applicable to publicly-maintainedschools, private schools are still subject to minimum operating requirements set by the state regarding healtldsafety, reporting, welfare and education quality standards. Consequently, private schools are also subject to external inspections. IV.1.4.USA.: Massachusetts/Bostonand Texas/Houston 72. Incontrast to the other countries described inthis report, education inthe U.S. has historically been the purview of states and local school districts, rather than the national government. States prescribe parameters such as the manner in which school districts are established and governed, age of compulsory student attendance, performance standards for students, licensing requirements for school personnel, school operating requirements, and provision of funding, through legislative statute and state board policy. District boards and administrations are then tasked with translating these parameters into policies and practices for the provision of education locally. The majority of states also authorize . the operation of publicly funded charter schools, which receive freedom from state and district regulation in exchange for compliance with their charters, including student performance. 73. Given the U.S. emphasis on local control, this analysis focuses on two state- district pairs: MassachusettdBoston and TexasEIouston. These education systems are characterized by strong standards and assessments at the state level, combined with autonomy and support at the school and district level that foster an array of curricular and instructional options. Inaddition, BostonPublic Schools (BPS) and HoustonIndependent School District (HISD) have been noted for effectively serving diverse populations that include a high percentage of low-income students, which increases these systems' relevance to Chile despite the relative wealth of the U.S. 74. In the past two decades, standards-based reform has become the dominant paradigm in U.S. education. B y national law, each state develops standards for what students should know and be able to do at each grade level and subject area. States are then required to assess and report student performance in different grades and subjects annually, for students as a whole and disaggregated by race/ethnicity, language status, and socioeconomic status, to spotlight any inequity in educational outcomes. Massachusetts has been lauded" for the clarity and caliber of its standards, termed Curriculum Frameworks, which encompass the arts, English, foreign languages, health, history and social studies, mathematics, and science and technology from Pre- Kindergarten (age 4) to grade 12 (age 18). Texas has been recognized18for linking its primary and secondary education system with postsecondary education and the l7"2006: The State of State Standards", Thomas B.FordhamFoundation,August 2006 "Closing the ExpectationsGap 2006", Achieve, Inc., February2006 23 workplace, through academically rigorous graduation requirements, the use of secondary school assessments for postsecondary admissions and placement decisions, and a longitudinal data system that enables the State to track individual student outcomes from Pre-Kindergarten to the postsecondary level. 75. Both of these state-district systems provide autonomy and support to a diverse portfolio of schools, from which students and families can choose to meet their educational needs and interests. Massachusettsand Texas both offer charter schools, and students inHoustonhave the option of attending either Texas or HISD charter schools, in addition to traditional public schools. HISD grants traditional schools authority over their curriculum, instructional methods, personnel, and budgets in exchange for the demonstrated attributes of strong leadership, a high-functioning team, and a vision and plan for the school. Massachusetts mandates participatory management at the school level, and BPS aligns this decision-making authority with its broader whole school improvement framework. Under BPS policy, each School Site Council must approve its school's whole school improvement plan and discretionary budget, among other functions. BPS then evaluates the schools using a rubric aligned to its school improvement framework. Furthermore, BPS provides intensive support to new and existing teachers and principals. Both BPS and HISDhave responsibility for intervening" inunder-performing schools andpersonnel. IV.1.5.Republicof Korea 76. The centralized nature of the education system in South Korea makes it the most distinct from the group of countries analyzed. The education administration consists of three layers of authority: the Ministry of Education and Human Resources Development (MOE&HRD) at the national level, the supervisors of education at the metropolitan and provincial level, and the district boards of education. MOE&HRD drafts, plans and coordinates national education policies, develops the national curriculum, publishes and approves school textbooks and teaching guides, provides administrative and financial support for schools, supports local educational agencies and operates the teacher training system. The Ministry delegates some budget planning processes and administrative decisions to municipal and provincial education authorities (MPEAs), or metropolitan and provincial offices of education (MPOEs), at the regional level. MPEAs and W O E S inturn delegate certain responsibilities to the local office of education (LOE). Under this structure, the role of individual education institutions is primarily to implement the policies and regulations defined by the government. Schools abide by nationally mandated subjects, contents, textbooks, time allocation, curriculum organization and implementation guidelines, teaching, assessment and reporting guidelines and school ''While determining and imposing consequences for performance has traditionally fallen to districts and more recently to charter authorizers, the U.S. government became dramatically more prescriptive in this area with the 2001 No Child Left Behind Act, which lays out a graduated set of sanctions for schools failing to meet state-establishedperformance targets. Students attending schools that fail to make Adequate Yearly Progress (AYP) are eligible to transfer to other schools or receive free tutoring services, for which districts must pay using a portion of their national funds. Schools that persistently fail to make AYP may be required to replace staff, adopt a new curriculum, decrease management authority at the school level, and ultimately face takeover by the state or another outside entity. Systems such as BPS and HISD must therefore comply with the accountability requirements set forth in the U.S. law, in addition to their own performance requirements. 24 administration guidelines as well as provincially determined staffing and operational guidelines. Even private schools, which account for roughly 30 percent of all education institutions," are subject to the national curriculum, student enrolment and staff regulations. 77. Recent trends in Korean education policy have started to move the education system towards a more decentralized framework. Inthis sense, recent curriculum reviews have aimed to "loosen the rigid and centralized curriculum framework" and encourage schools and individual teachers to become actively involved inthe decision and planning process for the curriculum (O'Donnell, 2002). In 1995, the Presidential Commission on Education Reform (PCER) recommended that, to improve the quality of school education, the process of increasing local self-government in education should continue, regional distinctiveness should be respected, and the autonomy of individual schools expanded. As a result, some schools were required to set up experimental School Management Committees comprising parents, teachers, principals, community leaders and education specialists with deliberative, consultative and decision-making powers. However, this initiative had limited success largely as a result of principals' concerns about the possibility of excessive parental interference in school internal affairs. Albeit small adjustments, the education system inSouth Korea remains highly centralized. IV.1.6. Spain 78. The education system in Spain distributes quality assurance roles and responsibilities among the State, Autonomous Communities, municipal authorities and education institutions. As stated in the constitution, the State retains authority to ensure the unity, homogeneity and equity of the education system and this power is held by the Ministry of Education. This right translates into a series of areas over which the State has exclusive competence, which include matters relating to the length of compulsory schooling, the levels and cycles of the system, minimum education standards, school operating requirement, staffing qualifications and levels, funding and inspection. Aside from its normative competencies, the State also holds executive responsibilities. Fundamental among these are the coordination and promotion of educational research and the High Inspection services that ensure compliance with legal requirements. Autonomous Communities assume all regulatory and executive responsibilities not included within the State's exclusive area of competence. Autonomous administrations' responsibilities include the setting up and authorization of establishments, personnel administration, expansion and development of education programs, student counseling, grants and loans. Although Town Councils do not have `education authority' status, Autonomous Communities can delegate powers to these municipal entities. Town Councils are usually responsible for the provision of land for building public establishments, conservation, maintenance and renovation of schools, development of programs for extra-curricular and complementary activities and supervision of compulsory school attendance. 2oThe majority of private schools inKoreaare inthe pre- and post-compulsory education levels. 25 79. Educational institutions in Spain maintain a great degree of autonomy. Schools' pedagogical, organizational and economic autonomy is officially asserted inthe country's regulations. Pedagogical autonomy is manifest in the schools' right to choose pedagogical programs and determine an educational project, thereby setting its own education priorities and objectives. Schools exercise organizational autonomy in their definition of annual programs and internal regulations. Annual programs establish the schools' organizational and curricular plan while internal regulations address student rights, responsibilities and disciplinary requirements that observe State regulations. Finally, school economic management autonomy materializes in schools' discretionary power over expense allocations, construction and material sourcing. Individual institutions are supervised in their administrative and managerial work at two different levels. The State's High Inspection Service supervises and enforces compliance with basic state regulations. This inspection is effected, not only at the school level, but also within the Autonomous Community as a whole. At the same time, the Autonomous Community itself carries out a technical inspection of schools. This technical inspection evaluates the achievement of educational objectives by looking at management, administration, functioning, results, legal requirement compliance and education quality. 26 V. Instructional visions for education provision and quality assurance 80. The previous section reviewed a broad spectrum of institutional frameworks for quality assurance in education and revealed wide differences across the systems. This variability in education quality assurance systems suggests the existence of alternative instructional visions, which has direct implications for the institutional arrangements for education quality assurance within a system. Based on the sample of countries reviewed, this report identifies four alternative instructional visions. This section introduces these visions and describes their main characteristics. 81. The experience of benchmark countries reviewed suggests that success in ensuring education quality does not depend on the application of a specific institutional framework. For example, while in Finland the institutional functions of policy-making and administrative support are separated between two autonomous institutions (The Ministry of Education is responsible for the former and FNBE, working under the auspices of the Ministry for the latter), the Republic of Korea entrusts both these duties to a single institution (the Ministry of Education). Similarly, while in England, Wales and Northern Ireland, teacher certification is carried out by independent professional bodies (General Teaching Councils), in New Zealand this function is fulfilled by an arm of the Ministry of Education (New Zealand Teacher's Council). Although the evidence indicates marked differences in quality assurance functions across countries, with the exception of Chile and, to a lesser degree, Spain, all countries analyzed have effective education quality assurance systems. 82. While they vary in how education quality functions are carried out and by whom, successful comparator countries are consistent in having a regulatory framework for quality assurance functions that encompasses most education system participants. In addition, one common factor among the majority of countries with successful quality assurance systems is the institutional separation of bodies responsible for policy making and service provision, and for monitoring and evaluation, and oversight. In other words, although the institutions responsible for quality assurance functions differ across countries, there is a statutory attribution of quality assurance responsibilities for each participant of the education system to a specific institution. In contrast, the review of in Chile's current education quality assurance (Table 4) indicates that in many cases the main quality assurance functions identified in the conceptual framework are either not explicitly defined and, consequently, developed, or they are weakly defined and carried outI 83. The review of the benchmark systems suggests that each system studied has adopted a particular instructional vision that then guides its institutional framework. Indeed, it appears that a country's success in ensuring quality is intrinsically tied to the consistent and comprehensive application of its instructional vision. Based on the review of the quality assurance systems in Section 111.2, we can distinguish four alternative instructional visions, which we have labeled: (i) limitedstate; (ii) contracts; (iii) quality 27 differentiated instruction; and (iv) managed instruction.21Each of these visions implies a different institutional distribution of quality assurance functions. We can place these visions along a continuum describing the degree of coupling between what is dictated by the central government and what is implemented at the school level. Broadly speaking, as sequenced above, a gradual movement from a limited state vision at one end of the continuumto one of managed instruction at the other implies an incremental expansion of central government control along with a simultaneous reduction inschool autonomy. Figure 8: Four Alternative Instructional visions Placed along a Continuum of School Autonomy-Central Government Control Control I I I 84. It is important to note that no country implements a perfect application of any one instructional vision. Although for the most part each country's education system exemplifies a particular instructional vision, each system allows for certain exceptions to the general model, or elements of other visions, to coexist within the larger structure. In England, for example, while the lion's share of the education system follows a differentiated instruction vision, some schools operate under conditions that are distinctive of a quality contracts vision. Similarly, while Finlandhas principally opted for a differentiated instruction vision, the State has held a very active role inthe maintenance of some special education and language schools. V.l. Analysis of alternative instructional visions 85. This subsection describes each of the instructional visions separately, analyzing each vision's main characteristics. The instructional frameworks of the education system in Chile and in benchmark countries are described in the context of the corresponding instructional vision. 86. Limited State. The limited state instructional vision i s based on the premise that market forces will act as a quality assurance instrument if the education system is allowed to function as a competitive market with minimal state intervention. In this system, the role of the central government is limited to: (i)establishing minimum operation requirements and reporting standards; (ii) financing schools on a per-student basis; and (iii)providinginformationto themarketto facilitateinformedschool choice. Schools 2' The concepts presentedinthis sectionwere stronglyinfluenced bythework ofDonaldR.McAdamsin `%at School Boards Can Do: Reform Governance for Urban Schools" (2006). 28 have discretion over their choice of performance standards, performance assessment mechanisms and the model of instruction. Meanwhile, students have the right to decide which school to attend. In theory, school choice, perfect information on the quality of education provided by schools and free entry and exit of schools should ensure that only good schools survive while poor performing schools are driven out of business for lack of demand. In this sense, it is market forces that define the quantity and distribution of schools inthe country. 87. Among the countries analyzed, Chile is the only one identified as having a limited state instructional vision. The Chilean education system is based on the premise that parents should have the freedom to select the most adequate school for their children, and that schools compete for students based on their quality. The government establishes minimum operation and reporting requirements regarding student attendance, finances public and private schools based on a per-student formula, and provides the market with information on school quality, arguably to an insufficient extent. While publicly financed schools must follow a national curriculum, to date the Government does not hold schools accountable for meeting established performance standards or adhering to specific instructional models. 88. Although Chile's instructional vision is identified broadly as limited state, since 1990, a number of education reforms have been implemented that in fact gradually introduced increased Government control ineducation. These reforms include: Mece Bhsica and Mece Media, two large-scale programs partially financed by The World Bank that supported infrastructure improvements in primary and secondary schools, respectively, as well as provided libraries, teaching materials, technical assistance, and support to school administration and teaching staffs; Targeted compensatory programs, including: P-900 (financial and technical assistance to under-performing primary schools, which reached around 11 percent of total primary school enrollment); Rural Education (targeted over 3,000 small rural schools by supporting teacher training and the provision of curricular and other teaching materials, which reached around 5.9 percent of total primary school enrollment); Montegrande (an innovation incentive that covered around 50 secondary schools and about 5 percent of total secondary school enrollment); and Liceo para Todos (technical and pedagogic support to underperforming secondary schools, which reached around 33 percent of total secondary school enrolment); Teacher Professional Development, including (i)initial teacher training reforms; a program to finance teacher internships abroad; and (iii)a teacher education curricular reform; Enlaces, an information and technology program that introduced computer labs in all secondary schools and 85 percent of primary schools in Chile, provided two years of training to teachers in IT, and developed a university network of technical support for the Enlaces program; Sistema de Medicidn de la Calidad de 10s Establecimientos Educacionales, or SZMCE,a national student learning assessment system introduced in 1988 that 29 has been regularly applied to all students in the 4*, 8*, and 10* grades every two years consistently.22 89. Inspite of these reforms, which represent a gradual shift from school autonomy to increased Government control, the fundamentals underpinning the limited role of the State in the education system in Chile remained unchanged throughout the past decades. Inthis sense, the system continued to bebased on the notions that parental school choice and competition among schools should lead to high education quality. As a result, one can conclude that the Chilean education system is most closely aligned with a limited state instructional vision. 90. Quality contracts. As we move along the continuum towards an instructional vision based on quality contracts, the distribution of quality assurance functions starts shiftingtowards the central government. Inthis second instructional vision, inadditionto the functions executed ina limited state vision, the state is responsible for: i)granting and revoking operating licenses; ii)establishing standards for performance and performance assessment; and iii)implementing performance assessment in schools. The power to grant and revoke licenses implies that under a quality contracts vision the state can actively influence the quantity and distribution of schools. Market forces however, continue to play an important role inthis allocation, as the decision of individual schools to apply for licenses is still demand driven. Even though schools are obliged to comply with statutory performance standards, under a quality contracts vision individual establishments still hold discretion over the instructional model and evaluation methodologies used to achieve such given standards. 91. New Zealand is one of the most compelling examples around the world of a coherent application of a quality contracts instructional vision. Statutory performance standards for all schools are set by the central government. State schools however, serving 96 percent of all children, establish individual school charters with specific goals and targets for student outcomes. Inorder to be allowed to operate, all schools, including private ones, must register with the Ministry of Education. Registration is dependent on the outcome of an evaluation of premises, equipment, staffing, and curriculum carried out every three years. When a school evaluation suggests poor performance, the state exerts pressure for improvement by performing discretionary reviews. Although external help may be provided by trustee's/principal' dteacher's associations or purchased from counseling agencies, the state does not intervene directly in poor performing schools. Finally, the student's right to school choice implies that market forces have a strong influence on the creation and distributionof schools around the country. 92. Differentiated instruction. Under the differentiated instruction vision there is a significant increase in the quality assurance functions served by the central government. In addition to the duties performed under a quality contracts vision, the state holds the following responsibilities: i)establishing standards for staff accreditation; ii)coordinating among schools to guarantee a balance of educational options; iii) intervening differentially ineducational establishments; and iv) providing diverse services for schools and professional development options for staff. The autonomy of schools under a 22Source: C. Cox, ed., (2003). 30 differentiated instruction vision is generally limited to defining their instructional model, though it can also include decision-making about human resources, such as the recruitment and termination of personnel. The state has a very active role in creating networks of information sharing so as to promote successful models and programs. Although the central government decides on the ultimate number and distribution of educational establishments, market forces continue to operate, to a lesser degree, because students are allowed to choose between schools. A key difference between a differentiated instruction vision and one based quality contracts is that inthe former when a school is performing poorly, the state actively intervenes to improve the quality of education providedrather than simply revoking its operating license. 93. A majority of the successful education systems analyzed here are based on a differentiated instruction vision. This is the case of the educational systems currently operating inEngland, Wales, Northern Ireland, Finland, Spain, MassachusettsBoston and Texas/Houston. In all these systems, the central government permits an array of instructional models to coexist and bases its intervention on school performance. Those schools achieving good performance are granted a high degree of administrative autonomy while those exhibiting poor performance are intervened by the state. School intervention options are diverse as the central government tries to account for differences inexisting instructional and administrative models. 94. Managedinstruction.The final instructional vision is one based on a system of managed instruction. Under this vision, the central government takes responsibility for practically all quality assurance functions. Two state duties that are unique to this system are: i)defining a single statutory instructional model; and ii)centrally assigning students to school. These functions consequently affect the way in which the state serves its other functions. Thus, school services and professional development options provided are not diverse, but uniformly targeted to the instructional model inplace. Similarly, intervention in poor performing schools is also uniform in nature. Market forces don't play any significant role ina managed instruction system and the role of schools is simply to act as implementingagencies for the policies centrally mandated by the state. 95. Among the countries analyzed, the Republic of Korea opted for an educational system based on an instructional vision of managed instruction. The central government establishes a national curriculum, performance standards and an instructional model. To support this model, the government publishes and authorizes permitted textbooks and provides them to students free of charge. Continual professional development and school improvement services are provided to ensure adherence to the instructional model. Finally, students are assigned to schools in their residential area by lottery, impeding market forces from influencing school allocation. V.2. The control-model coordinates analysis 96. Based on the descriptions presented above, we can place the four alternative instructional visions in a "control-model coordinates" graph (Figure 9). The number of existing instructional models will determine the instructional vision's placement along the "model axis." Meanwhile, the degree of school autonomy versus central government 31 control in any given system will determine the vision's positioning along the "control axis." Figure 9: Instructionalvisions ina "Control-Model Coordinates" Graph Schools Central Government 97. Three instructional visions are located inthe upper-rightquadrant of the "control- model coordinates" graph. Visions in this quadrant permit different instructional models to coexist in a relatively decentralized control system. The degree of school administrative/managerial autonomy decreases and the degree of state control increases as we move from a limited state vision to one of quality contracts and subsequently to a vision of differentiated instruction (i.e. a movement down the "control axis"). The managed instruction vision is the only one situated in the lower-left quadrant. In this quadrant, a single or small number of instructional models exist in a relatively centralized control system. 98. As Table 2 shows, the roles and responsibilities of Government vary depending on the instructional vision adopted. Indeed, in a system that adopts a limited state instructional vision, many of the responsibilities that governments would have to undertake in the other visions are instead carried out by the market. Similarly, while in the quality contracts vision, the Government develops operating requirements and performance standards and grants or revokes licenses to participants based on whether they do or do not meet these requirements and standards, in the differentiated instruction vision, the Government also intervenes differentially in schools to support their achievement of established standards. Inthe managed instruction vision, in contrast, the Government enforces the adherence to a unique curriculum by directly managing schools. 32 d vi w d c j r; +vi + r i & + vi w E m m I ri m 3 ri m 3 ri 'c 0 - 3 r i m r i m 3 ri L 3 ri VI. Policy options to strengthen the institutional arrangements for education quality assurance in Chile 99. Based on the conceptual framework developed for this report, its application to benchmark education systems and the identification of instructional visions for education quality provision and assurance, in this section we discuss policy options to strengthen the institutional arrangements for education quality assurance in Chile. We first present a set of general guidelines in which our proposed institutional allocation of roles and responsibilities i s grounded. 100. A key underlying guideline is to select an instructional vision and apply it consistently and comprehensively. A critical stumbling block faced by the Chilean education system in recent decades appears to have been a mismatch between its instructional vision, which was founded on a limited state vision, and its application to education provision and quality assurance functions. For example, the centralized definition of teacher compensation and weak education quality information systems are not fully consistent functions in a limited state instructional vision. 101. When selecting an instructional vision, it is important to realize that no single instructional vision is most effective in ensuring education quality. The international evidence reviewed for this report shows that countries with successful quality assurance systems have applied very different instructional visions. Indeed, there is evidence of successful countries in almost all the instructional visions identified here. The only exception to this rule is the limited state vision. Although we cannot discard the possibility that a proper application of a limited state instructional vision could lead to successful education quality assurance, we could not identify an exemplary country with a market instructional vision. One possible exception is the Netherlands, which has achieved impressive results in international assessments of student learning. However, in the Netherlands there is no explicit goal to achieve high performance levels among all students and, instead, from early rades students are tracked in different paths with varying performance expectations?' Thus, a second general guideline is that Chile may consider moving away from the limited state instructional vision. 102. As mentioned before, many instructional visions can potentially leadto successful quality assurance in education. Nonetheless, existing quality assurance mechanisms as well as historical, political and social factors, suggest that some models might be more cost- and time-effective than others in ensuring a quality education system for Chile. As Figure 9 shows graphically, the current limited state instructional vision places Chile's education system in the top right comer of the "control-model" coordinates' graph. In light of this starting point, a movement towards a managed instruction vision - in the opposite quadrant - would imply the most significant need for restructuring, and 23For a recent review of the Dutch education system, see Dutch Eurydice Unit, Ministry of Education, Culture and Science (2006). 35 therefore, the most cost- and time-consuming avenue towards an effective education quality assurance system. 103. Given that effective quality assurance is possible via more efficient avenues (Le. quality contracts or differentiated instruction), a more realistic option is for Chile to consider moving to a mix of the Quality Contracts and Differentiated Instruction visions. The coexistence of these visions within one system is actually not uncommon. InEngland and many districts of the U.S., for example, special character charter schools function under quality contract rules within dominantly differentiated-instruction systems. 104. The specific assignment of individual schools to one of the two instructional visions could be based on a number of alternative criteria. A first option is to maintain publicly-financed schools (municipal and private-subsidized schools) under a differentiated instruction vision, while allowing private-paid schools to function under a quality contracts vision (see Figure 10). This distinction would imply that private-paid schools would be managed in a Quality Contracts instructional vision, inwhich the main quality assurance mechanism is based on the granting or revoking of licenses to operate. Meanwhile, publicly-financed schools, when underperforming, would have the possibility to be intervened and supported by the State prior to closure. This distinction however, would (and perhaps should) not exclude private-paid schools from having to comply with the same operatingheporting requirements as well as student learning and school/staff assessment standards as the rest of the schools inthe system. Figure 10: Options for Chile's instructional vision I I I I Privately Publicly financed financed schools schools (municipal or private) 105. This first option for distributing schools between the Quality Contracts and Differentiated Instruction visions is based on the source of finance. A second option is to distribute schools by dependency, assigning municipal schools to a differentiated instruction vision while allocating all private schools, paid and subsidized, to a quality contracts' vision. A third option includes distributing schools based on their performance levels, irrespective of dependency or funding source. Under this criterion, high- performing schools would earn autonomy and be treated under quality contracts, while poor performing schools would lose autonomy and fall under the differentiated instructionrealm. 36 106. Equity considerations may weigh against the latter two options. Assuming private-subsidized schools are not allowed to select students, there is no technical reason why a municipal school should access more support than a private school. Similarly, from an equity perspective it is difficult to justify the use of public resources (Le. school interventionand support) to support private paid schools, which regularly select students. In contrast, the underlying notion to support the first option - that the decision for allocating schools into a Quality Contracts or Differentiated Instruction vision be based on the source of finance - is that any additional resources should be directed to ensuring high levels of education quality for all students, independent of hidher school choice (municipal or private). It is clearly important, however, that both types of schools (municipal and private) that receive public resources be subject to the same type of regulatory framework especially related to the ability to select students. VI.l. Implications for the institutional allocation of roles and responsibilities 107. As mentioned earlier, adopting an instructional vision has some important implications for the allocation of educational assurance functions across institutions. In this section, we first concentrate on the allocation of quality assurance functions across the three main institutions in the proposed reformed education system, namely, the Ministry of Education, the Superintendency of Education and the National Education Council. Then, we discuss options for the institutional allocation of roles and responsibilities for different administrative and managerial levels within the former two institutions (the National Education Council operates only at the central level). It is important to highlight that this report focuses on quality assurance functions in the education sector. In this sense, the discussion that follows does not address the many other functions that the Ministry of Education carries out related to education provision, such as provision of scholarships, norms and regulations, etc. 108. Allocation of functions across agencies. An important characteristic of successful education quality assurance systems involves clarity in the roles and responsibilities of each institution. Inthis sense, the mandate of the three main agencies could be distinguished as described in 109. Table3. Table 3: BroadMandates by Institution Policy and Oversight, Approvalof program design, measurement and curricular implementation reporting frameworks and intervention 37 110. The main notion behind this allocation of general functions is to separate the oversight, measurement and reporting responsibilities from those related to policy and programming as a mechanism to introduce accountability at the national agency level. The Council's main role also relates to accountability. This entity embodies a warranty against government policy and programming conflicting with societal interests and thereby infuses democratic principles into the system. 111. The analysis of benchmark education systems discussed in Section IV suggests that a regulatory framework that explicitly assigns quality assurance functions for all participants of the education system may be a necessary condition for effective quality assurance. Inthis sense, and in light of the broad mandates described above, we propose roles and responsibilities for each institution in each of the eight quality assurance functions that integrate the conceptual framework. Appendix B provides a detailed description of the proposed function allocation. 112. The allocation of functions across institutions and within levels that is suggested inAppendix Bremains faithful to the overarching institutional mandatesproposedin 113. Table 3. Taking, as an example, the function requirements to operate, the Ministry's purview would relate to the design and implementation of policy and regulations. Inthis case, the Ministry would define the operating requirements for each of the system's participants and propose the appropriate legal mandates to Parliament. Inthe specific case of teachers, the Ministry would grant and revoke teaching licenses based on compliance with the established requirements (which would be verified and reported to the Ministry by the Superintendency). In addition, the Ministry would be responsible for ensuring that existing law and regulations stay updated by continually revising them and proposing modifications. 114. The competencies of the Superintendency related to the requirements to operate function would be consistent with its mandate to `measure and report' in a similar way. Based on the legal requirements defined by the Ministry and sanctioned by Congress, the Superintendency would be responsible for monitoring and evaluating the compliance of relevant participants. The results of the evaluation would be employed to make recommendations to the Ministry licensure and/or accreditation of participants (e.g. teachers, school principals, schools). The Superintendency would also be responsible for analyzing the information gathered during regular evaluations to recommend to the Ministry of Educationmodifications to the administrative regulations of schools. 115. Function allocation by level within agencies. Having outlined the roles and responsibilities recommended for the three main national agencies in education, this section focuses on suggesting a possible distribution of quality assurance functions by the different administrative levels within each agency. Table 14 in the Appendix provides a detailed description of the recommended level distribution of functions within the Ministry of Education, Education Superintendency and National Education Council. This table also describes the roles and responsibilities of the education providers as this participant holds important quality assurance functions in the two instructional visions recommended. The suggested allocation of functions between agency levels varies by instructional vision. In other words, the relationship of the different agencies with the education providers will not be the same in the case of schools falling under a quality 38 contracts instructionalvision as inthe case of schools treated according to a differentiated instructionvision. 116. Although an analysis of institutional capacities is beyond the scope of this report, we recognize that the distribution of functions put forward implies a significant increase in state roles and responsibilities and will thus require considerable capacity building across the system. To carry out these recommendations effectively, it will be necessary for Chile to review the institutional recommendations in this report with an eye toward existing capacities and modify either the institutional recommendations or the existing institutional capacities so as to ensure efficiency. 39 VII. Conclusionsand Next Steps 117. The creation of a new Superintendency of Education in Chile implies a reallocation of functions across the Ministry of Education, the Superintendency, and other education institutions. The main objective of this report has been to present a conceptual framework for evaluating education quality assurance systems that may aid in the analysis and definition of policy options for the allocation of education quality assurancefunctions across education institutions inChile. 118. To do this, the report applied the conceptual framework to selected education systems throughout the world that are comparable to Chile's current institutional system but have achieved high levels of academic achievement among a majority of their basic education students. Among the systems selected as benchmarks using these criteria are: Finland,New Zealand, three subsystems of the United Kingdom, and two districts inthe United States. In addition, to expand the typology of comparison countries, the report also analyzed the education quality assurance systems in the Republic of Korea and Spain. 119. The review of these nine different education systems revealed the existence of four alternative "instructional visions" for the institutional design of an education system, which we have labeled as follows: (a) Limited State, (b) Quality Contracts, (c) Differentiated Instruction, and (d) Managed Instruction. An important result from the empirical review of education systems in the world is that one can identify successful countries in terms of education quality assurance ineach of the three latter visions. Thus, the decision to adopt a specific instructional vision should be based on social, political, cultural and historical reasons. However, once an instructional vision is adopted, the study indicates that its consistent application may be an important determinant of success ineducation quality assurance. 120. Countries that are most successful in ensuring quality tend to institutionally separate policy development and implementation functions from monitoring, evaluation and oversight functions. However, the distribution of roles and responsibilities for quality assurance across institutions varies depending on the instructional vision that is adopted. In this sense, this report's institutional guidelines and proposed policy options for Chile are based on a particular mix of instructional visions which are considered the most appropriate for Chile given its social, political and historical reality. 121. While the report makes some detailed suggestions, a necessary first next step for Chile is to first make a decision regarding the instructional vision that it wishes to adopt to guide its education quality assurance system. The Chilean education system of today falls under what we call a Limited State instructional vision. Given that, as the international evidence presented in this report indicates, other visions may obtain better results in assuring quality in education, one option for Chile is to consider the possibility of moving toward a combination of two visions: quality contracts and differentiated instruction. These visions capitalize on what Chile has already accomplished within the Limited State instructional vision, in terms of facilitating gains in the efficient use of 40 resources that result from the competition among schools and the per-pupil financing mechanism, by creating a larger quality assurancerole for the State. 122. Once an instructional vision is adopted-with broad support from all important stakeholders-the discussion on the institutional allocation of roles and responsibilities across participants will likely be more constructive. This report suggests dividing the responsibility for education quality control between the Minister of Education, the Superintendency of Education, and the National Council on Education, thereby introducing an external system of checks and balances. Along these lines, we propose that, in general terms, the Ministryof Educationbe the agency responsible for designing, implementing and intervening in policies and programs. The Superintendency of Education would be responsible for oversight-evaluation, measuring and publicizing results-and the national Council on Education would maintain its current mandate of approving curriculum and learning standards. 123. The main idea behinddividing the assignment of general functions is to introduce accountability by separating evaluation and oversight functions from policy and program development. The principal role of the Council will also be related to accountability. The Council can guarantee against possible conflicts between government programs and policies and social interests and, therefore, incorporate democratic principles into the system. 124. This report has taken an in-depthlook at the division of responsibilitybetweenthe Ministry, the Superintendency and the Council, and suggests options for the eight essential functions that make up the study's conceptual framework. Additionally, the report outlines possible divisions of labor amongst the various levels within a single agency-national, regional, and provincial-and among government institutions and sostenedores (an individual, private company or municipality that opens and operates a private subsidized school in Chile). The options for division of labor among institutions outlined in the report correspond to the application of a new mixed instructional vision for Chile, as it would mean introducing quality contracts for private unsubsidized schools and differentiated instruction for subsidized schools, be they public or private. 125. Comprehensively implementing this vision of mixed systems in Chile, as this report proposes, is a long-term goal, since it requires the institutional capacity that at the moment does not exist, or is very weak. Although implementing such a system consistently across the entire education system will no doubt take many years, beginning the process with an explicitly defined instructional vision is an essential condition for fully realizing the process. In other words, without a clear idea of to where the Chilean education system should move, it is highly unlikely that it would ever get there. 126. In order to achieve success in ensuring quality education in Chile, it will be necessary to strengthen the capacity of all of the educational institutions in the country, from the Ministry of Education, to the Superintendency, to school owners, all the way down to schools and teachers. Although an analysis of the institutional capacity of the various participants in Chile's education system is beyond the scope of this paper, we can say with confidence that in order to implement the policy options outlined inthis paper, it will be vital to strengthen the capacities of each of the participants for them to 41 successfully carry out their corresponding roles within an effective quality assurance system inChile. 42 References Aedo, C. 1997. "Organizacih Industrialde la Prestaci6nde Servicios Sociales." Working Paper Series R-302. Washington, D.C.: The Inter-American DevelopmentBank. Aedo, C. and 0.Larrafiaga. 1994. "Educaci6n Privadavs. Nblica en Chile: Calidad y Sesgo de Selecci6n." (Mimeo) Santiago: GraduateEconomicsProgram, ILADES/Georgetown University-Barber,Michael. 2004. "The Virtue of Accountability: system redesign, inspection and incentivesinthe era of informed professionalism." The Inaugural Edwin J. DelattreLecture, Boston University, November4. Ballou, Dale and Michael Podgursky. 1998. "Teacher Recruitment andRetention in Private andPublic Schools." Journal of Policy Analysis and Management, 17(3): 393-417.Boston Planfor Excellencewebsite: http://www.bpe.org/ Barber,Michael. 2004. "The Virtue of Accountability: systemredesign, inspectionand incentives inthe era of informedprofessionalism". The Inaugural Edwin J. Delattre Lecture, Boston University, November 4. Boston Public Schools. 2002. "Whole-School Improvement Rubrics: Theory Guiding the Essentials of Whole-School Improvement" Boston Public Schools website: http://www.boston.kl2.ma.us/ The Broad Education Foundation website: http://www.broadfoundation.orn/home.html Cox, Cristih (ed.). 2003. Politicas educacionales en el cambio de siglo. La reforma del sistema escolar en Chile. Santiago: Editorial Universitaria. DutchEurydice Unit, Ministry of Education, Culture and Science, 2006. "The Education System inthe Netherlands." The Hague (July). Eurydice. Online database, Eurybasewebsite http://www.eurydice.org/portal/page/portal/Eurydice FinishBasicEducation Act 628/1998, available online at: www.finlex fi/en/laki/kaannokset/ 1998/en19980628.pdf . FinnishMinistry of Educationwebsite: http://www.minedu.fi/OPM/?lang=en FinnishNational Board of Education.2002. "Evaluation and Outcomesin Finland: Main Results in 1995-2002". Helsinki, Finland. 43 FinnishNational Boardof Educationwebsite: http://www.oph.fi/english/frontpage.asp?path=447 Fuller,B.andElmore, R.F. 1996. Whochooses?Wholoses? Culture, institutions, and the unequal efsects of school choice.New York: Teachers CollegePress. GeneralLaws of Massachusetts website: http://www.mass.gov/legis/laws/mgl/ HoustonIndependentSchoolDistrict website: http://www.houstonisd.org/ Hoxby, C. 2000. "Would SchoolChoiceChangethe TeachingProfession?" (Mimeo) Cambridge, MA: HarvardUniversity. Institute of EducationSciences website: http://ies.ed.gov/ Kim, Gwang-Jo.2000. "Education Policies and Reform in South Korea". The World Bank. Washington, DC, U.S.A. KoreaEducation& ResearchInformationService (KERIS) website: http://www.keris.or.kr/english/index.jsp Korea Institute of Curriculum andEvaluation (KICE) website: http://www.kice.re.kr/kice/eng/intro/intro 1-2.j sp KoreanEducationalDevelopmentInstitute (KEDI)website: http://eng.kedi.re.kr/ KoreanMinistry of EducationandHumanResourcesDevelopmentwebsite: http://english.moe.go.kr/ Lee, Jinsoon. 2001. "Education Policy in the Republic of Korea: Building Block or Stumbling Block?" The World Bank. Washington, DC, U.S.A. Massachusetts Departmentof Educationwebsite: http://www.doe.mass.edu/ MassachusettsOffice of EducationalQuality and Accountability website: http://eqa.mass.edu/home/index.asp McAdams, DonaldR. 2006. "What School Boards CanDo: Reform Governancefor Urban Schools." New York: Teachers CollegePress. McEwan,P. andCarnoy, M.2000. "The effectivenessand efficiency of private schools in Chile's voucher system," Educational Evaluation and Policy Analysis 33:213-239. McEwan, P. andCarnoy, M. 1999."The impact of competitionon public school quality: longitudinal evidencefrom Chile's voucher system." (Mimeo) PaloAlto: Stanford University.Ministry of EducationandHumanResourcesDevelopment.2006. "Introduction on the Education System of the Republic of Korea 'I. 44 Mizala, A. andP. Romaguera. 2001. "Factores explicativos de 10s resultadosescolares en la educaci6nsecundaria en Chile." TrimestreEcon6mico272 (October-December): 515-549. No ChildLeft Behindwebpage: http://www.ed.gov/nclb/landing.jhtml New ZealandEducationAct 1989, available online at: http://www.legislation.govt.nz/libraries/contents/om~isapi.dll?client~=232262&i nfobase=pal-statutes.nfo&record={ E7A12E72)&softpage=DOC New ZealandEducationReview Office (ERO) website: http://www.ero.govt.nz/ero/publishing.nsf/Content/Home+Page New ZealandMinistry of Educationwebsite: http://www.minedu.govt.nz/ New ZealandQualifications Authority (NZQA) website: http://www.nzqa.govt.nz/ New Zealand State ServicesCommissionwebsite: http://www.ssc.govt.nz/display/home.asp New ZealandTeachers Council website: http://www.teacherscouncil.govt.nz/ New ZealandTreasury website: http://www.treasury.govt.nz/budgets/process/default.asp O'Donnell, Sharon. 2004. International Review of Curriculum and Assessment Frameworks: Comparative tables andfactual summaries - 2004. Qualifications and Curriculum Authority andNational Foundationfor EducationalResearch. OECD. 2004. Revisidn depoliticas nacionales de educacidn. Paris: Organisationfor EconomicCo-operationandDevelopment. OECD. 2003. "Attracting, Developing and Retaining Effective Teachers: Country Background Reportfor Finland". Qualificationsand Curriculum Authority. InternationalReview of Curriculum and Assessment FrameworksInternet Archive (INCA), online at: http://www.inca.org.uk Qualifications and Curriculum Authority (QCA). 2003. "International Review of Curriculum and Assessment Frameworks. Thematic Probe: Primary Education: an International Perspective". SpanishMinistry of Educationand Science website: http://www.mec.es/ Texas Administrative Code website: http://www.sos.state.tx.us/tac/ 45 Texas EducationAgency website: http://www.tea.state.tx.us/ UNESCO. 1998. "Supervisionfor the Self-Managing School: TheNew Zealand Experience". Paris, France. UNESCO. 2003. "Student Loans Schemes in the Republic of Korea: Review and Recommendations ". Bangkok, Thailand. United States NationalAssessment of Educational Progress webpage: http://nces.ed.gov/nationsreportcard/ UnitedStatesNational Center for EducationStatistics website: http://nces.ed.gov/ 46 c --. d -.< ? 2 0 -. 4 -. 4 --. 4 ? 5 ? --. 4 : ? --. 4 4 - : -. 4 5 --. d ? --. d ? B --. 4 ? Y VI 0 I -. 4 ? ? Z 0 Q d \ z . ' < ' < < ? \ ? z . d \ ? -. d - ? -. d ? < \ ? rn m < Z e < Z e 4 Z e 4 5; 4 5; I I . I < 2- e .5 Y yi ' 4 ? ' d ? ' 4 5 ' z< . . I Z 4 \ ? I h c, 0c 0 c C --. < ? --. C : --. < -. d ? ? -. d 5 e \ z 0 < \ ? I j L 4 zw 4 5. 2 4 Z Z w 4 5. c 4 : ? 5 Z Z w a 0 Y -. d za -. zda Y 8 -. 4 za d \ ? ?1* .z B --. -? 2 e d \ 5 --. d 5 d \ ? 8 3 l- 1 23 YI s C ' YD y P Y v) 3 9s C Eu .e 0 0 .I CI 0 a C 8 rc S -39 m -2 I 2 C ru 0 0 .e Y 0 C .I CI S cd Y CI 8 .I w v) ._3 3 C I m.. -2cd x .I 'CJ C 2Q Q) v) v) e Appendix C: Consultationprocess Having developed a conceptual framework, synthesized the experience of successful comparable countries and identified alternative instructional visions, the Bank team developed preliminary institutional recommendations for Chile. The team then traveled to Chile to undertake a broad consultative process. InChile, the team met with key stakeholders in Chile's education system to present the conceptual model and preliminary recommendations (see Table 15). The team met with each group separately inorder to gain a broadunderstanding of the political context inwhich the policy options should be grounded. Table 15: Consultation ProcessParticipating Organizations and Entities Senate Universidadde FIDE(Catholicprivateschools Centrode Education Chile organization) Estudios Commission Xblicos Universidad CONACEP (Privateschools Chamberof Cat6lica organization) Centrode Deputies (MIDEUC, Estudios Education Centro de Asociaci6n de Municipalidades para el Commission Estudios) public schools organization) Desarrollo Ministryof Universidad SecondaryEducationStudent Instituto Education Albert0 Leaders Libertad y Hurtado Desarrollo Ministryof UniversityStudent Leaders Finance Universidad Adolfo Ibiiiez Teachers Union SOFOFA (Business sector organization) Our analysis of Chile's education, quality assurance systems andthe identification o f instructional visions had strong resonance among those consulted. In this sense, a generalized discontent with the quality of the present education system and an appreciation o f the urge for immediate action were evidenced. More importantly, there was unanimous recognition of the potential for improvement embedded in ihe current tensions. The conceptual model presented was well received and its value recognized. The need to increase state roles and capacities in terms o f education quality assurance was consented upon. In regards to our specific recommendations, with very few exceptions, those consulted shared the view that a limited state instructional vision should 79 be abandoned infavor of some mix of the quality contracts and differentiated instruction visions. Finally, no one seemed to believe that a managed instruction vision would be an intelligent choice for Chile. Disagreement began to materialize once broad conceptual recommendations narrowed down to practical application aspects. Among the parties consulted, various views were expressed regarding the preferred distribution formula for assigning schools among instructional visions. Broadly speaking, the 3 main stances inthis regard were: i)a formula based on the source of financing assigning private-paid schools to a quality contracts vision and all other schools to a differentiated instruction vision; ii)a sustainer- based formula which would allocate all private schools (paid and subsidized) to a quality contracts vision and treat municipal schools according to a differentiated instruction vision; and iii)a performance-basedformula whereby, independent of finance source and sustainer type, all good performing schools would earn autonomy and fall under a quality contracts vision while poor performing schools would loose autonomy and be treated according to a differentiated instruction vision. Naturally, this disagreement translated into differing opinions on the optimal amount and distribution of state financial resources among education providers. A final area in which there was little consensus is inthe distribution of education quality assurance functions betweenthe different levels of the state apparatus (Le. national, regional, provincial and m~nicipal).~~ 24Although we recognizethe complexities posedby the existenceof diversepolitical and ideological standpoints ineducationfor the application of the recommendationspresented here, an analysis of political context implications is beyond the scope of this study. 80