WPS7012 Policy Research Working Paper 7012 School Autonomy and Accountability in Thailand A Systems Approach for Assessing Policy Intent and Implementation Gustavo Arcia Kevin MacDonald Harry Anthony Patrinos Education Global Practice Group August 2014 Policy Research Working Paper 7012 Abstract There is a consensus on the need for Thailand to reform its examines the effect of the gaps on various schooling out- education system to be able to compete with other high comes while controlling for covariates. The analysis finds performing countries in the region. In terms of learning significant differences between the Systems Approach for outcomes, the most recent evidence from the Programme Better Education Results indicators of policy intent and for International Student Assessment shows little improve- policy implementation in all areas assessed by the indica- ment over time. This paper uses the World Bank’s Systems tors. Schools in Thailand exercise more flexibility in their Approach for Better Education Results (SABER) approach personnel management in practice than what is intended in Thailand to contrast policy intent and policy imple- by policy; student assessments need to address issues of mentation in school autonomy and accountability. The content, reliability, and validity and school accountability policy implementation data were obtained from a survey needs to improve the interpretation of student assessments of school principals of the schools that participated in the to make schools more accountable. There is a positive asso- Programme for International Student Assessment and ciation between the Programme for International Student merged the data sets. First, the study analyzes the gap Assessment scores and school autonomy and accountability. between policy intent and policy implementation. Then it This paper is a product of the Education Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at hpatrinos@ worldbank.org . The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team School Autonomy and Accountability in Thailand: A Systems Approach for Assessing Policy Intent and Implementation Gustavo Arcia Kevin MacDonald Harry Anthony Patrinos 1 JEL No: H4,I20,I25,J24,O15 Keywords: educational policy; educational autonomy; school accountability; Thailand 1 We thank Sunee Klainin, Institute for the Promotion of Teaching Science and Technology, Thailand, for her coordination of the school survey and data coding, and Angie Rivera Olvera, for her assistance with the school-level dataset. We are indebted to Derek Pornsima, Advisor to the Secretary-General of KHURUSAPHA (Teachers’ Council of Thailand) and Kraiyos Pratawart, Quality Learning Foundation, for their support during the different phases of this study. At the World Bank Office in Bangkok we thank Vachraras Pasuksuwan for her invaluable assistance with the management and logistics of the SABER survey, and Lars Sondergaard, for his assistance and advice during the policy dialogue with the Government. We also would like to thank Benjalug Namfa, former Deputy Secretary General, Office of the Basic Education Commission, and Wongduan Suwansiri, Academic Officer, Office of the Basic Education Commission, for their participation and support during the phase of policy dialogue within the education sector in Bangkok. Gustavo Arcia and Kevin MacDonald are World Bank Consultants and Harry Anthony Patrinos is Manager, Education, World Bank. School Autonomy and Accountability in Thailand: A Systems Approach for Assessing Policy Intent and Implementation Summary • There are significant differences between the indicators of policy intent and implementation. • Schools in Thailand exercise more flexibility in their budget planning and management, and in personnel management than is intended by policy. • Student assessments need to address issues of content, reliability and validity. • School accountability needs to improve the interpretation of student assessments to make schools more accountable. • There is a positive association between reading scores and school autonomy in budgetary and personnel management. Thailand: Benchmarking Policy Intent and Implementation in School Autonomy and Accountability Indicator Policy Policy Implementation Intent School Autonomy in Budget Planning and Approval Emerging Established   School Autonomy in Personnel Management Latent Emerging   Participation of the School Council in School Finance Established Established   Assessment of School and Student Performance Advanced Established   School Accountability Established Emerging   Notes: Latent: reflects policy not in place or limited engagement; Emerging: reflects some good practice but still in progress; Established: reflects good practice with some limitations; and Advanced reflects international best practice 2 1. Introduction There is a general consensus on the need for Thailand to reform its education system to be able to compete with other high performing countries in the region (Patrinos et al. 2012). Education is crucial for economic growth (Jimenez, Nguyen and Patrinos 2012) and increasing the level of education of the labor force in rural areas contributed to 26 percent of the reduction in poverty. Nevertheless, Thai workers do not possess the skill set necessary to compete in the East Asian manufacturing markets (Bhaopichitr, Mala and Triratanasirikul 2012). In terms of learning outcomes, the most recent evidence from PISA (OECD 2013) shows little improvement. In mathematics, the annualized rate of growth in Thai scores did not change since 2009, while reading scores increased by 1 percent and science scores by 4 percent. Despite these modest advances in reading and science, Thailand had a lower than average share of high scorers, and a higher than average share of low scorers, than the average for the 64 countries participating in PISA. The Thai education system over-emphasizes inputs and has a large administrative structure that results in lower salaries for all, which reduces the probability of offering higher salaries for better teachers. This structural problem reduces equity and efficiency, and leads to a lack of higher- level and critical thinking among students (Fry and Bi 2013). Overall, there is a general consensus that Thailand will need to reconcile its intent in education system policies with the reality of the school level. One step in this direction is to assess the gap between policy intent and policy implementation, which is the main focus of this analysis. The objective of this analysis is to compare policy intent with policy implementation in school autonomy and accountability in Thailand, with the purpose of assisting policy dialogue based on school-level evidence. Benchmarking policy implementation can be very useful for decision makers because the difference between policy intent and policy implementation can help fine tune the incentives and disincentives required at the school level to ensure that education policy has its desired effect. Box 1—SABER. The Systems Approach for Better Education Results (SABER) is an initiative of the World Bank with the purpose of producing comparative data and knowledge on education policies and institutions. Its aim is to help countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data collected for the initiative. The SABER approach encompasses: (i) an analytical framework to guide policy dialogue and foster policy reforms; (ii) relevant new data and information on education policies and institutional practices from other countries in the world, and (iii) concise analyses of how well the policies and institutions in each domain are aligned toward achieving Learning for All. SABER’s systems approach recognizes that improving education requires strengthening all factors that improve learning, and that a system's policies for governance, accountability, information systems, financing rules, and school management have to be aligned to ensure good performance. In the process, SABER identifies some common policy and institutional threads that run through most of the more successful experiences, allowing users to use the information as a tool for their own policy dialogue. Source: http://saber.worldbank.org/index.cfm 3 Data for this analysis comes from the assessment of indicators of School Autonomy and Accountability using a scale developed under the World Bank’s Systems Approach for Better Education Results, or SABER (see Box 1). The assessment of the indicators of policy intent in School Autonomy and Accountability was done in the first trimester of 2011, and validated during the East Asia Regional Conference on Systems Approach for Better Education Results (SABER), June 6-8, 2011 (World Bank 2012). The assessment of the indicators for policy implementation was done in 2012, using data collected in the 226 schools that were sampled by the 2009 PISA. The report is organized as follows: Section 2 presents an overview of education in Thailand, setting the stage for the assessment of its education policies. Section 3 gives an overview of school autonomy and accountability through a conceptual framework that is used as the basis for the development of indicators of policy intent and policy implementation. Section 4 summarizes the results of the benchmarking on policy intent and discusses the content of the intent and implementation rubrics. This section presents the basic results of the two benchmarks. Section 5 presents the results of the implementation survey, and the links between the intent- implementation gaps with PISA results and selected covariates, and section 6 presents the conclusions and recommendations. 2. Education in Thailand: Searching for the path to high performance in education Education in Thailand is regulated by the Education Act of 2002, which establishes that all children should compulsorily study for nine years—six years of primary plus three years of lower secondary (OBEC 2003). For the past few years Thailand has made substantial gains in secondary education achievement, getting closer to the levels of the better performing countries in East Asia. UNESCO (2011) estimates that the school-life expectancy in Thailand is 12.3 years of education, which means that, on average, Thai youth are expected to finish secondary school. Still, the Ministry of Education considers that Thailand has structural problems that are manifested in different forms: • Education quality remains low due, mostly to the scarcity of good teachers. Based on learning assessments, there is a need to substantially improve learning in mathematics in primary school and in science in secondary school. • Non-formal education still falls short of adequate coverage, particularly in rural areas, and problems continue in the area of education quality, which vary significantly by schools managed by different agencies. • There is a need to improve primary and secondary education in order to produce students with higher levels of critical thinking and improved levels of problem-solving skills. • Inequalities persist in education funding, especially between urban and rural areas. Starting in 2002 school finances and school administration were decentralized at the Education Service Area level. The Education Service Area (ESA) is a division of the Ministry of Education responsible for the overseeing, monitoring, evaluation and dissolution of basic education institutions. The ESA also coordinates and promotes private institutions in the area and for the coordination with the Local Administrative Organizations (LAO, which are managed by the Ministry of the Interior). The ESA receives the transfers from the Ministry of Finance and 4 distributes the funds to the LOA or directly to schools. The LAO is the institution that will eventually administer schools autonomously. In the meantime, decentralization is in progress, transferring authorities from the ESA to the LAO as soon as it is feasible. Each LAO is administered by an Area Committee for Education, which includes community representatives, the Local Administrative Organization, the teachers’ association, the educational administrator associations, the parents’ association, and educational scholars. Geographically, the Education Service Area is above the local level but below the provincial level (UNESCO 2009). Under the decentralized framework schools are free to manage their non-salary expenditures. The Ministry of Education regulates policies and planning at the central level and each of the 76 provinces develops its own provincial plan, while local governments develop their own local policies. Teachers are managed under the rules of the Teachers Civil Service Commission, which now allows for some managerial tasks to be performed by the Education Institution Committee at the local level. Teacher management has also been decentralized to the ESA level, but the Commission for Teachers and Administrative Personnel still has substantial influence, leaving the ESA only with the responsibility for the recruitment and deployment of teachers. In terms of access, Thailand has made substantial progress in the last two decades. The gross enrollment rate for primary school is now above 100 percent; for lower secondary, it is 99 percent; and for upper secondary, 76 percent. In 2012 there were more than 5 million children enrolled in primary education, more than 4.7 million in secondary education and 2 million in tertiary education (UNESCO Institute of Statistics 2014). In addition, almost 1 million children attended vocational education (UNESCO 2011). Still, full coverage in basic education is not complete due to inaccessibility of some areas, and lower school access among the poor. 3. A Model for Assessing School Autonomy and Accountability Intuitively, good school management is a necessary but not a sufficient condition for learning. However, improper or inadequate management practices can be a substantial barrier to school and student performance. One can fix some managerial components and obtain no results or alter some other components and obtain good results. What combination of management practices are crucial for success are still under study, but the emerging body of practice point to a set of variables that foster managerial autonomy, the assessment of results, and the use of the assessment to promote accountability among all stakeholders (Patrinos, Velez and Wang 2013). The lessons learned in the last 30 years indicate that by combining managerial autonomy, assessing student learning, and being accountable to parents and other stakeholders tends to produce good school performance and increased learning (Bruns, Filmer and Patrinos 2011). Together, these factors work like a system; that is, they work as a set of interconnected variables that can improve learning when they operate in close coordination. Moreover, these factors have well identified components that all schools can manage on their own, such as the management of the operating budget, the responsibility for selecting and managing teachers and staff, the management of the school’s infrastructure, and the monitoring and evaluation of teacher performance and of student learning outcomes. School autonomy is a form of school management in which schools are given decision-making authority over their operations (Patrinos 2011). Autonomy may take different forms and may include the hiring and firing of personnel, and the assessment of teachers and pedagogical 5 practices. School management under autonomy may give an important role to the School Council—representing the interests of parents—in budget planning and approval, as well as a voice/vote in personnel decisions. 2 By including the School Council in school management, school autonomy fosters accountability (Di Gropello 2004, 2006; Barrera, Fasih and Patrinos 2009). School accountability is defined as the acceptance of responsibility for one’s actions. In school management accountability may take other additional meanings, including: (i) the act of compliance with the rules and regulations of school governance; (ii) reporting to those with oversight authority over the school and (iii) linking rewards and sanctions to expected results (Heim 1996; Rechebei 2010). Viewing the school as a closed system can lead to good insights on how to improve its performance. System performance is maximized when all variables are synchronized, allowing the system to close its internal loops. Under this framework, school management activities make sense when they contribute to system closure, otherwise they may have little impact on performance and become just a collection of isolated managerial activities. If school autonomy and school accountability regulate school management, managerial activities should foster an enabling environment for teachers and students, allowing for pedagogical variables, school inputs, and personal effort to work as intended, and helping achieve system closure. In that vein it is clear that for managerial factors to become critical the system has to achieve closure before their components reach a critical status. If a system closes, the sudden rupture in one of its components only leads to an imperfect solution. Therefore, defining a managerial system that can achieve closure is conceptually important. If the system is unable to close, are partial solutions effective? Yes, in a broad sense, in which schools can still function but their degree of effectiveness and efficiency would be lower than if the system closes. In this regard, the available evidence indicates that a school’s system can increase its chances of closure when it enforces enough autonomy to evaluate its results and use those results to hold someone accountable. The main conclusion of the lessons learned on school-based management is that schools can improve learning for all students when they interconnect managerial autonomy, student assessments, and accountability, by aligning the functions of the school councils, the policies for improving teacher quality, and the functioning of the Education Management Information Systems (EMIS). 3.1 Autonomy and Accountability as System Components School autonomy and school accountability allow a school to close its internal system and maximize performance. Figure 1 shows how they interact with other system components and how they help achieve system closure and maximize performance. School Councils are crucial for implementing school autonomy because they become the representatives of the school clients: parents and students. As such, the School Council can be a resource to school management in the process of tailoring school services (curricula, teaching materials, school 2 For ease of presentation the term School Council Is used to represent the school-level organizations that represent parents and that are often referred to Parent-Teacher Association, Parent Council, among other names. 6 calendar, and teacher selection) to the needs of students. A more active role of School Councils in school governance can make school autonomy more effective. Source: Arcia, Macdonald, Porta and Patrinos 2011 Teacher quality is interrelated with the assessment of teachers and students. The main objective of any assessment system is to monitor learning, which in turn is strongly linked to teacher quality. Hence, for the education system to achieve closure, one needs to link school and student assessment to teacher performance and teacher quality. Finally, information about student performance is linked to accountability because it is the mechanism in place to report on performance indicators at the school and system levels. In this vein, Education Management Information Systems (EMIS) help enforce accountability to the extent that it is fed data of good quality and it is used to produce reports that are informative to parents and society about the performance of the education sector. In summary, the interrelation between Autonomy, Assessment, and Accountability must be made operational by reinforcing the roles of school councils, policies aimed at improving teacher quality, and the operation of an Education Management Information System. Otherwise, there is a risk that the system may not reach closure. It must be pointed out that the above links are conceptual in nature, since the empirical evidence tends to confound these factors. However, in managerial terms it is clear that the point of contact between autonomous schools and their clients is primarily through the school council (Corrales 2006). Similarly, school assessments are the vehicles used by schools to determine their needs 7 for changes in pedagogical practices and to determine the training needs of their teachers. Both, pedagogical changes and teacher training are determinant factors of teacher quality (Vegas 2001). Finally, the role of EMIS on accountability has been well established and it is bound to increase as technology makes it easier to report on indicators of internal efficiency and on standardized test scores (Bruns, Filmer, and Patrinos 2011). 3.2 Linking the conceptual framework with SABER Indicators The last paragraph in the previous section helps define the types of indicators that would signal system closure and ensure better performance. Five indicators are used to benchmark school autonomy and accountability (SABER): 1. School autonomy in budget planning and approval – an increasing degree of local control over operating funds to better reflect the interests of parents and to improve operational efficiency 2. School autonomy in personnel management – giving the school more control over its teachers; depending on the laws, this control can range from teacher selection, to teacher contracting, and teacher salaries and bonuses 3. The participation of the school council (or the Parent Teacher Association as it is called in many countries) in school finance – dealing with their role in budget planning and management 4. The assessment of school and student performance – the regular measurement of teacher performance, student learning, or both; measuring school and student performance is a key precondition for ensuring accountability 5. School accountability to stakeholders – mechanisms in place to be held accountable to parents, local governments, and society at large Each of these indicators has a set of sub-indicators that give more details on the subject. Each indicator and sub-indicator is classified as Latent, Emerging, Established or Advanced. A Latent score indicates that the policy behind the indicator is not yet in place. An Emerging score indicates that the implementation of the program or policy is in progress. An Established score indicates that the program or policy is in operation and meets the minimum standards. An Advanced score indicates that the program or policy is in operation and reflects best practice. 3.3 Data Collection Process for the Policy Implementation Indicators All the 230 schools that participated in the PISA 2009 cycle were given a questionnaire for assessing school autonomy and accountability developed by the SABER initiative. The questionnaire was translated into Thai language with traces of modification as to fit to Thai education/schools system. Any modification was consulted and allowed by the SABER team. The translated questionnaire was verified by Ministry of Education staff to ensure that the translation correctly interpreted the original English content, and that the translation was compatible with the Thai school context to ensure that school principals would understand the questions and reduce the number of ‘not applicable’ responses. Every school in the sample was contacted through the PISA 2009 schools coordinators. They were informed about the project 8 and its objectives. School coordinators were asked to contact schools principals to answer the questionnaires. The questionnaire was reproduced and sent by registered post to the schools. One month was allowed for a response. A pre-paid envelope was provided to the schools along with a promise of a 500 Baht ($15) honorarium paid to the school for their collaboration. Most schools returned the questionnaire in time and those that did not were contacted by phone, through local education authorities and, for some schools, in person. In the latter case the survey staff had instructions to sit and wait until the questionnaire was completed. The answers to the questionnaires were keyed in the schools’ files and the schools’ PISA 2009 scores were matched to the questionnaire answers. The merged data has only the Schools ID since the school names are confidential. School principals at all 230 schools filled out the implementation rubric by choosing the cell in the rubric that most closely described their own situation. The implementation rubric is shown in the annex. The responses were coded using a simple scale with the following values: Latent = 1 Emerging = 2 Established = 3 Advanced = 4 These values are used as identifiers of the indicators for the subsequent analysis. The calculation of an implementation score for an indicator is done with a simple formula: Implementation Score = ( (Number of Latent scores * 1) + (Number of Emerging scores *2) + (Number of Established scores * 3) + (Number of Advanced scores * 4) ) / Number of school responses For example: For indicator 5.1 – defined as Guidelines for using school and student assessments for accountability—in the SABER rubric, 127 schools responded as follows: Indicator 5.1 Implemen tation Latent Emerging Established Advanced Total Score No Municipal Municipal Guidelines for guidelines. governments governments and Guidelines using school Assessments have guidelines schools have are published and student are used only for using the guidelines for online and assessments by central assessments, using the can be used for authorities but they cannot assessments and by the public accountability and school take personnel can take personnel to demand staff actions actions accountability 19 32 63 13 127 2.6 With these responses the score for indicator 5.1 can be calculated as follows: Indicator 5.1 = ((19*1)+(32*2)+(63*3)+(13*4))/127 = (19+64+189+52)/127 = 324/127 = 2.6 For the overall score the formula remains the same, as it adds up all the scores in each category, multiply them by their respective factors, and divide the sum by the total number of schools in the sample. 9 4. Benchmarking Policy Intent and Implementation In 2011, the SABER School Autonomy and Accountability Scale (SAAS) – which measures policy intent – was applied in Thailand to benchmark the existing framework of policies in this domain, and the results were discussed and validated with several institutions in the education sector. In 2012, a modified SAAS was applied in the 226 schools that participated in PISA 2009. The modified scale benchmarked the implementation of school autonomy and accountability policies, with the objective of comparing policy intent with policy implementation in school- based management. The results of the assessment of policy intent are summarized in Table 1 and discussed in more detail in their respective subsections. Clearly, the key issue for policy dialogue is the gap that may exist between the policy intent outlined in Table 1 and the implementation of these policies at the school level. Table 1: Thailand: Indicators of Policy Intent in School Autonomy and Accountability 2011 1. School Autonomy in Budget Planning and Approval Schools have autonomy over non-salary expenditures, and their budget management plans feed up to the Educational Service Area (ESA) level; education budgets are decentralized Emerging at the ESA and to continue to the Local Administrative Organization level; administrative  systems are in place and well established at the ESA level. 2. School Autonomy in Personnel Management Teachers are hired at the ESA level under the Teacher’s Civil Service Commission rules; Latent the selection process is bureaucratic and schools have little influence over their personnel choices; there is limited scope for implementing a system for managing teacher incentives  3. Participation of the School Council in School Finance Parents participate in school board committees and play an advisory role at the local level, and also contribute in fundraising activities; parents contribute up to 30% of total per Emerging student expenditure in public schools, and the private sector another 10%; parent and  private donations are not registered in the school budget 4. Assessment of School and Student Performance Office for National Education Standards and Quality Assessment (ONESQA) is an independently run public organization that manages the periodic evaluations of school performance; National Institute of Educational Testing Service (NIETS) is responsible for Advanced standardized testing in basic and secondary education; national test scores and rankings  are publicized and used in education policy and private results are available to students and parents 5. School Accountability School boards can request the office of the Auditor General to perform financial audits; school performance and learning outcomes are regularly made public; still, school Established accountability is hampered by the lack of power of parents over school personnel and for  the weak linkages between student performance and teacher and school accountability Source: World Bank 2013 10 4.1 School autonomy in budget planning and management The school autonomy in budget planning and management indicator has three sub-indicators: a. Level of autonomy in the management of the school’s operational budget b. Managerial autonomy of schools in managing non-teaching and teaching salaries c. Autonomy of principals to raise additional funds on top of what the school receives from the central and local governments A Latent indicator means that the school may not manage any money because the Ministry of Education or the local government sends goods directly to the school and pays for all services directly, without any intervention by the school. At the opposite end, an Advanced indicator means that school administrators may receive a transfer that is deposited into the school’s bank account, disburse payments to teaching and non-teaching staff; pay suppliers of goods and services, and manage the school flow of funds. In 2012, education funding in Thailand amounted to 441.5 million Baht ($13.6 million), or 18.7 percent of the government’s budget (Bureau of the Budget 2013). Of this amount, 341 million Baht were spent in basic education, and the rest in university education and other education- related expenditures. Funding is decentralized to the Educational Service Area (ESA), which may not correspond with geographical boundaries. Each ESA oversees the block transfers of operational funds to its schools. School principals have autonomy over the use of operational block transfers. Table 3 shows the results of benchmarking policy intent and policy implementation using the SABER rubric. A small panel of informants assessed policy intent, while the implementation score averages out the scores of 227 school principals. Table 2: Thailand: Policy Intent and Policy Implementation Scores for Indicator of Budget Autonomy Intent Implementation Sub indicator Intent Implementation Score Score 1.1 Management of the operational budget. School Established Established principals manage the operational   budget under ESA guidelines 3.0 3.3 1.2 Management of non- teaching staff & teacher Latent Emerging salaries. ESA delegates managerial   authority to school principals. 1.0 2.8 1.3 Raising additional funds for the school. School principals raise Established Established additional funds from parents and the   private sector. 3.0 3.0 Overall Emerging Established 2.3 3.0   Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level except for indicator 1.3, where the difference is not statistically significant. 11 In terms of policy intent schools should not manage teacher and non-teacher salaries, as they are centrally regulated by civil service rules, resulting in a Latent score of 1 (Table 2). In practice, however, school principals report that they do manage non-teaching and teaching staff salaries in coordination with their Education Service Area, resulting in a score of 2.7, which is the level of Emerging. This difference shows flexibility on the part of ESA authorities for budget management at the school level, which bodes well for a possible increase in school autonomy. However, this is the only significant discrepancy between policy intent and policy implementation found in the indicator of budget autonomy. 4.2 School autonomy in personnel management The school autonomy in personnel management indicator refers to the level of autonomy that the school has in the everyday management of the school’s human resources. It has three sub- indicators: a. School autonomy in the hiring and firing of teachers b. Role of the School Council—as representative of parents—in decisions relating to teacher tenure or teacher transfers c. Level of autonomy in the hiring and firing of school principals Teachers in Thailand are managed under the rules of the Teachers Civil Service Commission, which now allow for some managerial tasks to be performed by the Education Institution Committee at the local level. Teacher management has also been decentralized to the ESA level. However, the Commission for Teachers and Administrative Personnel retains administrative control over teachers. Moreover, at the ESA level teachers can only be recruited and deployed; they cannot be fired. As a result, teacher accountability is dissipated among the Civil Service Commission, the ESA, the Local Administrative Organizations, and the school, leading to situations where teacher accountability is low. Personnel expenditures in education for 2012 amounted to 231 million Baht, or 52 percent of total government expenditures in education. Most of these expenditures go to pay teacher salaries (Bureau of the Budget 2013). Teacher management—the hiring and firing of teachers, and the administration of the payroll—has been recently decentralized to the level of Education Service Areas. Teacher hiring is regulated by the Teacher’s Civil Service Commission rules, which tend to protect job tenure. The teacher selection process is based on compliance of curricular requirements—such as the completion of a university degree—and without provision for more local control over teachers. Because teacher hiring and deployment is bureaucratic, schools have little room for maneuvering when assigned inadequate teachers. Moreover, since teacher salaries are regulated by Civil Service rules, there is limited scope for implementing a system of rewards and sanctions that could improve teacher incentives. In terms of policy intent, legal authority to manage school personnel is centralized, leading to an overall Latent score. Although an ESA can appoint a teacher, the law requires centralized approval, leading to a score of Emerging. In a similar vein, parents—acting through the School Council—have no legal authority over teacher tenure or over a request for a teacher transfer, rendering a Latent score to this sub-indicator. Finally, in terms of intent, the law says that the Ministry of Education appoints school principals, leading to a Latent score. 12 The assessment of policy implementation shows less rigidity than the law currently allows, getting a benchmarking score of Emerging (Table 3). At the level of sub-indicator, municipal- level authorities seem to have more say over teacher hiring and firing than what the law allows, leading to a score of 2.7 that could be called Emerging, which is more autonomous than the Emerging score given to policy intent. The same pattern is observed in the other two sub indicators, where principals report more involvement by municipal-level authorities than the law requires. In summary, personnel management is more flexible than what the law allows, and that flexibility is significant because parents and schools may use it for designing better teacher incentives. Table 3: Thailand: Policy Intent and Implementation Scores for Indicator of Personnel Management Intent Implementation Sub Indicator Intent Implementation Score Score 2.1 School autonomy in teacher appointment and deployment Emerging Established decisions. ESA appoint and deploy 2.0 2.8   teachers under union or civil service agreements. 2.2 School Council’s role in teacher tenure or transfer. Councils are Latent Emerging 1.0 2.3 consulted and exert influence on transfer   requests. 2.3 Autonomy in the hiring and Latent Established firing of principals. ESA has 1.0 2.9   autonomy in principal hiring and firing. Overall Latent Emerging 1.3 2.7   Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level. 4.3 Participation of the School Council in School Finance The participation of the council in school administration is very important because it is where parents can exercise their real power as clients of the education system. If the council has to cosign payments, then it has purchasing power automatically. The use of an operational manual is extremely important in this area, since it allows Council members to adequately monitor school management performance, help the principal with cash flow decisions, and become a catalyst for seeking additional funds from the community. The use of detailed operation manuals by the School Council is also a good vehicle for increased accountability and for the institutionalization of autonomy. Policy intent for this indicator covers five sub-areas regarding the legal authority of the School Council: a. Participation in the preparation of the school budget b. Approval of the school budget c. Use of a manual outlining the participation of the School Council in school finances 13 d. Role of the School Council in the implementation of the school budget e. Acceptance of the Council-approved budget by higher level authorities as a guide for detailed fiscal transfers to the school. The more involved is the School Council in the planning and implementation of the school budget, the higher the probability for improving accountability. Overall, in terms of policy intent, Thailand gets an Established score (Table 4). Parents participate in school board committees, which have a strong advisory role at the local level, and also contribute in fundraising activities. Since schools are responsible for drawing their own plans for managing non-salary resources, the role of parents in financial planning is very important. Currently, parents contribute between 20 and 30 percent of the total per student expenditure in public schools (UNESCO 2009) and another 10 percent is donated by the private sector following the lead of parents. Although parent participation is significant, current policies force schools to keep two sets of books in order to manage a combined budget of government and private funds, creating problems of operational efficiency and transparency. In terms of policy implementation the participation of the School Council in school finance remains near the Established stage. At the sub-indicator level participation is higher than the law allows in the areas of budget preparation and approval, with an Emerging score for intent score and an Established – for implementation. Table 4: Thailand: Policy Intent and Policy Implementation Scores for Indicator of School Council Participation in School Finance Intent Implementation Sub-Indicator Intent Implementation Score Score 3.1 Participation of the School Council in budget preparation. Emerging Established 2.0 2.8   School Council has a voice in budget planning and preparation. 3.2 School Council's authority to Emerging Established approve the school budget. School 2.0 2.8   Council consulted before approval. 3.3 Manual for participation of Councils in school finances. There Established Established 3.0 2.8 are manuals, but participation by   Councils is less formal. 3.4 Role of the School Council in budget implementation. ESA and Established Emerging 3.0 2.7 schools inform parents on off-budget   items on a regular basis. 3.5 Use of budget prepared with School Council's participation. Established Established 3.0 2.8 Parent contributions used by ESA and   schools in budget planning. Overall Emerging Established 2.6 2.8   Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level. In the remaining three sub-indicators of School Council participation the intent and implementation scores are similar. The fact that schools must separate their real budget from the official budget means that School Councils have a say in budget planning and preparation, but it 14 is done with less certainty about the role of the council on budget management, and with lower chances for the real budget to be used as an input for education authorities above the school level. For the long term this gap reduces the transparency of school financing and reduces the potential for analyzing the efficiency of the school budget. 4.4 Assessment of School and Student Performance School assessments can have a significant effect on school performance because they force parents and teachers to agree on scoring rules and ways to keep track of them, allowing for accountability about student learning to emerge in a useful way. Measuring student assessment is another important way to determine if a school is effective in improving learning. A key aspect of school autonomy is the regular measurement of student learning, with the intent of using the results to inform parents and society, and to make adjustments to managerial and pedagogical practices. Without a regular assessment of learning outcomes school accountability is reduced and, with it, improving education quality becomes less certain. There are five sub-indicators for assessment: a. Existence and frequency of school and student assessment b. Use of school assessments for making school adjustments c. Frequency of standardized student assessments d. Use of student assessments for pedagogical and personnel adjustments e. Publication of school and student assessments Thailand has two government agencies in charge of system assessment: The Office for National Education Standards and Quality Assessment (ONESQA), which is responsible for school assessment, and the National Institute of Educational Testing Service (NIETS), which is responsible for student assessment. ONESQA is an autonomous agency responsible for conducting external evaluations of school quality in basic and post-secondary education, as mandated by the National Education Act of 1999 (Office of the National Education Commission 2003; Kingdom of Thailand 2000, 2003, 2004, 2009). Between 1999 and 2010, ONESQA conducted two complete rounds of school assessments in 40,000 schools, using an “amicable assessment model” to evaluate school performance. This approach compares a school’s performance with the standards set by the Ministry of Education using statistical data, indicators of quality, and a listing of the school’s strengths and weaknesses within an environment of cooperation in which consequences for low performance are framed in a positive way. Assessments yield a set of recommendations, which should be implemented by failing schools during a grace period. Failing schools are revisited to verify compliance with the recommendations of the assessment. Every school in the country is assessed every five years by a team of evaluators employed and trained by ONESQA. Thus, the third round of evaluations began in 2011. For this round of evaluation ONESQA assessments began covering four areas: (1) learning outcomes; (2) school administration and management; (3) student-centered teaching and learning practices; and (4) internal procedures for quality assurance. In 2010, an external evaluation committee reviewed ONESQA’s performance and gave it a Good rating (85 percent), one level lower than the maximum rating of Very Good (90-100 percent). Most of the recommendations received by ONESQA were operational in nature and aimed at improving performance efficiency. However, a recent analysis of Thai education states that 15 ONESQA’s evaluations are too rooted on bureaucracy and do not reflect school quality as related to student learning (Tangkitvanich and Sasiwuttiwat 2012: 8-9). While the first round of ONESQA school assessment indicated that 65 percent of OBEC schools did not meet the minimum standards, the second round of assessments in 2010 indicated that only 20 percent of schools in basic education failed to meet the minimum standards (UNESCO 2011: 30). However, learning outcomes declined during the same period, suggesting that the assessments did not reflect school quality (Tangkitvanich and Sasiwuttiwat 2012). For the third round of assessments ONESQA has developed a different methodology, which is aimed at producing recommendations for improving education quality. Table 5: Thailand: Policy Intent and Implementation Indicators of School and Student Assessment Intent Implementation Sub Indicator Intent Implementation Score Score 4.1 Existence and frequency of school and student Advanced Established assessments. Process well defined 4.0 3.6   but needs improvement in its implementation. 4.2 Use of school assessments for making school Advanced Established adjustments. Links between test 4.0 3.2   results and adjustments not always clear. 4.3 Frequency of standardized student assessments. Process Advanced Established 4.0 3.7 well defined but needs improvement   in its implementation. 4.4 Use of student assessments for pedagogical and personnel adjustments. Links between test Advanced Established 4.0 3.4   results and adjustments not well understood by principals and teachers. 4.5 Publication of school and student assessments. Process Advanced Established 4.0 3.6 well defined but needs improvement   in its implementation. Overall Advanced Established 4.0 3.5   Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level. The National Institute of Educational Testing Service (NIETS) was established in 2005, with the mission of implementing standardized student testing to measure learning outcomes in basic and vocational education. NIETS administers the Ordinary National Educational Test (O-NET), a standardized achievement test for grades 6, 9, and 12 in eight subject areas (Thai language, math, science, social science, religion and culture, health and physical education, art, career and technology, and foreign languages). Each year 950,000 students take O-NET in the 6th grade, 16 880,000 in the 9th grade, and 330,000 in the 12th grade. In addition, NIETS administers the National Test, a standardized test given to 3rd and 6th grade students. Local school systems administer their own tests to students in grades 2, 5 and 8. NIETS also administers two tests used to screen students for admission to the university: the General Aptitude Test (GAT) and the Professional and Academic Aptitude Test (PAT). These tests are given to students in Grade 12. The results of O-NET are published online, although the averages published without login-in are for the national level only. In general, the principals’ assessment of policy implementation for this indicator is generally lower than the Advanced score found in policy intent (Table 5). The lower score given by school principals may reflect the issues of reliability and validity, which lack meaningful analysis of test scores, or the low use of the results for making pedagogical and personnel adjustments because the school may exercise limited autonomy in these areas. It is important to point out that there is substantial room for improvement in the implementation of current school and student assessment policies. The qualifications to the existing system expressed by school principals reduce the rating of this sub-indicator to Established, reflecting that the value of the indicator is near the Advanced level. 4.5 School Accountability to Stakeholders Accountability is at the heart of school-based management. The systemic connection between budgetary and personnel autonomy, parent participation in the financial and operational aspects of the school, and the measurement of learning outcomes are all aimed to reinforce accountability. Only by being accountable to parents can education quality be sustainable. The following indicators address the aspects of accountability that can be implemented within the framework of school-based management: the presence of written guidelines for the use of the results of school and student assessments; the strength of national institutions in charge of school and student assessments; the use of analytical comparisons to inform the public and education stakeholders about the performance of schools and students; the rights of school councils to request financial audits; and the presence of written guidelines for the school council with which to assess school financial performance. The indicator of school accountability has five sub-indicators: a. School has guidelines for using school and student assessments for accountability b. There are national and/or regional systems for educational assessment c. There are reports with comparisons of school and student performance d. School Councils have legal authority to ask for financial audits e. There are manuals for the participation of the School Council on school audits The results of the assessment of policy intent are mixed. Two sub-indicators received an Emerging score: the presence of guidelines for interpreting school and student assessments, and the legal authority for school councils to request an audit also. Two other sub-indicators got an Advanced score: the national system for assessing schools and students, and the presence of an audit manual for school councils. School boards do not have direct legal authority to ask for financial audits to enforce financial transparency and accountability, but can request the office of the Auditor General to perform one. Still, school accountability is hampered by the lack of 17 power of parents over staffing decisions, which can be a very important factor in improving learning. Overall, the implementation of accountability needs improvement and it is rated as Established - (Table 6). The implementation indicators for the functioning of a national system of school and student assessment and for the manuals for interpreting school audits are significantly lower than the indicators of policy intent. The gaps in these indicators suggest problems with key aspects of accountability to parents, especially in the areas of parent information about the meaning of test results, and on the use of results by schools to make modifications to their pedagogical practices. Table 6: Thailand: Policy Intent and Policy Implementation Indicators of School Accountability Intent Implementation Sub Indicator Intent Score Implementation Score Score 5.1 Guidelines for using school and student assessments for Emerging Established accountability. ESA uses the 2.0 3.1   guidelines for teacher assessments and personnel management. 5.2 National or regional systems of educational assessments. ESA Advanced Established 4.0 3.1 autonomy in the interpretation of   guidelines may be inconsistent. 5.3 Comparisons of school and student performance reports. Comparisons done at National, Established Established 3.0 3.0 regional and municipal levels, and   previous years. Schools often publish summary results online. 5.4 School Council's authority to perform financial audits. Emerging Emerging Councils tend to rely on school-level 2.0 2.6   negotiations, even though they can ask for formal audits. 5.5 Manual for the participation of the School Councils in school Advanced Emerging audits. Manuals exist but personal 4.0 2.5   communications are preferred by Councils. Overall Established Emerging 3.0 2.8   Source: World Bank 2013 and school survey data. The differences between the intent and implementation scores are statistically significant at the 0.001 level, except for indicator 5.3. 4.6 Intent and Implementation Scores by Type of School Sponsorship The differences between the law’s intent and what happens on the ground may be due to many factors, but one that needs to be explored first is the type of school sponsorship. High schools in Thailand can be classified by type of institutional sponsor. Most schools are administered by the Office of the Basic Education Commission (OBEC), while others are administered by the Bangkok Metropolitan Administration (BMA), the Bureau of Local Educational Development (BLED), the Commission for Higher Education (CHE) also known as SATIT schools. Private 18 schools may or may not receive government funding, but those that do are under the supervision of the Office of the Private Education Commission (OPEC). Resources available to each type of school vary widely. As reported by Patrinos et al (2012), SATIT schools, which are governed by the Commission of Higher Education, have more resources and more teacher than other public schools. If school autonomy in the Education Service Areas vary in its interpretation by the schools—as measured by the SABER indicators— the impact on learning outcomes within the context of autonomy needs to be explored. After analyzing PISA data for 2006 and 2009 using a Oaxaca-Blinder decomposition methodology, Patrinos et al (2012) conclude that a decline in the reading scores among the poor was the net result of a decline in system quality, which was larger than the increase in student performance, especially among the poor. Relatedly, schools in Bangkok outperformed school in the rest of the country, and system performance was also a reflection of systems resources, which were correlated with student wealth. These findings have direct implications for the analysis of SABER indicators inasmuch they suggest a need to examine policy implementation by type of school and geographical area. Table 7 shows the different implementation and scores by type of school sponsorship. There are no significant differences among the SABER sub-indicators by type of school sponsorship, but the lack of a significant difference is related to the high variation found within the indicators, which tended to reduce the statistical significance of the average differences between school types (see Table A6 in the Annex). In terms of average sub-indicator scores, however, the average differences between intent and implementation are statistically significant. Table 7: SABER indicator scores for policy implementation by type of school sponsor, 2013 SABER SABER Indicator OBEC OPEC BMA BLED SATIT Implem. Intent 1a. Legal authority over management of budget 3.3 3.3 3.2 3.1 3.4 3.3 3.0 1b. Legal authority over management of salaries 2.8 2.9 2.7 2.6 2.5 2.8 1.0 1c. Legal authority to raise additional funds 3.0 3.1 3.2 3.0 2.8 3.0 3.0* 2a. Autonomy in teacher appointment decisions 2.8 2.9 2.7 2.7 2.9 2.8 2.0 2b. School council role in teacher tenure or transfer 2.3 2.2 2.2 2.5 2.2 2.3 1.0 2c. Autonomy in the hiring and firing of principals 2.9 2.9 2.5 2.8 2.8 2.9 1.0 3a. Participation of council in budget preparation 2.8 3.0 2.5 2.5 2.7 2.7 2.0 3b. School council authority to approve budget 2.8 2.9 2.8 2.7 2.8 2.8 2.0 3c. Manual for participation of council in finances 2.8 2.7 3.1 2.5 2.6 2.7 3.0 3d. Role of council in budget implementation 2.7 2.7 3.2 2.6 2.5 2.7 3.0 3e. Use of budget prepared with council 2.8 2.7 3.1 2.4 3.0 2.8 3.0 4a. Existence and frequency of assessments 3.6 3.5 3.8 3.6 3.6 3.6 4.0 4b. Use school assessments for decisions 3.2 3.2 3.2 3.4 3.2 3.2 4.0 4c. Frequency of standardized student assessments 3.7 3.7 3.8 3.7 3.7 3.7 4.0 4d. Use of assessments for adjustments 3.4 3.4 3.2 3.4 3.3 3.4 4.0 4e. Publication of school and student assessments 3.7 3.6 3.6 3.5 3.5 3.6 4.0 5a. Guidelines for assessments for accountability 3.1 3.0 3.1 3.2 3.2 3.1 2.0 5b. National or regional systems of assessments 3.2 3.1 3.2 3.2 3.0 3.1 4.0 5c. Comparisons of performance reports 2.9 2.8 2.7 2.9 3.3 3.0 3.0* 5d. Council authority to perform financial audits 2.6 2.8 2.2 2.8 2.4 2.6 2.0 5e. Manual for participation of council in audits 2.6 2.4 2.3 2.4 2.5 2.5 4.0 Source: Estimated by authors. Difference between the implementation and intent score is statistically significant at the 0.00l level except for 1c and 5c Note: OBEC = Office of the Basic Education Commission; OPEC = Office of the Private Education Commission; BMA = Bangkok Metropolitan Administration; BLED = Bureau of Local Educational Development; SATIT = Commission for Higher Education 19 This statistical significance can be summarized as follows: a. In budgetary autonomy, schools tend to behave more autonomously than what is intended by education policy, and sponsorship does not seem to matter b. In terms of personnel management, autonomy implemented at the school level is higher policy intent, regardless of school sponsorship c. In the area of parent participation in school finance the score for policy intent in Bangkok Metropolitan Administration (BMA) schools is higher than the scores of other schools, and higher than the score for policy intent d. In the area of school and student assessment policy implementation lags behind intent; there are no clear differences among school sponsors e. School accountability is mixed, with some policies showing higher implementation scores and others showing higher intent scores, with no clear differences found among sponsors 5. Is policy implementation of autonomy and accountability important for student learning? One of the key objectives of SABER is to assist governments in monitoring education policies conducive to improved education quality. Schools can be classified by type of school financing: public, private-subsidized, and fully private. In principle, classifying schools by the source of school financing should reflect the degree of autonomy that schools may have in selecting students and teachers, and in being more autonomous and, to some extent, more accountable. Table 8 shows the implementation scores for schools classified by its source of financing. Public schools are completely funded by with government funds; private-subsidized schools receive government funds to cover their operational expenses but can have significant autonomy in selecting students and teachers; private-independent schools are reliant on private tuition and are completely autonomous in their selection of students and teachers. The results of SABER implementation scores tend to be aligned with the source of school financing. Public schools tend to have lower implementation scores than private-subsidized schools, and both of these types of schools tend to show lower SABER scores than private- independent schools. 20 Table 8: Thailand: Implementation scores for SABER indicators of autonomy and accountability (2012) and average PISA Scores (2009) Personnel School School & Budget PISA School Type management council's Student Accountability N autonomy Score autonomy participation Assessment Public 3.0 2.6 2.7 3.6 2.9 425 201 (0.6) (0.7) (0.8) (0.6) (0.6) Private 3.3 3.6 3.1 3.4 3.0 403 19 subsidized (0.5) (0.6) (0.7) (0.7) (0.6) Private 3.7 3.7 3.3 3.5 3.3 432 6 independent (0.5) (0.5) (0.5) (0.6) (0.8) Total 3.0 2.7 2.8 3.6 2.9 423 226 (0.6) (0.8) (0.8) (0.6) (0.6) PISA 2009 3.2 2.8 2.9 3.5 2.9 535 21 Score ≥500 (0.6) (0.7) (0.9) (0.6) (0.8) PISA 2009 2.9 2.6 2.7 3.5 2.8 376 80 Score ≤400 (0.6) (0.8) (0.7) (0.7) (0.6) Source: Estimated by authors Note: Numbers in parenthesis are the standard deviations PISA scores also show a slightly different mix than the SABER scores. The PISA scores in public schools tend to be higher than in private-subsidized schools, but lower than in private- independent schools. Schools with an average PISA score above 500 points show significantly higher indicators of budget autonomy than schools with PISA scores below 400 points. Higher performing schools have more budgetary autonomy. Private subsidized schools have higher indicators of budget autonomy than public schools, but have lower average PISA scores. Similar statements could be made for the differences between indicators of personnel management and accountability, but the differences between schools with high PISA scores and schools with low PISA scores is smaller than in the case of budget autonomy. 5.1 Exploratory Analysis of the Association between School Autonomy and Accountability and Academic Achievement The scores in Table 8 are just an approximation, since there is a need to control for several school characteristics in the sample population. Table 9 presents the results of regressing the PISA score for reading on the average total score for the SABER indicators of autonomy and accountability as independent variables, after controlling for selected school characteristics. Overall, the results of the estimations show that PISA scores could be significantly associated with autonomy in personnel management. The overall impact of the SABER total score is significant, as shown in equation 12 in Table 9. A one-point increase in the total SABER indicator score accounts for an increase of 11 points in the PISA reading score. This could be very significant, as it suggests that schools with higher levels of autonomy and accountability have higher PISA scores. At the sub-indicator level the results consistently show significant test score impacts associated with budgetary autonomy (Equations 1 and 2 in Table 9) and personnel management autonomy 21 (Equations 3 and 4 in Table 9). The equations for budgetary autonomy indicate that a one-point increase in the indicator is associated with more than 19 points in the PISA reading score for the school. After adjusting for covariates, the one-point increase in the indicator is associated with an increase of 9.4 points in the PISA reading score. The equations for personnel autonomy also show similar associations. Schools that make personnel decisions at the school level tend to have higher PISA scores. Table 9 shows that a one-point increase in the implementation indicator of personnel autonomy is associated with an increase of 10.6 points in the school’s average PISA reading score. After adjusting for covariates, personnel autonomy at the school level is associated with an increase in the math test score of 9.5 points. Table 9 also shows that the indicators for school council participation, and for student assessment, are associated with higher reading scores. This effect, however, tends to be attenuated once the equation includes some key covariates. The results of Table 9, particularly in the area of personnel autonomy, are consistent with the results of a recent factor analysis of SABER indicators for East Asia (Arcia, Lewis and Patrinos 2012) which shows that teacher management is one of two broad factors that explain most of the variation in SABER indicators of policy intent in 20 areas of influence in education. 22 Table 9: Exploratory Analysis of the Association between School Autonomy and Accountability and Test Scores (PISA 2009) Variables 1 2 3 4 5 6 7 8 9 10 11 12 Overall SABER Score 11.6* Autonomy: Budget 19.2** 9.4* 15.0** Autonomy: Personnel 10.6** 9.5** 3.9 School Council 9.0* 4.8 1.6 Student Assessment 15.4** 5.3 9.3 School Accountability 9.7 1.0 -0.8 Controls NO YES NO YES NO YES NO YES NO YES NO YES Constant 363.4*** 367.3*** 392.8*** 361.5*** 396.3*** 384.5*** 367.1*** 379.6*** 393.3*** 396.3*** 331.1*** 367.8*** R2 0.043 0.430 0.0190 0.440 0.015 0.410 0.020 0.430 0.012 0.430 0.055 0.391 F 10.15** 23.9*** 4.37** 24.7*** 3.41* 23.4*** 4.68** 23.2*** 2.62 23.0*** 2.55** 28.3*** N 225 225 225 225 225 225 225 225 225 225 225 225 Source: Estimated by the authors Notes: Significance: * = 0.1, ** = 0.05, *** = 0.01; PISA scores have a mean of 500 and a standard deviation of 100; controls: total school enrolment; student-teacher ratio; quality school resources; index academic selectivity; public school 23 6. Conclusions Benchmarking policy intent in education is useful for assessing a country’s position on certain education areas relative to the policy position of other countries. For purposes of a more focused policy dialogue, benchmarking policy implementation can be of added utility to decision makers, since the difference between the value of an indicator of policy intent and the value of the same indicator in policy implementation can help fine tune the incentives and disincentives required at the school level to ensure that the policy has its desired effect. In terms of the differences between the indicators of policy intent in school autonomy and accountability, and the indicators of policy implementation, as reported by the school principals of 230 schools that participated in the 2009 PISA survey, the results show the following: 1. School principals are already managing budgets related to personnel in ways that go beyond what the law intends them to do. The difference between policy intent and policy implementation in budget management suggests that authorities at the Education Service Area level have found that important personnel management functions could be delegated to school principals, thus deepening school autonomy. 2. Personnel autonomy—the hiring and firing of teachers—also seem to be delegated to local areas under significant influence from school principals. In general, in all areas of personnel autonomy, school authorities are making decisions that are later corroborated by the Education Service Area. 3. In terms of parent participation through the school council, the indicator results show that parents already have more say over the school budget than what the law currently allows. Again, this bodes well for increasing autonomy at the school level. 4. In terms of school and student assessment, the results show an opposite trend. In theory, the current system of school and student assessment is at an advanced level. However, principals give it a lower rating because of perceived problems testing at the school level. 5. School accountability shows a mixed picture. The implementation indicators for the functioning of a national system of school and student assessment and for the manuals for interpreting school audits are significantly lower than the indicators of policy intent. The gaps in these indicators suggest problems with key aspects of accountability to parents. 6. In terms of the relationship between SABER indicators of policy implementation in school autonomy and accountability, and student learning, the results show that school autonomy budget planning and management, and in in personnel management are associated with higher PISA scores. These are key conclusions for education policy, since they seem to confirm the trend found in high performing countries, where giving schools more autonomy leads to higher learning outcomes. The exploratory regression results are important because their statistical significance reinforces the notion of evidence-based policy reform, which is at the core of the SABER approach. The results point towards a consistent relationship between budget and personnel autonomy on student learning. The evidence presented here is significant enough to suggest as a next step to include budget autonomy and personnel management autonomy as components of evidence- based policy dialogue. 24 References Arcia, G., L. Lewis and H.A. Patrinos. 2012. “Exploring the interrelationships among SABER indicators.” Human Development Network. Washington D.C.: The World Bank. Arcia, G., K. Macdonald, H.A. Patrinos and E. Porta. 2011. “School Autonomy and Accountability.” System Assessment and Benchmarking for Education Results (SABER). Human Development Network. Washington D.C.: The World Bank. Barrera, F., T. Fasih and H.A. Patrinos, with L. Santibáñez. 2009. Decentralized Decision- Making in Schools. The theory and evidence on School-based management. Washington D.C.: The World Bank. Bhaopichitr, K., A. Mala and N. Triratanasirikul. 2012. “Thailand Economic Monitor December 2012.” Bangkok: The World Bank. Bruns, B., D. Filmer and H.A. Patrinos. 2011. Making Schools Work: New Evidence on Accountability Reforms. Washington D.C.: The World Bank. Bureau of the Budget. 2013. “Thailand Budget in Brief. Fiscal Year 2012.” Bangkok. http://www.bb.go.th/bbhomeEng/budget_in_brief/budget_in_brief_2012.pdf. Fry, G.W. and H. Bi. 2013. "The evolution of educational reform in Thailand: the Thai educational paradox." Journal of Educational Administration 51(3): 290-319. Heim, M. 1996. “Accountability in Education: A primer for school leaders.” Research into Practice Series, Pacific Resources for Education and Learning. Honolulu: Hawaii Department of Education. Jimenez, E., V. Nguyen and H.A. Patrinos. 2012. “Stuck in the Middle? Human Capital Development and Economic Growth in Malaysia and Thailand.” Policy Research Working Paper 6283. Washington D.C.: The World Bank. Kingdom of Thailand. 2009. “The National Institute of Educational Testing Service (Public Organization) Thailand.” http://www.niets.or.th. Kingdom of Thailand. Office of the Prime Minister, Office of the National Education Commission (OBEC). 2003. Chapter 6, “Educational Standards and Quality Assurance,” in National Education Act B.E. 2542 (1999) and Amendments (Second National Education Act B.E. 2545 (2002). Bangkok. Kingdom of Thailand. Office of the Prime Minister, Office of the National Education Commission. 2003. National Education Act B.E. 2542 (1999) and Amendments (Second National Education Act B.E. 2545 (2002). Bangkok. See also Kingdom of Thailand. 2000. Royal Decree Establishing the Office for National Education Standards and Quality Assessment (Public Organization) B.E. 2543. Bangkok. Kingdom of Thailand. Office of the Prime Minister, Office of the National Education Commission. 2003. National Education Act B.E. 2542 (1999) and Amendments (Second National Education Act B.E. 2545 (2002). Bangkok. Kingdom of Thailand. 2000. Royal Decree Establishing the Office for National Education Standards and Quality Assessment (Public Organization) B.E. 2543. Bangkok. 25 Patrinos, H.A. 2011. “School-Based Management,” in B. Bruns, D. Filmer and H.A. Patrinos, Making Schools Work: New Evidence on Accountability Reforms. Washington D.C.: World Bank. Patrinos, H.A., E. Velez and C.Y. Wang. 2013. "Framework for the reform of education systems and planning for quality," Policy Research Working Paper Series 6701, Washington D.C.: The World Bank. Patrinos, H.A., K. Macdonald, S. Parandekar and S. Klainin. 2012. “Learning Outcomes in Thailand. What Can We Learn from International Assessments?” (Report N. 64801-TH). Human Development Network, East Asia and the Pacific Region. Washington DC: World Bank. Programme for International Student Assessment (PISA). 2013a. PISA 2012 Assessment and Analytical Framework. Mathematics, Reading, Science, Problem Solving and Financial Literacy. Paris: OECD. Programme for International Student Assessment (PISA). 2009. PISA 2009 Results: What Students Know and Can Do. Paris: OECD. Programme for International Student Assessment (PISA). 2009a. PISA Data Analysis Manual. SPSS Second Edition. Paris: OECD. Rechebei, E. 2010. “Accountability and Reality. Who Should Do What? And Who Should Be Accountable?” Research into Practice Series, Pacific Resources for Education and Learning. Honolulu: Hawaii Department of Education. Tangkitvanich, S. 2013. “Education system ills setting up future failure.” TDRI Insight (March). Tangkitvanich, S, and S. Sasiwuttiwat. 2012. “Revamping the Thai Education System: Quality for All.” TDRI Quarterly Review 27(2): 3-14. UNESCO Institute of Statistics. 2014. Data Centre. http://www.uis.unesco.org/DataCentre/Pages/BrowseEducation.aspx UNESCO. 2011. Thailand. World Data on Education 7th Edition 2010/11. Paris: International Bureau of Education, UNESCO. UNESCO. 2009. Education Financial Planning in Asia. Implementing Medium Term Expenditure Frameworks. Thailand. Bangkok: UNESCO. World Bank. 2013. “SABER—School Autonomy and Accountability. Thailand.” Human Development Network. Washington D.C. World Bank. 2012. Strengthening Education Quality in East Asia: System Assessment and Benchmarking for Education Results (SABER). Washington DC: World Bank. 26 Annex: Policy Intent and Policy Implementation Scores by Indicator and Sub-Indicator Table A1. Policy Intent and Policy Implementation Scores for Indicator of Budget Autonomy Sub indicator Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent Implementation 1.1 Management Management Management of School principals School principals of the of the the operational manage the manage the operational operational budget is at the operational operational budget budget budget is regional or budget under autonomously 3 3.3 centralized at municipal levels municipal the Ministry guidelines of Education 1.2 Management Management Management School principals School principals of non-teaching authority authority over manage teacher set and manage staff and teacher over teacher teacher and and non-teaching teacher and non- salaries and non- non-teaching salaries but must teaching salaries. 1 2.8 teaching salaries is at the to follow a salaries is regional or centralized pay centralized municipal levels scale 1.3 Raising Budget is School School principals School principals additional funds fixed by the principals can can raise can raise for the school Ministry of request more additional funds additional funds Education funds from from the private from any source, 3 3.0 and no regional and sector, and from including foreign additional municipal non-governmental governments and funding is governments. institutions. international permitted donors 27 Table A2. Policy Intent and Policy Implementation Scores for Indicator of Personnel Management Sub Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent Implementation Indicator 2.1 School Teachers are Municipal or Municipal or School autonomy in appointed by the regional regional principal teacher Ministry of governments governments appoint the appointment Education and appoint appoint and deploy school’s and deployed by the teachers under teachers under teachers. Union deployment Ministry’s office union or civil union or civil and civil decisions of human service service agreements. service 2 2.8 resources under agreements. agreement may a union or civil Appointments or may not service are subject to regulate the agreement. final review by appointments. central authorities. 2.2 School School Councils School School Councils The School Council’s do not have a Councils are appoint teachers. Council does role in voice in matters consulted over School Councils not supervise teacher of school the can also request a teachers tenure or personnel. appointment of transfer for non- because formal transfer teachers or may performance or for accountability have veto violations of mechanisms power; School personnel rules. already work Councils can well. request the Occasionally transfer for School 1 2.3 non- Councils use performance or those formal for grave mechanisms to violations of demand personnel rules. management The actions local/regional regarding government teachers. must take action. 2.3 Principals are Principals are Principals are Principals are Autonomy in appointed and appointed and appointed, appointed and the hiring deployed by the deployed by the deployed, and deployed by and firing of Ministry of Ministry of evaluated by municipal principals Education. Their Education. regional or authorities in performance is Their municipal consultation evaluated performance is authorities, which with the School centrally and evaluated also determine Council, or by 1 2.9 they can be regionally or by tenure, transfer, or the School transferred or municipal removal. Council alone. fired by Central inspectors, Municipal authorities. which authorities determines evaluate their tenure, principals to transfer, or determine 28 Table A2. Policy Intent and Policy Implementation Scores for Indicator of Personnel Management Sub Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent Implementation Indicator removal by tenure, transfer, Central or removal. authorities. 29 Table A3. Policy Intent and Policy Implementation Scores for Indicator of School Council Participation in School Finance Sub Implemen Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent Indicator tation 3.1 Participation No role for the School Council have School Council School Council has of the School School Council; a voice in the have a voice in the formal mechanisms Council in budget budgets are planning and planning and for participation in preparation prepared centrally preparation of the preparation of the the school budget if by the Ministry of budget at the school budget but final they so desire. Education level, but final responsibility falls Budget systems may 2 2.8 responsibility falls on the school function well on the school principal. enough that an principal. active role for the school council may not be necessary. 3.2 School Only the Ministry Municipal School Budget approval is School Councils Council's of Education or a Councils may be done at the school may be responsible authority to central consulted but the level. School for budget approval approve the Government office municipal councils may be unless the formal school budget has the authority to government is consulted but they systems function 2 2.8 approve the school responsible for the have no approval well enough to not budget. School approval of the responsibility. require their council does not school budget participation participate. 3.3 Manual for the School Councils There are manuals There are manuals School Council is participation of do not participate regulating the regulating the role not involved in the the School in school financial procedures for of the School school budget 3 2.8 Councils in school issues expressing the Council in the because formal finances Council's voice on preparation of the systems function budget issues. budget well. 3.4 Role of the The Ministry of Budget School Councils School Councils do School Council in Education or the implementation is supervise budget not receive specific budget local/regional supervised at the implementation and reports on the implementation government municipal level. occasionally request school budget since supervises budget School Councils formal audits from it is public implementation. only ask for the appropriate information readily 3 2.7 School Councils accounts on authorities. available. Formal do not have a role. additional funding supervisory systems from parents and work well. other off-budget funds. 3.5 Use of the Budgetary National and sub- National and sub- National and sub- budget prepared decisions are made national levels use national authorities national authorities with the School at the national and the budget sent by use the proposed use the proposed Council's sub-national levels the schools as budget as their main budget as the base participation without school recommendations source of for the final transfer 3 2.8 council for the final recommendation for of resources to the participation. allocation of the transfer of school. resources. resources to the school. 30 Table A4. Policy Intent and Policy Implementation Indicators of School and Student Assessment Sub Implemen Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent Indicator tation 4.1 Schools are Schools assessed Schools assessed every Schools are assessed Existence not assessed every few years. few years. All students every year. All students and and students Sample of assessed in primary and assessed in all or in frequency of do not take students assessed secondary schools selected grades of 4 3.6 school and standardized in primary and every few years. primary and secondary student tests. secondary school every year. assessments schools every few years. 4.2 Use of Schools are Ministry of Ministry of Education Ministry of Education school not assessed. Education analyzes school or municipal assessments Adjustments analyzes school assessments and makes governments analyzes for making made with assessment and direct recommendations school assessments. school different results sent to the to regional and local Results easily adjustments criteria. schools. Results offices and to schools. accessible to schools 4 3.2 used in Schools use the and the public. Schools pedagogical and information to make use the information to operational pedagogical and make pedagogical, adjustments. operational personnel, and adjustments. operational adjustments. 4.3 Students do Representative Student assessed in all Assessments of student Frequency not take sample of or in selected grades of learning in all or in of standardized students assessed primary and secondary selected grades of standardize tests. in primary and school every few years, primary and secondary 4 3.7 d student secondary or a sample of students school are done every assessments schools every assessed every year. year for all students in few years. the country. 4.4 Use of Schools are Ministry of Ministry of Education Ministry of Education student not assessed Education only analyzes all or municipal assessments and students analyzes standardized student governments analyzes for do not take standardized exit assessments and sends school assessments, pedagogical standardized exams and sends results and makes results easily and tests. the results to the recommendations to accessible to schools personnel schools. Schools regional and local and the public. Schools 4 3.4 adjustments may use the offices and directly to may use the results to make the schools. Schools information to make pedagogical and may use the pedagogical, personnel, operational information to make and operational adjustments. pedagogical and adjustments. operational adjustments. 4.5 Assessments Only student School assessments are Both school and student Publication are only assessments are available to the School assessments are made of school available to made public. Council and student public and are available and student the School assessment results are online. 4 3.6 assessments educational assessments are made public. authorities used only and to school internally. personnel 31 Table A5. Policy Intent and Policy Implementation Indicators of School Accountability Implemen Sub Indicator Latent = 1 Emerging = 2 Established = 3 Advanced = 4 Intent tation 5.1 Guidelines for There are no Municipal Municipal Guidelines are 2 3.1 using school and guidelines. governments governments and published student Assessments have guidelines schools have online and can assessments for are used only for using the guidelines for be used by the accountability by central assessments, using the public to authorities but they cannot assessments and demand and school take personnel can take accountability staff actions personnel actions 5.2 National or There are no Assessments Assessments are Assessments 4 3.1 regional systems national or are components components of a are components of educational regional of a national or national or sub- of a national or assessments systems of sub-national national system sub-national educational system but and there is a system with a assessments there is no strategy for their strategy for strategy for use their use. They their use are readily available online 5.3 Comparisons Analytical National National and National, 3 3.0 of school and comparisons comparisons municipal regional and student are never made for comparisons municipal performance made different types made for comparisons reports of schools, different types of made; schools regions, and schools, regions, produce student with previous and previous reports that are years. years. Schools given directly to publish summary parents; school results online. results published online 5.4 School School School School Councils School 2 2.6 Council's Councils Councils have have access to Councils have authority to have no access to detailed financial access to perform financial voice over detailed reports and can detailed audits the school financial reports demand an audit financial reports financial and can voice from the and can demand affairs their concerns appropriate an audit from to the Ministry government public or of Education office. private sources. 5.5 Manual for School There is a Schools Council Guidelines used 4 2.5 the participation Councils do manual have a manual by school and of the School not have a regulating the regulating their the School Councils in role in procedures for roles and Council are school audits school audits school audits by responsibilities available online the central or during and after and can be used municipal a school audit by the public to governments demand accountability. 32 Table A6. PISA scores by type of school sponsorship School Sponsorship PISA Reading N SD OBEC 421 130 49.7 OPEC 414 34 44 BMA 432 13 75.6 BLED 412 16 41.8 CHE 428 32 48.9 Total 421 230 51.2 PISA Math N SD OBEC 426 130 59.4 OPEC 407 34 54.4 BMA 431 13 69.4 BLED 413 16 39.7 CHE 431 32 63.1 Total 423 230 58.3 PISA Science N SD OBEC 427 130 55.6 OPEC 419 34 45.7 BMA 439 13 65 BLED 421 16 46.4 CHE 430 32 53.1 Total 426 230 53.2 Source: Calculated by the authors. None of the scores are significantly different from the others. 33 Table A7. T-Test of SABER Policy Implementation Indicators vs. Policy Intent Indicators, Paired Samples Test for all schools Paired Differences Std. 95% Conf. Int. Sig. Std. Error of the Difference (2- SABER Indicator Mean Dev. Mean Lower Upper t df tailed) 1a. legal authority over management of the operational budget 0.261 0.793 0.053 0.157 0.365 4.947 225 0.000 1b. legal authority over the management of non-teaching staff and teacher’s salaries 1.765 0.86 0.057 1.653 1.878 30.847 225 0.000 1c. legal authority to raise additional funds for the school -0.027 0.76 0.051 -0.126 0.073 -0.525 225 0.600 2a. school autonomy in teacher appointment and deployment decisions 0.823 0.739 0.049 0.726 0.92 16.739 225 0.000 2b. school council’s role in teacher tenure or transfer 1.301 1.07 0.071 1.161 1.441 18.279 225 0.000 2c. autonomy in the hiring and firing of principals 1.876 0.767 0.051 1.776 1.977 36.749 225 0.000 3a. participation of the school council in budget preparation 0.748 0.876 0.058 0.633 0.863 12.834 225 0.000 3b. school council's authority to approve the school budget 0.81 0.881 0.059 0.694 0.925 13.81 225 0.000 3c. manual for the participation of the school councils in school finances -0.252 0.99 0.066 -0.382 -0.122 -3.829 225 0.000 3d. role of the school council in budget implementation -0.301 0.979 0.065 -0.429 -0.173 -4.621 225 0.000 3e. use of the budget prepared with the school council's participation -0.217 1.088 0.072 -0.359 -0.074 -2.996 225 0.003 4a. existence and frequency of school and student assessments -0.398 0.64 0.043 -0.482 -0.314 -9.354 225 0.000 4b. use of school assessments for making - school adjustments -0.779 0.689 0.046 -0.869 -0.688 16.982 225 0.000 4c. frequency of standardized student assessments -0.288 0.582 0.039 -0.364 -0.211 -7.425 225 0.000 4d. use of student assessments for - pedagogical and personnel adjustments -0.593 0.756 0.05 -0.692 -0.494 11.793 225 0.000 4e. publication of school and student assessments -0.385 0.624 0.041 -0.467 -0.303 -9.28 225 0.000 5a. guidelines for the use of school and student assessments for accountability 1.137 0.628 0.042 1.055 1.219 27.22 225 0.000 5b. national or regional systems of - educational assessments -0.863 0.614 0.041 -0.943 -0.782 21.135 225 0.000 5c. comparisons of school and student performance reports -0.035 0.923 0.061 -0.156 0.086 -0.576 225 0.565 5d. school council authority to perform financial audits 0.611 1.032 0.069 0.475 0.746 8.893 225 0.000 5e. manual for the participation of the - school councils in school audits -1.491 0.865 0.058 -1.605 -1.378 25.905 225 0.000 Source: Calculated by the authors. 34 Annex 2: Number of Schools by Type of Sponsorship and Indicator Score 1a. legal authority over management of the operational budget SABER Latent Emerging Established Advanced Total Score OBEC 7 10 54 56 127 3.3 % within OBEC 5.50% 7.90% 42.50% 44.10% 100.00% OPEC 1 3 16 14 34 3.3 % within OPEC 2.90% 8.80% 47.10% 41.20% 100.00% BMA 1 7 5 13 3.2 % within BMA 7.70% 53.80% 38.50% 100.00% BLEDC 2 1 6 7 16 3.1 % within BLEDC 12.50% 6.20% 37.50% 43.80% 100.00% CHE 0 1 16 14 31 3.4 % within CHE 0.00% 3.20% 51.60% 45.20% 100.00% Unknown 0 1 3 1 5 3.0 % within Unknown 0.00% 20.00% 60.00% 20.00% 100.00% 1b. legal authority over the management of non-teaching staff and teacher’s salary Latent Emerging Established Advanced Total OBEC 13 22 67 25 127 2.8 % within OBEC 10.20% 17.30% 52.80% 19.70% 100.00% OPEC 3 4 21 6 34 2.9 % within OPEC 8.80% 11.80% 61.80% 17.60% 100.00% BMA 2 2 7 2 13 2.7 % within BMA 15.40% 15.40% 53.80% 15.40% 100.00% BLEDC 2 3 10 1 16 2.6 % within BLEDC 12.50% 18.80% 62.50% 6.20% 100.00% CHE 4 9 15 3 31 2.5 % within CHE 12.90% 29.00% 48.40% 9.70% 100.00% Unknown 1 1 2 1 5 2.6 % within Unknown 20.00% 20.00% 40.00% 20.00% 100.00% 1c. legal authority to raise additional funds for the school Latent Emerging Established Advanced Total OBEC 8 19 70 30 127 3.0 % within OBEC 6.30% 15.00% 55.10% 23.60% 100.00% OPEC 4 24 6 34 3.1 % within OPEC 11.80% 70.60% 17.60% 100.00% BMA 10 3 13 3.2 % within BMA 76.90% 23.10% 100.00% BLEDC 1 1 11 3 16 3.0 % within BLEDC 6.20% 6.20% 68.80% 18.80% 100.00% CHE 3 7 15 6 31 2.8 % within CHE 9.70% 22.60% 48.40% 19.40% 100.00% Unknown 0 1 2 2 5 3.2 % within Unknown 0.00% 20.00% 40.00% 40.00% 100.00% 35 2a. school autonomy in teacher appointment and deployment decisions SABER Latent Emerging Established Advanced Total Score OBEC 6 31 69 21 127 2.8 % within OBEC 4.70% 24.40% 54.30% 16.50% 100.00% OPEC 1 9 18 6 34 2.9 % within OPEC 2.90% 26.50% 52.90% 17.60% 100.00% BMA 2 1 9 1 13 2.7 % within BMA 15.40% 7.70% 69.20% 7.70% 100.00% BLEDC 2 2 11 1 16 % within BLEDC 12.50% 12.50% 68.80% 6.20% 100.00% CHE 0 7 20 4 31 2.9 % within CHE 0.00% 22.60% 64.50% 12.90% 100.00% Unknown 2 2 1 5 2.8 % within Unknown 40.00% 40.00% 20.00% 100.00% 2b. school council’s role in teacher tenure or transfer Latent Emerging Established Advanced Total OBEC 35 44 17 31 127 2.3 % within OBEC 27.60% 34.60% 13.40% 24.40% 100.00% OPEC 9 14 5 6 34 2.2 % within OPEC 26.50% 41.20% 14.70% 17.60% 100.00% BMA 3 7 1 2 13 2.2 % within BMA 23.10% 53.80% 7.70% 15.40% 100.00% BLEDC 2 7 4 3 16 2.5 % within BLEDC 12.50% 43.80% 25.00% 18.80% 100.00% CHE 7 15 5 4 31 2.2 % within CHE 22.60% 48.40% 16.10% 12.90% 100.00% Unknown 2 2 1 5 2.0 % within Unknown 40.00% 40.00% 20.00% 100.00% 2c. autonomy in the hiring and firing of principals Latent Emerging Established Advanced Total OBEC 8 21 69 29 127 2.9 % within OBEC 6.30% 16.50% 54.30% 22.80% 100.00% OPEC 3 6 18 7 34 2.9 % within OPEC 8.80% 17.60% 52.90% 20.60% 100.00% BMA 2 2 9 13 2.5 % within BMA 15.40% 15.40% 69.20% 100.00% BLEDC 1 2 12 1 16 2.8 % within BLEDC 6.20% 12.50% 75.00% 6.20% 100.00% CHE 1 4 25 1 31 2.8 % within CHE 3.20% 12.90% 80.60% 3.20% 100.00% Unknown 2 2 1 5 2.8 % within Unknown 40.00% 40.00% 20.00% 100.00% 36 3a. participation of the school council in budget preparation SABER Latent Emerging Established Advanced Total Score OBEC 14 27 62 24 127 2.8 % within OBEC 11.00% 21.30% 48.80% 18.90% 100.00% OPEC 1 6 20 7 34 3.0 % within OPEC 2.90% 17.60% 58.80% 20.60% 100.00% BMA 1 6 4 2 13 2.5 % within BMA 7.70% 46.20% 30.80% 15.40% 100.00% BLEDC 3 3 9 1 16 2.5 % within BLEDC 18.80% 18.80% 56.20% 6.20% 100.00% CHE 5 5 15 6 31 2.7 % within CHE 16.10% 16.10% 48.40% 19.40% 100.00% Unknown 3 1 1 5 2.6 % within Unknown 60.00% 20.00% 20.00% 100.00% 3b. school council's authority to approve the school budget Latent Emerging Established Advanced Total OBEC 16 16 72 23 127 2.8 % within OBEC 12.60% 12.60% 56.70% 18.10% 100.00% OPEC 4 3 19 8 34 2.9 % within OPEC 11.80% 8.80% 55.90% 23.50% 100.00% BMA 1 1 10 1 13 2.8 % within BMA 7.70% 7.70% 76.90% 7.70% 100.00% BLEDC 3 2 8 3 16 2.7 % within BLEDC 18.80% 12.50% 50.00% 18.80% 100.00% CHE 4 5 16 6 31 2.8 % within CHE 12.90% 16.10% 51.60% 19.40% 100.00% Unknown 2 2 1 5 2.8 % within Unknown 40.00% 40.00% 20.00% 100.00% 3c. manual for the participation of the school councils in school finances Latent Emerging Established Advanced Total OBEC 16 26 53 32 127 2.8 % within OBEC 12.60% 20.50% 41.70% 25.20% 100.00% OPEC 5 5 18 6 34 2.7 % within OPEC 14.70% 14.70% 52.90% 17.60% 100.00% BMA 4 4 5 13 3.1 % within BMA 30.80% 30.80% 38.50% 100.00% BLEDC 3 4 7 2 16 2.5 % within BLEDC 18.80% 25.00% 43.80% 12.50% 100.00% CHE 9 3 10 9 31 2.6 % within CHE 29.00% 9.70% 32.30% 29.00% 100.00% Unknown 1 2 1 1 5 2.4 % within Unknown 20.00% 40.00% 20.00% 20.00% 100.00% 3d. role of the school council in budget implementation Latent Emerging Established Advanced Total OBEC 14 45 35 33 127 2.7 % within OBEC 11.00% 35.40% 27.60% 26.00% 100.00% OPEC 3 10 14 7 34 2.7 % within OPEC 8.80% 29.40% 41.20% 20.60% 100.00% BMA 1 3 2 7 13 3.2 % within BMA 7.70% 23.10% 15.40% 53.80% 100.00% BLEDC 2 5 6 3 16 2.6 % within BLEDC 12.50% 31.20% 37.50% 18.80% 100.00% CHE 6 9 9 7 31 2.5 37 % within CHE 19.40% 29.00% 29.00% 22.60% 100.00% Unknown 2 2 1 5 2.8 % within Unknown 40.00% 40.00% 20.00% 100.00% 3e. use of the budget prepared with the school council's participation Latent Emerging Established Advanced Total OBEC 24 24 37 42 127 2.8 % within OBEC 18.90% 18.90% 29.10% 33.10% 100.00% OPEC 5 11 7 11 34 2.7 % within OPEC 14.70% 32.40% 20.60% 32.40% 100.00% BMA 1 2 5 5 13 3.1 % within BMA 7.70% 15.40% 38.50% 38.50% 100.00% BLEDC 5 3 5 3 16 2.4 % within BLEDC 31.20% 18.80% 31.20% 18.80% 100.00% CHE 4 5 9 13 31 3.0 % within CHE 12.90% 16.10% 29.00% 41.90% 100.00% Unknown 1 3 1 5 3.0 % within Unknown 20.00% 60.00% 20.00% 100.00% 38 4a. existence and frequency of school and student assessments SABER Latent Emerging Established Advanced Total Score OBEC 10 25 92 127 3.6 % within OBEC 7.90% 19.70% 72.40% 100.00% OPEC 3 10 21 34 3.5 % within OPEC 8.80% 29.40% 61.80% 100.00% BMA 3 10 13 3.8 % within BMA 23.10% 76.90% 100.00% BLEDC 1 5 10 16 3.6 % within BLEDC 6.20% 31.20% 62.50% 100.00% CHE 2 8 21 31 3.6 % within CHE 6.50% 25.80% 67.70% 100.00% Unknown 3 1 1 5 2.6 % within Unknown 60.00% 20.00% 20.00% 100.00% 4b. use of school assessments for making school adjustments Latent Emerging Established Advanced Total OBEC 3 12 66 46 127 3.2 % within OBEC 2.40% 9.40% 52.00% 36.20% 100.00% OPEC 4 19 11 34 3.2 % within OPEC 11.80% 55.90% 32.40% 100.00% BMA 2 7 4 13 3.2 % within BMA 15.40% 53.80% 30.80% 100.00% BLEDC 0 2 6 8 16 3.4 % within BLEDC 0.00% 12.50% 37.50% 50.00% 100.00% CHE 0 3 18 10 31 3.2 % within CHE 0.00% 9.70% 58.10% 32.30% 100.00% Unknown 2 1 2 5 3.0 % within Unknown 40.00% 20.00% 40.00% 100.00% 4c. frequency of standardized student assessments Latent Emerging Established Advanced Total OBEC 8 18 101 127 3.7 % within OBEC 6.30% 14.20% 79.50% 100.00% OPEC 2 6 26 34 3.7 % within OPEC 5.90% 17.60% 76.50% 100.00% BMA 1 1 11 13 3.8 % within BMA 7.70% 7.70% 84.60% 100.00% BLEDC 1 3 12 16 3.7 % within BLEDC 6.20% 18.80% 75.00% 100.00% CHE 1 7 23 31 3.7 % within CHE 3.20% 22.60% 74.20% 100.00% Unknown 2 3 5 3.2 % within Unknown 40.00% 60.00% 100.00% 4d. use of student assessments for pedagogical and personnel adjustments Latent Emerging Established Advanced Total OBEC 4 10 38 75 127 3.4 % within OBEC 3.10% 7.90% 29.90% 59.10% 100.00% OPEC 2 15 17 34 3.4 % within OPEC 5.90% 44.10% 50.00% 100.00% BMA 1 8 4 13 3.2 % within BMA 7.70% 61.50% 30.80% 100.00% BLEDC 0 2 5 9 16 3.4 % within BLEDC 0.00% 12.50% 31.20% 56.20% 100.00% CHE 1 5 8 17 31 3.3 39 % within CHE 3.20% 16.10% 25.80% 54.80% 100.00% Unknown 2 1 2 5 3.0 % within Unknown 40.00% 20.00% 40.00% 100.00% 4e. publication of school and student assessments Latent Emerging Established Advanced Total OBEC 1 4 33 89 127 3.7 % within OBEC 0.80% 3.10% 26.00% 70.10% 100.00% OPEC 3 6 25 34 3.6 % within OPEC 8.80% 17.60% 73.50% 100.00% BMA 1 3 9 13 3.6 % within BMA 7.70% 23.10% 69.20% 100.00% BLEDC 0 1 6 9 16 3.5 % within BLEDC 0.00% 6.20% 37.50% 56.20% 100.00% CHE 1 1 9 20 31 3.5 % within CHE 3.20% 3.20% 29.00% 64.50% 100.00% Unknown 1 2 2 5 3.2 % within Unknown 20.00% 40.00% 40.00% 100.00% 40 5a. guidelines for the use of school and student assessments for accountability SABER Latent Emerging Established Advanced Total Score OBEC 4 7 82 34 127 3.1 % within OBEC 3.10% 5.50% 64.60% 26.80% 100.00% OPEC 2 2 23 7 34 3.0 % within OPEC 5.90% 5.90% 67.60% 20.60% 100.00% BMA 1 10 2 13 3.1 % within BMA 7.70% 76.90% 15.40% 100.00% BLEDC 0 0 13 3 16 3.2 % within BLEDC 0.00% 0.00% 81.20% 18.80% 100.00% CHE 0 2 20 9 31 3.2 % within CHE 0.00% 6.50% 64.50% 29.00% 100.00% Unknown 1 3 1 5 3.0 % within Unknown 20.00% 60.00% 20.00% 100.00% 5b. national or regional systems of educational assessments Latent Emerging Established Advanced Total OBEC 15 76 36 127 3.2 % within OBEC 11.80% 59.80% 28.30% 100.00% OPEC 5 22 7 34 3.1 % within OPEC 14.70% 64.70% 20.60% 100.00% BMA 1 8 4 13 3.2 % within BMA 7.70% 61.50% 30.80% 100.00% BLEDC 2 9 5 16 3.2 % within BLEDC 12.50% 56.20% 31.20% 100.00% CHE 5 20 6 31 3.0 % within CHE 16.10% 64.50% 19.40% 100.00% Unknown 1 2 2 5 3.2 % within Unknown 20.00% 40.00% 40.00% 100.00% 5c. comparisons of school and student performance reports Latent Emerging Established Advanced Total OBEC 9 34 39 45 127 2.9 % within OBEC 7.10% 26.80% 30.70% 35.40% 100.00% OPEC 2 13 8 11 34 2.8 % within OPEC 5.90% 38.20% 23.50% 32.40% 100.00% BMA 1 6 2 4 13 2.7 % within BMA 7.70% 46.20% 15.40% 30.80% 100.00% BLEDC 0 5 7 4 16 2.9 % within BLEDC 0.00% 31.20% 43.80% 25.00% 100.00% CHE 0 5 11 15 31 3.3 % within CHE 0.00% 16.10% 35.50% 48.40% 100.00% Unknown 0 1 3 1 5 3.0 % within Unknown 0.00% 20.00% 60.00% 20.00% 100.00% 5d. school council authority to perform financial audits Latent Emerging Established Advanced Total OBEC 27 22 50 28 127 2.6 % within OBEC 21.30% 17.30% 39.40% 22.00% 100.00% OPEC 4 5 18 7 34 2.8 % within OPEC 11.80% 14.70% 52.90% 20.60% 100.00% BMA 5 2 4 2 13 2.2 % within BMA 38.50% 15.40% 30.80% 15.40% 100.00% BLEDC 2 1 12 1 16 2.8 % within BLEDC 12.50% 6.20% 75.00% 6.20% 100.00% CHE 10 3 13 5 31 2.4 41 % within CHE 32.30% 9.70% 41.90% 16.10% 100.00% Unknown 1 1 2 1 5 2.6 % within Unknown 20.00% 20.00% 40.00% 20.00% 100.00% 5e. manual for the participation of the school councils in school audits Latent Emerging Established Advanced Total OBEC 19 32 63 13 127 2.6 % within OBEC 15.00% 25.20% 49.60% 10.20% 100.00% OPEC 6 9 17 2 34 2.4 % within OPEC 17.60% 26.50% 50.00% 5.90% 100.00% BMA 4 1 8 13 2.3 % within BMA 30.80% 7.70% 61.50% 100.00% BLEDC 2 5 9 0 16 2.4 % within BLEDC 12.50% 31.20% 56.20% 0.00% 100.00% CHE 5 8 15 3 31 2.5 % within CHE 16.10% 25.80% 48.40% 9.70% 100.00% Unknown 1 1 2 1 5 2.6 % within Unknown 20.00% 20.00% 40.00% 20.00% 100.00% 42