94445 Systems Approach SABER for Better Education Results SABER - Tertiary Education Governance A Background Paper for the SABER Tertiary Education Domain Data Collection and Assessment Tool on Governance in Tertiary Education Revised: September 29, 2012 1 Acknowledgments This document was prepared jointly by a team comprising of Adriana Jaramillo, Sunita Kosaraju, Jamil Salmi (task team leader until January 2012), and Quentin Wodon (task team leader as of January 2012). The authors are grateful to Elizabeth King (Director, Education, Human Development Department) and Robin Horn (Manager, Education, Human Development Department) at the World Bank for continuous guidance and support. Colleagues from across the World Bank provided feedback that contributed to improving this document. The team is especially grateful for valuable suggestions received from Nina Arnold, Andreas Blom, Erik Bloom, Emmanuela DiGropello, Ariel Fiszbein, Sachiko Kataoka, Benoît Millot, Mamta Murthi, Michelle Neuman, Dena Ringold, Halsey Rogers, and Emiliana Vegas, as well as members of the COREHEG tertiary education thematic group of the World Bank. All remaining errors or omissions are the sole responsibility of the authors. 2 TABLE OF CONTENTS Executive Summary 1. Introduction 2. Governance, Autonomy, and Accountability 3. Eight Policy Goals 4. System-wide Governance Questionnaire 5. Institution-level Governance Questionnaire 6. Rubrics and Ratings 7. Conclusion Annexes 1. Models of Tertiary Education Governance 2. System-wide and Institution-level Indicators and Data 3. Existing Databases of Governance Indicators for Benchmarking Purposes 4. World Bank Institution-level Survey for the MENA Region 5. From Indicators to Indices: An Illustration with Institution-level Data 6. Web-based Institutional-level Questionnaire References List of Tables Table 1: Questionnaire for System-wide Governance Table 2: Questionnaire for Institution-level Governance Table 3: Scoring Rubric for a Policy Goal with Limited Empirical Evidence Table 4: Scoring Rubric for a Policy Goal with Some Empirical Evidence List of Boxes Box 1: Scope of data collection and analysis tool Box 2: Chilean Winter and the Balance between Autonomy and Accountability Box 3: Piloting Institutional-level Surveys in Jamaica List of Figures Figure 1: How does the operating context of institutions differ? A Simple Taxonomy Based on Autonomy and Accountability 3 EXECUTIVE SUMMARY Systems Assessment for Better Education Results (SABER) is designed to assess existing education policies of participating countries in order to enable comparisons between them and learning of best practices. The purpose of this background paper is to present SABER-Tertiary Education Governance, a tool for facilitating comparisons of both system-wide policies and practical implementation of these policies at the institution-level across and within countries in the tertiary education sector. This paper was prepared as a background papers for SABER Tertiary Education, one of the domains in the SABER initiative at the World Bank, itself launched within the context of the Bank’s new education strategy 2020 which sets the goal of Learning for All. Among the factors influencing tertiary education performance, recent research has identified governance as a key determinant. A good governance structure and favorable regulatory conditions can promote innovative behavior among tertiary education institutions, enable the development of strong quality assurance systems, and facilitate the design of effective financing mechanisms. This tool aims to assess tertiary education policy according to eight policy goals. Together these goals lay the foundation for a clear and coherent governance structure for institutions operating in a given country: 1. Clear Vision for Tertiary Education: The country or government has a vision and plan for the tertiary education sector, and a willingness to translate its vision into a concrete action plan. 2. Appropriate Regulatory Framework: The tertiary education system is governed by an appropriate regulatory framework including for private providers. 3. Capacity of the Tertiary Education Authority (TEA): The TEA has staff and resources to implement reforms, and to guide, support and monitor institutions. 4. Leadership, Management and Organizational Autonomy: The regulatory framework allows for sufficient organizational autonomy. This means that the TEA has an appropriate policy on the role and functions of the boards of higher education institutions as well as for the selection of their leadership, and the respective responsibilities of the leadership and the board. 5. Sufficient Institutional Autonomy: The regulatory framework allows for sufficient financial, staffing and academic autonomy in institutions. 6. Presence of performance-based and equity focused funding: Funding mechanism is performance-based, transparent, and promotes equity in student enrolment. 7. Checks on Quality and Relevance: The TEA has an independent quality assurance and accreditation agency for both public and private institutions. 8. Standards of Accountability: Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates. The tool proposed to assess these eight policy goals consists of two surveys. The first is aimed at assessing system-wide policies in the eight areas described above. It is meant to provide World Bank project managers and client governments with a fairly quick overview of the key policies impacting governance of institutions in their country. The second survey is to be completed by representatives of tertiary institutions. It aims to capture the extent to which system-wide policies have been adopted in institutions. 4 1. Introduction Systems Assessment for Better Education Results (SABER) is a World Bank program designed to assess existing education policies of participating countries in order to enable comparisons between them and learning of best practices. The purpose of this background paper for the SABER Tertiary Education domain is to present SABER- Tertiary Education Governance, a tool for facilitating comparisons of both system-wide policies and practical implementation of these policies at the institution-level across and within countries in the tertiary education sector. Comparisons of tertiary education1 systems across the world reveal wide variations in performance, despite often-similar funding levels and common country characteristics. They demonstrate that certain tertiary education systems consistently out- perform the others in many critical areas. Among the factors influencing performance, recent research has identified governance as a potential key determinant. A good governance structure and favorable regulatory conditions can promote innovative behavior among tertiary education institutions, enable the development of strong quality assurance systems, and facilitate the design of effective financing mechanisms. This is especially the case for research universities, where autonomy can lead to better management of resources (whether financial or human) with flexibility and agility to respond to changes in the global marketplace (Aghion 2008; Altbach and Salmi 2011). But it is also true for universities that focus on teaching and student learning. In a comparative study of European and U.S. universities, Aghion et al. (2007) found that governance and funding were important determinants of rankings. In a subsequent paper reporting on a survey of European universities, Aghion et al. (2008) find that research performance is positively linked to the degree of autonomy of universities, especially in terms of budget management, the ability to hire faculty and staff, and the freedom to set salaries. With respect to the composition of university boards, the report concludes that “having significant outside representation on the board may be a necessary condition to ensure that dynamic reforms taking into account long- term institutional interests can be decided upon without undue delay.” Autonomy however is not sufficient for good governance by itself. It needs to be combined with accountability to ensure that (public) institutions perform in a way that supports national objectives for quality, relevance, transparency and equity within the overall system. Fielden (2008) defines tertiary education governance as “all those structures, processes and activities that are involved in the planning and direction of the institutions and people working in tertiary education”. Governance can be analyzed at the level of a university, but it also needs to be analyzed at the level of systems. Indeed, university- level governance depends on the overall framework that regulates the broader tertiary education system and the interaction between institutions and the state. For public universities especially, where the main source of funding comes from the state, a national law often centrally defines the legal framework in which the institution operates, and this law may go as far as determining institutional goals. But even for private universities, a regulatory framework typically governs entry in the system as well as accreditation rules. 1 The tertiary education sector refers here to the colleges and universities (whether public or private) that provide associate, bachelor, or higher degrees after secondary schooling in a given country. 5 The fact that governance is now increasingly being taken into account in attempts to provide system-wide rankings of tertiary education systems is illustrated by a recent study by Universitas 21 (2012). That study considers four headings in assessing the performance of tertiary education systems: the resources available to the systems, the systems’ outputs (research and the education of the workforce), its connectivity (including international networks), and its environment (government policy and regulation, diversity and participation opportunities). System-wide governance relates to the macro level laws, structures, policies, and processes that are in place with the purpose of providing a favorable context in order for tertiary education institutions to be able to operate efficiently and effectively. These institutions are the actors that, by working in an integrated and articulated fashion, can produce the education outcomes that are most important to any nation- attainment, quality and relevance, equity, research, technology transfer, and positive values and behaviors. System-wide governance also refers to policies and practices that enable higher institutions to operate more autonomously, while promoting accountability. One critical challenge that confronts countries or states when developing a tertiary education system, and which is related to the strategic vision component of governance, is the extent to which the system should be oriented towards meeting high level research goals as opposed to providing teaching and access to a wide majority of people. The California Master Plan of 1960 is one example of an effective way of organizing a differentiated public education system to cater both to research excellence and to provide access to a large segment of the population (Altbach 2011). The plan, developed by Kerr (2001), established a three-tiered public tertiary education system that has been used as a model not only in the U.S. but also around the world. It makes clear distinctions between research universities that cater to elite researchers, offer Ph.D. programs, and have a clear research mission, traditional universities that offer bachelors and masters degrees where the main mission is teaching and there are no Ph.D. programs, and community colleges 2, which focus mainly on two- and four-year programs and are not involved in research. As a truly integrated system, there are important linkages and pathways between the three tiers, and the admissions processes and number of student placements available in each tier are defined in terms of the state’s economic and development needs. Thus, in California, research universities annually admit the top 100,000 high school students, while the second tier enrolls 450,000 students, and the third tier admits 3 million students. This differentiation allows the state to combine its needs for cutting-edge research with access to high quality, relevant education for a vast majority of students. Having a clear division of labor at the overall system levels also benefits the universities themselves, allowing them to focus on their mission and to align their goals with that mission. Thus, state universities are not expected to conduct cutting-edge research and their academic staffs are not obligated to maintain the same research intensity as faculty from the research campuses. A community college is not expected to produce the same results as a research university. Their missions are different and therefore their goals and expected results must be different. Having this clear distinction is absolutely essential for sound governance. The definition of each university’s mission is fundamental to the governance structure, and, correspondingly, this system allows the 2 The term ‘community college’ is commonly used in the United States and Canada to refer to institutions delivering degrees designated internationally as ISCED 5B. 6 state to differentiate funding and to align resources with the mission and goals of the different types of institutions. Another important role played by tertiary education institutions relates to their contribution to regional development. This is not an easy role to play, however. OECD (2008; see also Raza 2010) identifies four system-wide factors restricting the contribution of higher institutions to regional economic development: a lack of explicit orientation of public policy toward that goal; inadequate incentives structures for regional engagement; limits to autonomy and leadership; and a limited capacity of local and regional actors to determine strategic directions of higher institutions. These findings suggest again a need for the state to play a specific role in guiding and directing the tertiary education system, set an appropriate regulatory framework for new institutions to enter and existing institutions to operate, and align financial rewards with performance. Much has been said about the need for granting more autonomy to universities to enable them to adapt to a rapidly changing marketplace. But as already mentioned, autonomy needs to be paired with accountability. As noted by Henard (2009), as the role of tertiary education in contributing to economic growth becomes more important, so does the attention that must be paid to quality assurance in order to ensure the financial, academic and informational integrity of the tertiary education system. Huisman and Currie (2004) have documented the shift from input focused and rigid state-controlled systems of accountability to those that are broader, outcome based and allow for independence of institutions in day-to-day decision-making. This shift towards more outcome-based accountability has not only evolved from a changing relationship between government and universities, but from an the increased focus on efficiency, value for money spent, and globalization of education requiring greater accountability of tertiary education institutions. In such contexts, tracking the use of financial audits, corruption (whether financial, academic or information), and transparency are especially important. While it makes intuitive sense that good governance, and especially autonomy combined with accountability, may help in improving performance, it must be recognized however that empirical evidence to this effects remains limited. Good data on university- level as well as system-wide governance are hard to come by, especially in low and middle income countries. Because of lack of data, the relationship between governance and performance has not been analyzed in depth, and even the measurement of governance is not a settled issue (Wodon and Adoho 2012). As part of SABER’s broader program, the objective of this paper is to contribute to data collection and analysis of governance in tertiary education at the level of both systems and institutions. The paper is structured as follows. Section 2 briefly discusses the link between governance, autonomy, and accountability (a more detailed review of the various models of tertiary education governance that have been discussed in the literature is provided in Annex 1). Section 3 outlines the eight policy goals around which the data collection tools and assessments proposed in this paper are structured. The data collection questionnaires that are proposed for assessing the level of implementation of the eight policy goals at the level of the tertiary education system as a whole and at the level of institutions are then presented in sections 4 and 5. Section 6 discusses the issue of aggregating multiple indicators through rubrics and ratings. A conclusion follows. A series of five annexes are also provided to discuss some questions in more details. 7 2. Governance, Autonomy, and Accountability The purpose of this paper is to propose an assessment tool that will facilitate comparisons of both system-wide policies on tertiary education and practical implementation of these policies across countries for identification of best practices in the area of governance of universities and community colleges. The paper was prepared as part of a series of papers for the SABER initiative at the World Bank, itself launched within the context of the Bank’s new education strategy 2020 which sets the goal of Learning for All. SABER aims to support countries in systematically examining their education policies in a range of domains. Some of those domains are related to various education levels, such as Early Child Development at the beginning of life, or Tertiary education. Other domains are related to specific inputs in the education systems, such as teachers, finance, or learning standards. Each SABER domain provides a standardized diagnostic tool meant to facilitate the description and analysis of policies for a given country in a specific domain, with standardization facilitating cross-country comparisons and benchmarking, and in some cases within country comparisons. This specific tool on tertiary education governance takes as one of its core assumptions the fact that higher autonomy for universities and community colleges coupled with accountability mechanisms will tend to generate better results. This assumption may not necessarily be valid in all contexts, but it tends to reflect the literature on the performance of education systems (see Annex 1 for a brief synthesis of the literature related to models of tertiary education governance). Box 1: Scope of the Data Collection and Analysis Tool The framework provided in this paper includes both a system-wide data collection tool and a tool to be implemented at the level of institutions. The institution-level tool may itself be applied to both universities and community colleges. ISCED 5A, or university degree programs, are primarily theoretically based and are intended to provide sufficient qualification for gaining entry into advanced research programs and professions with high skills requirements. They are normally expected to be at least three years in duration although examples exist of 5A programs that are of shorter duration such as the university transfer programs offered by Canadian and American community colleges. ISCED 5B refers to programs that, like ISCED 4A programs, generally require successful completion of an upper secondary qualification or its equivalent for entry but which are generally shorter, more practical, technical or occupationally specific than ISCED 5A programs (OECD, 2004). While there are differences in overarching institutional mission and while the preparation of students varies between sectors, the main principles for good governance including vision, regulatory framework, capacity of tertiary education authority, autonomy and accountability remain important for both ISCED 5A and 5B institutions. There are however some differences in best practices between the institutions depending on the type of institution being considered. 8 Figure 1 shows in a stylized way how the relationship between the two concepts of autonomy and accountability may affect outcomes in the tertiary education sector. In the top-right quadrant, we have a tertiary education system consisting of “Self-managed, internationally competitive” colleges and universities. An example of such a system is Denmark, where universities have high levels of autonomy, and this is balanced by high levels of accountability. In this case, the government is detached from the process of actually managing universities and it concentrates instead on setting and monitoring policy goals and targets, and allocating budgets through which the aims are to be fulfilled (Wright and Orberg, 2008). For example, the Ministry of Education and Research determines national research priorities and budgets and who is responsible for what. The government funds research through grants and institutions have the opportunity to win competitive funds from the national research council and private organizations. At the same time, Danish institutions have a high degree of human resources autonomy to attract highly talented researchers through monetary incentives (Estermann and Nokkala, 2009). Figure 1: How Does the Operating Context of Institutions Differ? A Simple Taxonomy Based on Autonomy and Accountability Low autonomy High autonomy High accountability Centrally managed with high Self-managed and internationally quality competitive (Singapore, Canada) (Denmark) Low accountability Centrally managed with low Cowboys quality (Some (Some private institutions, for post- soviet countries) example in Latin America) Source: Authors. A tertiary education system in which institutions have a high level of autonomy without accountability can by contrast lead to negative outcomes such as low quality educational content, poor delivery and learning outcomes, and financial, academic or information fraud. Under high autonomy and low accountability, universities are free to do as they please and may behave like “cowboys”. Such cases of high autonomy without accountability are encountered among others in the Latin America and Caribbean region (see for example the brief discussion of the question of tuition at private universities in Chile and the recent reforms adopted by the government in Box 2). When tertiary education systems have a larger and increasing proportion of institutions that are private, this lack of balance may not only lead to poor quality, but also to a lack of equity. Private universities with full autonomy on setting tuition fees have flourished in Latin America but often without minimum standards for quality. When there is a high level of competition for admission in the few universities which are of high quality, this tends to 9 relegate students from lower socio-economic backgrounds to low quality schools that are overrepresented in high priced private institutions (The Economist, 2011) A third stylized scenario is that of a centrally managed system with low quality where public institutions in countries with low autonomy and low accountability are not performing well. This has been observed in some post-Soviet countries, as well as in South Asian countries, notwithstanding some of those countries’ flagship high quality institutions3. In this scenario the state controls universities and colleges to the extent that public institutions have almost no autonomy at all (Raza, 2010) At the same time, institutions are not held to high standards of accountability, whether these standards relate to the quality of the education provided or academic and financial integrity. Finally, institutions that have low autonomy but high levels of accountability may be considered as belonging to a centrally controlled system with high quality. This case would feature countries such as Singapore (even though recent reforms in Singapore devolved greater autonomy to the three national universities) and to some degree Canada. Even if the centrally controlled system with high quality may work for some countries, the approach suggested in this paper relies implicitly more on a model combining high autonomy and high accountability. In so doing, both the level of the tertiary education system as a whole and the level of specific institutions are considered. That is, while many other SABER domains focus almost exclusively on systems-wide indicators, this paper also includes indicators to assess the degree to which stated policies are implemented in practice at the institutional level. The fundamental reason to do this is that collecting data on institutions as well as on systems makes it feasible to assess whether governance regulations and policies as envisioned in the law and other regulatory instruments at the system level are indeed being implemented in the field at the level of institutions, where the rubber hits the ground. The value of looking at the system level as well as at the level of institutions is that one can then better identify whether specific policies are implemented or not, and whether they are working or not. This in turn may help to understand potential bottlenecks for improving governance, while still maintaining a focus on the role that national policies and regulations play in guiding institution-level practices – which is the main objective of SABER. 3 In India for example, many institutions are public but quality assurance is voluntary with less than 25 percent of institutions opting for it. This signals a lack of responsiveness to consumer (student) needs. Tuition fees are determined by the state and institutions do not have the autonomy to raise their own funds, nor to set faculty or staff salaries. An apex body called the University Grants Commission (UGC) is responsible for a range of functions (setting the strategic plan, allocating budgets, performing quality assurance and monitoring functions), but without power over funding, this body has no mechanism to enforce compliance (Raza, 2009) . 10 Box 2: Chilean Winter and the Balance between Autonomy and Accountability In May 2011, over 200,000 students took to the streets in protest of current state of affairs in Chile’s tertiary education system. Protests were focused around two main issues: First, that providers should not make a profit from education and second, that all students should have a right to tertiary education due to necessity of diploma for career. This was in the context of a country where demand for tertiary education had grown significantly over one generation. In 1980, the gross enrolment rate for tertiary education was just 12 percent and by 2009, it had jumped to about 60 percent. This took place despite the fact that the country had some of the world’s lowest levels of public funding for higher education (at 0.3 percent versus an OECD average of 1.0 percent), some of the longest degrees, and no comprehensive system of student grants or subsidized loans. Rising indebtedness of students due to increasing tuition fees, low quality, and a perception that Chile's education system was unfair – giving, as the BBC put it, “rich students access to some of the best schooling in Latin America while dumping poor pupils in shabby, under-funded state schools" were behind the protests (Long, 2011). In response to these protests, the government introduced a number of governance and financing reforms which are expected to take effect in 2013 in order to improve the level of information available to students prior to enrolling in university studies, as well as to improve the financial accessibility of tertiary education. Efforts aimed to balance the degree of financial autonomy enjoyed by public and private institutions with a measure of accountability. Prior to the protests, universities were setting tuition fees based on what competitors were charging rather than the cost of programming or the expected return to students upon graduation. In response, the government proposed a tuition reference rate system where institutions would be funded based on graduates’ performance in the labor market, adjusted by unemployment rates, students’ socioeconomic background, dropouts, and time-to-degree. While institutions may charge a fee higher than the tuition reference rate, they would then be responsible for financing 100 percent of the gap between the real tuition and the reference tuition, for example through scholarships and loans with the same conditions as government loans for students from the two lowest income quintiles, with a similar mechanism in place for middle-income students albeit at a rate of less than 100 percent (World Bank, policy brief, 2012). This is a good example of a reform that aims to promote both autonomy and accountability in tertiary education. 11 3. Eight Policy Goals On the basis of a review of the literature on best practices in tertiary education systems governance, this paper suggests a set of eight guiding principles or policy goals for systems interested in improving the performance of their universities and colleges through more autonomy coupled with strong accountability. This section briefly presents each of these eight policy goals as well as the rationale behind them. (1) Clear Vision for Tertiary Education: The country or government has a vision and plan for the tertiary education sector, and a willingness to translate its vision into a concrete action plan. A clear statement of priorities for the tertiary education sector is a first step in creating alignment between individual colleges and universities and national or regional objectives. It can also help in mobilizing political and public financial resources for support and investment in the sector. In its flagship study on Tertiary Education in the Knowledge Society, the OECD investigated the factors restricting tertiary institutions from contributing more significantly to regional economic development. The study showed that one key barrier was the “lack of explicit orientation of public policy toward that goal” (OECD, 2008). Furthermore, as noted by Fielden (2008), the articulation of a vision and strategy involves answering questions about the purpose of higher education within a country and the principal goals and targets to be set in terms of participation, as well as questions about the process by which these targets will be achieved including the types of institutions to be supported, the time period for such support, and the definition of the role of the private sector and broader community in achieving these objectives. At the institutional level, clarity of the mission and goals of a university is also essential for the effectiveness of institutional-level governance. A mission represents a shared understanding of the core purposes of the institution. While it is informed by an institution’s history, it must continually be re-interpreted to meet the contingencies of the times. The establishment and occasional modification of a university mission is a classic governance function that, depending on the governance model, may be fulfilled only by academic staff (sometimes only senior faculty) or by academic staff together with professional staff, students, and government agencies. (2) Appropriate Regulatory Framework: The tertiary education system is governed by an appropriate regulatory framework including for private providers. An appropriate regulatory framework would typically include a statement on tertiary education in national law as well as a legal framework stating eligibility of private providers for entrance into the tertiary education sector, certification requirements for operations, and the specific regulatory requirements that apply to for-profit and nonprofit institutions. The need to clarify roles and requirements in legislation is becoming increasingly important given the growing complexity of the tertiary education sector in which many different actors may be involved. As explained by Fielden (2008): 12 “The boundaries of a nation’s higher education system are not always easy to define since tertiary or higher education activities may well take place within the ambit of ministries or agencies not directly concerned with education.” What is also important is to set a framework for how institutions may adapt their operations to respond to changes in the demographics of the student population (e.g., for the inclusion of international students and adult learners on top of the more traditional national 18-24 cohorts) and the technology of education delivery (such as the move towards distance learning and work-study programs). An appropriate regulatory framework helps in clarifying how various stakeholders may operate within the system, allowing space for both state and private organizations without creating additional barriers to flexibility (Fielden, 2008). (3) Capacity of the Tertiary Education Authority (TEA): The TEA has staff and resources to implement reforms, and to guide, support and monitor institutions. The reform capacity of the tertiary education authority is another critical part of effective governance of the system. Reform capacity refers to the ability (for example in terms of staff size, skills, and power) of the tertiary education authority to develop and implement new reforms. However robust and well-thought out the vision and strategy of the system may be, little progress will be made without a competent, well-organized, and motivated agency involved in elaborating and monitoring policies and programs to realize stated national goals,. This has been recognized by the OECD which noted that a key barrier to a better contribution of tertiary education institutions to the broad public policy goal of regional economic development was the limited capacity of local and regional actors to determine the strategic direction of tertiary institutions (Raza, 2010; OECD, 2008)4. The instrument proposed in this paper thus assesses the capacity of the TEA to implement reforms (in terms of broad competency, sufficient staffing levels, and technical expertise for building the management capacity of individual colleges and universities). At the policy level, the instrument also captures the degree to which a tertiary education management information system (TEMIS) is used to formulate public policy. Finally, at the policy practice level, the instrument assesses the perceptions of institutional administrators regarding the competency of TEA staff and the frequency with which they report to the TEA on key personnel, academic, and financial data. 4 Such capacity has long been emphasized in the management literature. See for example the popular organizational change framework suggested by Waterman et al. (1980) asserting that productive organizational change is the product of the multiple relationships between 7 S’s: structure, strategy, systems, style, skills, staff, and what the authors refer to as superordinate goals or shared values. 13 (4) Organizational Autonomy, Leadership and Management: The regulatory framework allows for sufficient organizational autonomy. This means that the TEA has an appropriate policy on the role and functions of the boards of higher education institutions as well as for the selection of the leadership of higher education institutions, and the respective responsibilities of the leader. Organizational autonomy refers to the role and functions of the governing board of the university, and the selection modalities of the leadership team. Governing boards of tertiary institutions have a critical role to play in ensuring that the organization is able to adapt to changing needs of students and society, while also serving as an unbiased third party in cases where powerful stakeholder groups (i.e. students, faculty, alumni) attempt to block necessary institutional changes. In private institutions in the United States, governing boards are the ultimate governing authority, while at public institutions boards have a significant weight due substantial delegated authority given by governments to them (MacTaggart, 2011). Board input can help among others to form a fresh vision for the institution; focus attention on the most critical challenges; critique the prevailing wisdom from the vantage point of knowledgeable outsiders; and bring a new perspective (often from business world) to illuminate old problems. Further, board contributions based on formal authority can ensure regular monitoring of progress (or lack thereof) on institutional goals, which in turns makes it feasible to make tough decisions when the institution is divided on what option to choose (MacTaggart, 2011). Given the important role boards have in the strategic leadership of tertiary education institutions, this instrument assesses the composition of the board, the selection of board members, their level of independence, and the degree of clarity on the level of independence of the board. A second area of assessment under this policy goal relates to policies on the leadership of tertiary institutions. In particular, the instrument assesses the method by which the university leader and his/her team are selected, as well as policies on term limits. Strong leadership and strategic planning are key elements of a well-performing university. Management refers to the day-to-day but no less critical decisions of operating the institution, among others in the areas of admissions, registration, and certification of degrees for students; appointments, remuneration, and promotions of academic and other staff; construction and maintenance of facilities; scheduling of classes and assignment of staff to classes and laboratory spaces; the ability to secure sufficient resources to carry out the complex mission of the institution; and the allocation of scarce resources to various claimants. Management is sometimes undervalued and contrasted with leadership, which is generally portrayed as more visionary and more likely to alter the fundamental trajectory of the institution than mere management. Yet vision and the capacity to fundamentally change a university (or any institution or organization) require the ability to institutionalize change, to effectively create new resources by the efficient use of existing resources, and to make difficult managerial decisions on resource reallocation. Effective governance and management are thus inextricably woven together. Management without vision cannot adapt; leadership without effective management remains merely visionary (Johnstone, 2010). 14 A difficult question is whether the role of a university President should be primarily managerial, primarily academic, or both. The literature is divided on whether a university leader should be a top scholar, or whether the President should be a manager from outside of the academic world. How the university leader’s profile is defined is closely related to the governance model or trend a given university follows (e.g., academic or corporate – see annex 3 for a discussion). In some corporate governance models, the role of the university head is purely representative, depending on the powers given to the board. In most cases however, the President is responsible for day-to-day management of the university and for the implementation of its goals. More generally, the leadership and management orientation dimension in this paper refers to the head of the university and the available governing bodies, their composition, the process for selecting or appointing their members, as well as their roles and responsibilities, their reporting lines, their accountability measures, and the duration of their assignment. (5) Sufficient Institutional Autonomy: The regulatory framework allows for sufficient financial, staffing and academic autonomy in institutions. There is a strong international trend toward increasing the autonomy of university institutions by making them independent and self-governing. This emerged from the need to make universities more responsive to social and economic environments and more able to adapt to changing technologies and innovate. Academic autonomy and academic freedom are critical elements of well-performing universities. To be successful, Rosovosky (2001) observed that academic staff must be involved in the decision-making of the university. Research universities especially need the full involvement of academic staff in decision-making, and typically have a greater degree of professorial power and stronger guarantees of academic autonomy than other institutions (Altbach, 2011). This paper considers four principal features of institutional governance as it relates to the degree of autonomy of tertiary education institutions. These are (i) organizational autonomy, (ii) financial autonomy, (iii) staffing autonomy, and (iv) academic autonomy. Organizational autonomy was already discussed under the previous policy goal, but the other forms of autonomy must also be reviewed. Financial autonomy refers to an institution’s capacity to mobilize resources, manage its resources, and use them efficiently. By having control over their resources and facilities, tertiary institutions are in a better position to adapt and improve to meet the changing demands of society and the market. Staffing autonomy refers to the capacity and responsibility of the institution to recruit staff and set salaries, usually directly related to the legal status of staff (whether civil servant or employee of the university itself). Finally, academic autonomy refers to the degree to which an institution is at liberty of directing its own education and research strategy, for example choice in specialization in research or focus on teaching of specific topics and targeting of specific populations for enrolment. It also includes the ability of the institution to manage its academic profile and admission policy (including qualifications and size of student enrolment). Last but not least, it encompasses the degree of academic freedom enjoyed by tertiary education institutions in a given country. The grouping of autonomy factors into these four categories was first introduced by the European University Association (EUA) in its Lisbon declaration of 2007. This declaration results from discussions that took place among 700 universities and partners 15 during the fourth EUA Convention of Higher Education Institutions hosted by the five Lisbon universities and the Portuguese Rectors’ Conference. The selection of these four dimensions as the principle elements of institutional governance is further supported empirically by Aghion et al. (2007, 2008, 2009) in various studies focusing on the importance of autonomy for successful research and innovation at universities in Europe and America. The authors define tertiary institution performance as a ranking in the Shanghai Jiao Tong University (SJTU) ranking of world universities. In attempting to identify the factors that correlate with being ranked in the SJTU, the authors rely on factor analysis using a survey of European universities. They find that the autonomy factor is maximized for those European universities that share a number of characteristics, including that they (i) do not need to seek government approval of their budget; (ii) select their baccalaureate students in a manner independent of the government; (iii) pay faculty flexibly rather than based on a centralized seniority/rank based scale; (iv) control their hiring internally; (v) have low endogamy; (vi) own their own buildings; (vii) set their own curriculum; (viii) have a relatively low percentage of their budget from core government funds; and (ix) have a relatively high percentage of their funds from competitive research grants (Aghion et al., 2009). (6) Presence of incentive-based funding: Funding mechanism provides incentives for institutions to achieve system wide goals on institutional performance and equity among others. Incentives are transparent and known to all. Incentive-based funding involves the provision of funds to institutions based on their success in meeting certain performance targets (such as graduation rates, retention rates, or the employment rates of graduates) or contribution to certain system-wide goals such as equity (for example, enrolment of a certain number of minority or low income students into the first-year cohort). This policy goal also captures the extent to which the system allocates funds in a competitive way in the areas of research and investment. In their review of tertiary education funding allocation mechanisms, Salmi and Hauptman (2006) conclude that one of the most effective policy instruments to improve performance is the linking of funding formulas to performance measures for institutions. They write: “Tying the distribution of funds to institutions or students to performance measures can make a real difference in the ability of tertiary systems to achieve the goals of improved equity, quality, and efficiency” (Salmi and Hauptman, 2006). They caution however that the success of these policies in achieving improved results depends to a large degree on how the policies are designed and the problem that they intend to target. A number of OECD countries use this funding approach, including Denmark, the United Kingdom, Israel, and the Netherlands. These countries not only link funding allocations to the number of graduates of tertiary education institutions, but often go further. In Denmark for example, a ‘taximeter’ system was implemented to improve teaching outcomes. Institutions were funded based on the number of students who had passed exams and an additional quality assurance mechanism was instituted to ensure that standards remained high. Evaluations have shown that there are few negative effects of performance based funding, and indeed as a result of such reforms the management of the education sector tends to improve, with an increased focus on “value for money” (OECD, 2006). 16 (7) Checks on Quality and Relevance: The TEA has an independent quality assurance and accreditation function for both public and private institutions and it monitors the labor market relevance of degrees through an analysis of employment trends for graduates. The literature also emphasizes the importance of quality assurance mechanisms in a well-functioning tertiary education system. This is especially important given the significant surge in demand over the past 20 years for higher education degrees in low and middle income countries and the large supply of private providers (both brick and mortar and online) rushing to meet this demand in light of tightening public expenditure in the sector. The quality of the educational experience is what is at stake, with both private and public institutions aiming to strike the right balance between quality and cost. On the quality assurance side, the instrument proposed in this paper assesses both external quality assurance (through the presence and independence of accrediting bodies), and the degree to with quality assurance agencies guide institutions on developing strong internal quality assurance measures. On the relevance side, this instrument measures the extent to which institutions administer and publish student engagement and employment tracer surveys. In some countries, developing or strengthening quality assurance processes is a key step for improving higher education systems. External quality reviews (conducted in most cases by governments) prompt institutions to focus on monitoring and improved performance for “continuous improvement” (Green, 2011). As institutions devote more attention to internal assessments and evaluations, the relationships between student skills, knowledge and achievement, the requirements of the workplace, the requirements of professional bodies, and the marketability of students are all being brought into sharper focus (Achim et al. 2009). (8) Standards of Accountability: Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates. Finally, standards of accountability are adopted to ensure the financial, academic, and informational integrity of the tertiary education system. Components of the accountability framework proposed in this paper include indicators tracking the use of financial audits, measures against corruption (whether financial or academic), and measures for transparency. The need to ensure transparency and fight corruption is obvious enough, but its importance cannot be understated. For example, data collected by the World Bank Institute on the extent of corruption in a half dozen countries in Africa suggest that petty corruption may be highly prevalent in higher education (Tsimpo and Wodon, 2012a). In one of the countries, it appeared that close to half (44.1 percent) of the families with students in a tertiary education institutions made unofficial payments (these are the terms used in the survey) to the schools or its staff, and that the value of such payments was at about a third of the value of the official payments5. While some of 5 In addition, when asked whether the individual or any other person in the individual’s family had to make any unofficial payments to get your child admission in the school, 27.2 percent responded in the 17 these payments may be legitimate, so that these estimates are likely to be upper bounds of the incidence of petty corruption in tertiary education, it does appear that such corruption is widespread, and more so than in primary and secondary education. But the issue of accountability is broader than that of corruption and transparency. It also relates also to a broader shift from input focused and rigid state-controlled systems of accountability towards broader and outcome based systems that provide independence to institutions in their day-to-day decision-making. This push for more outcome-based accountability has not only evolved from the changing relationship between governments and universities, but also because of an increased focus on efficiency, value for money spent, and globalization of education requiring greater accountability of many new cross- border institutions (Huisman and Currie, 2004). As governments and ministries around the world promote more autonomy for public universities, the price to pay is often greater demand for accountability, and these demands are increasingly in the form of measurable demonstrations of achievement and progress towards the university’s goals. For instance, if the goal is to advance scholarship in basic science, then a measurable outcome might be the number of articles accepted in internationally-recognized, peer-reviewed journals in the university’s self-identified fields of scholarly emphasis. If the goal is applied scholarship, then the number and success rate of consulting contracts, grants, and patents applied for and awarded in a particular field might be measured. 4. System-wide Governance Questionnaire The eight policy goals identified above, together with policy levers which can help in achieving these goals and indicators which can be used to measure the effectiveness of these policy levers, are provided in more detail in table 1. While there would clearly be other ways to specify policy levers and indicators in order to measure governance, the questionnaire proposed in table 3 should work well in most contexts. It is worth noting that fairly different types of indicators are included in table 1. While some indicators are quantitative, others are objective qualitative indicators, and still others are subjective qualitative indicators. Quantitative indicators provide users with a tangible measure to compare governance across various dimensions of country systems and institutions. Data for these indicators are relatively easier to collect than qualitative data. Unfortunately, few governance indicators come under this category, for example the proportion of accredited institutions or programs in a country, or the proportion of budget allocated based on performance criteria, or the proportion of students who report corruption or academic fraud. Objective qualitative indicators describe key dimensions of governance in a non-numeric way. Qualitative indicators can capture in an objective manner the main characteristics of a tertiary education system’s affirmative. When asked whether it is common for parents to make unofficial payments to gain admission for their children, 43.0 percent of respondents said sometimes. 47.3 percent said frequently, and 6.5 percent said always. Finally, when asked how such unofficial payments occur more frequently, about one third of respondents responded in the affirmative for each of the three alternatives asked in the questionnaire (a school official indicates or asks for a payment, the parent or family member offer a payment on his/her own accord, or it is known before hand how to pay and how much to pay, so it is not discussed). 18 quality assurance set up (existence of an accreditation system) and key dimensions of organizational autonomy (existence of an independent Board, mode of selection of university leaders). Subjective qualitative indicators are constructed on the basis of expert judgments on key dimensions of system and institutional governance. The fact that many different indicators are included in table 3 begs the question as to whether some may be more important than others in order to improve system performance, and the same question could be asked at the level of any specific institution. While it makes intuitive sense that good governance, and especially autonomy combined with accountability, may help in improving performance, it must be recognized however that empirical evidence to this effects remains limited. Good data on university-level as well as system-wide governance are hard to come by, especially in low and middle income countries. Because of lack of data, the relationship between governance and performance has not been analyzed in depth, with the exception of a few studies by Aghion and others already mentioned earlier, and to a large extent even the measurement of governance is not a settled issue (Wodon and Adoho 2012). We will come back to the difficult question of how to aggregate and prioritize the information provided in the proposed system-wide and institution-level questionnaires in section 6. 19 Table 1: Questionnaire for System-wide Governance Policy Goals Policy Levers Indicators Measurement Goal #1. Setting a Clear Vision for Policy directions Existence of a strategic 0. No vision /plan. Tertiary Education vision and development plan 1. Partial development/reform plan but no implementation. The country or government has a for tertiary education 2. Partial vision/reform plan with implementation. vision and plan for the tertiary 3. Strategic vision/plan with implementation. education sector, a willingness to translate its vision into a concrete action plan. 20 Policy Goals Policy Levers Indicators Measurement Goal #2: Regulatory Framework Legal framework for Existence of tertiary 0. No tertiary education law exists. The tertiary education system is overall tertiary education law 1. Tertiary education law exists but is has not been revised in 10+ years. governed by an appropriate education system 2. Tertiary education law has been updated in last 10 years. regulatory framework including for private providers Legal framework for Regulation for the entry of 0. Private providers not allowed or heavily restricted. the regulation of private providers 1. No regulation for the entry/certification of private providers. private providers 2. Reasonable regulations to control entry/certification of private providers. Regulation for the operations 0. No regulation for the operations of private providers. of private providers 1. Unclear regulation for the operations of private providers but without clear distinction between for-profit and not-for-profit private institutions. 2. Clear regulatory framework for the operations of private providers including distinction between for-profit and not-for-profit private institutions. 21 Policy Goals Policy Levers Indicators Measurement Goal #3 Capacity of the Tertiary Tertiary Education Existence and capacity of 0. Inexistent or weak TEA. Education Authority (TEA) Authority Tertiary Education Authority 1. Small TEA with limited capacity and influence. The TEA has the capacity to guide, (TEA) 2. Reasonably well staffed TEA able to implement limited reforms. support, implement and monitor the 3. Highly credible/capable TEA able to implement controversial reforms. TEIs and the TE system Capacity building: the 0. TEA has no qualified staff for capacity building purposes. national or provincial 1. TEA has a few programs for building the management capacity of authorities have the know- tertiary education institutions. how and competencies to 2. TEA has extensive programs for building the management capacity of strengthen institutional tertiary education institutions. capacity in the sector Existence of Tertiary 0. No TEMIS. Information system Education Management 1. Incomplete TEMIS. Information System (TEMIS) 2. Complete system exists but is not frequently used for policy purposes. used for policy formulation 3. Comprehensive TEMIS exists and is frequently used for policy purposes. 22 Policy Goals Policy Levers Indicators Measurement Goal #4. Leadership, Management Governing Board Composition of Board 0. University faculty and staff only and Organizational Autonomy 1. University faculty, staff, and student representatives The regulatory framework allows for 2. University faculty, staff, students, and government officials sufficient organizational, autonomy. 3. University faculty, staff, students, and majority external participants (not This means that the HEA has an only government, but also members from private sector/ non-profit arena.). appropriate policy on the role and Selection of Board 0. Election by faculty and staff functions of the boards of higher 1. Appointment by external committee on basis of search committee education institutions as well as for Independence of Boards 0. No independence, the selection of the leadership of 1. Low degree of independence, higher education institutions, and the 2. high degree of independence (score 1, 2 or 3) respective responsibilities of both. Authority of Board 0. Board is purely consultative. 1. Board approves strategic plan. 2. Board approves strategic plan and annual budget. 3. Same as 3, but Board is also responsible for recruitment of University leader Responsibilities of Board and 0.Lack of clarity about the respective responsibilities of the Board and leadership leadership 1. Some clarity about the respective responsibilities of the Board and leadership 2. Full clarity about the respective responsibilities of the Board and leadership Selection of university leader 0. Selection of university leader is based on political appointment. Leadership 1. Selection of leader is based on internal election. 2. Selection of leader is based on competitive search (score 1, 2 or 3) Eligibility of external 0. Leader can be only an internal candidate. candidates 1. Leader can come from another academic institution in the country. 2. Leader can come from any academic institution in the world. Leadership term limits 0. Leader can serve only one term and cannot serve elsewhere afterwards. 1. Leader can serve 2 terms and cannot serve elsewhere afterwards. 2. No limits on terms either within or in other institutions. Selection of leadership team 0. Based on outside political appointment. 1. Based on elections. 2. At discretion of university leader. 3. By university leader based on competitive search 23 Policy Goals Policy Levers Indicators Measurement Goal #5 Institutional Autonomy Financial Autonomy Freedom/flexibility to borrow 0. No The regulatory framework allows for from commercial banks 1.Yes but with some restrictions sufficient financial, staffing and 2.Yes, with no restrictions academic autonomy in institutions. Freedom/flexibility to set 0.No level of tuition and fees for 1.Yes but with some restrictions students 2.Yes, with no restrictions Freedom/flexibility to issue 0.No bonds in financial markets 1.Yes but with some restrictions 2.Yes, with no restrictions Freedom/flexibility to retain 0.No surplus from annual budget 1.Yes but with some restrictions and self-generated funds 2.Yes, with no restrictions Freedom/flexibility to 0.No constitute and use an 1.Yes but with some restrictions endowment 2.Yes, with no restrictions Freedom/flexibility to 0.No purchase, own, and sell 1.Yes but with some restrictions buildings, facilities and 2.Yes, with no restrictions equipment Freedom/flexibility to use 0.No available resources (for 1.Yes but with some restrictions example with public funding 2.Yes, with no restrictions allocated as block grant rather than through line items or earmarked funds) Flexibility in procurement 0.No (this may range from very 1.Yes but with some restrictions restrictive rules and 2.Yes, with no restrictions procedures to rules and procedures as flexible as for private universities) 24 Policy Goals Policy Levers Indicators Measurement Goal #5 (Continued) Staffing Autonomy Civil service status of 0. Yes The regulatory framework allows for employees, especially faculty 2. No sufficient financial, staffing and Freedom/flexibility to recruit 0.No academic autonomy in institutions. faculty 1.Yes but with some restrictions 2.Yes, with no restrictions Freedom/flexibility to 0.No terminate non-performing 1.Yes but with some restrictions faculty 2.Yes, with no restrictions Freedom/flexibility to set 0.No salaries 1.Yes but with some restrictions 2.Yes, with no restrictions Academic Freedom/flexibility to define 0.No Autonomy academic structure and 1.Yes but with some restrictions programs 2.Yes, with no restrictions Freedom/flexibility to define 0.No course content 1.Yes but with some restrictions 2.Yes, with no restrictions Restrictions to academic 0.Yes freedom speech of faculty 1.Some 2.None Freedom to recruit students 0.No (enrolment size, academic 1.Yes but with some restrictions qualifications, etc.) 2.Yes, with no restrictions Assessment of students 0.No (learning outcomes), format 1.Yes but with some restrictions of exams 2.Yes, with no restrictions Academic partnerships with 0.No other institutions 1.Yes but with some restrictions 2.Yes, with no restrictions 25 Policy Goals Policy Levers Indicators Measurement Goal #6: Incentive-based Incentives for Funding allocation modalities 0. Share of recurrent budget allocated on the basis of performance. performance Performance take into account 1. Share of research budget allocated on a competitive basis. Funding mechanism is performance- performance targets on a 2. Share of investment funding allocated on a competitive basis. based, transparent, and promotes competitive basis 3. Share of recurrent budget allocated per students following a demand equity in student enrolment. model (i.e. grants, loans, vouchers). The TEA negotiates 0. No Performance agreements are in place and there is no plan to do so. performance targets with 1. Performance agreements are in place but not monitored institutions, monitors results, 2. Performance agreements are in place, are monitored, and acted upon and rewards institutions meeting/exceeding targets Equity promotion The government provides 0. The government provides needs-based scholarships or There is a student aid (grants, loans) to government-backed student loan scheme for needy students support recruitment and enrolment of minorities and 2. Government has a combination of both loan and grant funding. the poor. 26 Policy Goals Policy Levers Indicators Measurement Goal #7: Quality and Relevance External quality Existence of QA system and 0. No QA agency in place. The HEA has an independent quality assurance agency 1. Less than ¼ of institutions/programs are evaluated or accredited assurance and accreditation agency 2. Between ¼ and ¾ of programs are evaluated or accredited. for both public and private 3. More than ¾ of all programs are evaluated or accredited. institutions. Degree of independence of 0. The QA agency is part of a government agency and not independent QA agency 1. The QA agency is an autonomous public agency 2. The QA agency is autonomous, not public, and independently funded Internal quality Standards development (Yes 0. The QA agency has developed IQA standards and guidelines . assurance or No) 2. The QA agency has not developed IQA standards and guidelines. How comprehensive are IQA 0.IQA standards cover less than 2 of listed areas guidelines for institutions? 1.IQA standards cover between 3-5 listed areas Do the guidelines cover the 2.IQA standards cover over 5 listed areas following: 1. Institutional licensing 2. Institutional accreditation 3. Accreditation of programs 4. Assessing learning outcomes 5. Teaching methodologies 6. Research production 7. Facilities Labor market Participation in student 0. Very few institutions participate in surveys advisory service engagement and tracer 1. Most institutions participate in surveys but they do not publish the results surveys, publication of 2. Most institutions participate in surveys and publish the results results. Regular research reports on industry and employment trends released to public along with employment rate of recent graduates by occupation 0.No 2.Yes 27 Policy Goals Policy Levers Indicators Measurement Goal 8 # Accountability Anti-corruption Operation of an anti- 0.No such agency exists. Institutions are held to specific corruption agency competent 1. Agency exists but does not cover tertiary education. standards of transparency around for tertiary education 2. Agency exists and covers tertiary education. financial health, fraud, student Monitoring of corruption Share of users of services reporting they paid a bribe to receive attention engagement and employment of (financial, academic and from at least one of 9 different service providers in the past 12 months. graduates. information fraud). Global Percentage of students who report academic or information fraud. corruption barometer score: Financial integrity Financial audits (public 0. No audit conducted. institutions are independently 1. Audit conducted annually but results are not publicly available. audited annually and results 2. Audit conducted annually and results publicly available. of audit are publicly available) Are universities required to 0.No sanction cases of 1.Somehow embezzlement; inappropriate 2.Yes spending (failure to present justifications); misprocurement Academic integrity Are universities required to 0.No sanction : examination 1.Somehow fraud; Unethical behavior 2.Yes (sale of exams, nepotism,); Non-compliance with admission standards; Unethical management of faculty careers Transparency and Participation in accreditation 0. No publication of evaluation / accreditation results. openness and publication of results: 1. Very few institutions publish evaluation / accreditation results. 2. Most institutions publish evaluation / accreditation results. Publication of deliberations 0.No of governing boards 2.Yes Existence of fair/ transparent 0.No admissions policies 2.Yes Existence of fair/transparent 0.No examination policies 2.Yes 28 5. Institution-level Questionnaire As mentioned earlier, and as explained in more detail in Annex 2, it is especially important in the case of tertiary education to collect governance data at the institution level, apart from data at the level of whole systems as is done in other SABER domains. This is not only because of the need to assess whether system-wide policies are being implemented at the level of institutions, but also because of the high degree of heterogeneity between institutions. Following the structure outlined in the previous section for indicators of system-wide governance, this section provides a parallel questionnaire that can be implemented for assessing governance at the level of institutions. Some of the questions are meant to assess how institutions perceive system-wide governance, including whether they are actually aware of the policies being implemented at the level of the system. Other questions are meant to assess the quality of the governance of the institutions themselves, and whether they do correspond to what is mandated or suggested by the tertiary education authority and other system-level institutions. Before implementing the institutional-level questionnaire, it is necessary to implement the system-wide questionnaire because this facilitates a proper interpretation of the data collected at the level of institutions. Also, while collecting data at the level of institutions is recommended, it is not necessarily required, and whether this can be done depends on the budget available for data collection and analysis. The cost of implementing the institution-level questionnaire is higher than that of implementing the system-wide questionnaire simply because the institution-level questionnaire is longer and it must be collected from several institutions. In countries with a large number of institutions, sampling techniques can be used to come up with a representative sample of universities and community colleges, but for statistical analysis it is better to have – if possible – two dozen or more institutions sampled. Data collection for the system-wide questionnaire should be led by one or more in- country experts knowledgeable on the tertiary education system with access for the system-wide questionnaire to staff at the tertiary education authority in order to properly reference policy and process information not necessarily available in the public domain. By contrast, data collection for the institution-level questionnaire relies on interviews with university administrators. Apart from traditional interviews in person, web-based survey instrument may be used (see the discussion of the pros and cons of such techniques in Annex 6). This certainly reduces the cost of institution-level data collection, but it may require additional quality checks, and follow up calls to university administrators for clarification if needed. In table 2, three types of answers can be provided by university administrators, as indicated in the last three columns of the table. Some questions require simple yes-no answers. Other questions require answers with three or more levels. This is the case for questions on institutional autonomy, where the answer is typically one of three modalities: 1-No the decision is taken by another entity than the university, 2- Yes the decision is taken by the university but still needs approval from another entity, and 3- Yes, the decision falls entirely under the purview of the university. Finally, some questions require a response in the form of an estimate, typically in percentage terms. While many of the questions in the institutional level questionnaires were inspired by the work carried in the Middle East and North Africa region by Jaramillo et al. (2012), they have been reorganized along the eight policy goals identified earlier at the system- wide level. Also, following by Jaramillo et al. (2012), questions on participation have been added as optional, even though they are not part directly of the system-level questionnaire. 29 Box 3: Prototyping the Institutional-level Questionnaire in Jamaica The tertiary education sector in Jamaica is diverse, with 4 universities and over 40 public and private colleges. As part of the SABER assessment in Jamaica, all tertiary education institutions will be invited to fill the institutional-level questionnaire. About half the questionnaires will be filled through in-person interviews, and the other half through a web-based questionnaire, which will enable the team to test whether the quality of the information provided through a web questionnaire is similar to that obtained through in-person interviews. The selection of which institutions are assigned to each of the two groups will be based on a random allocation of the institutions according to characteristics, such as whether they are public or private, national or foreign-based, universities or community and teachers’ colleges, etc. If the responses of the two groups are similar in content and of the same level of quality in terms of valid responses and details in the responses, this would suggest that implementing the institutional-level questionnaire through the web may be a valid approach, which would save time and expense. 30 Table 2: Questionnaire for Institution-level Governance Goal#1: Setting a Clear Vision Policy lever Indicators No Measures Y/N Autonomy % Does the country have a strategic vision and development plan for tertiary education? X What are the main priorities mapped in the 1 Quality X national strategic vision? (Check all that apply) 2 Enrolment X System-wide 3 Equity X Policy 4 Internationalization X Directions 5 Research X 6 Teaching X 7 Skills for employment X 8 Accountability X 9 Other X Are the general missions of universities in the country formally stated? X 1 National Law for (Higher) Education X 2 Ministry of Education decree (or similar) X Where are they stated? 3 Report of a reflection committee or council X 4 Universities internal regulation X 1 The State, national level (Parliament, government) X 2 The State, regional level X 3 Civil society representatives X During their elaboration, which actor had a voice? 4 Industry & business representatives X 5 Universities representatives X Institutional 6 Syndicates/unions X Missions and Are the specific goals of your institution formally Goals stated? X 1 National Law for (Higher) Education X 2 MoE decree (or similar) X Where are they stated? 3 Report of a reflexion committee or council X 4 Universities internal regulation X 1 Reflecting the expectations of internal stakeholders (students, staff) X What was the major concern during their Reflecting the expectations of external stakeholders (State, donors, settlement? (Please choose only one option) 2 private sector, alumni) X 3 Being aligned with the institution missions nationally defined X 1 The State (Government, MoE, Parliament) X Which actors are in charge for monitoring their achievement? 2 Institutional leadership (Head or equivalent) X 3 Social council (e.g. civil society, industry & business representatives, X 31 associations, donors) 4 Senior Managers X 5 Governing Board (or equivalent) X 6 Other councils (e.g. Academic, Research, Student's Senate...) X 7 Private owners / Donors X 1 Quantitative surveys producing performance data X 2 Assessment reports conducted by the Governing Board X What are the elements used to measure Assessment reports conducted by the State (Parliament or achievement of institutional goals? 3 Government) X 4 Assessment reports conducted by an independent agency X 32 Goal#2: Regulatory Framework Policy lever Indicators No Measures Y/N Autonomy % 1 None X How many times has the legal status of the 2 One time X institution been modified in the last 10 years? 3 Two times X (Please choose only 1) 4 Three times X 5 More than three times X 1 None X Legal How many times has the mission legally 2 One time X framework entrusted to universities been modified in the 3 Two times X for TE last 10 years? (Please choose only one) 4 Three times X 5 More than three times X Reform of the law conducted by the National What is the event that better explains the last 1 Assembly/Parliament (or similar) X changes in the Legal Framework of the 2 Replacement of the executive (president, minister) X Institution? 3 Effects of a financial/economic crisis X 4 Alignment with international trend X Registration criteria were: 1 No entry regulations X 2 Overly complicated and/ approval process was long X 3 Clear and time to register were reasonable X Objective and measurable so as to minimize discretion and Legal 4 limit the scope for corruption. X What were the requirements to enter market framework 5 Openly published - they were easily accessed without delay X as a private provider? (For private for private Output-focused (i.e., designed to ask open questions about institutions only- select all that apply). providers the standards proposed rather than impose fixed national norms or ratios) so as to allow for flexible and diverse 6 delivery approaches X There are incentives and support (i.e. financial subsidies, tax holidays, provision of land, operational support) for 7 entry into this market X 33 Goal #3: Capacity of the Tertiary Education Authority Policy lever Indicators No Measures Y/N Autonomy % 1 HEA authority staff are professional, skilled and competent X HEA authority makes an effort to engage the sector, and Education How would you describe HEA staff? 2 communicates reforms in the sector X Authority HEA authority has successfully introduced reforms in the 3 sector (if applicable) X 1 personnel (full-time, part-time etc) X 2 academic (enrolment, retention, transition, graduation) X Information What data does your institution regularly students (composition- full-time, part-time, demographic system report to the HEA? (Check all that apply) 3 information) X 4 financial management X 5 infrastructure X 34 Goal #4: Leadership, Management and Organizational Autonomy Policy lever Indicators No Measures Y/N Autonomy % 1 Appointment by the Government X 2 Appointment by a Selection Committee X Selection process 3 Elections X 4 Competitive recruitment X 1 The State (Parliament, MoE or Regional authorities) X 2 Institutional leadership (Governing Board,...) X Who is implicated/involved in this process? 3 Senior Academic staff X 4 Private owners or donors X Head of the 1 s/he is not necessarily an academic X institution 2 s/he needs to have a Managerial profile X 3 s/he can be an external person from the Institution X What are the requirements to be selected? 4 s/he has to meet a full job specification X 5 s/he is not necessarily a member of a political party X 6 s/he has to propose a strategic vision for the institution X 1 The length is less than 4 years X What are the conditions of their mandate? 2 It's not renewable X 3 It's renewable but the number of mandates is limited X 1 Appointment by the Government X 2 Appointment by a Selection Committee X Selection process 3 Elections X 4 Competitive recruitment X 1 The State (Parliament, MoE or Regional authorities) X 2 Institutional leadership (Governing Board,...) X Who is implicated/involved in this process? 3 Semior Academic staff X 4 Private owners or donors X Deans 1 s/he is not necessarily an academic X 2 s/he needs to have a Managerial profile X 3 s/he can be an external person from the Institution X What are the requirements to be selected? 4 s/he has to meet a full job specification X 5 s/he is not necessarily a member of a political party X 6 s/he has to propose a strategic vision for the institution X 1 The length is less than 4 years X What are the conditions of their mandate? 2 It's not renewable X 3 It's renewable but the number of mandates is limited X Members of 1 Appointment by the Government X Selection process the 2 Appointment by a Selection Committee X 35 Governing 3 Elections X Board (or 4 Competitive recruitment X equivalent) 1 The State (Parliament, MoE or Regional authorities) X 2 Institutional leadership (Governing Board,...) X Who is implicated/involved in this process? 3 Semior Academic staff X 4 Private owners or donors X 1 s/he is not necessarily an academic X 2 s/he can be an external person from the Institution X What are the requirements to be selected? 3 s/he is not necessarily a member of a political party X 4 Others X 1 The length is less than 4 years X What are the conditions of their mandate? 2 It's not renewable X 3 It's renewable but the number of mandates is limited X 36 Goal #5: Institutional Autonomy Policy lever Indicators No Measures Y/N Autonomy % 1 The State (Government, MoE, Parliament) X 2 The Governing Board X 3 Instituional leadership (Head or equivalent) X Who approved the budget of the institution? 4 Instituitonal budget committee X 5 Senior Managers X 6 Private Owners X 7 Others X Funds from the Government (budget allocation, 1 grants, other) X X 2 Fees from students X 3 Funds from Private owners X Funds raised from Private companies (e.g. industries, What are the different revenue sources of 4 businesses) X the Institution? 5 Loans obtained from banks X Loans and grants obtained from International 6 organizations X 7 Funds from contracts (e.g. consulting, outsourcing) X 8 Revenues obtained from financial applications X Financial 1 Manage its assets (e.g. sell, buy, rent) X autonomy 2 Use unspent funds from one year to another X 3 Set the level of fees X Define the revenue structure of the institution (e.g. public/private sources, banks, assets interests, 4 endowments, fees) X Regarding the financial procedures, has the 5 Run a deficit X Institution autonomy to: Use a multi-year forecast (e.g. Medium Term 6 Expenditure Framework system) X Choose the budget allocation inside a block-grant 7 budget X Set the bonuses to be paid to private owners (for 8 Private institutions) X 1 Salaries/Recurrent spening X 2 Operational costs/ Recurrent spending X What is the proportion of spendings? 3 Recurrent spending/ Total spending X 4 Capital spending / Total spending X 1 Fees/ Total revenues X What is the proportion of revenues? 2 State-order scholarships/total revenues X 37 3 Stipends/Total revenues X 4 Research & Consultancy/ Total revenues X 1 Years of service in yoru institution X 2 Academic qualifications X How are salaries determined? 3 Years of rlevant experience X 4 Performance X 5 Others () X 1 Increasing the number of fee-paying students X 2 Increasing the fee levels X 3 Raising donations from the private sector X What measures are you taking to assure the 4 Conducting research and consultancy for fees X sustainability of university financing? Opening/closing programmes based on cost- 5 effectiveness X 6 Others X 1 Buildings X What type of assets is the University 2 Land X allowed to own? 3 Financial assets X 4 Equipments (Motorized, IT) and furniture X 1 Hiring new administrative staff open ended X 2 Dismissing administrative staff X 3 Hiring academic staff open ended X Does the University have autonomy for: 4 Dismissing academic staff open ended X 5 Developing training programs X 6 Promoting staff members X Assessing the performance of administrative and 1 academic staff X 2 Setting the salaries of academic staff X Does the University have autonomy for: 3 Setting the salaries of administrative staff X Staffing 4 Setting variable salaries (linked with performance) X autonomy Setting the Contracts of administrative staff and 5 academics (e.g. duration, benefits) X 1 Reporting measures X X What types of policies to provide 2 Compensations, salaries attached to performance X incentives/ enforcing of rules are used to 3 Providing justifications for expenditures X manage the staff in HR department, 4 Scorecards X financial department, academic department? Standardized sanctions in case of professional 5 misconduct X What is the composition of the overall staff 1 Administrative staff (as opposed to Academic) X in terms of: 2 Administrative Senior Staff X 38 3 Academic Senior Staff X 1 Civil servants (e.g. employed by the State) X What is the composition of the 2 Part-time contracts X administrative staff in terms of: 3 Fixed-term contracts X 1 Civil servants (e.g. employed by the State) X What is the composition of the academic 2 Part-time contracts X staff in terms of: 3 Fixed-term contracts X 1 Ph.D Holders X Ph.D holders having more than 10 years of service at What is the composition of the academic 2 the University X staff in terms of: PhD holders having more than 10 years of service in a 3 managerial position X 1 Structure of curricula (e.g. type of degrees) X 2 Introduction of new programs X 3 Types of courses (level, sector, mode) X 4 Number of hours per program per year X Has the university the autonomy to decide Assessment of students (learning outcomes), format of the: 5 exams X 6 Academic partnerships with other institutions X Academic Content of articles, speeches, other communications autonomy 7 by faculty? X The overall number of students admitted in the 1 University? (e.g. is the university able to refuse applications) X Has the University autonomy to determine: 2 The number of students per program? X The admission mechanisms (e.g. entry-exams, 3 analysis of academic grades of students)? X 39 Goal #6: Incentives based funding Policy lever Indicators No Measures Y/N Autonomy % Share of recurrent budget allocated on the basis of 1 performance measures. X 2 Share of research budget allocated on a competitive basis. X Incentives for Recurrent, Research, Investment, Equity Share of investment funding allocated on a competitive performance 3 basis. X Share of recurrent budget allocated per students following a 4 demand model (i.e. grants, loans, vouchers). X 40 Goal #7 Quality and Relevance Policy lever Indicators No Measures Y/N Autonomy % Assessing It takes place inside the University (e.g. assessing 1 External and programs, committee, participatory team...) X Internal If there is a Quality Assurance system, what It takes place outside the university under the 2 Quality is its form: responsibility of the Government X Assurance It takes place outside the university under the 3 responsibility of an independent agency X 1 Institutional licensing x 2 Institutional accreditation x 3 Accreditation of programs x If existent is/are the QA systems to address 4 Assessing Learning Outcomes x the following items 5 Teaching methodologies x 6 Research production x 7 Facilities x Internal 1 Action Plans are followed by Internal QA units X quality 2 Action Plans are followed by Deans X assurance Action Plans are followed by the head of the 3 What are the mechanisms used to follow up university X on evaluations' results? The Action Plans are regularely reviewed (every 2 4 years or more) X 5 Variable budget allocations are linked with results X 6 Others X 41 Goal #8: Accountability Policy lever Indicators No Measures Y/N Autonomy % 1 State (MoE, responsible agency) X 2 Academic staff X Are the budget documents available and/or 3 Administrative staff X reviewed by the following actors: 4 Students X 5 Medias, public audience X 1 Is it conducted by an external body ? X If the university is submitted to financial 2 Is the report available for inside actors? X audit: Financial 3 Is the report disseminated outside the university? X integrity The financial audit takes place every six month or Frequency of financial audit (Please choose 1 more X one only) 2 The financial audit takes place every year X 3 The financial audit takes place less than every year X 1 Embezzlement X Has the university applied standardized Inappropriate spending (failure to present sanctions against: 2 justifications) X 3 Misprocurement X 1 Examination fraud X Unethical behavior of faculty (sale of exams, Academic Has the university applied standardized 2 nepotism...) X integrity sanctions against: 3 Non compliance with admission standards X 4 Unethical management of faculty careers X 1 New graduates employment rate X 2 Average unemployment period after graduation X Has the university established tracking 3 Average salaries of new graduates X surveys to measure: 4 Main areas of market outlets X Average number of years required to complete a 5 degree X Transparenc 1 Institutional website X y and If existent, are these tracking surveys 2 Reports publicly disseminated X openness disseminated outside the university by : 3 Newsletters X 4 Communication brochures X 1 Institutional mission & goals X Are the following items disseminated by 2 Strategy of the university X one (or more) vector mentioned in the 3 Results of the accreditation process X question above? Institutional evaluations results conducted by internal 4 or external evaluators X 42 ADDENDUM: Degree of Participation in Decision-making Policy lever Indicators No Measures Y/N Autonomy % Does this group have a formal and active mode of representation ? X 1 Definition of the goals of the university X 2 Elaboration of the strategy X Do they have a voice when it comes to 3 Budget allocation X 4 Type and number of course X 1 Governing Board (or equivalent) X Students Do they have representatives in the 2 Academic council X following boards/councils? 3 Research council X 4 Administrative council X 1 In the Governing Board (or equivalent) X If any, are the representative members 2 In the Academic Council X elected ? 3 In the Research Council X 4 In the Administrative council X Does this group have a formal and active mode of representation ? X 1 Definition of the goals of the university X 2 Elaboration of the strategy X Do they have a voice when it comes to 3 Budget allocation X 4 Type and number of course X 1 Governing Board (or equivalent) X Academic Staff Do they have representatives in the 2 Academic council X following boards/councils? 3 Research council X 4 Administrative council X 1 In the Governing Board (or equivalent) X If any, are the representative members 2 In the Academic Council X elected ? 3 In the Research Council X 4 In the Administrative council X Does this group have a formal and active mode of representation ? X 1 Definition of the goals of the university X Administrative 2 Elaboration of the strategy X Do they have a voice when it comes to Staff 3 Budget allocation X 4 Type and number of course X Do they have representatives in the 1 Governing Board (or equivalent) X following boards/councils? 2 Academic council X 43 3 Research council X 4 Administrative council X 1 In the Governing Board (or equivalent) X If any, are the representative members 2 In the Academic Council X elected ? 3 In the Research Council X 4 In the Administrative council X Does this group have a formal and active mode of representation ? X 1 Definition of the goals of the university X 2 Elaboration of the strategy X Do they have a voice when it comes to 3 Budget allocation X 4 Type and number of course X 1 Governing Board (or equivalent) X Alumni Do they have representatives in the 2 Academic council X following boards/councils? 3 Research council X 4 Administrative council X 1 In the Governing Board (or equivalent) X If any, are the representative members 2 In the Academic Council X elected ? 3 In the Research Council X 4 In the Administrative council X Does this group have a formal and active mode of representation ? X 1 Definition of the goals of the university X 2 Elaboration of the strategy X Do they have a voice when it comes to Private sector 3 Budget allocation X representatives 4 Type and number of course X (e.g. leaders, 1 Governing Board (or equivalent) X ownders, Do they have representatives in the 2 Academic council X industry following boards/councils? 3 Research council X chambers) 4 Administrative council X 1 In the Governing Board (or equivalent) X If any, are the representative members 2 In the Academic Council X elected ? 3 In the Research Council X 4 In the Administrative council X 44 5. Rubrics and Ratings Countries that participate in a SABER domain receive an overall score or rating on their level of policy development in various components or policy goals of the particular policy domain being assessed. The ratings, which are also called “levels of development” indicate the extent to which a system has succeeded in achieving each goal. Four ratings are used for all SABER domains: “Latent” (poor performance), “Emerging” (insufficient performance), “Established” (adequate performance) and “Advanced” (outstanding performance). The advantage of the ratings is that they facilitate cross-country comparisons, and may help identify areas of priority for governments, although depending on country context, it is not necessarily the case that more attention would need to be necessarily developed to an area rated as latent, as opposed to another area rated as emerging. While these ratings are primarily meant to apply to higher education systems, they can also be used (with some caution) to rate individual institutions, to the extent that each of the eight policy goals also have meaning for individual institutions, with some exceptions such as the policy goal related to the regulatory framework. The ratings should not be used to rank countries, nor to name and shame. But they are expected to help in identifying key policy gaps or areas for improvements that governments may then want to tackle to improve their systems. Indeed, the idea behind SABER is that data collection and benchmarking can serve as an incentive to promote change and reform, including in tertiary education. Data collection and analysis should ultimately be oriented toward helping government as well as universities improve their performance and adopt measures to help them better serve the needs of their stakeholders. But where a country stands depends typically on its level of development, as well as historical factors which should not be an object of judgment. Furthermore, all SABER domains recognize that there is no single correct path that countries should follow in any one specific area. There are different ways for countries to address the policy goals and associated policy levers identified in the various domains. While the various domains compare countries using standardized data collection tools, they are not designed to present countries with a one-size-fits-all approach to policy development. Depending on country context, there may be different solutions or approaches to improve tertiary education systems6. In the case of tertiary education systems, the issue of rating governance arrangements as latent, emerging, established, or advanced, is however even less straightforward than it may be in other domains. This is because the evidence base on the relationship between tertiary education governance and outcomes that states may want to encourage is more limited than the evidence available, say, on policies related to primary and secondary teachers, or to early childhood development. For transparency, ratings should be based on clear rubrics or decision rules used for aggregating the answers to specific questions (the indicators) within each policy lever, and then for aggregating the assessment made at the level of policy levers into higher level assessments for policy goals. This is not easy when detailed empirical research on what matters in the various areas is often not available. For example, if one is to measure financial autonomy, how are the assessments for the policy levers under that policy goal to be aggregated, and for each policy lever, how is the information provided for each indicator to be aggregated at policy lever level. One possibility would be to consider all indicators under a policy lever as carrying equal weight, and all policy levers under a policy goal as also carrying the same weight, but that would almost certainly be sub-optimal given that some aspects probably matter more than others. 6 This paragraph is adapted with minor modifications from Neuman et al. (2012). 45 The position adopted at this stage for the SABER tertiary education governance tool is that although clear rubrics for establishing ratings will need to be proposed at some point, it is probably too early to do so now in any definitive way because more analysis is needed to understand what matters most for performance. Furthermore, even in terms of assessing within a metricthe contribution of different indicators, research is limited. Such analysis is being carried among others using existing institution-level data from the MENA region (see Adoho and Wodon, 212, as well as Annex 3), but this work remains at a preliminary stage. Nevertheless, in order to suggest the types of approach that could be adopted and given the operational need for summary measures of performance on each policy goal, a tentative scoring rubric based on three simple decision rules could be adopted provisionally: 1. Within a policy goal, when empirical research does not exist to provide evidence of priority of one indicator over another, all indicators in that goal could be weighted equally. If empirical evidence is available to suggest that one indicator may be more important t than another, weights would reflect this evidence accordingly. 2. Performance on each policy goal could be measured on a scale from zero to one, with cut off points for labels of latent, emerging, established and advanced set for the following intervals, respectively: 0-0.25, 0.26-0.50, 0.51-0.75, and 0.76-1.0. 3. Aggregate score of performance taking into account the various policy goals would not be computed, so that the World Bank is not perceived to be providing rankings of ‘best performing countries’ in terms of tertiary education governance (any such overall rankings would remain too arbitrary). Instead, interested parties could compare the relative strengths and weaknesses of various countries by looking at the data and scoring for the various policy goals taken individually. This simple and provisional approach is illustrated in tables 3 and 4 at the system-wide level. Table 3 provides a hypothetical scoring for Policy Goal #2 - Regulatory Framework. In this case, empirical evidence is not available, in the specific sense that it is not clear whether policy lever 1 (Legal framework for TE) is more or less important than policy lever 2 (Legal framework for private providers). Using equal weighting, and based on hypothetical survey scores (column A), a overall score for the policy goal is provided by weighting equally different policy levers within the goals, as well as different indicators within policy levers, when applicable. The hypothetical aggregate score is 0.6, which would mean that the country would be categorized as being in the “Established” phase of development on this particular dimension. It should be noted that this approach is not perfect. In the instance where a policy lever is composed of just one indicator with three possible answer options (i.e. in the case of “Existence of Tertiary Education Law” below), the weighting system does not allow for a classification in the third category “Established”. Instead, a country is assigned a classification as either Latent, Emerging, or Advanced. Given that in future, the proposed Governance survey will likely be absorbed into a broader tertiary education instrument, each policy lever will have several indicators and this discrepancy will have less impact to the respective classification assigned. A second example is provided in table 4, but in this case some information is available on some of the indicators as to whether they matter more than other indicators or not. The information is from Adoho and Wodon (2012), who conducted a multiple correspondence 46 analysis on data from a sample of 40 universities in the Middle East and North Africa. This analysis suggested that some indicators are more important than others to measure financial autonomy. Therefore the policy levers were weighted according to the empirical weights suggested by the MCA – while this does not measure impact, it does take into account information in the structure of data as to what might be a better measure of financial autonomy. 47 Table 3: Scoring Rubric for a Policy Goal with Limited Empirical Evidence Numeric Total Highest score for score Label possible Label for Weight Contribution Survey policy for of Policy Levers Indicators Measurement score on policy of policy from policy value (A) lever (A policy policy policy lever lever (C lever (D) divided goal goal lever (B) by B) (E) Legal Existence of 0. No law exists framework for tertiary 1. Law revised over TE education law 10 yrs ago 1 2 0.5 emerging 0.5 0.25 2. Law revised recently Regulation for 0.Heavy restriction the entry of 1.No regulation private 2.Reasonable 2 2 1.0 advanced 0.25 0.25 Legal providers regulation 0.6 framework for Established private Regulation for 0. No regulation for providers the operations the operations of of private private providers. providers 1. Unclear 1 2 0.5 emerging 0.25 0.1 regulation 2. Clear regulatory framework 48 Table 4: Scoring Rubric for a Policy Goal with Some Empirical Evidence Numeric Highest Total score for possible Weight Contribution score Label of Survey value policy Label for Policy Levers Measurement score on of policy from policy for policy (A) lever (A policy lever policy lever lever (C lever (D) policy goal divided by (B) goal (E) B) Freedom/flexibility to borrow from 2 2 1 advanced 0.03 0.03 commercial banks Freedom/flexibility to set level of tuition and fees for students 1 2 0.5 emerging 0.15 0.075 Freedom/flexibility to issue bonds in financial markets 1 2 0.5 emerging 0.1 0.05 Freedom/flexibility to 0.No retain surplus from 1.Yes with annual budget and self- restrictions 2.Yes with no 0.835 generated funds 2 2 1.0 advanced 0.1 0.1 restrictions Advanced Freedom/flexibility to constitute and use an endowment 1 2 0.5 emerging 0.08 0.04 Freedom/flexibility to purchase, own, and sell buildings, facilities and equipment 2 2 1.0 advanced 0.18 0.18 Freedom/flexibility to use available resources 2 2 1.0 advanced 0.36 0.36 Flexibility in procurement 2 2 1.0 advanced 0 0 49 6. Conclusion The governance of tertiary education, a key dimension of system health, is considered to have a strong impact on the performance of tertiary education institutions. Where a sound governance system is in place, decisions at the national and institutional levels are aligned, resulting in optimal resource allocation, use and outputs. This paper has set out a framework for beginning to analyze and understand governance trends and good practices in a more systematic way. Through the formulation of indicators that measure the key components of governance, this benchmarking framework can support policy-makers in assessing the strength of tertiary education governance in their own countries and compare it against reference countries in other parts of the world. The benchmarking of governance dimensions can also be used to assess the relative effects and merits of various types of institutions and governance modalities, for example when comparing states or provinces within a large federal system, or public and private universities within the same country. In Germany, for instance, the impact of the Excellence Initiative is likely to play out differently in each Land (State), depending on the scope and degree of governance reform under implementation in each particular State. In Chile, the absence of a level-playing field in terms of government regulations imposed on the public and private universities that receive public subsidies explains to a large extent the fact that the [private] Catholic University of Chile seems to be outperforming the [public] University of Chile (Bernasconi, 2011). As noted by Fiszbein et al (2011) there is a distinction between governance performance and governance policies, where the former is measured quantitatively, while the latter is measured qualitatively. This limits the possibility of measuring governance structures, policies and processes and assessing with certainty what aspects affect overall system performance and in which way their influence can be felt. It presents a challenge for analyzing the strength of the governance system, unlike other elements of benchmarking of tertiary education systems. For example, when looking at sustainable expansion or resource mobilization strategies, all indicators are of the quantitative type, making it easy to observe progress and carry out comparative analyses in an objective manner. Further work is therefore needed to test the validity of the proposed governance indicators and assess the impact of various governance structures and modalities on the performance of tertiary education systems and institutions. Three final considerations are important with respect to the desirable alignment among the main accountability dimensions of the governance benchmarking. First, one should look carefully at the gap between theory and practice. It is indeed not sufficient to ascertain what governance structures and processes exist on paper in a country. The analysis of tertiary education governance must always distinguish between what is expected to happen according to the prevailing laws and regulations and what actually occurs in the system as a whole and in individual institutions, which is also one of the reasons why collecting data at the level of institutions matters. Enforcement of and compliance with existing rules are also a critical dimension of accountability. Second, the performance of tertiary education institutions may be negatively affected by divergences between the regulations and the results components of the accountability framework. Governance reforms in various parts of the world have tried to address this lack of alignment by encouraging greater institutional autonomy and putting more emphasis on results than on compliance with regulations per se. 50 Thirdly, it is worth noting that the comparative approach suggested in this paper tracks only the average performance of institutions in the system (as well as the performance of specific institutions) but does not measure in a global manner how different one system is from another. That is, the analysis may tell which countries are doing better than others on particular indicators, but it does not give a single measure of how much better it is doing – because such a measure is unlikely to be available. As emphasized at the beginning of this paper, this is an initial exploration of benchmarking issues and the set of indicators proposed need to be tested and expanded further. An important part of this process is to map proposed indicators to the evidence base. One of the challenges of this type of mapping exercise is that there is currently no comprehensive database on most governance aspects discussed in this paper. As a result, much of the data to be gathered in this study requires specific country surveys relying on expert judgments. Research linking university governance to performance and datasets accompanying this research would also be valuable for the next phase of this work. 51 Annex 1: Models of Tertiary Education Governance7 Governance Models for Systems Different models of tertiary education governance exist today in different countries. Neave and Van Vught (1994) describe a continuum in tertiary education governance models, at one end of which is the “state-control model” where the government seeks to control its universities closely, and at the other end is the “state-supervising model” where it monitors and regulates them at a distance. As shown by Fielden (2008), countries are moving from the control model to the supervisory model in virtually all aspects of their relationship with their universities given the growth in demand for tertiary education and the impracticality of a central body effectively managing day-to-day operations of university institutions. As systems become more complicated and modern management principles are adopted in institutions, the need for autonomy grows. Table 1 provides a summary of how various dimensions of governance are likely to be observed in the two ideal-types of system-level governance: the government-driven and the autonomous and government-steered models. Table A3.1: Implications for Institutions of Models of Tertiary education Governance System Level Context, Leadership & Autonomy Accountability Participation Mission and Management Goals Government- Government- Government- Centrally Central audits Mainly on driven or defined appointed managed budget consultation basis controlled missions and president Central QA policies Central control of new programs Nationally-driven and curriculum curriculum Central HR Low management accountability-no links between performance and rewards Autonomous Self-defined Governing Competitive External audits High participation and mission boards led funds allocation of stakeholders Government- throughout the steered or Strategic plans Autonomy to Independent decision-making supervised prepared by introduce new external QA process Institutions programs and set curriculum Performance-based salaries HR autonomy Source: Jaramillo et al. (2012), based in part on Fielden (2008). A review of recent reforms in East Asian countries confirms the trend toward the state- supervised model of governance where governments award tertiary education institutions increased autonomy to enable them to manage their operations independently (Reza, 2010). This push for decentralization of higher management started in the mid 1990s, beginning with the 7 This annex is based on Jaramillo (2012). 52 Republic of Korea and followed by Indonesia and Thailand. A second wave of reforms took place in the last decade with Japan and Singapore extending autonomy to their public tertiary education institutions. This pattern of decentralization of control from governments to institutions has not only been witnessed in many emerging economies but it has been taking place in developed countries. In 2008, the OECD prepared a three-volume synthesis of the main findings and lessons learned on the basis of twenty-four reviews of national tertiary education systems conducted in the previous five years (Santiago et al., 2008). The report showed that countries differed widely in their economic and social characteristics, as well as their approaches to tertiary education. But the report does recommend a move towards state supervision as opposed to control in a majority of OECD countries. This is confirmed by a recent report prepared by the European Association of Universities that looks at governance patterns in 27 European countries (Eastman et al, 2011). Under the state supervision model, the main responsibility of the state is to establish regulations and incentives to foster autonomy and accountability within higher institutions. The state does so by elaborating a vision for the future of tertiary education, providing overall guidance and policy-making, establishing a tertiary education law that that sets a level-playing field for all higher institutions operating in the system, creating a favorable regulatory framework for private tertiary education institutions, developing and maintaining a quality assurance system, encouraging tertiary education institutions to develop a strategic plan by which they set their own policies and direction, and allocating public resources to higher institutions and students in the system on the basis of transparent performance criteria. Through these mechanisms, the state engenders a context for and culture of autonomy among tertiary education institutions. Well managed autonomous institutions are typically characterized by the presence of an independent university board with external representation, the appointment of the leadership team according to professional criteria, the ability to set human resource policy around recruitment and retention of staff as well including salary levels, academic autonomy and financial autonomy. While tertiary education institutions are given autonomy by the State, or more precisely by a tertiary education state agency with a strategic purpose, in return they must be accountable for their use of public resources, the alignment of their operations with public policy goals, and their overall performance. In the words of John Millett, former Senior Vice President of the Academy for Educational Development, “accountability is the responsibility to demonstrate that specific and carefully defined outcomes result from tertiary education and that these outcomes are worth what they cost” (MOHE 2008). For universities and their leaders, accountability represents the ethical and managerial obligation to report on their activities and results, explain their performance, and assume responsibility for unmet expectations. At the very minimum, tertiary education institutions should be legally required to fulfill the following two basic dimensions of accountability: (i) integrity in the delivery of education services, and (ii) effective use of financial resources. In addition, many stakeholders have a legitimate claim to expect a cost-effective use of available resources and the best possible quality and relevance of the programs and courses offered by these higher institutions (Salmi, 2007). Higher institutions should also maintain accountability through internal quality assurance mechanisms, regular reporting on academic results and relevance of programs, financial audits, and appropriate instruments to prevent and punish corruption, all aspects to which the data collection tools presented in this paper pay attention. 53 Governance Models for Institutions It was mentioned in the previous section of this annex that there has been a gradual shift from government-driven to government-steered tertiary education governance in the last two decades or so. But what exactly are we trying to achieve with sound governance practices? Before presenting the types of indicators that can be used to measure and assess governance at the level of higher education systems in section 3, and at the level of institutions in section 4, it is useful to first review briefly what the goal– well managed and well governed higher education institutions and systems, would look like. This is done in this section following work by Jaramillo et al. (2012). We first provide a bit more details on the various types of governance frameworks that have been discussed in the literature, what is expected from proper governance mechanisms. While the discussion is presented mostly at the level of institutions, which is where the rubber hits the road, it also matters for system-wide governance. On the basis of a review of the literature, including the Guidelines and Good Practice Codes revised by the OECD, the European University Association, the Guide for Members of Tertiary education Governing Bodies in the U.K. (CUC, 2006), the Australian Universities, and the West Coast Guidelines, four main dimensions of governance appear to be especially important: (1) the vision, that is the overall context, mission, and goals of a higher education system or of an institution; (2) the leadership and management orientation of institutions and the regulations which are adopted at the level of a system to guide the selection of individuals in a position of leadership; (3) the autonomy of institutions and the regulations that govern various forms of autonomy; and finally (4) the accountability of institutions and their participatory mechanisms, as well as the regulations which govern accountability at the level of the higher education system. In table A3.2, selected features of these four dimensions are illustrated with respect to the governance models that they most relate to, following a typology proposed by Trakman (2008) in a study of the governance models of universities in Australia, the United Kingdom, and the United States. In that study, Trakman identified four distinct models of university governance: academic, corporate, trustee, and representational. The academic-driven governance models are the most traditional and are based on the assumption that universities should be governed by academic staff. There are several ways of adopting such an approach; e.g., by granting decision powers to the academic council or senate, by having important representation of academic staff on governing boards, or by appointing a prominent academic as president or head of the institution. Under academic governance, academic staffs have the most representation and the greatest voice in defining the mission and management of the university. A classic example of this model is Oxford University, where academic staffs have openly rejected any kind of corporate governance. Corporate governance, on the other hand, has emerged as a response to financial crises and the need for universities to manage financial resources more responsibly. This model is prevalent in Australia, the U.S., and the U.K., and it emerged as a response to the need to improve public university management. corporate governance operates under the assumption that applying corporate approaches, such as financial accountability, helps to improve outcomes. This model usually implies that the head of the university is a professional corporate manager, and not an academic. Trustee governance, in contrast with representational governance, gives management powers to a “trustee,” usually in the form of a board of trustees. Its members are not elected from within the institution, nor do they represent the different stakeholders. The board of trustees usually has fiduciary responsibilities and due diligence in protecting the trust, including 54 disclosing any factors that might constitute a conflict of interest with that trust. Representational Governance occurs when governance is vested in a wide array of stakeholders, including students, academic staff, alumni, corporate partners, government, and civil society. In the U.S., private tertiary education institutions have been well developed for over a century, in many cases with funding from philanthropic donors, following corporate practices such as governance by boards of trustees, and significant organizational and legal autonomy. A chief executive officer and a chief operating officer usually serve on the board as the senior management team. This model is widely used in public and private universities and is now common outside the U.S., notably in Australia. It is clear that the adoption of a governance approach is an important decision in a given time and context for an institution or university system. It is also evident that the need to modify and adjust it to changing times is an important element of success. It is also worth mentioning that some institutions have adopted an amalgam model (Birnbaum, 1991) which aims to provide a combination of academic, corporate, trustee, and representational governance. The advantage of the amalgam model is that it incorporates when it works well the strengths of each model to better suit the needs of an institution at a given point in time or to meet specific objectives. Table A3.2: University governance models and dimensions of institutional autonomy Vision Leadership & Autonomy Accountability Management High autonomy in High accountability Mission all three areas: in financial and Corporate defined by Results-based academic, financial HR/staffing, but management and HR/staffing limited participation High internal Mission defined academic High academic Academic mostly by academic Variable accountability, and autonomy staff high participation of academic staff High external Mission defined by accountability and Representational consensus among all Variable Limited autonomy high participation of stakeholders stakeholders High internal Mission defined in accountability, and Trustee consultation with Results-based Variable variable levels of trustees participation Source: Jaramillo et al. (2012), based in part on Trakman (2008). To a large extent, the four models just described all belong at least in spirit to the class of governance models that are autonomous and government-steered or government –supervised, as opposed to government-centered to use the classification presented earlier in table A3.1. Yet the models are clearly not the same, and in terms of the five dimensions of governance suggested by Jaramillo et al. (2012), there are marked differences between the models, as illustrated in table A3.2. The point of view taken in this paper is that the corporate, academic, representational, and trustee models all have specific strengths, so that none of the models is necessary better than the others – what works well depends on the nature of the institution considered, as well as the context in which it operates. But it is important to keep these various models when looking in more details at the indicators proposed to assess governance according to various dimensions being proposed. 55 Annex 2: System-wide and Institution-level Indicators and Data SABER is designed to assess existing policies of participating countries in order to enable comparisons between them and learning of best practices8. The global scope of the SABER initiative necessitates an approach that allows for a large amount of data to be collected from countries in a relatively short period of time. While there is scope to collect information on the execution and implementation of policies, most SABER domains do not verify outcomes at the household or institution level. That is, while these domains recognize the valuable role of such data and, where possible, secondary data from household or institution-level level surveys are incorporated into the analysis, the emphasis is placed almost exclusively on system-wide frameworks, indicators, and in the case of country reports analysis and recommendations. The approach here is a bit different, in that this paper proposes to collect and analyze data for both the tertiary education system as a whole, and the institutions that operate within this system, such as individual universities. The fundamental reason to do this is that collecting data on institutions as well as on systems makes it feasible to assess whether governance regulations and policies as envisioned in the law and other regulatory instruments at the system level are indeed being implemented in the field at the level of institutions, where the rubber hits the ground. The value of looking at the system level as well as at the level of institutions is that one can then better identify whether specific policies are implemented or not, and whether they are working or not. This in turn may help to understand potential bottlenecks for improving governance, while still maintaining a focus on the role that national policies and regulations play in guiding institution-level practices – which is the objective of SABER, given SABER’s focus on state-level policies. This point can of course be made about any of the SABER domains since there may well be substantial differences between policies and their implementation in other domains as well, ranging from teacher policies to financing. But the issue of the correspondence between national policies and their implementation on the ground seems especially important in the case of tertiary education for two reasons. Firstly, as emphasized by Jaramillo et al. (2012), tertiary education institutions are especially complex organizations, called to perform a variety of tasks that vary from managing assets and personnel - like other organizations- to managing academic and research production, student affairs, community interactions, and government functions. Unlike in basic and secondary education, identifying indicators to assess comparable tertiary education outcomes is quite complex, and there have been very few attempts to measure tertiary education institutions’ performance that can be used to make valid international comparisons. Because tertiary education is more complex and contextualized than primary and secondary education, and because there is such a wide range of institutions, it is not sufficient to adopt universal criteria to assess governance and performance. Tertiary education institutions have diversified goals; therefore, their governance and performance cannot easily be measured with a standardized or "one size fits all" approach, and the usefulness of indicators at the system-wide level is likely to be more limited. Looking at university-level governance and identifying different patterns and "fitness for purpose" is a critical step toward understanding how tertiary education institutions could improve their performance. This of course can to some extent be said of lower levels of education as well, but it seems especially the case for tertiary education. 8 This paragraph is adapted with minor modifications from Neuman et al. (2012). 56 Secondly, the fact that private service provision is large at the higher level in many countries adds to the degree of heterogeneity present in the institutions and makes it even more necessary to go beyond the role of the state in steering the tertiary education level in order to factor in as well dimensions of governance that are decided and implemented at the level of institutions. Tsimpo and Wodon (2012b) provide data on the market share of private providers in tertiary education in sub-Saharan Africa on the basis of data from 14 nationally representative household surveys, and show that this market share is higher than at the primary and secondary level. The role of private providers is also important in the data collected for the MENA region by Jaramillo et al. (2012). The state of course has an important role in regulating the entry into the system by private providers, as well as their operations, and this can be reflected in system-wide assessments of governance for the tertiary education sector. But because private providers are especially active at the level of tertiary education, it is important to assess whether there are differences in governance mechanisms between public and private universities, and when feasible to also look at differences between for-profit and non-profit private universities. Furthermore, if most universities were public, knowing the rules that govern public universities might be enough, since pretty much the same rules would apply to most universities. But this is not enough when many universities are private. Said differently, while it is likely that in many countries many of the rules of the games in tertiary education will apply in the same way to public and private universities, it is likely that public universities will in addition be subject to additional rules that apply only to them, while private universities may have more autonomy in that those additional areas. While these distinctions can be looked at the system level – different rules may need to be designed for different types of providers and this can be recognized in system-wide assessments, such assessments clearly would benefit from data collection at the level of institutions. That is, collecting data only at the level of systems may not be enough to understand the various governance mechanisms at work in different types of universities. For the above two reasons, it seems especially important to collect governance data at the level of institutions in tertiary education, apart from system-wide data on policies. The good news is that in the specific case of tertiary education, this is also feasible at low cost, at least in most countries. This is because there are typically much fewer tertiary education institutions at the level of a country, or at the level of a state within a country. In the study on governance at the institution level carried by Jaramillo et al. (2012) for four countries in the Middle East and North Africa, data collection costs were minimal, and the information collected on about 10 institutions per country proved very valuable. As mentioned earlier, data at the institution level allow the assessment of practices in addition to policies, which are the focus of system-wide assessments under the standard SABER approach. They also enable the assessment of gaps between policies and practices, which are important to identifying reform needs. Other SABER domains do recognize the distinction between policies “on paper” and the realities of access, coverage and service delivery in a country. Many countries in the world have well-defined policies, but still have poor outcomes due to resource constraints, flawed service delivery and/or a lack of quality assurance mechanisms. But in part because of the cost of collecting data for a sufficiently large set of institutions (or households), it is often not practical to implement the SABER tools for the other domains to the level of institutions. Finally, there is one more reason to collect data at the level of institutions in the case of governance in tertiary education which relates to the need to better understand the various dimensions of governance themselves. That is, because the availability of in-depth empirical 57 research on how to measure governance in tertiary education (not to speak of the impact of governance on performance indicators, which is a much more complicated endeavor) remains fairly limited, collecting data at the level of institutions may help for better measurement not only at the institution level, but also at the level of systems. A partial example of how this could be done is provided in annex 3, but essentially the idea is that if one wants for example to measure university autonomy at the system level, it is useful to analyze first what may be the main components of autonomy at the level of institutions, given that this should inform how one would measure policies related to system-wide autonomy. Said differently, collecting data at the level of institutions may help in assessing system-wide parameters, especially when the available evidence on what matters and what does not remains rather limited. Now, the fact that this paper proposes to complement system-wide policy data with data at the level of institutions does not mean that this always needs to be done. In some countries, it may well be sufficient to collect system-wide data only. Also, the suggestion to collect institution-level data in this paper does not imply that the importance of collecting systems-wide policy data is in any way diminished. As explained in the SABER-Teacher module by Vegas et al. (2012), analyzing system-wide policy frameworks matters for at least three reasons9: x First, policy frameworks provide a sense of what is possible in a system. Policies clarify the expectations of a system (the goals it aims to) as well as its theory of action (the specific actions and associated conditions that are deemed necessary to achieve a determined set of goals). Any activity that takes place within the system does so within the boundaries set by the policy framework, which may promote certain types of activities and prevent others. Thus, understanding the limits that policies may pose to the pool of possible options to improve educational practices is a first step towards systemic improvement. x Second, the analysis of policy frameworks allows for a better understanding of where to focus improvement efforts. The analysis of the internal consistency of a specific policy framework allows assessing the likelihood that it will achieve the expected outcomes, and help direct policy interventions where they are most necessary. For example, an education system may decide that the most effective way to improve learning and teaching is to have a set of performance-based incentives for universities. However, for such a policy to function properly, other policies may be needed, such as having mechanisms to assess performance and student learning, and perhaps different salary scales for university professors based on performance in order to make performance- based incentives palatable. Assessing the internal consistency of the theory of action of a system may thus help direct policy efforts towards those areas where the system needs greater consistency in order to improve the likelihood of achieving its goals. x Third, the analysis of policy frameworks may support a more thorough understanding of implementation gaps. In order to understand why a certain policy is not producing the expected results, it is important to be able to assess whether this is due to a fault in the implementation process, to a mismatch between a policy and its context, or to a lack of internal consistency within the policy. While the information to be collected through this diagnostic tool does not allow carefully assessing these options, it should provide a starting point. 9 The three bullet points that follow are adapted with minor modifications from Vegas et al. (2012). 58 Annex 3: Existing Databases of Governance Indicators for Benchmarking Purposes Examples of systematic use of indicators to assess the governance characteristics of tertiary education are few and far between. To date, there is no comprehensive database on the state of governance of tertiary education systems and institutions. However, a small number of organizations have conducted surveys on the topic including the OECD, the EUA, the EURYDICE Network and the World Bank (see for example the survey implemented in the Middle East and North Africa region presented in annex 4). In 2003, the OECD published a study titled Changing Patterns of Governance in Tertiary education, based on surveys of university governance in 15 member institutions of the OECD’s Institutional Management in Tertiary education Program (IMHE). Table A1 illustrates some key dimensions of autonomy enjoyed by universities in 14 countries. Table A1 - Extent of Autonomy Experienced by Universities in 14 OECD countries Source: OECD (2003) The table shows that, overall, institutions have autonomy over expenditure of budgets and management of academic staff. This is in part because governments have been withdrawing from their traditional role of direct management of universities, allowing for greater institutional autonomy. At the same time, however, governments are introducing new mechanisms (i.e. through funding and quality assurance) to ensure that institutions remain accountable for their performance. 59 In 2009, the European University Association, an organization that represents and supports more than 800 universities in 47 countries, published the results of an exploratory study on the enabling conditions for university autonomy (Estermann and Nokkala, 2009). The purpose of the study was to provide the foundation for a Europe-wide database of comparable information on different aspects of university governance and autonomy. It also aimed to bring the institutional perspective (i.e. what autonomy really means in practice) into the European debate in order to influence reforms at the policy level. The EUA identified the same four key dimensions driving institutional autonomy as the ones used in the benchmarking framework, including (i) organizational autonomy, (ii) financial autonomy, (iii) staffing autonomy, and (iv) academic autonomy. Within each category, the EUA identified features determining each dimension. So for example, in the case of the first dimension of organizational autonomy, the key features are internal academic and administrative structures, the composition of the institution’s governing bodies, and the ability of the institution to select its executive leadership. The study, which covered 34 universities, relied on an online questionnaire addressed to the National Rector’s Conferences (NRC) in countries with universities belonging to EUA. The questionnaire focused on the legal status of institutions, institutional strategies, management and governing structures, financial issues, students, human resources, intermediary bodies and overall autonomy. The study revealed that the framework and conditions under which European universities operate vary greatly between and sometimes within countries, with some noteworthy trends. First, in terms of organizational autonomy, universities in almost all the systems under review had regulations in place allowing for institutional autonomy with regulations requiring accountability to the State. In the majority of countries, universities are relatively free to decide on administrative structures. In addition, there is a trend toward the inclusion of external members in the university decision-making processes, especially where universities have dual governance structures (academic and corporate) (Estermann and Nokkala, 2009). In terms of staffing autonomy, the analysis reveals that, in some countries, universities are gaining a greater flexibility in their staffing autonomy, in particular as staff is generally directly paid and/or employed by the university rather than by the government, as the following examples illustrate: x Czech Republic- Individual tertiary education institutions determine the number of academic staff in each rank. Institutions also now have the authority to set the length of contract for staff (i.e. fixed term versus indefinite) x Denmark – The government’s policy on Strategy in the Global Economy calls for universities to have increased freedom in attracting highly talented researchers through monetary incentives x France- Introduced a new law in 2007 that gives institutions autonomy in human resource management x Hungary- Introduced a law in 2005 titled the Act on Tertiary education which gave increased scope for autonomy in staffing. As a result, institutions now “decide on matters of employment, freely select staff, and designate their duties based on institutional regulation and expectations regarding performance and quality of work” (Euridyce, 2008, 22) x Finland, Romania, Austria have established performance criteria and/or introduced performance-based pay systems in their tertiary education institutions. 60 However, many universities are still restricted in their ability to set salaries for individual staff members. This power continues to lie in the government’s domain. Furthermore, in almost half of the countries surveyed, all or the majority of staff continue to have civil servant status. In terms of financial autonomy, in the majority of countries institutions receive funding via block grants. Most universities across Europe have the autonomy to borrow funds, but there are restrictions on the amount allowed. In the majority of countries analyzed, institutions can collect tuition fees and administrative fees from students. The study noted the general finding that Western European countries had more autonomy in the use of funding that they receive, but less autonomy in setting tuition and administrative fees in comparison to their Eastern European counterparts. Finally in terms of academic autonomy, the main issues are institution’s ability to determine their own academic profile. In general, most governments across Europe require that institutions obtain some type of approval before establishing new degree programs. In a third of the countries surveyed, universities are able to regulate the number of students per discipline. Similarly in 2009, the Eurydice Network published a report titled “Governance in Tertiary education” which provides information on and analyses of 33 European tertiary education systems. The paper provides a useful collection of data on the governance framework of the countries surveyed. In particular, the document provides in depth description and comparative analysis on the following: x national strategic policies for tertiary education with a focus on funding and staffing of institutions; x scope of external regulation (i.e. at national /international level) and discussion on the state of institutional autonomy as it relates to academics and human resource issues; x analysis of public funding mechanisms and their influence on performance, competition and accountability of institutions to society; and x resource mobilization in the context of private financing. Overall, the study concludes that there is a trend toward less prescription regulatory frameworks at the national level, leading to the strengthening of academic self-governance while maintaining accountability to the state. Countries have developed a range of models suiting the demands and traditions of their nations. In 2011, the EUA published a follow up to this report, titled University Autonomy in Europe II- The Scorecard which elaborates on the original report by presenting a country-wise ranking on each of the four dimensions of autonomy. Countries receive a score out of 100 based on the relative “freedom from restriction” enjoyed by institutions in specific areas of organizational, financial, staffing and academic autonomy and are classified into four groups (high, medium high, medium low, and low) . The purpose of this scoring card is to focus attention on the issue of university autonomy and to inspire policy reform on specific regulations identified as being “restrictive” to university autonomy (Estermann et al., 2011). 61 Annex 4: World Bank Institution-level Survey for the MENA Region An institution level data collection tool was recently developed by Jaramillo et al. (2012) for a study on the benchmarking of universities in the Middle East and North Africa (MENA) region. Many aspects of this questionnaire were incorporates in section four of this paper, but it is useful in this annex to outline some of the key features of the original questionnaire and its structure. The MENA questionnaire was organized along four dimensions, namely vision, leadership and management, autonomy, and accountability. Each dimension is briefly discussed in turn below. The vision (context, mission, and goals) dimension for the institution-level tool developed by Jaramillo et al. (2012) takes into account three sets of indicators: (i) the definition of the university mission, the process followed, and the stakeholders involved; (ii) the definition of the goals, their implementation, and the monitoring mechanisms used to evaluate them; and (iii) the legal framework and national context in which the university operates. Taking these various aspects into account, the structure of the questionnaire for the institution-level screening card dimension related to vision (context, mission, and goals) is outlined in Figure 2. Figure 2: Vision: Indicators and sub-Indicators Source: Jaramillo et al. (2012). Questions about the leadership and management dimension at the level of institutions include the clarity of the bodies’ mandates; the alignment of their mandate with the university’s mission, goals, and legal framework; the accountability measures for governing bodies; and mechanisms for measuring governing bodies’ performance. The screening card for the leadership and management orientation dimension is designed to identify the extent to which a university followed results-based management aligned with the NPM practices. These indicators include: the type of university leader (i.e., president, rector, or chief executive officer) and the process for 62 appointing that person; the roles and responsibilities assigned, including the legal functions; his/her lines of accountability; and the mechanisms for evaluating his/her performance. Indicators also include the management structures, departments, or units and their roles, responsibilities, lines of accountability, and mechanisms for performance evaluation. To get a good picture of the management structure of a university, the most critical departments and functions, such as human resources, budgeting and accounting, academics, procurement, and legal services, are also taken into account. Taking these various aspects into account, the structure of the questionnaire for the institution-level screening card dimension related to management orientation is outlined in Figure 3. Figure 3: Leadership and Management Orientation: Indicators and sub-Indicators Source: Jaramillo et al. (2012). Financial autonomy at the level of institutions is defined by indicators such as the ability of universities to: set tuition fees, accumulate reserves, and keep surplus on state funding; borrow money; invest money in financial or physical assets; own and sell the land and buildings they occupy; deliver contractual services; and attract funds on a competitive basis. Academic autonomy takes into account: the responsibility for curriculum design; the extent to which universities are autonomous to introduce or cancel degree programs and to determine academic structure; the overall number of students; admissions criteria; admissions per discipline; evaluation of programs; evaluation of learning outcomes; and teaching methodologies. Human Resources autonomy relates to: the recruitment procedures for the appointment of senior academic staff; the status of employees (whether they are considered civil servants); and the 63 procedure for determining salary levels, salary incentives, and workloads; human resources policies; career development policies; and performance management. Taking these various aspects into account, the structure of the questionnaire for the institution-level screening card dimension related to autonomy is outlined in Figure 4. Figure 4: Autonomy: Indicators and sub-indicators Source: Jaramillo et al. (2012). The accountability dimension indicators at the level of institutions look at academic accountability, accountability to civil society or social responsibility, and financial accountability. The indicators addressed are: clarity in the definition of accountability lines at all levels (academic staff, managerial staff, administrative staff, and governing bodies); the process for evaluating the completion of institutional goals; dissemination of information on institutional goals, student achievements, graduate insertion in the labor market, institutional evaluations (internal and external), and accreditation; methods used for evaluating the performance of students, teaching staff, administrative staff, and managerial staff; the process for auditing university accounts; and the processes for risk management and dealing with misconduct. Taking these various aspects into account, the structure of the questionnaire for the institution- level screening card dimension related to accountability is outlined in Figure 5. 64 Figure 5: Accountability: Indicators and sub-indicators Source: Jaramillo et al. (2012). One last component of the screening card implemented in the MENA region by Jaramillo et al. (2012) relates to participation. In the MENA study, participation is presented as a fifth dimension, but it could as well be considered as one aspect of accountability, which is the approach used here. Clearly, voice and accountability are key elements for good governance, as access to information on the government’s decisions and participation in policy development are important elements of democratic and open societies. Freedom of expression and availability of free media are also important in monitoring public policy making and implementation. Improving transparency and accountability are often obstacles that need to be overcome to succeed in improving governance, including in the area of education governance. It is thus important for each stakeholder to assess the representation mode and voice in the decision- making process that the stakeholder has (or lacks). Given this, the structure of the questionnaire for the institution-level screening card dimension related to participation is outlined in Figure 6. It should be emphasized however that not “any” form of voice or participation is necessarily positive for universities. For example, a professional selection process for the leadership is likely to be more appropriate than elections, even if these may seem to be more democratic. 65 Figure 6: Participation Indicators and sub-Indicators Source: Jaramillo et al. (2012). 66 Annex 5: From Indicators to Indices: An Illustration with Institution-level Data One of the very few data sets that can be used in order to look at various dimensions of tertiary education governance at the level of institutions in order to come up with overall indices of governance is the data set collected by Jaramillo et al. (2012) for universities in the MENA region using the institution-level tool described in section 4. The database includes information on universities in four countries or territories: Egypt, Morocco, Palestine, and Tunisia. The selection of the universities that participated in the study was purposeful in order to have representation from different types of institutions in each country, including public and private institutions, older and relatively newer ones, as well as smaller and larger universities. While the universities may not be fully representative of the four countries, they account for a large share of total tertiary level enrollment in each country: 78 percent in Palestine, 60 percent in Morocco, 46 percent in Tunisia, and 36 percent in Egypt. Using these data, Adoho and Wodon (2012) show how simple statistical methods can be used to come up with aggregate indices of governance. Instead of weighting various parts of the data equally or arbitrarily, they use multiple correspondence analysis (MCA) to assess whether specific questions in the questionnaire should be given more weight when coming up with aggregate statistical measures of, among others, financial, academic, and human resources or staffing autonomy. MCA is a technique used for analyzing relationships between variables taking categorical values (Greenacre, 1984). This type of analysis is often used to uncover the main underlying factors that capture the variance in a data set, typically with visualization along two or three dimensions. Here, the technique is used simply to obtain aggregate indices of various types of autonomy (as well as other dimensions of governance), with the indices constructed in such a way that they explain a large share of the variance in the underlying variables. Essentially, the first factor obtained from the MCA as applied to the set of variables in the questionnaire under, say, financial autonomy is interpreted as a measure of the overall financial autonomy of institutions, and the same holds for the application of the MCA to the other dimensions. Details are discussed in Adoho and Wodon (2012), but a few results can help in suggesting how this works. For example, it can be shown than for the questionnaire used by Jaramillo et al. (2012) and the specific data set collected for the four MENA countries, the questions on whether a university has autonomy to define its revenue structure, set the level of student fees, and run a deficit are among the most important variables associated with the overall index of financial autonomy. Other variables, such as the autonomy to set bonuses to be paid to private owners and own financial assets as well as land also contribute substantially to the overall index. Some of the variables that contribute the most to the overall index of academic autonomy are whether universities can determine admission mechanisms, decide the types of courses provided, determine the overall number of students to be admitted and the number of hours of programs per year, as well as the structure of the curricula. Finally, key contributors to the index of human resources autonomy include whether universities have the autonomy to hire and dismiss staff (administrative and open ended), and whether they are authorized to set the salaries and variable contributions paid to staff. Of course, the analysis is purely data driven, and the results are valid only for one questionnaire and one data set, but replicating this type of analysis to other settings, and combining the results with additional information (from a more theoretical or à priori point of view) about what matters would help in suggesting appropriate weights for 67 different indicators in order to aggregate the information first at the level of policy levers and then at the level of policy goals. While the approach is no panacea, it is likely to be useful. As one example of the types of results that emerge from such analysis, table 8 provides a comparison of the indices of financial, human resources, and academic autonomy obtained for the universities in the various countries, and for private and public universities in the MENA dataset. The results from the MCA have been scaled in order to be expressed in percentage terms between values of zero and one. It can be seen that private universities have, as expected, much more autonomy than public universities, especially in the area of human resources. For the overall index of autonomy that takes into account academic autonomy, staffing autonomy, and financial autonomy, the average level of autonomy of private universities, at 90 percent of the maximum value, is twice the average level for public universities, at only 46 percent. Table 8: Estimates of Autonomy for Universities in the MENA Region Country Status Indices of autonomy All Egypt Morocco Palestine Tunisia Public Private Academic Autonomy 0.62 0.57 0.74 0.29 0.48 0.65 0.55 Human Resources Autonomy 0.76 0.27 0.75 0.36 0.30 0.88 0.55 Financial Autonomy 0.69 0.57 0.81 0.55 0.52 0.84 0.65 All Dimensions of Autonomy 0.80 0.49 0.86 0.42 0.46 0.90 0.65 Source: Adoho and Wodon (2012). While the above illustrates how data collected at the level of institutions can be used to assess various dimensions of governance within and between countries in terms of implementation (the above does not assess system-wide regulations and policies), a somewhat similar approach could in principle be used for comparing data obtained at the level of tertiary education systems when data from different systems become available. In fact, some of the information obtained at the level of institutions could perhaps even also be used to inform decisions on the weights that would apply for system-wide ratings. For example, if it turns out that in the analysis of institution-level data, the ability to set the level of student fees appears to be ten times more important than the ability to own buildings for financial autonomy as measured at the level of institutions, then it would probably make sense (unless there is a clear reason not to do so) to also place more weight on rules governing student fees at the system-wide level than on rules governing building ownership. Of course, statistical analysis can go only so far, and any statistical analysis should be combined with good judgment. But such analysis can be helpful in resolving the complex issue of assigning weights for the rating of policy goals. 68 Annex 7: Web-based Questionnaire for Institutional-level Indicators The following pages include a print out of the survey monkey questionnaire used to assess governance practices at tertiary education institutions. In fact, two separate surveys were developed- one aimed at private institutions and a slightly modified one aimed at public institutions, given practical differences in the operating context of these entities. The questionnaire that follows is that developed specifically for private institutions. 69 SABER-Tertiary Education- Governance Survey for Private Institutions Preliminary questions and information before you start the survey -This survey is designed to capture trends on governance policies and practices at tertiary education institutions across Jamaica. Research has shown that governance is the single biggest driver of performance of institutions. This survey will try to assess governance practice at both the system level, as well as those practices specific to your institution. -The survey will take between 60-90 minutes to complete. It is expected that you will need to take a break from the survey mid-way through to gather information or consult a colleague. Survey results are automatically saved by page. You may return to the last completed page to resume where you left off. -Please be assured that the survey results will remain confidential, unless your institution wishes to share them with other institutions participating in the project. 1. What type of institution do you represent?  Public/ Regional  Private Setting a Clear Vision for Tertiary Education The purpose of this set of questions is to capture data on the following policy goal- Setting a Clear Vision for Tertiary Education: the country or government has a vision and plan for the tertiary education sector, a willingness to translate its vision into a concrete action plan 2. Does the country have an official policy or plan for the future development of the tertiary education sector?  Yes  No 3. What are the overarching goals of the government as articulated in its official policy/plan for development of tertiary education sector?  Labour market demand for graduates  Research  Equity and Access  Teaching and Academic Quality  Enrolment growth  Other (please specify) 70 Page 1 SABER-Tertiary Education- Governance Survey for Private Institutions 4. Are university and colleges in Jamaica required to formally state their institutional mission statements?  Yes  No 5. Where is the general mission statement of your tertiary institution stated?  National Law for (Higher) Education  Ministry of Education decree (or similar)  Report of a reflexion committee or council  Universities internal regulation 6. During the formulation of the general mission statement of your institution, which actors had a voice?  The State, national level (Parliament, government)  The State, regional level  Civil society representatives  Industry & business representatives  Universities representatives  Syndicates/unions 7. Are the specific performance goals of your institution required to be formally stated?  Yes  No 8. Where are the specific performance goals of your institution required to be stated?  National Law for (Higher) Education  Ministry of Education decree (or similar)  Report of a reflexion committee or council  Universities internal regulation 71 Page 2 SABER-Tertiary Education- Governance Survey for Private Institutions 9. What was the major concern during the finalization of the institutional performance goals? (Please choose only one option)  Reflecting the expectations of internal stakeholders (students, staff)  Reflecting the expectations of external stakeholders (State, donors, private sector, alumni)  Being aligned with the institution missions nationally defined  The State (Government, MoE, Parliament) 10. Which actors are in charge for monitoring achievement of institutional goals?  The State (Government, MoE, Parliament)  Institutional leadership (Head or equivalent)  Social council (e.g. civil society, industry & business representatives, associations, donors)  Senior Managers  Governing Board (or equivalent)  Other councils (e.g. Academic, Research, Student's Senate...)  Private owners / Donors 11. What are the tools used to measure achievement of institutional goals?  Quantitative surveys producing performance and efficiency data  Assessment reports conducted by the Governing Board  Assessment reports conducted by the State (Parliament or Government)  Assessment reports conducted by an independent agency 12. What is the frequency with which achievement of institutional goals is monitored?  Multiple times per fiscal year  Once per fiscal year  Once every 2-5 years or more  No specified time frame 13. Is monitoring of institutional goals mandatory? (either by external government regulation or internal institutional regulation)  Yes  No Regulatory Framework 72 Page 3 SABER-Tertiary Education- Governance Survey for Private Institutions The next set of questions pertain to the following policy goal- Regulatory Framework : The tertiary education system is governed by an appropriate regulatory framework including for private providers 14. How many times has the legal status of the institution been modified in the last 10 years? (Please choose only 1)  None  One time  Two times  Three times  More than three times 15. How many times has the mission legally entrusted to universities been modified in the last 10 years? (Please choose only one)  None  One time  Two times  Three times  More than three times 16. What is the event that better explains the last changes in the Legal Framework of the Institution?  Reform of the law conducted by the National Assembly/Parliament (or similar)  Replacement of the executive (president, minister)  Effects of a financial/economic crisis  Alignment with international trend Other (please specify) 73 Page 4 SABER-Tertiary Education- Governance Survey for Private Institutions 17. What were the requirements to enter market as a private provider?  No entry regulations  Overly complicated and/ approval process was long  Clear and time to register were reasonable  Objective and measurable so as to minimize discretion and limit the scope for corruption.  Openly published - they were easily accessed without delay  Output-focused (i.e., designed to ask open questions about the standards proposed rather than impose fixed national norms or ratios) so as to allow for flexible and diverse delivery approaches  There are incentives and support (i.e. financial subsidies, tax holidays, provision of land, operational support) for entry into this market Capacity of the Tertiary Education Authority (TEA) The next set of questions pertain to the following Policy Goal- Capacity of the Tertiary Education Authority (TEA): The TEA has the capacity to guide, support, implement and monitor the TEIs and the TE system 18. Does a Tertiary Education Authority exist in your country?  Yes  No 19. In what form does the Tertiary Education Authority exist?  Independent agency of government, responsible for overseeing colleges and universities  Multi-stakeholder led group  Government agency Other (please specify) 20. HEA authority staff are professional,skilled and competent  Yes  Somewhat  No  Do not have enough information/exposure to make a judgement 74 Page 5 SABER-Tertiary Education- Governance Survey for Private Institutions 21. HEA authority makes an effort to engage the sector, and communicates reforms in the sector  Yes  Somewhat  No  I don't have enough information/exposure to comment 22. HEA authority has successfully introduced reforms in the sector  Yes  Somewhat  No  Not applicable 23. What data does your institution regularly report to the HEA? (Check all that apply)  personnel (full-time, part-time, staff, faculty, etc)  academic (enrolment, retention, transition, graduation, learning)  students (composition- full-time, part-time, demographic information)  financial data  changes to physical infrastructure (i.e. construction of new buildings, major renovations etc) Leadership, Management and Organizational Autonomy The next set of questions pertain to policy goal- Leadership, Management and Organizational Autonomy: The regulatory framework allows for sufficient organizational autonomy. This means that the TEA has an appropriate policy on the role and functions of the boards of tertiary education institutions, and the respective responsibilities of the leader. 24. Head of Institution -- What is the selection process  Appointment by the Government  Appointment by a Selection Committee at the institution  Elections  Competitive recruitment 75 Page 6 SABER-Tertiary Education- Governance Survey for Private Institutions 25. Head of institution-- Who is implicated/involved in this process?  The State (Parliament, MoE or Regional authorities)  Institutional leadership (Governing Board,...)  Senior Academic staff  Private owners or donors of/to the institution if it is private 26. Head of institution -- What are the requirements to be selected?  s/he must be an academic  s/he must have a Managerial profile  s/he must be an internal person to the Institution  s/he must meet the full job specification  s/he must be affiliated to a political party 27. Head of institution -- What are the conditions of their mandate?  The length is less than 4 years  It's not renewable  It's renewable but the number of mandates is limited 28. Deans -- Selection process (Select all that apply)  Appointment by the Government  Appointment by a Committee  Elections  Competitive recruitment  By last high authority (president/vice president) appointment 29. Deans -- Who is implicated/involved in this process?  The State (Parliament, MoE or Regional authorities)  Institutional leadership (Governing Board,...)  Senior Academic leadership  Private owners or donors 76 Page 7 SABER-Tertiary Education- Governance Survey for Private Institutions 30. Deans-- What are the requirements to be selected?  s/he must be an academic  s/he must have a Managerial profile  s/he must be an internal person to the Institution  s/he must meet the full job specification  s/he must be affiliated with a political party 31. Deans-- What are the conditions of their mandate?  The length is less than 4 years  It's not renewable  It's renewable but the number of mandates is limited  It's renewable but the number of mandates is not limited 32. Members of the Governing Board (or equivalent)-- Selection process  Appointment by the Government  Appointment by a Selection Committee based on credentials  Elections  Appointed by institutional leadership 33. Members of the Governing Board (or equivalent) -- Who is implicated/involved in this process?  The State (Parliament, MoE or Regional authorities)  Institutional leadership (Governing Board,...)  Senior Academic staff  Private owners or donors 34. Members of the Governing Board (or equivalent)-- What are the requirements to be selected?  s/he is not necessarily an academic  s/he can be an external person from outside the Institution  s/he is not necessarily a member of a political party  Others 77 Page 8 SABER-Tertiary Education- Governance Survey for Private Institutions 35. Members of the Governing Board (or equivalent)-- What are the conditions of their mandate?  The length is less than 4 years  It's not renewable  It's renewable but the number of mandates is limited  It's renewable but the number of mandates is unlimited Institutional Autonomy The next set of questions pertains to the policy goal on Institutional Autonomy: The regulatory framework allows for sufficient financial, staffing and academic autonomy in institutions 36. Who had the final say on the budget of the institution?  The State (Government, MoE, Parliament)  The Governing Board  Institutional leadership (Head or equivalent)  Institutional budget committee  Senior Managers  Private Owners  Others 37. What are the total revenues for your institution for the fiscal year ended March 31, 2011? JMD 78 Page 9 SABER-Tertiary Education- Governance Survey for Private Institutions 38. What are the different revenue sources of the Institution? Please specify amount in JMD for fiscal year ending March 31, 2011. Funds from the Government (budget allocation, grants, other) Fees from students Funds from Private owners Funds raised from Private companies (e.g. industries, businesses) Loans obtained from banks Loans and grants obtained from International organizations Funds from contracts (e.g. consulting, outsourcing) Revenues obtained from financial applications Other 39. Regarding the financial procedures, has the Institution autonomy to:  Manage its assets (e.g. sell, buy, rent)  Use unspent funds from one year to another  Set the level of fees  Define the revenue structure of the institution (e.g. public/private sources, banks, assets interests, endowments, fees)  Run a deficit  Use a multi-year forecast (e.g. Medium Term Expenditure Framework system)  Choose the budget allocation inside a block-grant budget  Set the bonuses to be paid to private owners (for Private institutions) 40. What is the total amount of recurrent spending for fiscal year ended March 31, 2011? Staff Salaries Operational Costs Other (recurrent) Total recurrent spending 41. How much did your institution spend in JMD on capital expenditure: 79 Page 10 SABER-Tertiary Education- Governance Survey for Private Institutions 42. What is involved in determining the salary of faculty/staff?  Years of service in your institution  Academic qualifications  Years of relevant experience  Performance  Others () 43. What measures are you taking to assure the sustainability of university financing?  Increasing the number of fee-paying students  Increasing the fee levels  Raising donations from the private sector  Conducting research and consultancy for fees  Opening/closing programmes based on cost-effectiveness  Others 44. What type of assets is the University allowed to own?  Buildings  Land  Financial assets  Equipments (Motorized, IT) and furniture 45. Staffing autonomy: Does the University have autonomy for:  Hiring new administrative staff open ended  Dismissing administrative staff  Hiring academic staff open ended  Dismissing academic staff open ended  Developing training programs  Promoting staff members 80 Page 11 SABER-Tertiary Education- Governance Survey for Private Institutions 46. Does the University have autonomy for: Yes- but needs approval from No- decision is taken by another entity Yes- entirely somebody else Assessing the performance    of administrative and academic staff Setting the salaries of    academic staff Setting the salaries of    administrative staff Setting variable salaries    (linked with performance) Setting the Contracts of    administrative staff and academics (e.g. duration, benefits) 47. What types of policies are used to manage the staff in HR department, financial department, academic department?  Reporting measures  Compensations, salaries attached to performance  Providing justifications for expenditures  Scorecards  Standardized sanctions in case of professional misconduct 48. How many staff does your institution have? Administrative staff (as opposed to Academic) Administrative Senior Staff Academic Senior Staff 49. Are any of your administrative staff civil servants (employed directly by the State) Number of staff 50. How many of your administrative staff are Full-time Part-time 51. Are any of your academic staff civil servants (i.e. employed by state)?  Yes  No 81 Page 12 SABER-Tertiary Education- Governance Survey for Private Institutions 52. How many academic staff are civil servants? Number of staff 53. How many academic staff in each type of contract: Full-time Part-time 54. Please give a breakdown (by approximate percentage) of number of academic staff at each qualification level Ph.D Holders Ph.D holders having more than 10 years of service at the University PhD holders having more than 10 years of service in a managerial position 55. Academic Autonomy: Has the university the autonomy to decide the: Yes- but needs approval from No- decision is taken by another entity Yes- entirely somebody else Structure of curricula (e.g.    type of degrees) Introduction of new    programs Types of courses (level,    sector, mode) Number of hours per    program per year Assessment of students    (learning outcomes), format of exams Academic partnerships with    other institutions Content of articles,    speeches, other communications by faculty? 82 Page 13 SABER-Tertiary Education- Governance Survey for Private Institutions 56. Has the University autonomy to determine: Yes- but needs approval from No- decision is taken by another entity Yes- entirely somebody else The overall number of    students admitted in the University? (e.g. is the university able to refuse applications) The number of students per    program? The admission    mechanisms (e.g. entry- exams, analysis of academic grades of students)? Incentives based funding: The next set of questions pertains to policy goal on Incentives based funding: Funding mechanism is performance based, transparent and promotes equity and choice in student enrolment 57. Recurrent, Research, Investment, Equity Percent of recurrent budget allocated on the basis of performance measures. Percent of research budget allocated on a competitive basis. Percent of investment funding allocated on a competitive basis. Percent of recurrent budget allocated per students following a needs-based model (i.e. grants, loans, vouchers). Quality and Relevance The next set of questions pertains to the policy goal on quality and relevance: The TEA has an independent quality assurance and accreditation agency for both public and private institutions 83 Page 14 SABER-Tertiary Education- Governance Survey for Private Institutions 58. If there is a Quality Assurance (QA) system, what is its form:  It takes place inside the University (e.g. assessing programs, committee, participatory team...)  It takes place outside the university under the responsibility of the Government  It takes place outside the university under the responsibility of a specially designated independent agency Other (please specify) 59. If existent is/are there QA systems to address the following items  Institutional licensing  Institutional accreditation  Accreditation of programs  Assessing Learning Outcomes  Teaching methodologies  Research production  Appropriateness of physical infrastructure 60. Is there are a feedback loop in place to respond to QA assessment results?  Action Plans are followed by Internal QA units  Action Plans are followed by Deans  Action Plans are followed by the head of the university  The Action Plans are regularely reviewed (every 2 years or more)  Variable budget allocations are linked with results  Others Accountability The next set of questions pertains to policy goal on Accountability: Institutions are held to specific standards of transparency around financial health, fraud, student engagement and employment of graduates 84 Page 15 SABER-Tertiary Education- Governance Survey for Private Institutions 61. Financial integrity: Are the budget documents available and/or reviewed by the following actors:  State (MoE, responsible agency)  Academic staff  Administrative staff  Students  Medias, public audience 62. If the university is submitted to financial audit:  Is it conducted by an external body ?  Is the report available for inside actors?  Is the report disseminated outside the university? 63. Frequency of financial audit (Please choose one only)  The financial audit takes place every six month or more  The financial audit takes place every year  The financial audit takes place less than every year 64. Has the university applied formal sanctions against:  Embezzlement  Inappropriate spending (failure to present justifications)  Misprocurement 65. Academic integrity: Has the university applied formal sanctions against:  Examination fraud  Unethical behavior of faculty (sale of exams, nepotism, academic dishonesty...)  Non compliance with admission standards  Unethical management of faculty careers 85 Page 16 SABER-Tertiary Education- Governance Survey for Private Institutions 66. Transparency and openness: Has the university established tracking surveys to measure:  New graduates employment rate  Average unemployment period after graduation  Average salaries of new graduates  Main areas of market outlets  Average number of years required to complete a degree 67. If existent, are these tracking surveys disseminated outside the university by :  Institutional website  Reports publicly disseminated  Newsletters  Communication brochures 68. Are the following items disseminated by one (or more) vector mentioned in the question above?  Institutional mission & goals  Strategy of the university  Results of the accreditation process  Institutional evaluations results conducted by internal or external evaluators Performance metrics Various indicators assessing efficiency of operations of the institution 69. What is the average length of time to degree (in years) for fiscal year 2011. Years (decimals are acceptable): 70. What is the number of years set in the standard definition of graduation rates for your institution? (Note: graduation rate refers to the proportion of students who enter and graduate within a certain number of years) Years 71. What is the average graduation rate of students in your institution? Graduation rate: Page 17 86 References Achim, M.I., Cabulea, L., Popa, M., Mihalache, S. 2009. “On the role of benchmarking in the higher education quality assessment,” V.11(2). Romania: Annales Universitatis Apulensis Series Oeconomica. Aghion, P. M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir, 2007. “Why Reform Europe’s Universities.” Bruegel Policy Brief. Brussels: Bruegel. Available via the Internet: aei.pitt.edu/8323/ Aghion, P. M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir, 2008. Higher Aspirations: an Agenda for Reforming European Universities”. Brussels. (http://www.bruegel.org/publications/publication-detail/publication/1-higher-aspirations- an-agenda-for-reforming-european-universities/). Aghion, P., M. Dewatripont, C. Hoxby, A. Mas-Colell, and A. Sapir, 2009. “The Governance and Performance of Research Universities: Evidence from Europe and the U.S.” National Bureau of Economic Research. Working Paper No. 14851, April 2009. Aghion, P., M. Dewatripont, M. Hoxby, A. Mas-Colell, and A. Sapir, 2010. “The governance and performance of universities: Evidence from Europe and the US.” Economic Policy: 7-59. Alden, J. and G. Lin. 2004. “Benchmarking the Characteristics of a World-Class University: Developing an International Strategy at University Level.” London: The UK Tertiary education Leadership Foundation. Altbach, P. 2007. “Academic Freedom: International Realities and Challenges.” In Tradition and Transition: The International Imperative in Tertiary education, 49–66. Rotterdam, The Netherlands: Sense. Altbach, P. and J. Salmi, 2011. The Road to Academic Excellence: the Making of World-Class Research Universities. Directions in Development; World Bank, Washington, D.C. Anderson, D. and Johnson, R. 1998. “University Autonomy in Twenty Countries”. Centre for Continuing Education, The Australian National University, Canberra: ANU. Anderson, J.A., 2005. Accountability in Education. IAE and IIEP Education Policy Series No. 1. Paris and Brussels. Association of Governing Boards (AGB), 2007. “AGB Statement on Board Accountability.” Washington, DC: Association of Governing Boards of Universities and Colleges. 87 Baye, A. and Mainguet, C. , 2006. “Defining a framework of indicators to measure the social outcomes of learning.” Paper in “Measuring the effects of education on health and civic engagement: Proceedings of the Copenhagen Symposium). Bernasconi, A.. 2011. “Private and public pathways to world class research universities: the case of Chile”. Altbach, P. and J. Salmi, eds. “The Road to Academic Excellence: Emerging Research Universities in Developing and Transition Countries”. Washington DC: The World Bank. Birnbaum, R. 1991. Faculty in Governance: The role of Senates and Joint Committees in Academic Decision Making. New Directions for Tertiary education, 18 (3), pp. 8-25 Bloom, D., Hartley, M. and Rosovsky, H. 2006. “Beyond Private Gain: The Public Benefits of Higher Educaion” in International Handbook of Tertiary education edited by Altbach, P. and Forrest, J. The Netherlands, Kluwer Press. Cajas, F., 2010,. “Gobernabilidad de la educación superior en Centro América”. Report produced for the World Bank. Unpublished. Campbell, D., 2006. “What is education’s impact on civic and social engagement?” Measuring the effects of education on health and civic engagement: Proceedings of the Copenhagen Symposium). OECD. Clark, B.R. 1983. “The Tertiary education System: Academic Organization in Cross-National Perspective.” Berkeley: University of California Press. Clark, B.R. 1998. “Creating Entrepreneurial Universities: Organizational Pathways of Transformation.” Issues in Tertiary education Series XVI. Oxford: IAU Press. Crellin, M., Aaron, Darrell, Mabe, David, Wilk, Courtney. 2011. “Catalyst for Completion: Performance-Based Funding in Higher Education A case study of three states.” Boston: Policy and Research. CUC (Committee of University Chairs). 2006. “Governance Questionnaire.” Bristol: CUC Publication. CUC (Committee of University Chairs). 2009. “Guide for Members of Tertiary education Governing Bodies in the UK”. Bristol: CUC Publication. Currie, J, Vidovich, L and Yang R. 2008. "Countability Not Answerability?" Accountability in Hong Kong and Singapore Universities. Asia Pacific Journal of Education, v28 n1 p67- 85 Mar. DEST (Department of Education, Science and Training). 2000. “Benchmarking: A Manual for Australian Universities”. Canberra. 88 DEST (Department of Education, Science and Training). 2008. “Commonwealth Grant Scheme Guidelines Nͼ 1”. Canberra: 13-21. ENQA (European Association for Quality Assurance in Tertiary education). 2007. “Standards and Guidelines for Quality Assurance in the European Tertiary education Area.” Helsinki. Estermann, T., and T. Nokkala, 2009. “University Autonomy in Europe I: Exploratory Study.” Brussels: EUA Publication. Estermann, T., T. Nokkala, and M. Steinal, 2011. “University Autonomy in Europe II The Scorecard”. Brussels: EUA Publication. European University Association. 2007. “The EUA Lisbon Declaration – Europe’s Universities beyond 2010: Diversity with a Common Purpose” Brussels: EUA. Eurydice, the Information Network on Education in Europe. 2008. “Tertiary education Governance in Europe: Policies, structures, funding and academic staff. Brussels: Eurydice European Unit, European Commission. Fielden, J. 2008. “Global Trends in University Governance.” World Bank Education Working Paper Series, March 2008. Washington D.C.: World Bank. Fiszbein, A., D. Ringold, and F. H. Rogers, 2011. “Making Services Work: Indicators, Assessments and Benchmarking of the Quality and Governance of Human Service Delivery in the Human Development Sectors.” Policy Research Working Paper No. 5690. Washington DC: The World Bank. Geiger, R. (2005), Knowledge and Money: Research Universities and the Paradox of the Marketplace, Stanford University Press, Stanford. Green, Madeleine. “Lost in Translation: Degree Definition and Quality in a Globalized World”. September 2011. The International Association of Universities and NAFSA. Available online: http://dx.doi.org/10.1080/00091383.2011.599288 HEA (Tertiary Education Authority) & IUA (Irish Universities Association). 2007. “Governance of Irish Universities: A Governance Code of Legislation, Principles, Best Practice and Guidelines.” Tertiary education Authority and Irish Universities Association. Dublin. Henard, Fabrice, 2009. Good Governance and the Transformation of Tertiary education Institution: A review of Tertiary education Governance Literature, Governance Guidelines and Quality Assurance Guidelines. Paris: OECD Huisman, J. and J Currie, 2004. “Accountability in Tertiary education: Bridge Over Troubled Water? “ Tertiary education 48: 529-551. 89 IGOPP (Institute for Governance of Private and Public organizations). 2007. “Report of the Working Group on University Governance.” Montreal. IUQB (Irish Universities Quality Board). 2007. “A Framework for Quality in Irish Higher Universities: Concerted Action for Institutional Improvement.” Dublin. Jaramillo, A., et al. 2012 Universities through the Looking Glass: Benchmarking University Governance to Enable Tertiary education Modernization in MENA, World Bank Publication, Washington DC Johansen, L. N. et al. 2003. “Recommendations for Good University Governance in Denmark.” Committee University Boards in Denmark. Copenhagen. Johnstone, B and Marcucci, P., 2010. Center for Comparative and Global Studies in Education of the State University of New York at Buffalo. International Comparative Tertiary education Finance and Accessibility Project (ICHEFAP). Johnstone, D. B. 2010. “Autonomy and Accountability: The Search for Balance in University Governance and Management”. State University of New York at Buffalo [unpublished]. Kerr C. 2001. The Uses of the University. Cambridge, MA: Harvard University Press. Long, Gideon. Chile student protests point to deep discontent. August 11, 2011. BBC. Minnesota Office of Tertiary education (MOHE). 2008. Minnesota Measures: 2008 Report on Tertiary education Performance. St. Paul, MN: Minnesota Office of Tertiary education, http://www.ohe.state.mn.us/pdf/MinnesotaMeasures2008.pdf. Accessed January 2009. Morley, L., M. Eraut, S. Aynsley, D. MacDonald and J. Shepherd (2006), “Establishing the Needs of Employers and Related Organisations for Information about Quality and Standards of Tertiary education Provision and Student Achievement in England”, Tertiary education Funding Council for England (HEFCE), Bristol. OECD, 2003. Education and Policy Analysis. Paper 3- Changing patterns of governance in tertiary education. < http://www.oecd.org/dataoecd/0/20/35747684.pdf> OECD. 2005. “Guidelines for Quality Provision in cross-border Education.” OECD/UNESCO. Paris. OECD. 2006. Funding Systems and Their Effects on Higher Education Systems : Country Study Denmark. Available online: http://www.oecd.org/dataoecd/20/43/38307998.pdf OECD, 2008. Tertiary education for the Knowledge Society Volume 1, Paris: OECD 90 Patrinos, Harry Anthony. 2012. Chapter 4 “Student Assessments” and Chapter 5 “Teacher Policies” in Strengthening Education Quality in East Asia. Washington DC: The World Bank. Penn, J. 2007. “Assessment for ‘Us’ and Assessment for ‘Them.’” Inside Tertiary education. June 26, 2007. Raza, R., 2010. “Tertiary education Governance in East Asia,” A paper prepared for the East Asia Flagship Report on Tertiary education. World Bank, Washington D.C. Rosovosky, H. 2001. Some thoughts about University Governance. In W. Hirscht and L. Webber (eds), Governance in Tertiary education: The University in Flux. London: Economica. Saint, W. 2009. “Guiding Universities: Governance and Management Arrangements around the Globe.” Human Development Network. Washington, DC: World Bank. Salmi, J. and Hauptman, A.M. 2006 Innovations in Tertiary Education Financing: A comparative evaluation of allocation mechanisms. World Bank: Washington DC. Salmi, J. 2007. “Autonomy from the State vs. Responsiveness to Markets.” Tertiary education Policy, Vol. 20, No. 3, Palgrave MacMillan: 223-242. Salmi, J. 2007. “The Growing Accountability Agenda: Progress or Mixed Blessing?”. International Tertiary education Number 50, Winter 2007. Salmi, J., and Saroyan, A. 2007. “League Tables as Policy Instruments: Uses and Misuses.” Tertiary education Management and Policy. Volume 19, No. 2. OECD, Paris. Salmi, J. 2009. “The Growing Accountability Agenda in Tertiary education: Progress or Mixed Blessing?” Education Working Papers Series N° 16. Washington, DC: World Bank. Salmi, J., 2009. The Challenge of Establishing World-Class Universities, Directions in Development, World Bank, Washington, D.C. Saltmarsh, J. and Hartley, M. 2011. To serve a larger purpose. Engagement for democracy and the transformation of tertiary education. Philadelphia: Temple University Press. Santiago, Paulo, Tremblay, K , Basri, Ester and Elena Arnal. 2008. Tertiary education for the Knowledge Society: Thematic Review of Tertiary education. OECD Paris. Schofield, A. et al. 2008. “A Guide for New Clerks and Secretaries of Governing Bodies of Tertiary education Institutions in the UK.” Leadership Foundation for Tertiary education. London. 91 Singh, M. 2007. “The Governance of Accreditation.” Tertiary education in the World 2007 – Accreditation for Quality Assurance. GUNI Series on the Social Commitment of Universities 2. Global University Network for Innovation: 96-108. Snowdon, K., 2005. “Without a Roadmap: Government Funding and Regulation of Canada’s Universities and Colleges.” Research Report W31. Canadian Policy Research Networks, Ottawa, Canada. Tavenas, F., 2003. “Quality Assurance: A Reference System for Indicators and Evaluation Procedures.” Brussels: EUA Publication. The Economist. 2011. The struggle to make the grade: If only more of the region’s higher- education institutions were like the University of São Paulo. Available through: The Economist. [Accessed Oct 10, 2012] Trakman, L., 2008. Modeling University Governance. University of New South Wales. Waterman, Robert H. Jr. , Peters, Thomas J and Julien R. Phillips. June 1980. “Structure is not Organization.” Available online: http://www.tompeters.com/docs/Structure_Is_Not_Organization.pdf Williams, R., G. de Rassenfosse, P. Jensen, and S. Marginson, 2012. U21 Ranking of National Tertiary education Systems : A project sponsored by Universitas 21. Institute of Applied Economic and Social Research, University of Melbourne World Bank, 2011. Skills and Research for Productivity and Growth: Tertiary education in East Asia Main Report. Human Development Department, East Asia Region. Wright, S. and Orberg, J.W. 2011. “Autonomy and control: Danish university reform in the context of modern governance” Learning and Teaching. Volume I, Issue 1, Spring 2008 92