62251 Quality Assurance in Higher Education: A Comparison of Eight Systems Key Messages practices in evaluating, teaching and learning in all types of HEIs, private and public, vocational and research-oriented, Quality in higher education is difficult and for traditional students as well as for second-chance or to measure. Incentives to improve (or lifelong learners. not) are often different for public and private institutions, and approaches Issues in and Approaches to Quality Assessment in to quality assurance have to be Higher Education adapted to different situations. Quality in higher education is inherently difficult to Poland is reforming its higher measure. Unlike the manufacturing sector, the production education institutional framework function in education is much more complex and results are and the World Bank conducted a often not readily discernible in a timely, objective or useful review, comparing strategies followed manner. The field of quality assurance in higher education by (largely) OECD countries with is still in a state of adolescence, with varying and shifting regard to quality assessment, to help approaches and confusion in both objectives and the country assess educational terminology. Incentive structures for improving (or not services at its higher education improving) quality also generally differ between public and institutions. private institutions. Based on the research, some of the Common approaches to quality assurance in higher most comprehensive quality education are generally classified as summative (that is, assessment frameworks for public entail a judgment regarding whether an institution is universities were found in the Irish, meeting certain criteria) or formative (that is, encourage an Scottish and Hong Kong educational institution to identify its own strengths and deficiencies and systems; for private universities, the develop plans to address the latter). Approaches currently in Austrian and Swiss systems present use include: a) minimum standard accreditation; b) interesting examples. accreditation of excellence; c) supportive evaluation; d) audits of internal quality assurance processes; e) comparative evaluation of the state of a discipline; f) benchmarking between institutions; and g) rankings, such as Introduction those established by the Center for World-Class Universities and the Institute of Higher Education of The Government of Poland (GoP) is currently engaged in Shanghai Jiao Tong University. reforming its higher educational institutional framework and sought World Bank assistance in assessing the quality of educational services at its higher education institutions (HEIs), specifically post-graduate courses and training. The World Bank prepared a study describing and comparing strategies followed by (largely) OECD countries in regard to quality assessment, with the purpose of identifying best ECA Knowledge Brief While summative approaches such as accreditation are both Comparison of Seven Countries and one Cross- common and popular, they are often tricky to apply well at Country System reasonable cost. Decisions must be made whether to accredit an institution and/or a program, with differing In order to systematically analyze a range of country quality criteria and levels of expertise needed for each choice. assurance systems, four criteria were drawn from the Criteria chosen are often proxies for quality, and since methodologies outlined above. They were: accreditation implies either a reward or sanction, institutions are encouraged to “put on their best face,” rather than be up The object and nature (formative or summative) of front about weaknesses and plans to address them. evaluation. The relative role of HEIs, agencies and The traditional criticism of formative approaches is that governments. they do not entail clear penalties and are only marginally based on objective criteria of quality. This criticism is The consequences and impact of decisions and/or mitigated by the fact that it is difficult to identify objective recommendations. criteria for both summative or formative evaluations and The costs of HEI quality assurance systems in that in practice, penalties under summative approaches tend relation to the expected benefits. to be infrequently applied. Formative approaches are also clearly superior at promoting improvement. Considering the A sample of HEI quality assurance systems from different relative advantages and disadvantages of both approaches, it countries was then selected. The systems described range appears that no single or rigid approach is best. Rather, the from institution-centered and/or formative systems to others methodology employed should be adapted to different types that are agency-driven and often require certification of of institutions and situations. some kind. This sample was chosen as representative of the possibilities - not all systems are necessarily good or even Overall, a good quality assurance system should do three sustainable. A summary of findings for each of the systems things: examined is presented below. Ensure that HEIs and programs satisfy at least a The Swiss system is a hybrid. It requires a strict minimum level of quality in order to protect accreditation procedure for private institutions eligible for students. federal support, based on predefined criteria. The system is Contribute to the improvement of all institutions relatively flexible, however, in that it encourages public and programs, whatever their level, and encourage universities to develop their own rigorous internal quality HEIs to develop their own internal quality culture. improvement culture. Fulfill both of the objectives above at a reasonable cost, which should not exceed the estimated Historically, the German system has endeavored to benefits of the quality improvements. accredit all programs of study - a gigantic task that is also questionable from a scientific and didactic perspective. A further issue in designing quality assessment systems is Moreover, the system is very agency-centered and, the question of who does the assessments - the HEIs therefore, does not sufficiently encourage HEIs to make themselves, external agencies, or governments? It is widely improvements on their own. Recent reforms have moved agreed that quality assurance and quality improvement are a the system toward a less agency-driven and more formative shared responsibility between HEIs and quality assurance approach. agencies (and in many cases governments), although there are widely diverging views regarding who should have the The French system is comprehensive - it looks at the leading role. The chosen solution depends on the degree of governance, management and quality assurance systems of trust governments have in their HEIs to guarantee at least a institutions; covers the performance of research units minimum level of quality or to make permanent efforts to (departments, institutes, laboratories); and examines the improve. The perceived independence of quality assurance quality of all teaching and learning programs. The agencies from influence, both from governments and HEIs, institutional framework has a weakness, however, in that it also affects their credibility. is mainly agency-driven and does not sufficiently encourage quality improvement. Institutions and units are under pressure to receive good marks and, therefore, do not openly acknowledge weaknesses or take advantage of external support to work towards improvements. ECA Knowledge Brief The Irish system is probably the most comprehensive The Hong Kong system reflects influences from Australia, within the sample and is largely institution-driven. The law New Zealand and the United States, and emphasizes the makes it compulsory for institutions to develop rigorous responsibility of institutions in promoting quality in internal systems of evaluation and improvement of teaching teaching and learning. It also emphasizes the supportive role and learning, research, outreach at the level of departments, of national agencies which, in the case of Hong Kong, is the as well as the efficacy of university services. The law, Quality Assurance Council of the University Grants however, also requires evaluations of these internal quality Committee. The approach to quality stems from the assurance measures on a regular basis by an external recognition that HEIs in Hong Kong have distinct and agency. Overall, the Irish system presents an interesting and varied missions. Therefore, the system does not attempt to efficient division of responsibilities for quality assurance, straitjacket institutions through a single set of quality making institutions responsible for the quality of all their standards or objectives, but recognizes that each institution services but also making sure that they take their has its own objectives appropriate to its mission. responsibilities seriously. The Scottish system is based on the same principles as the Conclusion Irish system, where the objective is to make institutions responsible for promoting improvements. The primary A number of “best practice” lessons can be garnered from difference is that in Scotland, the main system is focused on this review of different national quality assurance systems teaching and learning, research being evaluated through for HEIs. other channels. The system is quite well developed, and for First, a robust quality assessment system should those interested mainly in improving teaching and learning examine the strategies followed by an HEI in the at the Bachelors and Masters levels in public and private light of the HEI’s intended purposes. institutions, it is worth examining closely. Second, quality assessment should focus on quality The Austrian system has in place a partial system of assurance processes more than on pre-defined accreditation of universities. It contributes to creating more criteria. transparency in the private sector by determining which Third, the system should be as much institution- universities should be allowed to call themselves ‘private driven as agency-driven, which means that internal universities’. The system is therefore clearly summative and quality assurance procedures are an important agency-led. It contributes indirectly to improving quality by element of quality assurance. fixing the minimum quality standards for accreditation; however, the internal quality processes imposed as criteria Fourth, the system should be as light as possible - it for accreditation remain very superficial. should push the concerned HEI to do a great part of the work. The EUA institutional evaluation program was set up to Finally, the system should be adapted to the types serve European University Association (EUA) members and of HEIs in the country. The higher the standard of help HEIs, on a voluntary basis, to measure where they are an institution - as measured by the quality of its regarding governance, capacity for change and quality research performance, staff and students - the more assurance. Universities write self-evaluation reports, advisable it is that quality assurance be conducted including SWOT analyses. Visiting panels of five experts internally by the institution itself. (including a student) then judge the self-evaluation reports and make recommendations. The impact of EUA The systems which seem to satisfy the above-mentioned evaluations nevertheless depends very much on the conditions are the Irish, Scottish and Hong Kong systems, at institutions, which are free to ignore the reports or actively least for public universities. For private universities, the follow-up on the experts’ recommendations. The best aspect Austrian and Swiss systems are worth examining with of the EUA system is that it is quite good in promoting a respect to accreditation. dynamic of change and quality improvement. ECA Knowledge Brief Summary of System Types Formative Summative Evaluation/QA audit, Accreditation follow-up: Programmes (and Weaker Stronger Institutions institutions) Gov/ agency/ High Role Role of HEIs Low agen. >=< 0 < < < < = < > < < High Cost Low (previous) Germany Germany Scotland (private) (private) Republic IEP/EUA (public) Austria France (new) Swiss Swiss Hong Kong Irish UK Systems 17 Source: Powerpoint Presentation on Higher Education Quality Assurance by Dr. Luc Weber, Poland, May 2010. About the Author This Knowledge Brief is based on a background paper prepared by Dr. Luc Weber, consultant to the World Bank. It was prepared by Sara Bin Mahfooz and Kate Hovde (consultants) under the guidance of Alberto Rodriguez, Acting Senior Manager for the Education Sector, Europe and Central Asia Region of the World Bank. “ECA Knowledge Brief” is a regular series of notes highlighting recent analyses, good practices and lessons learned from the development work program of the World Bank’s Europe and Central Asia Region http://www.worldbank.org/eca