VERIFICATION AND MONITORING OF RESULTS AND STRATEGIC PURCHASING DISCUSSION PAPER SEPTEMBER 2022 Rafael Cortez Meaghen Quinlan-Davidson / VERIFICATION AND MONITORING OF RESULTS AND STRATEGIC PURCHASING Rafael Cortez Meaghen Quinlan-Davidson September 2022 Health, Nutrition, and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. For information regarding the HNP Discussion Paper Series, please contact the Editor, Martin Lutalo at mlutalo@worldbank.org or Erika Yanick at eyanick@worldbank.org. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. © 2022 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition, and Population (HNP) Discussion Paper Verification and Monitoring of Results and Strategic Purchasing Rafael Corteza and Meaghen Quinlan-Davidsonb Economist, Health, Nutrition, and Population Global Practice, The World Bank, Washington a Senior DC, USA Professor-Researcher. Department of Economics. University of the Pacific (Peru) b PhD Candidate, Institute for Global Health, University College London, UK This paper is part of a series of products prepared under the Advisory Services and Analytics (ASA) “Strengthening the strategic purchasing of health care services to improve health system efficiency and equity (P170294)” conducted by the Health, Nutrition, and Population Global Practice and supported by the Korean-World Bank Trust Fund Abstract: The aim of the study was to highlight the different verification processes that four countries and one province take to monitor efficiency, quality, coverage, financial protection, and health outcomes in health systems. The literature review focused on the experiences of England and Sweden, illustrating a comprehensive verification process. Norway, the Canadian province of Ontario, and Turkey collected data on health care provider performance but did not publicly report it. Different instruments were used. Performance measures of patient-reported experiences, compliance with clinical guidelines, and waiting times have become common measurement-based indicators. To improve verification processes, it is necessary to maintain accountability between providers and governments to ensure that the overall objectives of health care are achieved. Monitoring effective service coverage includes measuring the population in need of the service using administrative records from service providers, determining the effectiveness of service coverage using selected indicators, and monitoring equity in access to quality health services using data disaggregated by inequality dimensions. Verification of results is essential within the context of institutional arrangements for the purchasing of health care services to providers. There is autonomy over several significant decision areas such as staffing (numbers and skill mix); financial management (ability to take loans); the scope of activities and capital investments; governance mechanisms that make providers accountable to purchasers; and conditions that balance the power between purchasers and providers. Within this context, monitoring and verification of results is critical to enhance the performance of service providers and ensure value for money within health expenditure. This would be strengthened by previously agreed standards between providers and the implementing agency. Setting strong monitoring and verification procedures has become a key factor in the success of Results-Based Financing programs in general, strengthening health information and governance structures are the most valuable “spillover effects” of such programs. Keywords: Monitoring of results, indicators, strategic purchasing, health systems Disclaimer: The findings, interpretations, and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. 3 Correspondence Details: Rafael Cortez, World Bank, 1818 H Street, NW, Washington, DC 20433, USA; telephone: 202-458-8707; fax: (202) 522-0050, e-mails:rcortez@worldbank.org /cortez_ra@up.edu.pe; website: www.worldbank.org/hnp. 4 Contents ACKNOWLEDGMENTS ................................................................................................ 9 INTRODUCTION........................................................................................................... 10 METHODOLOGY ............................................................................................................. 11 CONCEPTS AND INDICATORS ................................................................................ 12 EFFICIENCY .................................................................................................................... 12 QUALITY ........................................................................................................................ 13 COVERAGE ..................................................................................................................... 14 FINANCIAL PROTECTION ................................................................................................ 15 HEALTH OUTCOMES ....................................................................................................... 15 COUNTRY CASE STUDIES......................................................................................... 16 ENGLAND ....................................................................................................................... 16 SWEDEN ......................................................................................................................... 24 ONTARIO, CANADA ........................................................................................................ 27 NORWAY ........................................................................................................................ 33 TURKEY ......................................................................................................................... 37 DISCUSSION .................................................................................................................. 40 CHALLENGES AND OPPORTUNITIES ................................................................................ 41 REFERENCES................................................................................................................ 44 Figures Figure 1. Key Actors, Relationships and Policy Objectives Needed to Develop Strategic Purchasing in Health Care ..........................................................................................10 Figure 2. Flow of Budgeted Funding in the Health and Care System (2018–2019) .........17 Figure 3. Organization of Swedish Health System ............................................................25 Figure 4. Financial Flow of Swedish Health Care System ................................................26 Figure 5. Organization of the Canadian Health System ....................................................28 Figure 6. Organization of Norway’s Health System ..........................................................34 Figure 7. Main Financial Flows in the Norwegian Health Care System ...........................36 Figure 8. Overview of the Turkish Health System ............................................................38 Figure 9. Financial Flows in the Turkish Health System...................................................39 5 Tables Table 1. Oversight Metrics 2019/20 21 Table 2. NCAPOP List of Indicators (2017–2019) 23 Table 3. Models within the Ontario Health System 29 6 Acronyms A&E Accidents and emergencies BPT Best practice tariffs CCG Clinical commission group CCM Comprehensive care model (Ontario) CHT Canada Health Transfer CIP Cost-improvement programme CQUIN Commissioning for Quality and Innovation DHSC Department of Health and Social Care (UK) DRG Diagnosis-related group EMS Emergency Medical Services (Canada) EU European Union FFS Fee for service FHG Family Health Group (Ontario) FHN Family Health Network (Ontario) FHO Family Health Organization (Ontario) GP General practitioner HIG Health Implementation Guide (Turkey) HEFPI Health Equity and Financial Protection Indicators HRG Healthcare resource group HTP Health Transformation Programme (Turkey) HQIP Healthcare Quality Improvement Partnership LHIN Local health integration networks MFF Market Forces Factor MoH Ministry of Health MOHLTC Ministry of Health and Long-Term Care (Ontario) NCAPOP National Clinical Audit and Patient Outcomes Programme NHS National Health Service NHSE National Health Service England NHSI National Health Service Improvement NPS National Patient Survey (Sweden) NTPS National Tariff Payment System (UK) OECD Organisation for Economic Co-operation and Development OHIP Ontario Health Insurance Plan OMA Ontario Medical Association OOP Out-of-pocket PbR Payment by results (UK) P4P Payment for performance PHU Public Health Unit (Canada) PMS Performance Measurement System QBF Quality-Based Financing RBF Results-Based Financing 7 RHA Regional Health Authority (Norway) SDI Service Delivery Indicator SSI Social Security Institution (Turkey) SUD Sudden unexplained death SUS Secondary Uses Service SALAR Swedish Association of Local Authorities and Regions TFC Treatment function code UHC Universal health coverage USAID United States Agency for International Development WBG World Bank Group WHO World Health Organization 8 ACKNOWLEDGMENTS This report was prepared by a World Bank team led by Rafael Cortez (Senior Economist, HNPHN) and Meaghen Quinlan-Davidson (Consultant, University College London). This report was part of the products prepared for the World Bank’s Health, Nutrition, and Population Global Practice under the Advisory Services and Analytics (ASA) “Strengthening the strategic purchasing of health care services to improve health system efficiency and equity” and supported by the Korean-World Bank Trust Fund. The team worked under the overall supervision of Tania Dmytraczenko (Health, Nutrition, and Population, Practice Manager, HECHN). The main task as well as this report benefited from helpful peer review comments by Moulay Driss Zine Eddine El Idrissi (Lead Economist, HECHN), Roberto Lunes (Senior Economist, HHNGE), Olena Doroshenko (Senior Economist, HECHN), Elvira Anadolu (Senior Health Specialist, HECHN), and Ahmet Levent Yener (Practice Leader, HECDR). The team is grateful to Jane Brodie for her editorial services The authors are grateful to the World Bank for publishing this report as an HNP Discussion Paper. 9 INTRODUCTION Strategic purchasing of health services has emerged as a critical component in the improvement of health system performance and a guide for health financing reforms. Countries have increasingly focused on this topic to achieve universal health coverage (UHC) (Hanson et al. 2019) and ensure transparency and accountability within the health system (Bramesfeld et al. 2016). In this manner, the health system can incorporate patient needs, ensure that resources are allocated efficiently, and guarantee good quality health care services (Sanderson, Lonsdale, and Mannion 2019). According to the World Health Organization (WHO), strategic purchasing is an active process wherein funds are systematically allocated to health care providers. This involves a “continuous search for the best interventions to purchase, the best providers to purchase from, and the best payment mechanisms and contracting arrangements to pay for such interventions” (Mathauer, Dale, and Meesser 2017). It involves evaluating “the population health needs, the planning and design of health care services, qualifying and selecting appropriate providers, and incentivizing and managing providers to ensure good performance” (Sanderson, Lonsdale, and Mannion 2019). Hanson et al. (2019) posit that failure to address purchasing questions such as what services should be covered, which providers should deliver them, and how they should be paid for could lead to inefficiencies, lower quality, and reduced coverage. Strategic purchasing involves relationships between three groups of actors: patients and purchasers; government and purchasers; and providers and purchasers. Three key policy objectives are associated with these relationships: patient empowerment, effective stewardship, and improved provider performance (Sanderson, Lonsdale, and Mannion 2019) (figure 1). Figure 1. Key Actors, Relationships, and Policy Objectives Needed to Develop Strategic Purchasing in Health Care Source: Sanderson, Lonsdale, and Mannion 2019. 10 Indeed, the way providers are paid can create powerful incentives that influence their behavior; and changes in behavior affect the efficiency, quality, coverage, health outcomes, and finances of clients and the health care system (WBG 2007). Verification protocols are procedures that provide evidence that health care services and health care providers are meeting agreed-upon (between strategic purchaser and provider) conditions as detailed in performance-based agreements or contracts and are often linked to payment for performance. Ideally, a verification system should be developed, and data collected on a regular basis on specific indicators, providing information on inputs and outputs/results. Indicators must be specific, measurable, and verifiable. Too many indicators make verification complex. In some cases, the actual suitability of indicators can only be tested once operationalized (WBG 2012). As such, strategic purchasers are under pressure to measure provider performance and evaluate whether they are delivering services that cover all citizens, are efficient, of quality, are financially viable and sustainable, and are improving health outcomes. This is done to ensure that strategic purchasers are getting value for money and to determine whether provider performance needs to be improved (WBG 2012). Despite this important as well as widespread interest in evaluating provider performance, uncertainty exists about the methods that are used by different strategic purchasers to verify provider performance on efficiency, quality, coverage, financial protection, and health outcomes. This could be attributed to the fact that different stakeholders use different definitions for these constructs. Moreover, there are several methods used to measure these outcomes by strategic purchasers, and it is uncertain how well these methods represent these constructs (Hussey et al. 2009). Given this gap in the literature, the aim of the report is to explore how strategic purchasers verify the monitoring of results of health care providers on efficiency, quality, coverage, financial protection, and health outcomes. The report is organized as follows: Section 2 provides definitions and examples of indicators typically used for efficiency, quality, coverage, and financial protection as offered by the World Bank Group (WBG), WHO, United States Agency for International Development (USAID), the European Union (EU), and the Organisation for Economic Co-operation and Development (OECD). Section 3 is a rapid assessment of verification processes in the following countries: England; Sweden; Ontario, Canada; Norway and Turkey. Section 4 includes a discussion on challenges, lessons learned, recommendations, and conclusions. METHODOLOGY To investigate how international organizations define and measure concepts related to efficiency, quality, coverage, financial protection, and health outcomes, we conducted a desk review of definitions and indicators put forth by international organizations. This involved reviewing documents and websites of international organizations on the topics. To better explore how verification of monitoring of results within strategic purchasing works in practice, we conducted a desk review of country cases using available documents related to health system organization, strategic purchasing, and verification protocols to measure provider performance in England; Sweden; Ontario, Canada; Norway and Turkey. 11 CONCEPTS AND INDICATORS Outcomes typically tied to provider performance include efficiency, quality, coverage, financial protection, and health outcomes. Yet how these concepts are defined and measured vary. The following section provides a summary of selected organizations that offer definitions of these concepts and examples of indicators used to measure these concepts as a way to understand how these are conceptualized. EFFICIENCY Health sector efficiency seeks to capture the extent to which the inputs to the health system, in the form of expenditure and other resources, are used to secure valued health system goals. The pursuit of efficiency is a central objective of policy makers and managers, and to that end better instruments for measuring and understanding efficiency are urgently needed. Ensuring efficiency also reassures payers that their money is being spent wisely and that their claims on the health system are being treated fairly and consistently. Inefficient use of health system resources poses serious concerns including the following (European Observatory on Health Systems and Policies 2016):  Denying health gains to patients who have received treatment because they do not have the best possible care available within the health system’s resource limits.  Denying treatment to other patients who could have benefited from treatment if the resources had been better used.  Sacrificing loss of consumption opportunities elsewhere in the economy.  Reducing society’s willingness to contribute to the funding of health services, thereby harming social solidarity, health system performance, and social welfare. The WBG (2018) defines efficiency as comparing levels of inputs used in the health system in relation to the output produced to measure health system efficiency (WBG 2018). Examples of indicators used by WBG to measure efficiency include the following:  Hospital bed occupancy rate  Mean number of doctors per bed  Outpatient visits per day According to the WHO (Chisholm and Evans 2010), health system efficiency is defined as “attaining the highest level of health possible with the available resources (together with the realization of other key health system goals around financial protection and responsiveness to the needs of service users).” The WHO also identifies that often “efficiency is also assessed in (more intermediate) terms as the amount or mix of service outputs that can be produced within a fixed budget.” The organization also discriminates between technical efficiency or improving the way in which inputs to health care are optimized (“doing things the right way”), and allocative efficiency, or how well the outcomes of health care provision are distributed among the population (“doing the right thing”) (Chisholm and Evans 2010).  An example of an indicator for technical efficiency is total health expenditure per capita (public and private) (Tandon et al. 2000). Heredia-Ortiz (2013) defines efficiency in terms of productive efficiency or using technologies and techniques of production to ensure the highest possible output for the available inputs (reducing waste); technical efficiency or ensuring that the desired output is produced with the least cost combination of inputs (i.e., keeping hospital length of stay 12 down to a level that ensures safe and appropriate discharge, thus saving money); and allocative efficiency or ensuring that the mix of goods and services provided (given technical and productive efficiency) is the mix that is of most value to society (maximizing social value). Examples of indicators include the following (Heredia-Ortiz 2013):  Health worker attendance rates  Amount of time spent with patients in one visit  Percentage of drugs purchased by the Ministry of Health (MoH) that are generic  Average length of stay in the hospital The European Commission (2019) defines efficiency as “how well a health care system uses its resources (inputs) at its disposal to improve population health (outcome) and attain related goals.” The European Commission goes on to say that efficiency is “the ratio between health system inputs (costs, in the form of labour, capital or equipment) and either outputs (e.g., number of patients treated) or health outcomes (e.g., life years gained).” It also discriminates between technical efficiency or “the capacity of an entity within the health care system to produce its chosen outputs given its resources” and allocative efficiency, or “deploying the right mix of outputs (or inputs) that maximizes welfare according to societal preferences.” Examples of indicators include the following:  Number of health workers  Nurse to physician ratios  Costs per bed/doctor  Number of hospitalizations/discharges  Number of bed-days  Length of stay for specific conditions The OECD (2010) defines efficiency as “a comparison of inputs with outputs or outcomes of the health care system to assess the degree to which goals are achieved while minimizing resource usage.” They state that “efficiency measures could be assessed at three levels: the disease, sub-sector, and system level. The disease level approach focuses for each disease on the gains in the health status brought by the health care system. The sub-sector approach focuses on the gains brought specifically by hospitals, out-patient care and pharmaceuticals, while the system level approach relies on a holistic view” (OECD 2010). Examples of indicators include the following:  Disease-specific average length-of-stay in hospital  Number of cases per available acute care bed  Occupancy rate for acute care beds QUALITY Quality of care, according to the WHO, OECD, and WBG (2018) is defined as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. It implies that quality of care can be measured, is ultimately aimed at health improvements rather than simply increasing service inputs or refining system processes, and should reflect the desires of key stakeholders, including service users and communities.” It includes curative and preventive care, as well as community and facility-based care. Characteristics of health systems indicative of quality include effectiveness, safety, people-centeredness, timeliness, equity, integration, and efficiency (WHO, OECD, and WBG 2018). 13 The WBG (2017) collected and analyzed data on quality health care through the Service Delivery Indicators (SDIs) in eight African countries (Kenya, Madagascar, Mozambique, Nigeria, Senegal, Tanzania, Togo and Uganda). The categories on quality include provider competence and knowledge, proxies for effort, and availability of key infrastructure and inputs. The following include examples of SDI indicators (WBG 2017):  Knowledge and ability: o Adherence to clinical guidelines (percentage) o Clinical management of maternal and newborn complications  Effort: o Caseload o Absence from health facility (percentage) • Key infrastructure and inputs: o Drug availability (all), first-level facilities (percentage) o Minimum equipment, first-level facilities (percentage) The OECD (2017) measures quality through the following domains: primary care, acute care, mental health care, cancer care, patient safety, and responsiveness and patient experiences. The OECD uses the following health care quality indicators, among others (OECD 2017):  Primary health care quality indicators: Asthma hospital admission rates  Acute care: Rates of death within 30 days following hospital admissions for acute myocardial infection  Mental health care: Inpatient suicides among people diagnosed with a mental disorder  Cancer care: Breast cancer five-year net survival  Patient safety: Number of hospital-acquired infections  Responsiveness and patient experience: Consultation skipped due to costs COVERAGE The WHO’s (2018) Global Reference List of 100 Core Health Indicators includes service coverage indicators. The aim of the Global Reference List is to “serve as a normative guidance for the selection of standard indicators and their definitions that countries and partner stakeholders can use for monitoring in accordance with their respective health priorities and capacity. Service coverage indicators reflect priorities across the spectrum of health services including reproductive, maternal, newborn, child and adolescent, immunization, HIV [human immunodeficiency virus], TB [tuberculosis], malaria, neglected tropical diseases, NCDs [noncommunicable diseases], mental health and substance abuse” (WHO 2018). Examples of indicators include the following:  Coverage of diarrhea treatment  Postpartum care coverage—women  Antiretroviral therapy (ART) coverage  Treatment coverage for drug-resistant TB  Coverage of services for mental health disorders  Treatment coverage for alcohol and drug dependence The WHO and WBG (2017) define effective service coverage as “the proportion of people in need of services who receive services of sufficient quality to obtain potential health gains.” Effective coverage indicators illustrate a country’s endeavor to ensure that the 14 population receives quality health care services that meet their needs. It is the preferred indicator for monitoring service coverage within UHC. As the WHO and WBG (2017) point out, one of the challenges to effective coverage is the lack of available indicators, due to a lack of resources in collecting data on the topic or defining an indicator for a particular health service. When this is the case, “other indicators associated with effective coverage must be used” (WHO and WBG 2017). Service coverage indicators, defined as “the proportion of people in need of a service that receive it, regardless of quality,” are more commonly used than effective coverage indicators. This is problematic as it does not measure the quality of services (WHO and WBG 2017). FINANCIAL PROTECTION According to the WBG (2019), financial protection means that people no longer need to sell assets or borrow to meet health payments. It is beneficial for everyone and frees individuals from making precautionary savings, stimulating expenditures on other goods and services. Financial protection directly reduces the number of people living in poverty, stimulates economic growth, and boosts human security. Expanding financial protection immediately reduces the probability of people falling into poverty by paying for health services out-of-pocket (OOP). Indeed, while OOP spending can be catastrophic and impoverishing to people at all income levels, people living close to the poverty line can be pushed into poverty even by small expenditures. Ways in which countries can advance financial protection include (a) drawing on funds from prepaid and pooled sources with subsidies for people who cannot afford to contribute to ensure financial protection for guaranteed services; and (b) providing a guaranteed set of health services to all people at an affordable price, aiming toward zero or nominal OOP payments along with strong protection mechanisms (WBG 2019). The WBG has the Health Equity and Financial Protection Indicators (HEFPI) dataset (Wagstaff et al. 2019). Examples of financial protection indicators include the following:  The incidence of catastrophic health expenditures (defined alternately as expenditures exceeding 10 percent of consumption or income, or as expenditure exceeding 25 percent of consumption or income)  The incidence of impoverishing expenditures (expenditures without which the household would have been above the poverty line but because of the expenditures is below the poverty line (with two absolute poverty lines (US$1.90 per day and US$3.20 per day in 2011 purchasing power parity [PPP] dollars and one relative poverty line—50 percent of median consumption or income)  Mean annual household per capita OOP expenditure in 2011 PPP dollar terms HEALTH OUTCOMES As part of the WHO’s (2018) Global Reference List of 100 Core Health Indicators, there are health status indicators that include core indicators including mortality by age, sex, and cause (WHO 2018). Examples of indicators include the following:  Maternal mortality ratio  AIDS-related mortality rate  Adolescent birth rate  Malaria incidence rate  Cancer incidence, by type of cancer 15 COUNTRY CASE STUDIES The following section provides a summary of countries’ (England; Sweden; Ontario, Canada; Norway; Turkey) health care systems, identifying strategic purchasers and providers, and determining the instruments each country uses to verify health care provider performance. The objective is to explore how strategic purchasers in these countries are verifying provider performance. ENGLAND Overview of Health Care System As part of the United Kingdom, England’s National Health Service (NHS) was established in 1948 with services available to all residents. Funded mainly through general taxation, services are free at point of use and provided through public clinics and hospitals. NHS England (NHSE) does receive some funding from private medical insurance and OOP payments. For example, there is cost-sharing for certain services (i.e., dental care and pharmaceuticals) and charges for prescription medications (Cylus et al. 2015; HQIP 2017). Despite its mandate to provide comprehensive health services, coverage for specific health services varies (Cylus et al. 2015). NHSE has its own advisory, planning, and monitoring framework for the health system. In fact, as part of the NHS Constitution, patients have the right for the NHS to “monitor and make efforts to continuously improve the quality of health care they commission or provide. This includes improvement to safety, effectiveness, and experiences of services” (HQIP 2017). Funding for NHSE is directly allocated by the UK government to the Department of Health and Social Care (DHSC). See figure 2 for the flow of funding to the NHSE (DHSC 2019). In 1990, England introduced a split between purchasers and providers of health services. The DHSC has overall financial control, oversight (including strategic direction and policy development), and leadership on NHSE’s performance and delivery of services (DHSC 2019). Strategic Purchasers and Providers The DHSC allocates funds to NHSE, which are then distributed—through weighted capitation—to clinical commission groups (CCGs) and specialist and primary care services. The delivery and quality of NHS and care services—as established in the Health and Social Care Act 2012—is under the remit of the NHSE and NHS Improvement (NHSI). These organizations are responsible for the following (Cylus et al. 2015):  Improving health outcomes and quality of care through national leadership  Managing and administering resources to CCGs  Purchasing primary care and specialized services The NHSE is responsible for not only contracting and purchasing primary health care services but also some nationally based functions previously under the remit of the DHSC. It is also responsible for ensuring that the local health services are financially sustainable (DHSC 2019). The NHSE strategic purchasers, or commissioners, as they are called in England, are composed of CCGs and the NHSE. CCGs are led by general practitioners (GPs). They purchase services for urgent and emergency care; elective hospital care; community 16 health; mental health; and maternity, newborn, and children’s health care. Having GPs lead the CCGs is strategically important as they engage more with patients on a frequent basis and usually have a better idea of patients’ needs. CCGs are supported in purchasing the services by commission support units, strategic clinical networks, and multi- professional advisory groups (“clinical senates”). Wherein CCGs purchase primary care services, NHSE instead purchases the services to avoid conflicts of interest (as GPs are both purchasers and providers) (Cylus et al. 2015). Figure 2. Flow of Budgeted Funding in the Health and Care System (2018–2019) Source: DHSC 2019. Note: NHS = National Health Service. These services are purchased from a range of providers (that must be registered with a regulating body) from the public and private sectors, including public hospitals (NHS Trust and Foundation Trusts), and community and mental health providers. The Health and Social Care Act 2012 requires that CCGs and NHSE continuously improve service quality, health outcomes, effectiveness, safety, patient experience, and meet the local population’s needs (Health and Social Care Act 2012). CCGs are also pushed to purchase services that are of better value, cost-effective, and reduce inequalities (HQIP 2017). Instruments Used to Verify the Monitoring of Results To create incentives for quality and efficiency within the health care system, the NHSE uses a National Tariff Payment System (NTPS), formally known as the payment by results (PbR) system. The NTPS ties provider performance to certain quality and efficiency goals (Cylus et al. 2015). 17 The NTPS is a system that pays “NHS health care providers a standard national price or tariff for each patient seen or treated” (NHS 2013). There are two features to the system: nationally determined currencies and tariffs that are agreed upon between NHSE and NHSI. According to the NHS (2013), currencies are a health care unit for which a payment is made. These units can be made up of a number of different forms, ranging from time periods covering outpatient attendance to a hospital stay, a year of care for a long-term condition, to a bundle of services for groups of patients and treatment for an individual patient (NHS 2013; NHSE and NHSI 2019a). Tariffs are the “set prices paid for each currency” (NHS 2013). The NTPS covers all types of NHS health care, whether they are commissioned by CCGs, NHSE, or local authorities on behalf of the NHS, and are physical or mental health services delivered in primary care or through acute care. In 2013, the NTPS represented approximately 60 percent of acute hospital income and one-third of primary care trust budgets (NHS 2013). It was developed to ensure that patient choice was supported and that the required standards of care for efficiency and quality were met by rewarding health care providers for meeting these standards at a lower cost; to reduce waiting times; and to focus discussions between commissioners and providers on quality and efficiency instead of on price. The NTPS does not cover (a) public health services; (b) primary care services (payment is determined by regulation or directions); (c) personal health budgets; (d) integrated health and social care; and (e) incentives and sanctions specified in the contract (e.g., Commissioning for Quality and Innovation [CQUIN] payments, which are based on provider performance after the provider’s income has been determined by the NTPS). It should be noted that maternity services and emergency care as of 2019/20 are not included in currencies and national prices, as the prices for these services are nonmandatory and subject to local pricing (NHSE and NHSI 2019b). For example, the currency for admitted patient care, outpatient procedures, and accidents and emergencies (A&E) is the healthcare resource group (HRG). This is “clinically similar groups of conditions or treatments that consume similar levels of NHS resources,” and each HRG covers a period of care, from admission to discharge (NHS 2013). To categorize patients into case-mix groups around HRGs requires the use of the following codes: diagnosis codes, procedure codes, treatment codes, and investigation codes. This process is collected through grouper software, developed by NHS Digital. This allows “tariffs to be set at a sensible and workable level” (NHS 2013; NHSE and NHSI 2019b). Also, the currency for outpatient attendance is based on clinical specialty, with the attendance type defined by treatment function codes (TFC) (NHS 2013; NHSE and NHSI 2019c). The NTPS relies on collected patient-level data, requiring information on clinical activity and its associated currency. For example, when a patient is discharged from the hospital, his or her care is translated into codes. The NHS uses two classification systems: The International Classification for Diseases-10 for diagnoses and the Office of Population Censuses and Surveys-5 for operations, procedures, and interventions. For outpatient appointments, TFC are recorded. Along with the patient’s age and length of stay at the hospital, this information is sent to the Secondary Uses Service (SUS), a national database that “allows commissioners to pay providers for the work they have done, or to adjust any regular monthly payments for actual activity undertaken” (NHS 2013). In fact, NHS Digital has helped improve the completeness and quality of SUS data by developing 18 dashboards for users to monitor provider performance, including coverage and quality (NHS Digital 2019). Data and indicators that the SUS regularly collects include examples of the following (NHSE and NHSI 2019a):  Total number of days that the patient received advanced respiratory support during adult critical care  For neonatal patients, the level of special or intensive care provided (0 = Normal care; 1 = Special care; 2 = Level 2 intensive care [high dependence]; 3 = Level 1 intensive care)  Number of days spent in pediatric critical care unit Traditionally, tariff prices have been based on reports by NHS providers on the average costs of services in the mandatory reference costs collection. In practice, however, final tariff prices may not reflect national averages as adjustments are made to the average of reference costs and reflect the market forces factor (MFF). Indeed, these reference costs are usually made three years earlier, so that adjustments are applied to the current prices and reflect efficiencies and pressures of the system (NHS 2013). The tariff also accounts for the complexity of the case, the expense based on the region of the country, length of stay in the hospital, and specialized services (NHS 2013). The NHS has also developed best practice tariffs (BPTs) to incentivize providers to improve the health outcomes of cohorts of patients (NHS 2013; NHSE and NHSI 2019b). Examples of BPTs for acute stroke care include the following:  Rapid brain imaging: £399  Direct admission and 90 percent of spell spent in an acute stroke unit: £1,026  Alteplase: £840 Prices for health care services are set in this way to ensure that the patient is receiving quality health care, delivered in the most efficient manner. According to NHSE and NHSI (2019b), national prices should demonstrate efficient cost. These prices should also illustrate to the commissioners that they are getting value for money with their budgets, giving them the tools to make decisions about the best mix of services, incentivizing and encouraging providers to reduce their cost and provide more efficient and effective services. Based on evidence and cost trends reported by providers, for 2019/20 the NHS has set an efficiency factor of 1.1 percent (NHSE and NHSI 2019b). Commissioning for Quality and Innovation (CQUIN) is a national framework for quality improvement schemes as decided at the local level. CQUIN is used by commissioners to financially reward health care providers for providing quality services. The schemes for 2019/20 are categorized under four key areas: prevention of ill health, mental health, patient safety, and best practice pathways. The following are examples of indicators that CQUIN schemes collect (NHSE and NHSI 2019c):  Prevention of ill health: Achieving an 80 percent uptake of flu vaccinations by frontline clinical staff  Mental health: Achieving 80 percent of adult mental health inpatients receiving a follow-up within 72 hours of discharge  Patient safety: Achieving 80 percent of older inpatients receiving key fall- prevention actions 19  Best practice pathways: Sentinel Stroke National Audit Programme six-month review for all discharged stroke patients At the same time, the NHS has prioritized the importance of oversight and how regional teams review performance, determining the support needed to ensure sustainability, collaboration, and integration. As such, the NHS Oversight Framework is an annual assessment of every CCG to ensure support and dialogue between NHSE, NHSI, CCGs, providers, and sustainability and transformation partnerships. Table 1 provides examples of oversight metrics and the organizations responsible for the oversight by five key areas: new service models, preventing ill health and reducing inequalities, quality of care and outcomes, leadership and workforce, and finance and use of resources (NHSE and NHSI 2019a). For a full list of the indicators, please see the NHS Oversight Framework 2019/20 (NHSE and NHSI 2019a). Cost-improvement programmes (CIPs) are also implemented within the NHSE to ensure that health care providers are productive, and services are efficient and effective. According to the NHS, provider performance report for the first quarter of 2019, the NHSE reported that with CIPs, providers achieved savings of £3.2 billion or 3.6 percent. Also, NHSI reported that in the fourth quarter, provider productivity was estimated at 2.3 percent, well above last year’s productivity of 1.2 percent. For this financial year, the cost- improvement schemes (both recurrent and nonrecurrent) linked to operational productivity program areas were estimated to be £1.8 billion, up from the £1.5 billion achieved in 2017/18 (NHSI 2019). The Getting It Right First-Time program also works to promote efficiency and productivity gains within the health system. The program aims to promote the reduction of unwarranted variations in the way services are delivered to improve quality and productivity. To support this work, the provider sustainability fund has dedicated £19 million (NHSI 2019). The provision of quality services is an important policy area and priority for the NHSE and NHSI. As evidence, the NHS Standard Contract between purchasers and providers explicitly requires health care providers to undertake local clinical audits and participate in national clinical audits (HQIP 2017). These audits focus on a “quality improvement cycle that involves measuring the effectiveness of health care against agreed and proven standards for high quality and acting to bring practice in line with these standards so as to improve the quality of health care and health outcomes” (HQIP 2017). Local clinical audits are usually undertaken by the health care providers in the facility being audited; however, there may be times where strategic purchasers appoint a third-party auditor to carry out the audit. At the same time, NHS-commissioned health care services are required to publish an annual quality account, which includes information about service quality. Information includes the number of clinical audits and national confidential enquiries that the provider has participated in (participation rate); the number of national and local clinical audit reports reviewed by the Provider Board; and actions the provider intends to take to improve the services. For 2020/21, the following criteria have been specified to be included in the quality accounts (HQIP 2017):  Coverage: Collects data from at least 70 percent of eligible services nationally  Data: Collected on individual patients  Comparisons of providers (trusts, hospitals, networks) 20  Plan to recruit patients during the following financial year  Public reporting: Comparison of providers’ performance published within 12 months of completion of most recent clinical event  Outcomes and processes of care being audited must be based on rigorous evidence (including National Institute for Health and Care Excellency [NICE] quality standards and guidelines) Table 1. Oversight Metrics 2019/20 Areas Indicators Oversight New Service Models Integrated primary care and Patient experience of GP services CCGs community health services Acute emergency care and Percentage of patients admitted, CCGs and transfers of care transferred, or discharged from A&E within providers four hours Personalization and patient Personal health budgets CCGs choice Preventing ill health and reducing inequalities Smoking Maternal smoking at delivery CCGs Obesity Percentage of children 10–11 years CCGs classified as overweight or obese Falls Injuries from falls in people 65+ years CCGs Antimicrobial resistance Antimicrobial resistance: appropriate CCGs prescribing of antibiotics in primary care Health inequalities Proportion of people on GP severe mental CCGs illness register in receiving physical health checks in primary care Quality of care and outcomes General Provision of high-quality care: hospitals CCGs and providers Evidence-based interventions Maternity services Neonatal mortality and stillbirths CCGs Cancer services Cancers diagnosed at an early stage CCGs Mental health Improving access to psychological CCGs and therapies—recovery providers Learning disability and Reliance on specialist inpatient care for CCGs autism people with a learning disability and/or autism 21 Areas Indicators Oversight Diabetes Diabetes patients that have achieved all the CCGs NICE-recommended treatment targets: three (HbA1c, cholesterol, and blood pressure) for adults and one (HbA1c) for children People with long-term Estimated diagnosis rate for people with CCGs conditions and complex dementia needs Planned care Patients waiting 18 weeks or less from CCGs and referral to hospital treatment providers Leadership and workforce Quality of leadership CCGs and providers Finance and use of resources In-year financial performance CCGs and providers Source: NHSE and NHSI 2019a. Notes: GP = General practitioner; CCGs = Clinical commission groups. NICE = National Institute for Health and Care Excellency; A&E = Accidents and emergencies. The NHS Standard contract also explicitly states that all health care providers participate in the National Clinical Audit and Patient Outcomes Programme (NCAPOP). These are centrally commissioned audits managed and developed by the Healthcare Quality Improvement Partnership (HQIP) on behalf of NHSE. They are national clinical audits “that measure provider performance against national quality standards or evidence-based best practice and allow for comparisons to be made between provider organizations to improve the quality and outcomes of care” (HQIP 2017). Table 2 provides examples of the indicators that NCAPOP measures at the national (national improvements in quality and health outcomes); system (how the project supports policy development and system management); local (how the project stimulates quality improvement); and public (how the project is used by the public and the demand for it) levels. Further information can be found at HQIP’s Impact Report 2019 (HQIP 2019). In January 2019, the NHS Long Term Plan was published. A new service model was developed that will be implemented over the next five years, so that “every patient will have the right to online “digital’ GP consultations, and redesigned hospital support will be able to avoid up to a third of outpatient appointments—saving patients 30 million trips to hospital and saving the NHS over £1 billion a year in new expenditure averted. GP practices will be funded to work together to deal with pressures in primary care and extend the range of convenient local services, creating genuinely integrated teams of GPs, community health, and social care staff. Now expanded community health teams will be required under new national standards to provide fast support to people in their own homes as an alternative to hospitalization, and to ramp up NHS support for people living 22 in care homes. Within five years over 2.5 million more people will benefit from ‘social prescribing,’ a personal health budget, and new support for managing their own health in partnership with patients’ groups and the voluntary sector” (NHS 2019). Table 2. NCAPOP List of Indicators (2017–2019) Health Level of Level of Indicators Indicators outcome system system Stroke National Median % of days as an Local Team-level output inpatient on which downloads physiotherapy is received National National Median delay between Local Consultants who Vascular symptom and carotid registered on the NVR IT Registry endarterectomy system carcinoembryonic antigen (CEA) test Neonatal: National Rate of babies born at <32 Local Follow-up assessments National weeks gestation who have for at-risk neonates in Neonatal a temperature recorded district general hospitals Audit within an hour of admission Programme within the recommended (NNAP) range from 36.5°C to 37.5°C Maternal, National Perinatal mortality rate Local Establishment of a Newborn, multidisciplinary team to and Infant, identify recurring themes Clinical and develop actions to Outcome reduce stillbirth rates Review Program (CORP) Chronic National Secondary care: inpatient Local Number of action plans obstructive mortality submitted by hospitals pulmonary that participated in the disease secondary care audit (COPD) Dementia National Percentage of patients with Local Number of regional dementia receiving quality improvement nutritional assessment workshops to increase awareness of how to apply quality improvement methods to local audit findings Prostate National Percentage of men who had Local Number of action plans cancer a multiparametric MRI developed to ensure that (mpMRI) senior clinicians and other members of the Multidisciplinary Team 23 Health Level of Level of Indicators Indicators outcome system system (MDT) complete accurate data on cancer stage and tumor grade for submission to NPCA Cardiac National Percentage of cases that Local Number of hospitals had radial artery access conducting pilot programs to improve timeliness of treatment Mental National Sudden unexplained deaths Local Number of safety score Health (SUD) of inpatients cards produced annually for each NHS trust in England Source: HQIP 2019. Notes: NVR = Network video recorder; MRI = Magnetic resonance imaging; NPCA = National Prostate Cancer Audit; NHS = National Health Service. SWEDEN Overview of Health Care System The Swedish health care system is organized in three levels: the national level or the Ministry of Health and Social Affairs, where overall health and health care policy is developed and coordination occurs between the other government agencies; the regional level, comprising twelve county councils and nine regional bodies responsible for health financing and service delivery to all Swedish citizens; and the local level, comprising 290 municipalities, responsible for the care of the disabled and elderly. The Swedish Association of Local Authorities and Regions (SALAR) represents the local and regional authorities (Glenngard 2017) (figure 3). To cover the health care services provided, health care at the regional (county council) and local (municipality) levels is funded through proportional income taxes. It is also funded through state grants (financed by national income taxes or indirect taxes) and user charges. Health care services vary by council as they are tailored to meet their catchment area needs and are generally allocated based on a formula that accounts for diagnosed illnesses and socioeconomic conditions of the county (Anell, Glenngard, and Merkur 2012; Glenngard 2017). Similarly, payment to health care providers varies by county council. For primary health care providers, payments are widely based on capitation, complemented with performance-based payments and a fee for service (FFS). Meanwhile, physicians, nurses, and other health care providers tend to be salaried, while hospital payments are based on “global budgets or a mix of global budgets, case-based and performance-based payment.” Most hospitals are owned and operated by county councils (Anell, Glenngard, and Merkur 2012). At the primary and specialist levels, patients are charged flat-rate payments, with the national ceiling for OOP capped at US$120 over a period of 12 months (Glenngard 2017). 24 Patients under 20 years of age are exempt from these user fees (Anell, Glenngard, and Merkur 2012). Figure 4 illustrates the financial flows (Anell, Glenngard, and Merkur 2012). Figure 3. Organization of the Swedish Health System Source: Glenngard 2017. As county councils are responsible for the health care services in their area, relationships between purchaser and purchaser-provider, as well as the number of different private providers, vary by county council. Based on reforms in the 1990s, some counties opted to introduce a purchaser-provider split model, while other counties maintained the “traditional system of fixed annual allocations to hospitals and primary care services.” The purchaser- provider split model included allocation of resources to purchasers based on their residents’ needs, “negotiated and per-case payment schemes to providers, and total cost responsibility for providers through the use of internal transfer prices for services” (Anell, Glenngard, and Merkur 2012). For private providers to be reimbursed for their services, they must have an agreement with the county council. If they do not, the costs of the service falls on the patient. Health care providers at hospitals enter into financial and activity contracts with the purchaser, usually based on fixed prospective per-case payments and complemented by price or volume ceilings and quality components. These prices are based on historical costs and negotiations between purchasers and providers and vary by county councils. Payment to primary care providers is regulated through conditions for accreditation, such as minimum clinical competences. Payment is made via capitation for registered patients (Anell, Glenngard, and Merkur 2012). County councils also set copayment rates, which range from US$5.5 for hospitalization per day to US$33.0 for primary care physician visit (Glenngard 2017). School-age children are exempt from paying for vaccinations and health exams and consultations at primary care clinics. Also, women attending regular antenatal primary care checkups are exempt from user fees. Strategic Purchasers and Providers As strategic purchasers, county councils and municipalities are responsible for controlling the cost of health care services. As such, before strategic purchasers enter into a contract with a private provider, a tendering process occurs, with cost used to evaluate the provider. 25 Health service cost is also controlled through “global budgets, volume caps, capitation formulas, and contracts, as providers retain responsibility for meeting costs with funds received through those prospective payment mechanisms” (Glenngard 2017). County councils also prioritize quality of health care services as it relates to implementing evidence-based clinical guidelines and coordination, such as patient satisfaction, continuity, enrollment of national registers with performance-related payment linked to this (both in the public and private spheres) (Glenngard 2017). Figure 4. Financial Flow of Swedish Health Care System Source: Anell, Glenngard, and Merkur 2012. Note: OTC = Over the counter. 26 “County councils, in their role as purchasers, use contracts to allocate resources and tasks to health care providers.” This includes stipulations (financial, organizational, and quality requirements) that primary health care providers must comply with to practice and be paid. The requirements are decided upon by the county council (Glenngard 2019). For outpatient specialist care, health care providers (public and private) receive fixed, prospective, per-case payments (based on diagnosis-related groups, or DRGs). This is supplemented with quality components and price and volume ceilings (Glenngard 2017). Hospitals are paid and reimbursed through global budgets or a mix of payment mechanisms including global budgets, DRGs, and pay for performance (Glenngard 2017). In 2010, the country implemented a market reform of primary care. Since then, the objective of accessibility to services was achieved. However, due to challenges associated with accurate reporting and monitoring, it is not clear how the reform has affected efficiency and quality of health care services (Glenngard 2017). Instruments Used to Verify the Monitoring of Results County councils use results from the National Patient Survey (NPS) to allocate resources to pay for performance to health care providers. They also use performance measurement systems (PMS) to monitor health care providers. Between 15 and 80 indicators that are mostly structure and process measures are used continuously to monitor health care provider performance. Examples of structure and process measures include the following (Glenngard 2019):  Proportion of newborns who receive home visits  Compliance with clinical guidelines regarding prescription of antibiotics and/or other drugs Meanwhile, examples of outcome measures used (Glenngard 2019):  Avoidable hospitalization rates among elderly The Primary Care Quality framework, which is a collaboration between county councils led by the Swedish Association of Local Authorities and Regions (SALAR), is also used by several county councils to allocate resources to pay for health care provider performance. Approximately 80 indicators are used that focus on productivity, working methods, and quality improvements. The most common are on accessibility, coverage rate, prevention, and compliance with various clinical guidelines. In fact, it was reported that in 2016/17, 14 of the 21 county councils used pay for performance to promote quality improvements (Glenngard 2019). ONTARIO, CANADA Overview of Health Care System The Canadian health system is a universal single-payer public system for all residents. The primary responsibility of the Canadian health system lies with the provinces and territories. As set out in the Canada Health Act 1985, the federal government cofunds the health system in each province and territory. To receive federal funds, provincial and territory health programs must be universal, publicly administered, accessible, and portable across provinces and territories, and provide comprehensive coverage (Allin and Rudoler 2017). In 2010, 71 percent of Canada’s health spending was publicly funded. However, Canada’s health care delivery system is largely private. Most physicians are independent contractors who are reimbursed by the provincial or territorial health plan on 27 an FFS basis. Almost all hospitals are owned and operated by private not-for-profit entities (Hutchison and Glazier 2013). Figure 5 provides an overview of the organization of the Canadian health system. Strategic Purchasers and Providers The Canada Health Transfer (CHT) provides federal funding to provincial and territory health systems. The CHT transfers funds on an equal per capita basis and includes both cash and tax point transfers. In Ontario, the CHT transfers funds to the Ministry of Health and Long-Term Care (MOHLTC) health care, with payments transferred to (Allin and Rudoler 2017):  Local health integration networks (LHINs), which are responsible for hospitals, long- term care homes, community care, community support services, community health centers, and addiction and mental health agencies.  Emergency Medical Services (EMS)  Public Health Unit (PHU)  Agencies  The Ontario Health Insurance Plan (OHIP): the government-run health plan, paid through taxes. OHIP pays for primary care and specialists, with most basic and emergency services covered. Figure 5. Organization of the Canadian Health System Source: Adapted from GP Marchildon “Canada: Health System Review,” Health Systems in Transition, vol. 15 no.1, 2013. P.22 28 To improve access, quality, and continuity of care, the Primary Care Reform was implemented in Ontario in the 1990s to replace traditional FFS to GPs and introduce a menu of payment models. The most common forms of remuneration were FFS, capitation, and salary; and these models had the following common components (Marchildon and Hutchison 2016; Sweetman and Buckley 2016):  Patient enrollment (rostering)  Group requirement of three or more physicians for most models  Provision of after-hours care for most models  Physician choice of payment model The models included in the reforms are the following, with individual physicians voluntarily enrolling in one model at a time. The level of payment rates and levels in these models are determined by the government of Ontario and the Ontario Medical Association (OMA) (Sweetman and Buckley 2016) (table 3):  Family Health Group (FHG), which consists of FFS payment, a small rostering fee, after-hours premiums, comprehensive care premiums for a select set of fee codes, and incentives and bonuses (payments for performance [P4P]), group requirement, optional rostering.  Comprehensive Care Model (CCM), which has similar components as the FHG but for physicians on their own (solo).  Family Health Organization (FHO): A blended capitation model with about 70 percent remuneration. This includes a restriction on noncapitated billings for GPs remunerated by capitation.  Family Health Network (FHN): Similar to the FHO model but with a smaller capitated basket with capitation representing over 60 percent of total remuneration. This includes a 50 percent reduction in capitation payments if one physician has more than 2,400 patients. Table 3. Models within the Ontario Health System Model (year Professional After-hours Description Remuneration Rostering established) team requirement Family Health Three or more FFS (blended Yes Limited Required Group (2003) physicians with targeted providing incentives and primary care bonuses) to rostered patients with after-hours coverage provided through a combination of limited direct service and telephone advisory services Comprehensive Solo physician No Not required Care Model providing (2005) primary care to rostered 29 patients with some after- hours care Family Health Three or more Capitation Limited Required Network (2002) physicians (blended with providing targeted primary care incentives and to rostered bonuses) patients with after-hours coverage provided through a combination of limited direct service and telephone advisory services Family Health Same as Same as above Limited Required Organization above but with a (2007) broader basket of primary care services included in the capitation model Rural and Special Salary (blended Limited Required Northern arrangement with targeted Physician for primary incentives and Group (2004) care bonuses) physicians dealing with dispersed population in rural and northern areas Nurse Teams led by Salary payment Yes to the Limited Not required Practitioner– nurse and FFS clinic Led Clinic practitioners (collaborating (2007) (with physicians) collaborating physicians) to provide primary care services in communities with high numbers of patients not attached to a family physician Family Health Interprofessio Blended Yes Yes Required (as Team (2005) nal teams that capitation (FHO per go beyond or FHN model) 30 doctors and or blended remuneration nurses to salary model) include other (physicians), health salary (other providers health (e.g., nurse professionals practitioners, dieticians, pharmacists, social workers, psychologists, occupational therapists) Sources: Hutchison and Glazier 2013; Glazier, Hutchison, and Kopp 2015; Marchildon and Hutchison 2016; Price et al. 2015. Notes: FFS = Fee for service; FHO = Family Health Organization; FHN = Family Health Network. To counterbalance incentives provided by any one remuneration system, this was blended with elements of other systems. Furthermore, pay-for-performance financial incentives have been offered in all models to encourage desirable behaviors including the provision of after-hour coverage for rostered patients, the provision of targeted services (e.g., mental health care, palliative care, cancer screening), and the establishment of key primary care infrastructure including the implementation of electronic medical records (Marchildon and Hutchison 2016). Between 2007 and 2009, total payments to primary care physicians increased by 32 percent (compared to a 23 percent increase in overall provincial government health care expenditures), related mainly to the introduction and spread of the new reimbursement models. Mean payments per full-time equivalent primary care physician (unadjusted for inflation) increased by 31 percent between 2005 and 2009, compared to an increase of 25 percent for all Ontario physicians. By 2015, less than a quarter remained in traditional FFS, and almost half of those FFS physicians provide specialized services (e.g., palliative care, sports medicine, hospitalist care, psychotherapy) rather than full-service primary care (Marchildon and Hutchison 2016). Instruments Used to Verify the Monitoring of Results Ontario lacks a coherent system for ongoing primary care performance measurement and feedback at the practice, organization, and system (community, regional, and provincial) levels (Hutchison and Glazier 2013). In its 2011 annual report, the provincial auditor general concluded that the province had not received value for money for its significant new expenditures on primary care doctors (Marchildon and Hutchison 2016). At the same time, it is not clear how practice patterns, access, cost-effectiveness, or quality have changed as a result of the reform. Indeed, there was no performance measurement system linked to the reforms, thereby preventing an ongoing systematic evaluation of outcomes. Rather, evaluations on dimensions have been conducted on a piecemeal basis by external actors, including academics and organizations that are arm’s length from the MOHLTC. For example, Hurley and colleagues (2011) at McMaster University found that pay-for-performance incentives led to an increase (mostly modest) over baseline levels in the provision of four of five preventive services but that the special payments for the provision of priority services (e.g., obstetrics, palliative care, home visits) above specified 31 thresholds had no effect (Hurley et al. 2011; Marchildon and Hutchison 2016). Glazier and colleagues (2015) at the Institute for Clinical Evaluative Studies found improvements in cervical cancer screening and diabetes care between 2004/05 and 2011/12 in all organizational and payment models, with FHT and the capitation-based models (FHOs and FHNs) outperforming blended FFS and traditional FFS on almost every measure (Glazier, Hutchison, and Kopp 2015; Marchildon and Hutchison 2016). In a five-year evaluation of FHTs, organizational structures and processes associated with high primary care performance were strengthened between 2009 and 2012, but patient-reported outcomes across multiple domains of patient experience (access, care coordination, patient and family centeredness, prevention and health promotion, and support for management of chronic conditions) were unchanged (Marchildon and Hutchison 2016). A longitudinal evaluation of population-based data (approximately 10.7 million patients) found that patients in team-based capitation practices were more likely to receive appropriate diabetes care and to be screened for cervical, breast, and colorectal cancers in the final year of the study period than those patients in enhanced FFS (FHG and CCM) models, even after adjustment for patient and physician characteristics. Patients in team- based capitation practices showed greater improvement in recommended diabetes care and cervical cancer screening than patients in nonteam capitation practices and enhanced FFS practices (Kiran et al. 2015). Outcome indicators and provider performance have until recently not been reported. There has been a push to report on quality, including access, patient-centered care, integrated care, and population health. Incentives are used to encourage collaboration between physicians. They are also used for after-hours payment and for “access bonuses,” or when patients stay within the primary care group and do not seek a GP in another group (Sweetman and Buckley 2016). According to Marchildon and Hutchison (2016), the reforms have not yet produced the level of improvement in access and quality of care that the provincial government originally expected. As a result, the provincial government agenda is now focused on containing costs while potentially broadening the reforms to include potential structural changes that could require more direct accountability of primary care teams to the provincial government (Marchildon and Hutchison 2016). Current pay-for-performance incentives also need to be reviewed given their modest or, in some cases, nonexistent impact. The “access bonus” component of capitation-based blended payment models is administratively cumbersome, and it penalizes physicians serving marginalized populations. Moreover, the access bonus is unaffected by enrolled patients’ use of emergency departments and fails to discourage unnecessary emergency department use (Hutchison and Glazier 2013). The MOHLTC provides Cumulative Preventive Care Bonuses to physicians who provide and maintain a certain level of preventive care for their enrolled patients. Physicians may earn an annual bonus if they report on the following five preventive categories with examples of indicators (MOHLTC 2019):  Influenza vaccine: Percentage of target population (those 65+ years) who have received the influenza vaccine appropriate for that influenza season.  Pap smear: Percentage of the target population (females 21–69 years) who are sexually active and who have received a pap smear within 42 months prior to March 31.  Mammography: Percentage of target population (female patients 50–74 years) who have received a mammogram within 30 months prior to March 31. 32  Childhood immunizations: Percentage of target population (enrolled patients 30– 42 months of age) who have received all of the ministry-supplied immunizations as recommended by the National Advisory Committee on Immunisation.  Colorectal cancer screening: Percentage of target population (enrolled patients ages 50–74 years) who have received a fecal occult blood test within 30 months prior to March 31. At the same time, Health Quality Ontario—an organization comprising doctors, nurses, patients, and other areas of the health care sector appointed by the MOHLTC—leads the province on providing evidence about health care quality. The organization reports and is accountable to the public, government, health care providers, and other organizations on how the health system (all levels) is performing in terms of effectiveness, efficiency, and affordability; identifying best practices; and translating the evidence into standards, recommendations, and tools (HQO 2019). In terms of health system performance, it measures the following elements of quality with examples of indicators (HQO,2017):  Effective: Premature mortality (potential years of life lost)  Timely: After-hours access to a primary care provider  Patient-centered: Home visits by a doctor in the patient’s last 30 days of life  Safe: Use of antipsychotic medications in long-term care homes  Efficient: Total health spending per person  Equitable: Same-day or next-day access to a primary care provider variation by region NORWAY Overview of Health Care System The Norwegian health system is universal for all residents, covering planned and acute primary care, hospital and ambulatory care, rehabilitation, and outpatient prescription medicines. It also covers dental care services for those under 18 years of age and prioritized groups. Financed through national, county, and municipal taxes, the government is responsible for providing health care to the population. Part of the general tax revenue is made up of national insurance contributions, which contribute to financing the National Insurance Scheme. The country does not have a defined benefits package except for new and costly treatments and technologies (Lindahl 2017; Ringard et al. 2013). The national budget and specifications regarding the allocation of the budget, provider fees, OOP payments, and ceilings are set each year by the Ministry of Health and Care Services (Lindahl 2017; Ringard et al. 2013). Clinical guidelines, new health technologies, and the national quality indicator system is under the remit of the Directorate of Health. It has been in charge of the National Patient Safety Program and has a system for reporting and learning about adverse events in hospitals. No single authority oversees fee-setting for providers other than hospitals (Lindahl 2017). Primary care is the responsibility of municipalities and specialized care under the remit of Regional Health Authority (RHA). Public retirement funds, sick leave payment, and reimbursement for additional health care costs are financed by social security contributions. Private insurance exists in Norway as a way to gain swifter access to health care services and have more options in terms of private health care providers. Private insurance is provided by for-profit insurers (Lindahl 2017). Set copayments (as determined by the government) exist for GP and specialist visits, except for those under 16 years of age, antenatal and postnatal care visits, sexually 33 transmitted infection (STI) prevention and treatment, and hospital admissions and inpatient treatment. There are two ceilings for OOP payments (Lindahl 2017). Services provided through primary health care are the responsibility of municipalities and conform to legislation, government directives, and quality requirements set by the Directorate for Health. Municipalities contract with individual GPs (who are self-employed), and physicians are incentivized to certify as specialized GPs and see as many patients as possible per day (Lindahl 2017). Strategic Purchasers and Providers The annual budget is distributed among the purchasing organizations including RHAs, municipalities, and counties. According to Ringard and colleagues (2013), purchasing and provision are often integrated. For example, “long-term care facilities are owned by the municipalities, and health care providers working in these facilities are employed by the municipalities” (Ringard et al. 2013). Municipalities and counties receive block grants from the government, which are the main source of funding. Distribution of these resources is based on the General-Purpose Grant Scheme. This involves a weighting system (with age as the most important weight) that adjusts the resources based on local variation for need and demand in services and cost difference (Ringard et al. 2013). Municipalities pay GPs in a combination of capitation, FFS, and OOP from patients. Only 5 percent of GPs are salaried. GP financing is determined by the MoH and Norwegian Medical Association. As part of the FFS scheme, fees are provided for medial reconciliation, taking part in care coordination, and for continuity of care for complex needs patients. GPs are paid a small fee from municipalities for after-hours care. Other payments come from the national FFS system and OOP from patients (Lindahl 2017). Specialized, including acute hospital, care is the responsibility of RHAs, which are state- owned corporations that report to the MoH. The budget, as well as the aims and priorities, are also provided by the MoH. Hospitals and self-employed specialists provide outpatient specialist care. Hospital-based specialists are salaried. The financing for specialized care is determined by the MoH and Norwegian Medical Association. See figure 6 for the organization of the health system (Lindahl 2017). For hospital care, a combination of block grants and activity-based funds are allocated to the RHAs, while block grants mainly finance specialist care. For RHAs, the amount of funds for each block grant is contingent on the type of service provided. For example, Ringard and colleagues (2013) report that in 2010, somatic care was given the most weight, followed by psychiatric care, ambulance/patient transport, and substance abuse treatment. Activity-based funding is contingent on the number of patients treated. This is based on a diagnosis-related group (DRG) system (Ringard et al. 2013). See figure 7 for the main financial flows of the Norwegian health care system (Ringard et al. 2013). Figure 6. Organization of Norway’s Health System 34 Source: Lindahl 2015, 2017. Public hospitals are state- and publicly owned corporations. A few hospitals are for-profit and some are owned by nonprofit humanitarian organizations provided by publicly funded services. All health care providers at public hospitals are salaried (Lindahl 2017). Different areas of the health care system, either systematically or individually, are audited by the National Board of Health Supervision. This board is also supposed to be informed when serious adverse events occur. Fines and warnings can be issued to facilities and health care providers. County governors perform local audits. Based on the National Strategy for Quality Improvement in Health and Social Services (2005–2015), which focuses on efficacy, safety, efficiency, patient-centered care, care coordination, continuity and equality in access to health care, 54 national clinical registries have been established for specific diseases and 15 national health registries (Lindahl 2017). The national program for health care quality indicators includes results from national patient experience surveys; quality indicators for survival rates, infection rates, and waiting times, among others; as well as indicators specific to health outcomes. However, results from these surveys and indicators are not available publicly or tied to health care provider performance (Lindahl 2017). Health care providers are licensed and authorized by the Registration Authority for Health Personnel. According to Lindahl (2017), most health care providers do not need to be reevaluated or reauthorized once they receive their license. Only GP specialists need to be recertified. The health care workforce is audited by the Norwegian Board of Health (Lindahl 2017). 35 Figure 7. Main Financial Flows in the Norwegian Health Care System Source: Ringard et al. 2013. Notes: NIS = National Insurance Service; OOP = Out-of-pocket; GP = General practitioner. The quality of services provided through RHAs, hospitals, municipal providers, and the private sector is under their remit. “There is no requirement for accreditation or reaccreditation, although some hospitals or hospital departments are accredited” (Lindahl 2017). Mandatory agreements between hospitals and municipalities include care coordination incentives. Hospitals are state funded while primary care is municipality funded. Lindahl (2017) reports that municipalities will be fined daily if patient length of stay in a hospital is longer than when they are ready for discharge (Lindahl 2017). Instruments Used to Verify the Monitoring of Results Resources allocated by the MoH consider the population’s needs and activity level (i.e., allocation by the RHA). Yet there is no penalty for overspending and Parliament will often increase funding above the budget allocation. Between 2006 and 2010, resources allocated to child and adolescent psychiatric care, alcohol and drug abuse treatment, and ambulatory care increased (Ringard et al. 2013). Meanwhile, the country has experienced a decrease in lengths of inpatient stay, pointing to greater hospital efficiency, while increases have been observed in outpatient and day-care rates (Ringard et al. 2013). The government is investigating the quality of preventive care, as measured through vaccination rates; quality of chronic care conditions can be measured by looking at avoidable hospital admission rates for asthma, chronic obstructive pulmonary disease (COPD), and diabetes-related complications. Several patient safety indicators have 36 recently been established in Norway as part of the OECD’s Health Care Quality Indicators (HCQI) project (Ringard et al. 2013). In 2014, the Quality-Based Financing (QBF) program was introduced as a pay-for- performance scheme in the country and includes 32 indicators that measure performance, improvement, and patient experience. “The main objective of the program is to use financial incentives to motivate the hospitals to increase overall quality and patient safety delivered to patients in their care. The scheme targets RHAs and was introduced across all four regions and therefore includes all public secondary care providers, and also private hospitals with a contract with the RHA. The RHA budget is reallocated based on the hospitals’ achievement on the chosen set of quality indicators” (Olsen and Brandborg 2016). Examples of indicators include the following (Olsen and Brandborg 2016): Outcome indicators  Five-year survival rate for colon cancer, per health region  Five-year survival rate for breast cancer, per health region Process indicators  Thrombolysis treatments  Waiting time violations Patient satisfaction  Nursing staff  Physicians The country also has the National Quality Indicator System as a way to monitor quality and improve health service governance. It is not tied to any financial objectives, although the QBF utilizes the system by “attaching reimbursement to hospital performance on the indicators included in the pay for performance scheme” (Olsen and Brandborg 2016). The system includes 100 indicators that measure quality in secondary care, long-term care, and dental care (Olsen and Brandborg 2016). TURKEY Overview of Health Care System In 2003, the government implemented health system reforms through the Health Transformation Programme (HTP) as a way to improve primary health care and progress toward UHC. This included improving the pooling and redistribution of services, which involved consolidating several insurance schemes into one: the Social Security Institution (SSI). It also involved developing consistent payment methods to providers at the national level (OECD 2014). The provision of health care services is under the remit of the MoH in Turkey. The MoH sets health policies; implements national health strategies; and delivers primary, secondary, maternal and child, and family planning health services. This is delivered through hospitals, dispensaries, and health care facilities (Tatar et al. 2011). At the provincial level, each province has a health directorate that implements MoH health policies. The directorates are led by physicians. They plan and implement health care services. Meanwhile the private sector also operates within the health care system, providing services on a contractual basis (Tatar et al. 2011). Figure 8 provides an overview of the Turkish health system. 37 Strategic Purchasers and Providers The SSI and the Ministry of Finance are responsible for public health financing and resource allocation, while the MOH is responsible for the planning and delivery of health care services (Tatar et al. 2011). The private sector is also a purchaser of health services. The SSI and Ministry of Finance, as purchasing organizations, are annually audited by the Turkish Court of Accounts, while the private sector sets its own rules and only needs to comply with commercial and tax regulation laws (Tatar et al. 2011). The MoH’s Health Implementation Guide (HIG) determines the prices and package of services to be purchased. Figure 8. Overview of the Turkish Health System Source: Tatar et al. 2011. Notes: SSI = Social Security Institution; OOP = Out-of-pocket; NGO = Nongovernmental organization. The purchaser and provider functions are split. The government allocates resources in two ways: through annual budgets (university and public) and a revolving fund, or pooled funds from the SSI and patients on FFS basis, paid retrospectively. Thus, health care providers are paid a salary and also paid for their performance through the revolving fund. To determine how much the provider is paid through the revolving fund, there is a formula that the hospital uses, which is also based on prices set by the HIG. Figure 9 illustrates the financial flow of the Turkish health system. Note that the Green Card was subsumed under the SSI in 2012. 38 Revenue collection, pooling, and purchasing functions are all integrated in the SSI. Services are purchased from public and private providers through a global budget and service contracts. Wherein the SSI contracts with the private sector, “contracts are negotiated and concluded between the SSI and providers.” It should be noted, however, that not all private health care providers are contracted by the SSI, with the SSI contracting for specific services such as surgeries and cardiovascular diseases (Tatar et al. 2011). Instruments Used to Verify the Monitoring of Results Through the HTP, physicians are paid through a mixed payment system of capitation and pay for performance (revolving fund). The revenue generated from the revolving fund, which determines how much a physician will be paid, depends upon two factors: the institution’s rating and the physician’s rating. For the institutional rating, each hospital or facility receives a score between 0 and 1, which is then multiplied by the total percentage of revenue the facility/hospital can allocate to staff bonuses. Note that health facilities/hospitals can allocate 40 percent of their total revenues to performance-based payment (Bump et al. 2014). According to the OECD (2014), there are approximately 49 indicators upon which the score is calculated. The institutional performance score is based on examples of the following: 1. Access to examination rooms 2. Hospital infrastructure and process 3. Patient and caregiver satisfaction 4. Institutional productivity 5. Average length of stay 6. Share of the doctors working full-time Tatar and colleagues (2011) also state that institutional performance includes a quality coefficient. Indeed, the MoH has developed approximately 150 criteria for health facilities to be assessed in terms of quality. This includes “access to health care services, administration, information management, laboratories, radiology, operating rooms, clinics, patient and staff safety, infection control and prevention, intensive care units, dialysis centers, institutional safety, pharmacy, emergency room, kitchen, laundry, patient records and the mortuary” (Tatar et al. 2011). Quality is assessed three times per year and through patient surveys, given twice per year. Hospitals are also trying to meet quality standards by obtaining International Organization for Standardization certification from the Institute of Turkish Standards (Tatar et al. 2011). The physician’s individual rating is determined on the number of clinical procedures (i.e., number of invasive clinical procedures per physician) performed and the job title. The MoH sets the point level for each clinical procedure. The score is then adjusted for number of days worked in the year, and part-time or full-time status (Bump et al. 2014; Tatar et al. 2011). Physicians can be penalized for not providing services as well. Providers who fail to provide antenatal and postnatal services can be penalized approximately US$220 (Tatar et al. 2011). Results from the pay-for-performance schemes are not available in the public domain (Tatar et al. 2011), and the OECD states that “compared to other OECD countries, there is less evidence of a quality culture in Turkish health care than elsewhere” (OECD 2014). Figure 9. Financial Flows in the Turkish Health System 39 Source: Tatar et al. 2011. Notes: SSI = Social Security Institution; OOP = Out-of-pocket. DISCUSSION This section highlights the different verification processes that countries take to monitor efficiency, quality, coverage, financial protection, and health outcomes. England and Sweden appear to have more comprehensive verification processes while Norway, the province of Ontario, and Turkey are collecting data on health care provider performance but not publicly reporting it. The report also finds that the countries use different instruments to measure provider performance with no standard instrument used. Further, most focus on measuring quality and health outcomes, and few concentrate on efficiency, 40 coverage, and financial protection. The following section provides a summary of challenges and opportunities in verifying provider performance. CHALLENGES AND OPPORTUNITIES The first is whose perspective you take: the government in its role as policy maker and purchaser of services (principal) or the providers who are subject to the control (agent)? Agency theory is commonly used to describe relationships between actors in health care systems. Principals and agents may very well have different expectations and be motivated by different things. Goal alignment between different actors is often described as one of the most important tasks of management control systems (Glenngard 2019). What is regarded as an appropriate set of management controls can also be assumed to depend on the intended overall role of the governance model from the controlling part: To ensure external accountability for the use of allocated resources and adherence to tasks stipulated in agreements between providers and governments? Or to support learning, innovations, and quality improvement in the health system? Management controls can be used in different ways and for different purposes in health care (e.g., to offer support to quality improvements and for external accountability) (Glenngard 2019). In more general terms, the management control package used can be characterized by the degree of formalization and type of controls used: coercive or enabling. Degree of formalization refers to the occurrence of structured formal rules, procedures, and instructions. Coercive types of control refer to procedures to force compliance while enabling types provide individuals with knowledge about lessons learned from experience. Organizations may pursue different, potentially conflicting objectives by using managements controls in an enabling way. Principals assigning governance an enabling role should invite agents to participate in the design and use of the management control systems. Using such a participatory process can provide agents with increased capacity to do their tasks through increased knowledge about the organization’s goals and how to reach them. Moreover, by involving agents, principals can improve the agents’ attitudes toward the assigned task (Glenngard 2019). Finally, what is regarded as an appropriate set of management controls could also depend on contextual factors: when and where. Principles for governance and management are subject to continuous change, reflecting the priorities and trends that generally apply in the society. This applies not least to organizations operating in the public sector where various reforms are introduced on a regular basis as a solution to identified problems and political priorities. Each new reform entails new demands on the organizations that operate here, a fact implying that governance models need to be adapted. Swedish health care is no exception to policy makers’ ambitions to improve the health care system by introducing continuous reforms (Glenngard 2019). In practice, a mix of governance structures is used in most contexts. Studies on Swedish primary care after the choice reform have shown that it is necessary to maintain accountability relationships between providers and governments to ensure that the overall objectives of health care are achieved. This applies not least in a situation with increased choice for individuals. Increased choice may suffice to achieve increased accessibility and responsiveness to individual needs and preferences, but not to achieve important goals from a population perspective. Specifying requirements that providers must comply with to be allowed to practice care and be publicly reimbursed, and follow-up of such requirements using performance measures of patient-reported experiences, compliance 41 with clinical guidelines and waiting times have become common types of controls following the choice reform. Hence, the overall governance structure can be described as a combination of the “choice and competition” model and the “hierarchy and targets” model where the latter is associated with external incentives and monitoring by government. One critique following the change in the overall governance structure is that providers are subject to a heavy administrative burden and perhaps too tight controls, as they are supposed to act in accordance with evidence-based clinical guidelines, targets, and clinical performance indicators set by governments aiming at greater systematization in health care. Limiting health care workers’ professional autonomy using targets and clinical performance indicators set by governments can be described as a coercive type of control (Glenngard 2019). There are three key challenges associated with monitoring effective service coverage, which is defined as service coverage that results in the maximum possible health gains. The first challenge is accurate measurement of the population in need of the service. Administrative records from service providers and self-reported prior diagnosis are often unreliable sources of information, as those who do not have access to health services remain undiagnosed. A full assessment of population needs requires alternative sources of data, such as a set of survey questions or biomarkers collected in a household health examination survey. Because few conditions requiring treatment can be diagnosed in this way, this substantially limits the set of effective coverage indicators that may be reliably monitored (WHO and WBG 2017). Determining effectiveness of service coverage—that is, the degree to which services result in health improvement—is a second challenge. For some indicators, it is possible to directly measure quality of care. For example, monitoring of treatment for hypertension can include measurement of whether hypertension is effectively controlled, and monitoring of cataract surgical coverage can include measurement of current visual acuity. However, measuring effectiveness of care is more complicated than measuring service provision (WHO and WBG 2017). The third key challenge is to monitor equity in access to quality health services. Making sure that no one is left behind as countries strive for UHC requires access to data disaggregated by inequality dimensions, such as wealth or geographical location. Disaggregated data are commonly available for Reproductive, Maternal, Child, Neonatal, and Child Health (RMNCH) interventions, malaria prevention, and water and sanitation services in low- and middle-income countries but may not be available for other health topics and indicators required for UHC monitoring. Therefore, investments are needed in data collection, especially for conducting regular household health examination surveys and developing electronic and harmonized facility reporting systems. In addition, it is crucial to build capacities for analyzing and reporting health inequality data. Only then can countries tie this information to the policies they are implementing to improve health equity (WHO and WBG 2017). A key challenge, according to Sanderson, Lonsdale, and Mannion (2019) for purchasers is that system responsiveness, equity of access, and resource efficiency might conflict with one another. For example, consolidation of a service in one main location to enhance cost-effectiveness and increase quality is very likely to have an adverse impact on equity of access for patients living farther away. Decisions aimed at improving provider performance will therefore require purchasers to make trade-offs (Sanderson, Lonsdale, and Mannion 2019). 42 According to the literature, there are three factors that may influence the response of providers to purchasing decisions and how these decisions are expressed in practice. The first is the degree and type of autonomy that providers have when they are deciding how to meet the requirements of purchasers. Policy makers might give providers autonomy over a number of significant decision areas, for example: staffing (numbers and skill mix); financial management (ability to take loans); the scope of activities (which services are offered and where); and capital investment (size and location of buildings, technology mix). The rationale for granting such autonomy is based on the idea of creating market competition between different types of providers (public, private, and third sector), and incentivizing innovative and efficient choices by providers by giving them a right to retain “surplus” resources. The primary emphasis here is on efficient resource utilization, although policies of this type are also intended to enhance quality of care and responsiveness to patient needs. This insight suggests that a second key factor likely to shape provider behavior and performance are the kinds of governance mechanisms used to make providers accountable to purchasers, and the effectiveness of those mechanisms in mitigating opportunism. The third and final factor likely to affect how providers respond to purchasing decisions picks up on this idea of creating a balanced relationship. More specifically, we are interested in the balance of power between purchasers and providers and the moves that largely autonomous providers, whether they are public, private, or third sector, might make to protect or enhance their power relative to purchasers (Sanderson, Lonsdale, and Mannion 2019). As Loening and Tineo (2012) argue, independent verification is a key mechanism to enhance the performance of service providers. For the funding entity, it mitigates the risk of misuse of funds, providing assurance and evidence that funds have been used for the intended purpose. Verification is also directly affected by the reporting capacity and internal control systems of service providers and/or implementing agencies. Due to the pressure to get paid, service providers expect verification to be carried out as soon as they have delivered the service. Collaboration of all parties (funders, implementing agency, service providers, communities, and verification agent) is critical in ensuring timely disbursement. At the beginning of the project, the verification agent should agree on the verification protocol (including the format for the verification reports and sampling methodology), and make sure that it is acceptable to all parties. Debriefing on preliminary findings at the end of each verification exercise then gives the service provider and the implementing agency an opportunity to respond and take corrective measures while the verification agent prepares its report (WBG 2012). Verification approaches in any country are often driven by the accuracy of data in the health information systems and the status of the governance structures in place in the health sector. The first lesson is that as any Results-Based Financing (RBF) program matures, however, and the quality of data improves, the verification process may also need to evolve, particularly if the government eventually takes over responsibility for funding the effort. The second and related lesson is that the RBF program in general, and the verification process specifically, can and should be a lever to strengthen health information and governance structures. Findings from many RBF programs worldwide suggest that this is one of the most valuable “spillover effects” of such programs (WBG 2010 and 2012). 43 REFERENCES Allin, Sara, and David Rudoler, 2017. “The Canadian Health Care System.” In International Profiles of Health Care Systems, ed. Elias Mossialos et al., 21–30. Commonwealth Fund, May. Anell, A., A. H. Glenngard, and S. Merkur. 2012. “Sweden: Health System Review.” Health Systems in Transition 14 (5): 1–159. Bramesfeld, A., M. Wensing, P. Bartels, H. Bobzin, C. Grenieer, M. Heugren, D. Jaffe Hirschfield, et al. 2016. “Mandatory National Quality Improvement Systems Using Indicators: An Initial Assessment in Europe and Israel.” Health Policy 120: 1256– 69. Bump, J., S. Sparkes, M. Tatar, and Y. Celik. 2014. Turkey on the Way of Universal Health Coverage through the Health Transformation Program (2003–13). Washington, DC: World Bank Group. Cashin, C., L. Fleisher, and T. Hashemi. 2015. Verification of Performance in Results- Based Financing (RBF): The Case of Afghanistan. Washington, DC: World Bank Group. Chisholm, D., and D. B. Evans. 2010. Improving Health System Efficiency as a Means of Moving Towards Universal Coverage. Geneva: World Health Organization. Cylus, J. R., E. Richardson, L. Findley, M. Longley, C. O’Neill, and D. Steel. 2015. “United Kingdom: Health System Review.” Health Systems in Transition 17 (5): 1–125. DHSC (Department of Health and Social Care). 2019. Annual Report and Accounts 2018– 19 (for the period ended 31 March 2019). London: Department of Health and Social Care. Dohmen, P. J. G., and E. M. van Raaij 2019. “A New Approach to Preferred Provider Selection in Health Care.” Health Policy 123: 300–305. European Commission. 2019. “Tools and Methodologies to Assess the Efficiency of Health Care Services in Europe: An Overview of Current Approaches and Opportunities for Improvement.” Luxembourg: European Union (EU). European Observatory on Health Systems and Policies. 2016. Health System Efficiency: How to Make Measurement Matter for Policy and Management. Geneva, WHO. Glazier, R. H., B. Hutchison, and A. Kopp. 2015. A Comparison of Family Health Teams to Other Ontario Primary Care Models, 2004/05 to 2011/12. Toronto: Institute for Clinical Evaluative Sciences. Glenngard, A. H. 2017. The Swedish Health Care System. New York: Commonwealth Fund. ———. 2019. “Pursuing the Objectives of Support to Providers and External Accountability through Enabling Controls—A Study of Governance Models in Swedish Primary Care. BMC Health Services 19: 114. 44 Hanson, K., E. Barasa, A. Honda, W. Panichkriangkrai, and W. Patcharanarumol. 2019. “Strategic Purchasing: The Neglected Health Financing Function for Pursuing Universal Health Coverage in Low- and Middle-Income Countries Comment on ‘What’s Needed to Develop Strategic Purchasing in Healthcare? Policy Lessons from a Realist Review.’” Int J Health Policy Manag 8 (8): 501–504. Health and Social Care Act. 2012. London: Stationery Office. http://www.legislation.gov.uk/ukpga/2012/7/contents/enacted. Heredia-Ortiz, E. 2013. “Data for Efficiency: A Tool for Assessing Health System’s Resource Use Efficiency.” Bethesda, MD: USAID’s Health Finance and Governance Project, Abt Associates Inc. HQIP (Healthcare Quality Improvement Partnership). 2017. Using Clinical Audit in Commissioning Healthcare Services. London: HQIP. ———. 2019. Impact Report: February 2019. London: HQIP. HQO (Health Quality Ontario). 2017. Health System Performance in Ontario—2017 Highlights. Toronto: HQO. ———. 2019. Measuring Up 2019: A Yearly Report on How Ontario’s Health System is Performing. Toronto: HQO. Hurley, J., P. DiCicca, J. Li, and G. Buckley. 2011. “The Response of Ontario Primary Care Physicians to Pay-for-Performance Incentives.” Working paper. Hamilton, ON: McMaster University Centre for Health Economics and Policy Analysis. Hussey, P. S., H. de Vries, J. Romley, M. C. Wang, S. S. Chen, P. G. Shekelle, and E. A. McGlynn. 2009. “A Systematic Review of Health Care Efficiency Measures.” Health Serv Res 44 (3): 784–805. Hutchison, B., and R. Glazier. 2013. “Ontario’s Primary Care Reforms Have Transformed the Local Care Landscape: But a Plan Is Needed for Ongoing Improvement.” Health Affairs 32 (4): 695–703. Kiran, T., A. Kopp, R. Moineddin, and R. H. Glazier. 2015. “Longitudinal Evaluation of Physician Payment Reform and Team-Based Care for Chronic Disease Management and Prevention. CMAJ 187 (17): E494–502. Lindahl, A. K. 2015. Norwegian Knowledge Centre for Health Services. The Norwegian Health Care System, 2014. In the International Profiles of Health Care Systems, 2014, Edited by Elias Mossialos and Martin Wenzl London School of Economics and Political Science Robin Osborn and Chloe Anderson.The Commonwealth Fund ———. 2017. The Norwegian Health Care System. New York: Commonwealth Fund. Loening, Esther, and Luis Tineo. 2012. Independent Verification in Results-Based Financing. OB Approaches, No 43. Washington, DC. World Bank. https://openknowledge.worldbank.org/handle/10986/16175 License: CC BY 3.0 IGO. 45 Marchildon, G. P., and B. Hutchison. 2016. “Primary Care in Ontario, Canada: New Proposals after 15 Years of Reform.” Health Policy 120: 732–38. Mathauer, Inke, E. Dale, and B. Meesser. 2017. “Strategic Purchasing for Universal Health Coverage: Key Policy Issues and Questions. A Summary from Expert and Practitioners’ Discussions.” Health Financing Working Paper no. 8. Geneva: World Health Organization. MOHLTC (Ministry of Health and Long-Term Care). 2019. Information and Procedures for Claiming the Cumulative Preventive Care Bonus. Toronto: MOHLTC. NHS (National Health Service). 2013. A Simple Guide to Payment by Results. London: NHS. ———. 2019. Long Term Plan. London. NHS Digital. 2019. SUS+ PbR Reference Manual: Implementation of National Tariff/Payment by Results (PbR) in the Secondary Uses Service (SUS). London, NHS Digital. NHSE and NHSI (National Health Service England and National Health Service Improvement). 2019a. NHS Oversight Framework 2019/20. London: NHSE and NHSI. https://www.england.nhs.uk/wp-content/uploads/2019/10/nhs-oversight- framework-rev-oct19.pdf. ———. 2019b. 2019/20 National Tariff Payment System: A joint publication by NHS England and NHS Improvement. London: NHSE and NHSI. ———. 2019c. Commissioning for Quality and Innovation (CQUIN): Guidance for 2019– 2020. London: NHSE and NHSI. NHSI (National Health Service Improvement). 2019. Performance of the NHS Provider Sector for the Year Ended 31 March 2019. London: NHSI. OECD (Organisation for Economic Co-operation and Development). 2010. Health Care Systems: Efficiency and Policy Settings. Paris: OECD Publishing. ———. 2014. OECD Reviews of Health Care Quality: Turkey 2014: Raising Standards. Paris: OECD Publishing. ———. 2017. Health at a Glance 2017: OECD Indicators. Paris: OECD Publishing. https://doi.org/10.1787/health_glance-2017-en. Olsen, C. B., and G. Brandborg. 2016. “Quality Based Financing in Norway: Country Background Note: Norway.” Norwegian Directorate of Health. Price, D., E. Baker, B. Golden, and R. Hannam. 2015. “Patient Care Groups: A New Model of Population Based Primary Healthcare for Ontario.” Toronto: Primary Health Care Expert Advisory Committee confidential draft, February 11. 46 Ringard, A., A. Sagan, I. Sperre Saunes, and A. K. Lindahl. 2013. “Norway: Health System Review.” Health System in Transition 15 (8): 1–162. Sanderson, J., Chris Lonsdale, and R. Mannion. 2019. “What’s Needed to Develop Strategic Purchasing in Health Care? Policy Lessons from a Realist Review.” Int J Health Policy Manag 8 (1): 4–17. Sweetman, A., and G. Buckley. 2016. “Payment Innovations for General Practice Physicians in Ontario, Canada.” Country Background Note: Canada (Ontario) and Paris: OECD. Tandon, A., C. Murray, J. Lauer, and D. Evans. 2000. “Measuring Overall Health System Performance for 191 Countries.” GPE Discussion Paper Series no. 30. Geneva: WHO. Tatar, M., S. Mollahaliloglu, B. Sahin S. Aydin, A. Maresso, C. Hernández-Quevedo. 2011. “Turkey: Health System Review.” Health Systems in Transition 13 (6): 1–186. Wagstaff, A., P. Eozenou, S Neelsen, and M. Smitz. 2019. The Health Equity and Financial Protection Indicators Database 2019. Washington, DC: World Bank. WBG (World Bank Group). 2007. Public Ends, Private Means: Strategic Purchasing of Health Services. Washington, DC. ———. 2010. “Results-Based Financing for Health (RBF): Verification at a Glance: A Series of Snapshots of Experiences in Verifying Performance Linked to Financial Incentives for Results-Based Financing (RBF) Programs from Selected Countries.” Washington, DC. ———. 2012. OBA Approaches: Verification in Results-Based Financing. Washington, DC. ———. 2017. Service Delivery Indicators. Washington, DC. ———. 2018. Service Delivery by Health Facilities in Papua New Guinea. Washington, DC. ———. 2019. High-Performance Health Financing Universal Health Coverage: Driving Sustainable Inclusive Growth in the 21st Century. Washington DC. WHO (World Health Organization). 2018. 2018 Global Reference List of 100 Core Health Indicators (plus health-related SDGs). Geneva: WHO. WHO and WBG (World Health Organization and World Bank Group). 2017. Tracking Universal Health Coverage: 2017 Global Monitoring Report. World Health Organization and World Bank. License: CC BY-NC-SA 3.0 IGO. WHO, OECD, and WBG (World Health Organization, Organisation for Economic Co- Operation and Development, and the World Bank Group). 2018. Delivering Quality Health Services: A Global Imperative for Universal Health Coverage. Geneva. 47 Yip, W., and R. Hafez. 2015. Improving Health System Efficiency: Reforms for Improving the Efficiency of Health Systems: Lessons from 10 Country Cases. Geneva: WHO. 48 The aim of the study was to highlight the different verification processes that four countries and one province take to monitor efficiency, quality, coverage, financial protection, and health outcomes in health systems. The literature review focused on the experiences of England and Sweden, illustrating a comprehensive verification process. Norway, the Canadian province of Ontario, and Turkey collected data on health care provider performance but did not publicly report it. Different instruments were used. Performance measures of patient-reported experiences, compliance with clinical guidelines, and waiting times have become common measurement-based indicators. To improve verification processes, it is necessary to maintain accountability between providers and governments to ensure that the overall objectives of health care are achieved. Monitoring effective service coverage includes measuring the population in need of the service using administrative records from service providers, determining the effectiveness of service coverage using selected indicators, and monitoring equity in access to quality health services using data disaggregated by inequality dimensions. Verification of results is essential within the context of institutional arrangements for the purchasing of health care services to providers. There is autonomy over several significant decision areas such as staffing (numbers and skill mix); financial management (ability to take loans); the scope of activities and capital investments; governance mechanisms that make providers accountable to purchasers; and conditions that balance the power between purchasers and providers. Within this context, monitoring and verification of results is critical to enhance the performance of service providers and ensure value for money within health expenditure. This would be strengthened by previously agreed standards between providers and the implementing agency. Setting strong monitoring and verification procedures has become a key factor in the success of Results-Based Financing programs in general, strengthening health information and governance structures are the most valuable “spillover effects” of such programs. ABOUT THIS SERIES: This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author/s whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@ worldbank.org) or HNP Advisory Service (askhnp@worldbank.org, tel 202 473- 2256). For more information, see also www.worldbank.org/hnppublications. 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org