H N P D i s c u s s i o n P a p e R 77047 VERIFICATION IN RESULTS-BASED FINANCING (RBF): The Case of the United Kingdom Cheryl Cashin, Petra Vergeer January 2013 Verification in Results-Based Financing (RBF): The Case of the United Kingdom Cheryl Cashin Petra Vergeer January 2013 ii iii Health, Nutrition and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population (HNP) Family of the World Bank's Human Development Network (HDN). The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. Enquiries about the series and submissions should be made directly to the Editor, Martin Lutalo (mlutalo@worldbank.org). Submissions undergo informal peer review by selected internal reviewers and have to be cleared by the Task Team Leader's Sector Manager. The sponsoring department and author(s) bear full responsibility for the quality of the technical contents and presentation of material in the series. Since the material will be published as presented, authors should submit an electronic copy in the predefined template (available at www.worldbank.org/hnppublications on the Guide for Authors page). Drafts that do not meet minimum presentational standards may be returned to authors for more work before being accepted. For information regarding the HNP Discussion Paper Series, please contact Martin Lutalo at mlutalo@worldbank.org or 202-522-3234 (fax). © 2013 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. iv i Health, Nutrition and Population (HNP) Discussion Paper Verification in Results-Based Financing (RBF): The Case of the United Kingdom Quality and Outcomes Framework (QOF) Cheryl Cashina, Petra Vergeerb a Provider Payment Track, Joint Learning Network for Universal Health Coverage, Results for Development Institute, Washington D.C., United States b Health, Nutrition, and Population, World Bank, Washington D.C., United States Abstract: Paying health care providers performance based incentive payments is one form of results-based financing (RBF). Verifying that providers have reached set performance thresholds is a crucial part of RBF program implementation and key to maintaining the transparency, fairness, and viability of the programs. The National Health Service of the United Kingdom has been implementing one of the largest RBF initiatives worldwide, the Quality and Outcomes Framework (QOF), since 2004. The QOF provides incentive payments for primary care providers to improve quality of care and patient experience. Its verification process is well developed and relies on a highly sophisticated clinical information system as the foundation for performance data. This case study describes the process for verification of achievement rates under the QOF and identifies lessons learned applicable to other RBF programs. The case study is part of a broader analysis, which includes multiple country case examples, to expand knowledge about the verification process and practices to address the immediate design and implementation needs of RBF programs. Although the QOF verification process requires an institutional capacity and data infrastructure that may be less developed in lower-and middle-income countries, there are important lessons for countries of all income levels considering or implementing RBF programs for health care providers. We found that well-designed RBF verification can contribute to health system strengthening such as improving availability and use of health information, and opening a structured dialogue between purchaser and providers. While verification costs are likely to be high, using risk-based sampling criteria for selecting providers and indicators could be more cost-effective. While the balance between validity and affordability and between transparency and confidentiality can be tricky, it is essential to maintain the transparency and objectivity of the verification process. Keywords: Verification, quality, health system, incentives Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. ii Correspondence Details: Petra Vergeer, World Bank, 1818 H Street, NW, Washington DC, 20433, Tel: 202-473-7181, email: pvergeer@worldbank.org iii Table of Contents 1. INTRODUCTION ................................................................................................................. 1 2. QOF OVERVIEW ................................................................................................................. 1 3. BACKGROUND: RBF IN THE U.K. .................................................................................. 3 4. MAJOR CHARACTERISTICS OF THE VERIFICATION METHOD ........................ 6 UNITED KINGDOM: QUALITY AND OUTCOMES FRAMEWORK (QOF) ........................................... 7 4.1 OBJECTIVES OF THE QOF VERIFICATION PROCESS ................................................................ 8 4.2 DATA SOURCES AND FLOWS .................................................................................................. 8 4.3 THE QOF REVIEW VISIT ...................................................................................................... 11 4.3.1 Assessor Team ............................................................................................................. 11 4.3.2 Process for the QOF Review Visit ............................................................................... 12 4.3.3 Targeting GP Practices and Indicators for the QOF Review Visit ............................. 12 4.4. PRE-PAYMENT VERIFICATION ............................................................................................. 16 4.5. POST-PAYMENT VERIFICATION ........................................................................................... 16 4.6 HOW IS THE QUANTITY OF SERVICES VERIFIED FOR PRE- AND POST-PAYMENT VERIFICATION? .......................................................................................................................... 17 4.7 HOW IS THE QUALITY OF SERVICES VERIFIED FOR PRE- AND POST-PAYMENT VERIFICATION? ................................................................................................................................................... 18 5. FINDINGS OF THE VERIFICATION METHODS ....................................................... 19 5.1 HOW ARE VERIFICATION FINDINGS USED? ............................................................................ 20 6. VERIFICATION COSTS ................................................................................................... 20 7. LESSONS LEARNED......................................................................................................... 22 REFERENCES ............................................................................................................................ 25 ANNEX 1. TEMPLATE FOR VERIFICATION VISITS IN NHS WESTERN CHESHIRE ....................................................................................................................................................... 29 ANNEX 2. NOTTINGHAMSHIRE COUNTY PCT SELF-ASSESSMENT FORM ........ 44 iv 1. INTRODUCTION Paying health care providers incentive payments based on performance is one form results-based financing (RBF). In RBF programs, verifying that providers have reached performance thresholds is a crucial part of program implementation and key to maintaining the transparency, fairness, and viability of the programs. The National Health Service (NHS) of the United Kingdom (U.K.) has been implementing the Quality and Outcomes Framework (QOF), one of the largest RBF initiatives worldwide, since 2004 to provide incentive payments for primary care providers to improve quality of care and patient experience. The verification process established to implement the QOF is well developed and relies on a highly sophisticated clinical information system with electronic medical records as the foundation for performance data. Although the QOF verification process requires an institutional capacity and data infrastructure that may be less developed in lower-and middle-income countries, there are important lessons from the U.K. QOF experience for countries of all income levels considering or implementing RBF programs for health care providers. The objectives of this case study are: to provide a detailed description of the process for verification of achievement rates under the QOF; and to generate possible lessons for other RBF schemes to make verification more cost-effective and useful in supporting overall performance improvement and health system strengthening. This case study is part of a broader analysis of multiple country case examples to expand knowledge about the verification process and practices to address the immediate design and implementation needs of RBF programs. The U.K. QOF and its verification process are evolving, so this case study only provides a snapshot of the system over a limited period of time (2006-2010). 2. QOF OVERVIEW In its 2000 NHS Plan for Reform and Investment, the U.K. government made a historic commitment to investing in the NHS. Over the next ten years, spending on the NHS was increased by 43 percent in real terms (Government of the U.K. 2000). This infusion of resources into the NHS was accompanied by measures to increase accountability and set standards for providers. Quality-based contracts and performance targets, some of which were tied to financial incentives, became a key feature of the approach to reforming the NHS. As part of the NHS reforms launched in the 2000 strategy, a new contract between Primary Care Trusts (PCTs), the local branches of the NHS, and GP practices was negotiated in 2004. The new contract included a voluntary RBF program based on the Quality and Outcomes Framework (QOF). 1 The QOF is an expensive program, costing the NHS about £1 billion per year, but published studies on the results of the QOF raise questions about whether gains in quality of care and health outcomes are significant (U.K. National Audit Office 2008, Serumaga, et al. 2011, Doran, Kontopantelis, et al. 2011, Campbell, et al. 2007). Nonetheless, the QOF is widely credited with improving the availability and use of data and information in primary care in the 1 Since the QOF began in 2004, several local initiatives to improve the prescribing practices have been introduced by PCTs, such as the West Sussex Prescribing Incentive Scheme (NHS West Sussex 2010). These smaller initiatives are sometimes coordinated with but not part of the QOF and are not considered in this case study. 1 U.K., which is driven, at least in part, by the automated clinical data system put in place to support the QOF and the highly detailed and participatory verification process (Campbell, et al. 2007). Better availability and use of data may have a cross-cutting effect on improving processes of care and ultimately patient outcomes over the longer term (Clemmer 2004). The method for carrying out this case study involved a desk review of the published policies for verification under the QOF, supplemented by an in-depth analysis of the documented experience of a convenience sample of ten PCTs 2 in the U.K. that had published QOF reports. 3 A search was completed of all QOF reports, and only ten PCTs were found with reports publicly available of sufficient detail to describe key features of the verification process. The availability and detail of QOF reports varied across the PCTs, so the QOF period reviewed was not constant, ranging from 2006/07 to 2009/10, and the information available was not completely consistent across all ten PCTs. Table 1 provides an overview of the PCTs included in the study. Table 1. Overview of PCTs Reviewed PCT # of GP QOF Period Annual Expenditure QOF Expenditure Practices Reviewed on QOF per GP Practice Brent 70 2008/2009 Not specified Not specified Bromley 51 2008/09 £6,017,000 £118,000 Heart of Birmingham 76 2007/08 Not specified Not specified Teaching PCT City and Hackney 44 2009/10 Not specified Not specified Kirklees 74 2008/09 £2,333,584 £106,000 Northamptonshire 82 2009/10 £13,200,000 £161,000 Nottinghamshire 96 2008/20009 Not specified Not specified County 2009/2010 Oldham 50 2009/10 £4,500,000 £90,000 Solihull 31 2006/07 Not specified Not specified Western Cheshire 40 2006/07 Not specified Not specified Source: PCT QOF annual reports. 2 The ten PCTs reviewed include Brent, Bromley, Heart of Birmingham Teaching PCT, City and Hackney, Kirklees, Northamptonshire, Nottinghamshire County, Oldham, Solihull, and Western Cheshire. 3 Direct input on the study was intended to be obtained from PCT staff and other stakeholders through key informant interviews. No PCT staff contacted agreed to be interviewed, however, although several did provide comments on the accuracy of the study, which were incorporated into the final draft. 2 3. BACKGROUND: RBF IN THE U.K. 4 The new contract between the NHS and GPs, including the QOF pay-for-performance program, had a range of ambitious objectives: to increase productivity; redesign services around patients; improve the skill mix in primary care; create the culture and governance structure to improve quality of care; extend the range of services available; and improve recruitment, retention, and morale (U.K. National Audit Office 2008). The main underlying objective of the QOF was to create an accountability mechanism for the planned infusion of new resources in the primary care sector (Government of the U.K. 2000). The initial program included 146 targets in 4 domains--clinical, organizational, patient experience, and additional services. The contract is re-negotiated regularly (in 2006, 2008, 2009 and 2010), and QOF indicators and targets are updated as agreed between the NHS and the General Practitioners Committee of the British Medical Association. In 2009, the National Institute for Clinical Excellence (NICE) was given the role of advising on future indicators for the QOF. A crucial part of the new process is the creation by NICE of an independent Primary Care Quality and Outcomes Framework Indicator Advisory Committee, which reviews existing indicators and recommends new ones in a participatory way (Rawlins and Moore 2009). The 2009/10 contract included 134 indicators. Each indicator has a maximum point value, and practices accumulate quality points according to their performance on the indicators, up to a maximum of 1,000 points. Achievement of points for most of the indicators is triggered at lower and upper target thresholds of attainment (percent of eligible patients reached). For other indicators payment is received when an action is confirmed, for example production of a relevant disease register. A sample of indicators in each domain with their point value is presented in Table 2. 4 This section is adapted from Cashin C. 2011. Major Developments in Results-based Financing (RBF) in OECD Countries: Country Summaries and Mapping of RBF programs. World Bank, Washington, D.C. 3 Table 2. Examples of Indicators in the Four Performance Domains of the 2009/10 U.K. QOF Domain Indicator Clinical Care (example—secondary prevention of coronary heart disease) The practice can produce a register of patients with coronary heart disease (4 points) The % of patients with newly diagnosed angina who are referred for exercise testing and/or specialist assessment (7 points) The % of patients with coronary heart disease whose notes have a record of blood pressure in the previous 15 months (7 points) The % of patients with coronary heart disease in whom the last blood pressure reading (measured in the previous 15 months) is 150/90 or less (17 points) The % of patients with coronary heart disease whose notes have a record of total cholesterol in the previous 15 months (7 points) The % of patients with coronary heart disease whose last measured total cholesterol (measured in the previous 15 months) is 5 mmol/l or less (17 points) The % of patients with coronary heart disease with a record in the previous 15 months that aspirin, an alternative anti-platelet therapy, or an anti-coagulant is being taken (unless a contraindication or side-effects are recorded) (7 points) The % of patients with coronary heart disease who are currently treated with a beta blocker (unless a contraindication or side-effects are recorded) (7 points) The % of patients with a history of myocardial infarction (diagnosed after 1 April 2003) who are currently treated with an ACE inhibitor or angiotensin II antagonist (7 points) The % of patients with coronary heart disease who have a record of influenza immunization in the preceding 1 September to 31 March (7 points) Organizational The blood pressure of patients aged 45 and over is 10 recorded in the preceding 5 years for at least 65% of patients (10 points) The practice supports smokers in stopping smoking by 2 a strategy which includes providing literature and offering appropriate therapy (2 points) There is a record of all practice-employed clinical staff having attended training/updating in basic life support skills in the preceding 18 months (4 points) The practice offers a range of appointment times to patients, which as a minimum should include morning and afternoon appointments five mornings and four afternoons per week, except where agreed with the PCO (3 points) There is a system for checking the expiry dates of emergency drugs on at least an annual basis (2 points) Patient Experience The length of routine booked appointments with the doctors in the practice is not less than 10 minutes (33 points) The % of patients who, in the appropriate national survey, indicate that they were able to book an appointment with a GP more than 2 days ahead (35 points) Additional Services (example—cervical screening) The % of patients aged from 25 to 64 (Scotland from 21 to 60) whose notes record that a cervical smear has been performed in the last five years (11 points) The practice has a system for informing all women of the results of cervical smears (2 points) The practice has a policy for auditing its cervical screening service, and performs an audit of inadequate cervical smears in relation to individual smear-takers at least every 2 years (2 points) The practice has a protocol that is in line with national guidance and practice for the management of cervical screening, which includes staff training, management of patient call/recall, exception reporting and the regular monitoring of inadequate smear rates (7 points) Source: (NHS Employers 2009) 4 The points are distributed in a way that more heavily weights indicators that have a higher estimated workload, many of which are closer to outcomes. For example, overall recording of patients with coronary heart disease is worth 4 points, while the percentage of patients with specific diagnostic information recorded is worth 7 points, and the percentage of patients with measured blood pressure below an acceptable threshold is worth 17 points. Patient experience indicators have high point values (over 30 points), while organizational indicators tend to have point values below 10 (U.K. NHS 2009). Incentive payments to GP practices are calculated on an annual basis. Either annual payments are made within three months of the end of the QOF year, or intermediate “aspirational� payments are made during the QOF year with a lump-sum payment made at the end of the year based on actual achievement (NHS Birmingham 2008). Practices are paid a flat rate based on the points they achieve (£127 per point in 2010/11). The reward is capped at a maximum of 1,000 points and the corresponding total bonus amount. Payments are adjusted for practice size and disease prevalence relative to national average values (Mason, et al. 2008). The program was criticized for not adequately compensating the extra work required to achieve quality targets in deprived areas (Hutchinson 2008), but this situation improved when the original payment formula was corrected in 2009 to better account for variations in disease prevalence (NHS Employers 2011). The QOF allows practices to “exception-report� (exclude) specific patients from data collected to calculate achievement scores. Patient exception reporting applies to those indicators in the clinical domain of the QOF where level of achievement is determined by the percentage of patients receiving the designated level of care. Patients can be excluded from individual indicators if, for example, they do not attend appointments or where the recommended treatment is judged as inappropriate by the GP (such as medication that cannot be prescribed due to side- effects). Table 3 provides the full set of exception-reporting criteria. Table 3. Criteria for Exception-Reporting Under the QOF Exception-Reporting Criteria 1 Patients who have been recorded as refusing to attend reviews - who have been invited on at least three occasions during the preceding 12 months. 2 Patients for whom it is not appropriate to review the chronic disease parameters due to particular circumstances (for example terminal illness or extreme frailty). 3 Patients newly diagnosed or who have recently registered with the practice or who should have measurements made within three months and delivery of clinical standards within nine months (for example blood pressure or cholesterol measurements within target levels). 4 Patients who are on maximum tolerated doses of medication whose levels remain sub-optimal. 5 Patients for whom prescribing a medication is not clinically appropriate such as those with an allergy, contraindication or who have experienced an adverse reaction. 6 Where a patient has not tolerated medication. 7 Where a patient does not agree to investigation or treatment (informed dissent) and this has been recorded in their medical records following a discussion with the patient. 5 9 Where the patient has a supervening condition which makes treatment of their condition inappropriate (for example, cholesterol reduction where the patient has liver disease). 8 Where an investigation service or secondary care service is unavailable. Source: NHS (2009A). Although it is a voluntary program, nearly all GP practices in the U.K. participate. In 2009 the program covered 8,229 GP practices and 99.7 percent of registered patients (The NHS Information Centre, Prescribing Support Unit 2009). Of the PCTs reviewed for this study, participation of GP practices appeared to be 100 percent, with the exception of NHS Brent where 12 practices (17 percent) failed to submit achievement data for the 2008/2009 QOF period, and thus effectively were not participants in the program (NHS Brent 2009). The reach of the QOF is also significant as a source of financing for GP practices. The average additional income from the QOF per GP practice was £74,300 in 2004-05 and £126,000 in 2005- 06, or about 25 percent of practice income. The size of the reward is large by international standards, and to date no other country experimenting with quality incentives is tying as large a proportion of income to quality of care (Campbell, et al. 2007). 4. MAJOR CHARACTERISTICS OF THE VERIFICATION METHOD Verification is carried out by QOF teams under the PCTs, which manage the contracts with GP practices under the supervision of the Strategic Health Authority, the local representation of the NHS (Figure 1). The QOF teams report to the PCT boards and are often accountable to another body within or related to the PCT. In NHS Kirklees, for example, a QOF Assurance Panel has been established to oversee the QOF process. This panel consists of PCT managers from the Finance, Contracting, Patient Care and Clinical Governance departments of the PCT (NHS Kirklees 2009). In NHS Northamptonshire, the QOF validation process is accountable to the PCT’s Contracts Approval Panel (NHS Northamptonshire 2009). 6 Figure 1. Structure of the Primary Care System Under the English National Health Service Source: (Cashin 2011) PCTs oversee the assessment of performance and calculation of scores, and carry out a three- pronged verification process: (1) Review visits to all GP practices at least once in three years (“QOF review visit�); (2) Pre-payment verification of achievement (“pre-payment verification�); and (3) Post-payment verification of 5 percent of practices randomly selected (“post-payment verification�). The first prong of the verification process has a supportive function and is focused on reviewing the practice’s expected achievement, identifying barriers to improvement, and assessing data quality. The second prong of the verification process is intended to confirm the validity of the data and other evidence submitted for the QOF payment. The third prong of the verification process has solely an audit function as part of the anti-fraud system. The NHS provides a significant amount of guidance to PCTs and GP practices for carrying out the verification process and undertaking annual review visits (NHS 2003, NHS 2003, NHS PCC 2009a, NHS PPC 2009b). Nonetheless, there is wide variation in the arrangements between PCTs and GP practices for the verification process (Audit Commission 2011). There is very little information on the time and resources used to implement the verification process. QOF reports indicate that the review visits are conducted over a 3-month period, with preparation and review of evidence submitted by practices followed by up to a full day for each GP practice visit (typically during the middle of the QOF year). The pre-payment verification of QMAS data and review of other supporting evidence submitted by GP practices takes about one to five weeks 7 (starting the week immediately following the end of the QOF year), and the post-payment verification visits are conducted over the one -two month period after payments are made. 4.1 OBJECTIVES OF THE QOF VERIFICATION PROCESS From the beginning of the QOF implementation, the verification process was intended to serve both a verification and a support function. The balance has evolved since the QOF began in 2004 from more of a “light touch� approach to a more rigorous verification process, which may reflect the changing political and financial circumstances facing the NHS and the large amount of public funds spent on the QOF (Pulse 2007). The more “light touch� approach was reflected in the first manual on the QOF verification process, which stated: “QOF assessment is an opportunity for both practices and PCTs to gain a greater understanding of their performance and their ability to improve quality of care� (NHS 2003). The stated objectives of the QOF review visits were to: (1) Review the practice’s current achievement and provide an assessment of likely achievement by the end of the QOF period; (2) Confirm that data collection and quality are accurate; (3) Discuss the practice’s aspiration for the following year. As the objectives of the QOF verification process evolved, objectives that focus on value-for- money can be seen in the specific policy statements of some PCTs. The NHS Northamtonshire verification policy, for example, specifies the following objectives tailored to local conditions (NHS Northamptonshire 2009): (1) To ensure equity across practices by introducing a rigorous countywide performance framework; (2) To ensure value for money and check actual achievement against GP practice claims and challenge where appropriate; (3) To facilitate an improvement in the quality of healthcare provided within general practice. 4.2 DATA SOURCES AND FLOWS Data for QOF verification come from three sources: (1) the GP practice clinical data system; (2) supplemental evidence supplied by GP practices; and (3) a national survey on patient experience. The achievement calculation, verification, and payment under the QOF are highly automated and use the electronic medical record in the GP clinical data system as its foundation for most indicators. The cornerstone of the clinical data used for QOF verification is the Quality Management Analysis System (QMAS). QMAS is a national system based on data from the GP clinical data systems that are anonymized to protect privacy. The GP clinical systems must be compliant with national system specifications and compatible with the QMAS. Early in the QOF 8 implementation, PCTs were expected to provide resources to upgrade the clinical systems of those GP practices that did not have compliant systems (U.K. Department of Health 2003). QMAS is directly linked to electronic medical records. Providers use electronic medical records to record patient-level data directly during the consultation. Each month the information for patients who meet the predefined criteria in the QMAS business rules for QOF indicators is automatically extracted from anonymized patient records in the GP practice clinical data system. The data are grouped into achievement levels and submitted electronically to QMAS (NHS PCC 2009). The QMAS uses this data to automatically calculate achievement and payment amounts. The QMAS generates achievement reports, which are sent to the GP practices to approve through an “Achievement Declaration.� After the GP practice approves the achievement report, it is sent by the QMAS to the PCT for pre-payment verification. Once pre-payment verification is completed, the PCT sends an approval for payment to the QMAS. The QMAS generates and sends payment details to the PCT payment agency, which makes the approved payment to the GP practice (see Figure 2). There are no patient-specific data in QMAS, because this is not required to support the QOF. For example, QMAS captures aggregate information for each practice on patients with coronary heart disease and on patients with diabetes, but it is not possible to identify or analyze information about individual patients (The NHS Information Centre, Prescribing Support Unit 2009). Patient-level analysis is part of verification, but this must be done on the local computer system of the GP practice (see 3.6 and 3.7 below). If there is a QOF review visit to the GP practice in that year, it takes place prior to the generation of the achievement report by the QMAS at the end of the QOF period. Interim reports are generated that are used by QOF assessors during the review visits. Thus the PCT can identify any errors or weaknesses in the data, and the GP practice can predict its achievement and payment level and take any remedial actions (R. McDonald 2009). 9 Figure 2. Information Flows for QOF Performance Achievement Calculation, Verification and Payment GP Practice GP Clinical System Payment Clinical System Other Data Achievement Achievement (Web-based Reports Data entry) Approve achievement and submit on-line “Achievement Declaration� QMAS Payment details PCT Payment Agency GP Practice’s achievement PCT approves submission achievement for payment PCT Conducts pre-payment verification Source: Adapted from U.K. Department of Health (2003) Supporting information is submitted by the GP practices to the PCTs through other channels to calculate achievement rates for non-clinical indicators. For example, data relating to most of the organizational indicators cannot be automatically extracted from the QMAS, so practices enter organizational data manually using forms on the QMAS website. The Oldham PCT Trust, for example, has developed an innovative electronic submission and support tool for the non-clinical indicators that is also being used to support other primary care improvement initiatives and has gained regional and national attention (Claridge and Beecroft 2010). Two indicators related to patient experience are generated from a quarterly patient mail/online survey, the “GP Patient Survey� (NHS 2010). Data for individual GP practices are submitted to the PCTs by the market research firm that implements the survey. The PCTs then manually enter the data into the QMAS. The patient survey serves other purposes for the NHS beyond QOF patient experience indicators (Ipsos MORI 2011). 10 Thus, at the end of the annual QOF period, the Practice’s final incentive payment for the year is calculated based on four separate data submissions to QMAS from the practice: (1) Disease Register. The GP Practice clinical system automatically submits Disease Register data to the QMAS at the end of the year. (2) Clinical Submissions. The practice’s clinical achievement is calculated from the data that are automatically extracted by the QMAS from the practice’s electronic medical records. (3) Non Clinical Submissions. The practice’s non-clinical achievement is calculated from the most recent non-clinical data submission made by the practice on QMAS on the date of the calculation. (4) The Practice List size. The number of individuals registered with the practice, or practice list size, is uploaded onto the QMAS at the beginning of each QOF period. When all of the clinical and non-clinical information are successfully entered into QMAS, and the practice has submitted a declaration that the data is correct, the scores are calculated automatically by specialized software (Checkland 2004). Practices can access QMAS and run reports to assess their performance whenever they wish. 4.3 THE QOF REVIEW VISIT The QOF review visit is an early step in the verification process, meant to give both GP practices and PCTs “early warning� of any issues related to data, reporting, or predicted performance achievement levels. When the QOF began in 2004 it was required that the PCTs conduct the review visits for all practices each year. The annual visit requirement was relaxed, and now it is recommended that PCTs conduct pre-payment assessment visits to each practice at least once in three years (U.K. National Audit Office 2008). Most PCTs also require a self-assessment form to be completed by all GP practices, whether or not they are visited in the current QOF year (NHS Kirklees 2009, NHS Brent 2008, Coulson 2011). 4.3.1 Assessor Team The QOF review visit is intended to be carried out by a team of QOF assessors, including one PCT manager (typically the QOF Lead), one external clinician, and one layperson (patient representative). The verification team members should participate in a two-day national standardized training. Extensive guidance is available for assessors in the form of guidance documents, web-based materials, and periodic educational seminars. In practice, there is wide variation in both the composition of the verification teams and their preparation. Some assessors have found their role and training lacking in clarity and focus, which some PCTs have addressed by drafting annual job descriptions for assessors (Audit Commission 2011). Even with ongoing training and standard job descriptions, some PCTs have found it challenging to maintain consistency in how assessors apply the verification methodologies, particularly consistency between lay assessors and clinicians (NHS Oldham 2012). 11 There also has been difficulty for some PCTs to recruit assessors, particularly lay assessors, which often are excluded from the verification teams (Audit Commission 2011, NHS Oldham 2012). The reasons for difficulty in recruiting assessors are not known, but it may be an issue of cost, or the reluctance of clinicians to serve in an auditing role for other clinicians. Some PCTs have addressed the latter issue by recruiting clinical assessors from outside of the local health authority, which may also contribute to objectivity (Audit Commission 2011). 4.3.2 Process for the QOF Review Visit The process for the QOF review visit starts with the scheduling of visits by the PCT team. According to the guidance documents, practices should be given at least two months’ notice to allow sufficient time to prepare documentation. In addition to QMAS data, practices are required to submit written evidence for key areas of verification. GP practices should submit their supporting information to the PCT one month before the visit date in the format of the most up to date achievement report form. Based on the information submitted by the practice, the PCT team should identify a selective list of topics, including matters for clarification or verification and areas for future development. The guidance documents suggest that the first part of the QOF assessment visit cover the review and verification of the Practice’s level of achievement at indicator level. The second part of the visit should be developmental, and the aim is to discuss the contractor’s future plans within the QOF, including the following year’s goals. Following the visit, the PCT QOF lead should draft a report of the visit setting out the main findings, conclusions, and subsequent actions. PCTs are expected to give the Practice the opportunity to see the report in draft and to challenge any factual errors and comment on its opinions and conclusions. Maintaining the consistency, objectivity, and standardization of the review visit has been a challenge for some PCTs. Some local health authorities have taken steps to ensure that the verification process is standardized. The Oldham and Western Cheshire PCTs, for example, developed templates for pre-payment verification visits. NHS Western Cheshire template is the same for both pre-payment and post-payment verification visits (see Annex 1). Nottinghamshire County PCT developed a Self-Assessment Form to standardize the review visit process (see Annex 2). There is wide variation in how PCTs follow up after QOF review visits. Some PCTs produce detailed visit reports with action plans and follow these up with return visits. In other cases, little or no action is taken in response to verification findings (Audit Commission 2011). 4.3.3 Targeting GP Practices and Indicators for the QOF Review Visit Most PCTs do not have the resources to conduct QOF review visits for all practices or to assess all QOF indicators, so they are moving from annual visits for all practices to one visit every three years per practices (U.K. National Audit Office 2008). Of the PCTs reviewed for this study, one PCT (Nottinghamshire County) has almost completely eliminated QOF review visits and replaced them with GP practice self-assessments due to concerns about the cost-effectiveness of the QOF review visits. QOF review visits were conducted on a three-year rolling basis until 2011, when NHS Nottinghamshire replaced them with practice self-assessments, except for new practices, and a patient record audit on a five percent random sample of GP practices (Coulson 2011). 12 Table 3 shows the diversity across PCTs in the two levels targeting both GP practices and indicators for QOF review visits. Of the PCTs reviewed, the percentage of GP practices visited for QOF review ranged from 5 to 100 percent in one annual QOF period. In general among the reviewed PCTs, targeting of GP practices for visits appears to be moving toward “risk-based� approaches, with higher risk practices more likely to be selected for QOF review visits. The risk-based criteria for selecting GP practices for visits mainly focus on low achievement outliers, but Northamptonshire included additional criteria such as outliers for exception reporting, significant organizational changes, or a new contract. Of the ten PCTs reviewed, four reported using risk-based targeting to select GP practices for QOF review visits (Heart of Birmingham, Northamptonshire, Nottinghamshire, and Oldham), two PCTs visit 100 percent of practices each year (Bromley and Solihull), three PCTs do not visit all practices and do not report using targeting (Brent, City and Hackney, and Western Cheshire), and one PCT visits a random selection of practices (Kirklees, with Nottinghamshire moving in that direction by 2011). The two PCTs that visit all practices are relatively small, with only 51 and 31 practices, respectively. The movement toward a risk-based approach to selecting a sub-set of GP practices for review visits may reflect a combination of a greater understanding of the QOF among GP practices (for example Heart of Birmingham), and the resource-intensity of such visits (for example Bromley and Nottinghamshire). The Bromley PCT annual QOF report stated the following: “it must be clear that the continuation of such a time intensive visiting program must demonstrably provide added value to both practices (in supporting them to achieve improved patient quality and outcomes) and the PCT in providing a level of assurance that the achievement of QOF indicators is leading to improved patient quality and outcomes.� (NHS Bromley 2009) Regarding targeting of indicators for verification, the verification team originally was required to cover all the domains for which the GP practice intended to submit an achievement claim. This guidance and what happens in practice appear to be evolving. The disease areas that are verified depend on the visit agenda, but the verification team may choose some indicators at random (NHS 2003). When it still conducted QOF review visits, Nottinghamshire County PCT coordinated practice-specific agendas by analyzing each practice’s QMAS data and written evidence one month prior to the visit. The indicators for review were selected on the basis of failure to submit adequate evidence one month before the visit or high/low QMAS percentages one month prior to the visit (Gash 2009). This approach was later changed, with a standard agenda used for all QOF review visits and assessment of all indicators, and the assessment visit team not reviewing QMAS data prior to the visit in order to carry out an unbiased review (Gash 2009). 13 Table 3. Frequency and Criteria for QOF Review Visits in Study PCTs PCT # (%) of Practices Criteria for Selecting Practices to be Criteria for Selecting Domains and Indicators to Visited Visited be Verified Brent 7 (10%) in 2007/2008; None Not specified increased to 100% in 2008/2009; then 30% on a 3-year rolling bases in 2009/2010 Bromley 51(100%) None 11 indicators selected based on the objectives: • To support the PCT goal to develop and improve services provided in primary care for people with long term conditions, ensuring QOF indicators are being used to best effect. • To follow up on issues arising from the evaluation of 2007/08 QOF, and their applicability to individual practice’s clinical care. Domains and indicators selected: Clinical domains: dementia, coronary heart disease, and diabetes. 4 organizational indicators; 5 records indicators Exception reporting Heart of 34 (45%) In previous years all practices were Emphasis on clinical indicators with less emphasis on Birmingham visited to ensure they understood the organizational domains. Teaching PCT QOF process and to provide support. Starting in 2007/08 PCT prioritized visits to practices whose performance was at the lower end of the performance scale, and several were randomly selected. In 2008/09 those practices not visited in the previous year were visited. City and Hackney 13 (30%) Not specified Not specified Kirklees 19 (25%) Random All non-clinical and patient experience domains and an agreed set of indicators from the clinical domain. Northamptonshire Not specified. Criteria for Year 1 (2007/08): Not specified 14 • Questions about the adequacy of demonstration of achievement • Outliers for prevalence in clinical domains • Outliers for use of exception reporting • Concern about capacity to provide high quality services • Low (or unexpectedly high) QOF achievement in previous year relative to PCT average • Significant organizational changes • New contract Those not visited in Year 1 will be considered for visits in future years. Nottinghamshire 38 (40%) 3-year rolling visit schedule. Initially selected on the basis of failure to submit County Year 1 (2008/09)—“lower end of the adequate evidence one month before the visit or achievement table� plus two higher high/low QMAS percentages one month prior to the scoring practices visit. Subsequent 2 years—those practices that This approach was later changed to selecting all were not previously visited indicators with no targeting. Visits replaced by practice self- assessments and 5% random patient Currently 3 records indicators and 4 clinical record review in 2011. indicators verified in 5% random sample patient record review. Oldham 12 (25%) Bottom 25% of performers • Disease areas with low prevalence compared to national prevalence; • Practice’s register validation process; • Clinical activity related to development and maintenance of disease registers; • Pathway of patient care Solihull 31(100%) None Not specified Western Cheshire 12 (30%) Three-year cycle 10 clinical indicators and 11 non-clinical indicators selected. Source: PCT QOF annual reports. 15 4.4. PRE-PAYMENT VERIFICATION Pre-payment verification is a routine check by PCTs of the QMAS data and other supporting evidence prior to final approval of the GP practice’s achievement report. According to national guidance, pre-payment verification should focus on (U.K. Department of Health 2003) the following: • Inexplicably low or high numbers of patients on disease registers given the PCT average prevalence (a result of not coding or miscoding patient records), or unusually high levels of exception reporting; • Evidence of a GP practice systematically and inappropriately referring patients to secondary care in order to maximize quality achievement points; • Substantial unexplained variation between expected achievement and achievement; • Suspected fraud or other illegality. Some guidance is given to PCTs on verification of both clinical and non-clinical indicators. For example, a patient’s inclusion on a disease register may be verified through a review of other supporting clinical evidence in the patient record, such as the prescription of disease-specific drugs. The PCTs seem to vary substantially, however, in how they carry out pre-payment verification. NHS Oldham, for example, reported that pre-payment verification consists of an analysis of the end-of-year clinical QMAS data for all practices by the Clinical Governance Team to identify areas of high exception reporting and/or unusual patterns of activity, which is triangulated with a general overview of prevalence and achievement. The team also completes a full status report on the organizational indicators for all practices (Claridge and Beecroft 2010). In the Birmingham PCT, pre-payment verification has focused on exception reporting. Exception reporting for specific areas and individual clinical indicators is checked against PCT and national averages and where a significant variance is identified the practice is asked to provide evidence that the level of exception reporting is both justified and accurate (NHS Birmingham 2008). The Brent PCT, on the other hand, reported focusing primarily on organizational indicators in its pre-payment verification (NHS Brent 2008). 4.5. POST-PAYMENT VERIFICATION Post-payment verification is a re-verification of the achievement QMAS data and submitted evidence for the previous QOF period. The guidance for carrying out post-payment verification stipulates that at a minimum, four areas should be examined in detail (NHS 2003): (1) Substantial discrepancies between the QOF pre-payment assessment report and the original achievement claim submitted; (2) High or low prevalence rates for disease areas compared to PCT or national averages that cannot be explained by related practice demographics; (3) High or low rates of exception reporting; (4) Sudden large changes in figures, particularly one month to the next. 16 A random five percent sample of GP practices should be checked thoroughly in a post-payment verification process as part of counter-fraud measures (U.K. Department of Health 2003). The post-payment verification should draw as much as possible on written material provided for the QOF review visit, if one was conducted at that practice during the most recent QOF period. PCTs appear to vary in how they carry out the post-payment verification process. There is no formal guidance on the composition of the post-payment verification team or what is considered to be a high, low, or substantial discrepancy. In many cases, the PCTs appear to use external audit agencies for this function, which may or may not be supplemented by assessors from the PCT (Coulson 2011). Other PCTs invite assessors from external PCTs (NHS Solihull 2007). The PCTs also vary in their approach to selecting indicators for post-payment verification. In the West Chestershire PCT, for example, the external audit firm that was contracted to conduct the verification visit focused on prevalence rates, exception rates, ten clinical indicators, and eleven non-clinical indicators to confirm achievement rates. Five patient records were reviewed (NHS Western Cheshire 2009). In the Bromley PCT, the clinical areas checked were selected based on a review of all of the clinical achievements of all practices. A selection of indicators from different clinical domains was tested in each practice. Several non-clinical domains were also selected for verification in each practice (NHS Bromley 2009). 4.6 HOW IS THE QUANTITY OF SERVICES VERIFIED FOR PRE- AND POST-PAYMENT VERIFICATION? The performance of GP practices in the QOF clinical domain is related to the share of registered patients in each disease area who receive the required services (“quantity of services�) and in the appropriate way (“quality of services�). Verifying the quantity of services requires verification of both the denominator (for example number of patients eligible for the service), and the numerator (the number of patients who received the service). Given that achievement of clinical indicators is directly related to the number of patients who are eligible for the service, the practice’s disease registers for priority conditions form an important backbone of the QOF and are an area often targeted for accuracy checks during verification (NHS PCC 2009). Some PCTs have developed protocols for placing patients on the registers for different diseases. Getting accurate disease registers is also of great importance to the GP practices, and some consulting services have sprung up with various electronic tools to help practices capture all eligible patients for their disease registers (Oberoi Consulting 2011, Insight Solutions 2011). There also is an emphasis on checking the accuracy and levels of exception reporting, since this is one area that is potentially vulnerable to gaming. 5 Verification of the number of eligible patients who received the service and exception reporting are mainly carried out through review of QMAS data during pre-payment verification and checks of patient records during QOF review and post-payment verification. Exception reporting is typically verified in the reviewed PCTs by comparing GP practice exception reporting rates for key indicators to PCT-wide and national averages, and identifying and further assessing evidence for the outliers. NHS Oldham, for example, examines practices more closely that have exception reporting rates and disease prevalence that fall outside thresholds established by the PCT. The QOF team requests more 5 There is some evidence of gaming relating to both exception reporting and disease registers (Gravelle 2010). 17 information on the clinical rational of the practice when the exception reporting rate is more than twice the national average, and when disease prevalence is either less than 80 percent or more than 120 percent of the national average (NHS Oldham 2012). NHS Nottinghamshire checks exception reporting for all practices for selected clinical indicators. Where higher than average exception reporting is detected, the PCT asks the practice to provide more detailed information for each patient through an Exception Reporting Query form (Coulson 2011). 4.7 HOW IS THE QUALITY OF SERVICES VERIFIED FOR PRE- AND POST-PAYMENT VERIFICATION? The quality of services is mainly verified through a random selection of patient records during QOF review and post-payment verification visits to verify the validity of a subset of the indicators. NHS Nottinghamshire, for example, devotes particular attention to the indicators within the clinical domain that require evidence of care planning and multi-disciplinary review (for example asthma, mental health, dementia, epilepsy and depression). Practices are asked to provide anonymized care plans so that the Clinical Assessor can check the content in line with the national guidance. Patient specific reports can only be generated manually on the practice’s local computer, and this is typically done by the QOF assessors on the day of the verification visit. The patients’ details in these reports are anonymous, and individual patients are identified only by a uniquely generated, random number and not by name (NHS 2003). Nonetheless, some concerns remain about patient confidentiality in the QOF verification process, and national guidance has been conflicting. 6 To help standardize the verification of the quality of services provided, the NHS has developed the QOF Assessor Validation Report clinical audit tool. This software randomly selects twenty patients and displays QOF-related entries from each patient’s record for the previous two years, including age, sex, observation type (e.g. blood pressure), medication, clinical notes, diagnosis, and co-morbidities. Each patient encounter is recorded, so assessors are able to link the diagnosis with all prescriptions and other services. It is not clear, however, how widely the PCTs use this tool, as it was not mentioned in any of the QOF annual reports. The non-clinical domains are considered to be measures of the structural aspects of quality of services. The supporting evidence required for verification of non-clinical indicators varies by PCT, but up to 40 policies and reports may have to be submitted by GP practices each year. For example, as evidence to verify achievement of “Education 6� indicator (“The practice conducts an annual review of patient complaints and suggestions to ascertain general learning points which are shared with the team�), the Northamptonshire PCT requires submission of a report or minutes of a minimum of two team meetings that summarize patient complaints and identify learning points and any changes that were made as a result of the complaint (NHS Northamptonshire 2009). For verification of the “Management 5� (“The practice offers a wide 6 The Department of Health guidance states that for QOF verification purposes data may be released to PCTs in patient identifiable form, but the reasons why must be documented and there must be a clear audit trail. However, the NHS confidentiality Code of Conduct states that patient consent should be sought if the use of de-identified data is not feasible (Gash 2009). 18 range of appointment times�) indicator, the practices are required to submit a practice leaflet showing the range of appointment times offered. 5. FINDINGS OF THE VERIFICATION METHODS In general, the PCTs reviewed report remarkably little discrepancy between reported achievement levels and verification results. The QMAS based on electronic medical records is highly reliable, and the uncovered incidence of fraud is very low. No reports were found of QOF payments being substantially reduced as a result of verification. In NHS Nottingham, for example, £17,000 was recovered in 2008/09. In NHS Northamptonshire, more robust verification only saved about £60,000 out of over £13 million in QOF payments (NHS Northamptonshire 2007). Even in the area of exception reporting, which has been noted as a potential area of gaming in the QOF, the average exception reporting rate has remained low on average, at approximately only 5 percent of patients (The NHS Information Centre, Prescribing Support Unit 2009). Discrepancies were typically found to be either valid or related to misunderstanding of the rules. For example, the Bromley PCT found a significant number of practices with exception reporting rates more than twice the national average, so exception reporting has been a major focus of their verification activities. Although most of the reasons were found to be unrelated to fraud, such as difficulty removing patients from the patient list when they are no longer registered with the practice, exception reporting continues to be a central focus of verification visits in this PCT (NHS Bromley 2009). There is evidence at the national level that problems with invalid exception reporting exist (Gravelle 2010), and the levels vary widely by indicator. Therefore, it is not clear whether the problem was less of an issue for the reviewed PCTs or if the verification process is not sensitive enough to identify gaming and fraud related to exception reporting. The verification process itself may contribute to keeping rates of gaming and fraud low. In addition, the overall low rates of discrepancy found during the verification process may be explained by a number of other factors. First, it is possible that the targets are not particularly challenging, and the QOF is mostly paying GP practices for what they have been doing all along (Hutchinson 2008). Second, it is possible that the verification process, particularly the automated reporting through the QMAS and the electronic medical record is a well-functioning system that reduces errors and is difficult to game. Finally, the GPs have had a significant increase in income tied to the introduction of the QOF and consistently achieve most of the possible bonus amounts. It is possible that further gains in income that would be possible through misreporting of performance indicators are marginal and simply not worth it. On the other hand, it is not too surprising that the discrepancy between reported achievement levels and evidence reviewed during the verification process is low given that GP practices are responsible for both paper and electronic patient notes, and are likely to ensure that the two correspond. Patterns of performance suggest some low level gaming/fraud does occur (Carey I 2009). Some discrepancies are found during pre-payment verification, and the NHS provides detailed guidance on how to resolve them. If during pre-payment verification the payment value for the practice calculated by QMAS is incorrect due to discrepancies in the data held on the GP practice 19 clinical system, the verification team may request additional evidence, such as a QMAS interim report submitted after the data on the GP clinical system has been corrected. Where no agreement can be reached it may be necessary to invoke local dispute resolution processes, and if these do not resolve the situation then the formal dispute resolution process is used. Guidance to PCT assessors is that assessors should only investigate further if they identify significant areas of concern and the GP practice cannot give a credible explanation (NHS 2003). Low rates of discrepancy are also found in the post-payment verification process. Although a number of PCTs reviewed noted deficiencies in the compilation of supporting evidence for non- clinical indicators (NHS Bromley 2009), only NHS Brent found significant discrepancies during post-payment verification visits. The five practices visited for random post-payment checks were not able to provide evidence to support all claims made relating to the non-clinical indicators. The visits also highlighted concerns with clinical achievement (NHS Brent 2009). 5.1 HOW ARE VERIFICATION FINDINGS USED? The verified QOF achievement data for GP practices are available on the QOF web site and open to the public after pre-payment verification and final approval. Individual GP practice achievement levels are shown for all indicators and also can be shown in comparison to the PCT and national averages. The extent to which patients use this information to make decisions about enrolling with a GP practice, however, is unknown. Information about post-payment verification typically is not made public, with the names of GP practices audited not even mentioned in QOF annual reports. Other consequences of the verification process appear to focus more on dialogue with the practices to improve data reporting and overall performance. In general, it appears to be rare for a PCT to reduce payments or withhold payments to GP practices as a result of pre-payment verification findings (Audit Commission 2011). Only one PCT reviewed reported withholding payments to GP practices as a result of pre-payment verification (NHS Brent 2008). The same PCT also found significant discrepancies during post-payment verification. In response to these post-payment discrepancies, the PCT recommended follow-up with the practices to examine further evidence, establish a working group to make the post-payment verification process more rigorous, and initiate “claw back� of funds where appropriate, although it is not clear whether any practices were actually required to return any QOF payments (NHS Brent 2009). Overall, it seems that the results of the QOF pre-payment verification process are more oriented toward developmental support to GP practices than consequences for discrepancies and fraud. Some PCTs provide additional support for low scoring practices in the form of supplemental monitoring, visits or telephone support, or other ways to help identify and address the reasons for low performance (Gash 2010). 6. VERIFICATION COSTS The costs of QOF verification include the start-up costs to establish the data system, the costs to the PCTs of carrying out the verification process, and the costs to GP practices of complying with the verification process. No cost estimates for any aspect of the design, implementation or compliance with the QOF verification system are publicly available, but the costs are likely to be 20 substantial. In 2004 alone, £30 million was made available to PCTs to upgrade clinical data systems and to provide systems for non-computerized practices (U.K. National Health Service, 2004). The cost of carrying out verification visits is also likely to be substantial. One PCT that carries out annual QOF review visits for all of its 100 GP practice reported having a team of 20 assessors, including 11 GPs (Audit Commission 2011). The costs to the GP practices of complying with verification are reduced by the highly automated data submission through the QMAS, but 25 percent of the information required for payment and verification during practice visits is not generated by the QMAS (NHS 2004a). The additional evidence required is in the form of specific reports prepared by the GP practice or inspections made by the verification team. The QOF guidance documents outline the types of evidence required for non-clinical indicators, which includes, for example, a “report on the results of a survey of a minimum of 50 medical records of patients who have commenced a repeat medication,� and a report of “the results of a survey of the records of newly registered patients.� There are at least 15 such reports that are specified in the guidance documents, with about half that need to be generated each QOF period and half that are one-off reports of policies and procedures that would not change every QOF period (NHS 2010). As a means to detect fraud and “claw back� over-payment to GPs, the QOF verification process is unlikely to be cost-effective given the low level of discrepancies found. The verification process does, however, appear to have additional value in some cases as a vehicle for dialogue between PCTs and GP practices to understand achievement levels and identify support needed to improve performance, although it is not clear how this value weighs against the additional cost. Furthermore, there is no estimate of potential fraud that has been deterred by the verification process, which would further add to its overall cost-effectiveness. 21 7. LESSONS LEARNED The verification process for the QOF is evolving, with lessons learned incorporated into the verification procedures each year. Some PCTs conduct evaluations of their QOF verification process and seek feedback from GP practices on the verification process (NHS Kirklees 2009, Gash 2010). The NHS Primary Care Commissioning (PCC) also sponsors periodic regional events to revisit the program of education and support for PCTs in carrying out QOF verification. Nonetheless, some questions are raised by the U.K. QOF experience with RBF verification. First, given the emphasis in the QOF initiative on improving accountability, there is a remarkable lack of transparency around verification. Reports may be publicly available, but it cannot be confirmed that this is widespread. None of the PCT websites visited for this review had a readily visible link to QOF results in general, or verification processes and results in particular. The PCTs varied widely in the detail around verification reports, with some as short as two pages. Second, it is unclear how costly the QOF verification process is, but it appears to be elaborate and highly labor-intensive. Given the high levels of GP performance and low rates of discrepancy and gaming, it is not clear whether the verification process contributes to low rates of gaming and fraud, and thus whether the investment in verification is cost-effective. Some PCTs are using the verification process for developmental dialogue with the GP practices, but the impact of this process on quality and outcomes has not been measured. The PCTs seem to be naturally moving to a more streamlined and targeted verification process, using risk-based targeting to select sub-sets of practices and indicators for the QOF review visit, which is the most intensive pre-payment verification. This approach combined with the random post-payment verification on a small share of GP practices may prove to be a more cost-effective approach as targeting sharpens the focus of verification resources toward the highest risk practices. In spite of the likely high costs and unclear role in accountability and fraud prevention, the QOF verification process seems to contribute to overall health system strengthening in the U.K. Improvement in data availability and use has been almost universally identified as a key positive “spillover effect� of the QOF and its verification process. Rates of recording may be increasing for all risk factors, even those without a QOF incentive payment attached (Sutton and McLean 2006), although recent research finds a different trend (Doran 2011). 7 7 The author conducted a longitudinal analysis of achievement rates for 42 activities (23 included in incentive scheme, 19 not included). There was no overall effect on the rate of improvement for non-incentivised indicators in the first year of the scheme, and by 2006/07 achievement rates for those indicators were significantly below those predicted by pre-incentive trends. 22 A number of PCTs emphasize the value of the QMAS beyond QOF verification. One report stated: “The system was established to support payments to GP practices under the QOF. However, its potential to provide information is recognized, for example, national prevalence and exception data has the potential to support commissioning, public health, governance, and performance management (NHS Solihull 2007).� Another important byproduct of the 3-pronged verification approach of the U.K. QOF is the opportunity for ongoing dialogue between the providers and the purchaser to support performance improvement. A notable feature of the U.K. QOF pre-payment verification process is how it is leveraged to provide support to GP practices to improve their data quality, as well as their overall performance. The separation between pre-payment “developmental� verification and post-payment “audit� verification creates a useful division between the different functions and allows a more cooperative approach. One PCT specifically noted the positive effect of the process on the relationship between the providers and PCT (NHS Oldham 2012). Key lessons learned from the U.K. QOF verification experience must be generated in light of the original objectives of the QOF, which were: to bring more money into the primary sector with greater accountability; and to improve quality (but given the already high rates of performance, this objective did not prove to be overly challenging). Against this backdrop, some key lessons from the U.K. QOF verification experience include the following: 1. Well-designed RBF verification can contribute to health system strengthening. Even when quality and performance levels are already high, the RBF and its accompanying verification process can contribute to health system strengthening in other ways. The QOF has been credited with strengthening the availability and use of health information and creating the opportunity for structured dialogue between the purchaser and providers in the spirit of supportive supervision. 2. Risk-based targeting of verification may increase its cost-effectiveness. Verification is likely to be costly, and developing risk-based criteria for selecting providers and indicators for verification may be more cost-effective. In particular, the combination of a paper-based review of automated data, risk-based targeting of a small set of providers for more intensive pre-payment verification, and the credible threat of random post-payment verification may prove to be most cost-effective. 3. It is essential to maintain the transparency and objectivity of the verification process. The transparency and fairness of verification is critical to its credibility and acceptance by providers as a basis for their payment and developmental dialogue with the purchaser. It is an obvious point, but the balance between validity and affordability, and between transparency and confidentiality is not always easy to maintain in the U.K. QOF verification. Although transparency is questioned because of the lack of publicly available information, PCTs appear to be attempting to standardize their verification processes and engaging the right assessors with the right skills, including patient representatives. It appears particularly important to ensure: 23 • Clear objectives for verification, which may evolve over time • Standardized tools and processes for verification and valid business rules • Independent assessors that have standardized job descriptions and appropriate training on an ongoing basis to address system changes. 24 REFERENCES Audit Commission. 2011. Paying GPs to Improve Quality. London: Audit Commission. Campbell, Stephen, David Reeves, Evangelos Kontopantelis, Elizabeth Middleton, Bonnie Sibbald, and Martin Roland, 2007. "Quality of Primary Care in England with the Introduction of Pay for Performance." New England Journal of Medicine 357, no. 2: 181- 190. Carey I., et al. 2009. "Blood Pressure Recording Bias During a Period When the Quality and Outcomes Framework was Introduced." J Hum Hypertens 23, no. 11: 764-770. Cashin, Cheryl. 2011. Major Developments in Results-based Financing (RBF) in OECD Countries: Country Summaries and Mapping of RBF Programs. World Bank, Washington, D.C. Checkland, K., M. Marshall. and S. Harrison. 2004. "Re-thinking Accountability: Trust versus Confidence in Medical Practice." Quality and Safety in Health Care 13: 130-135. Claridge, Tanya, and Pat Beecroft. 2010. Quality and Outcomes Framework 2009-10. NHS Oldham. Clemmer, Terry P. 2004. "Monitoring Outcomes with Relational Databases: Does it Improve Quality of Care?" Journal of Critical Care 19, no. 4: 243-247. Coulson, Julie. 2011. Quality and Outcome Framework Review and Assessment Process: Guidance for GP Practices. Nottingham City: NHS Nottinghamshire. Doran, Tim. 2011. "Effect of Financial Incentives on Incentivized and Non-Incentivized Clinical Activities: Longitudinal Analysis of Data from the U.K. Quality and Outcomes Framework." BMJ 342: d3590. Doran, Tim., et al. 2011. "Effect of Financial Incentives on Incentivised and Non-incentivised Clinical Activities: Longitudinal Analysis of Data from the UK Quality and Outcomes Framework." BMJ 342: d3590. Gash, Glenna. 2009. Nottinghamshire County Quality and Outcomes Framework Annual Report 2008-2009. NHS Nottinghamshire. _____. 2010. Nottinghamshire County Quality and Outcomes Framework Annual Report 2009- 2010. NHS Nottinghamshire. Government of the U.K. 2000. "The NHS Plan." Gravelle, Hugh., Matt Sutton, and Ada Ma. 2010."Doctor Behavior under a Pay for Performance Contract: Treating, Cheating and Case Finding?" The Economic Journal: F129-F156. 25 Hutchinson, Brian. 2008. "Pay for Performance in Primary Care: Proceed with Caution, Pitfalls Ahead." Health Care Policy 4, no. 1: 10-15. Insight Solutions. 2011. Data Quality Assessment. http://insightsol.co.uk/fliers/Data_Quality_Assessments_Service_Flier_May_11.pdf (Accessed August, 2011). Ipsos MORI. 2011. GP Patient Survey Results – National Reports and Data. http://www.gp- patient.co.uk/results/ (Accessed September 2011). Mason, Anne., Simon Walker, Karl Claxton, Richad Cookson, Elizabeth Fenwick, and Mark Sculpher. 2008. The GMS Quality and Outcomes Framework:Are the Quality And Outcomes Framework (QOF) Indicators a Cost-Effective Use of NHS Resources? York: Centre for Health Economics, University of York. McDonald, Ruth. 2009. "Pay for Performance in Primary Care in England and California: Comparison of Unintended Consequences." Annals of Family Medicine 7, no. 2: 121-127. _____. 2010. "Nottingham University Business School." Financial Incentives for Quality Care in UK Primary Medical Care. Jaipur. NHS (National Health Service). 2003. QOF 5% Random Counter Fraud Checks. London: U.K. National Health Service Primary Care Contracting. _____. 2003. Practice QOF Assessment Visits. London: U.K. _____. 2004a. General Guidance on QOF Annual Review Process. U.K. National Health Service. _____. 2004b. National Programme for Information Technology: GMS IT-QMAS Bulletin. London: U.K. _____. 2007. Minutes: PCT Board Meeting July 4, 2007. NHS Northamptonshire. _____. 2007. "General Medical Service Contracts: Monitoring of the Quality and Outcomes Framework." Solihull. _____. 2008. Summary of Report to Professional Executive Committee. NHS Birmingham. _____. 2008. Update on Commissioning GP Services. NHS Brent. _____. 2009. "QOF Annual Report 2008/2009." Brent. _____. 2009. GMS & PMS Contract: Quality Outcomes Framework (QOF) Annual Report 2008/09. Bromley. 26 _____. 2009. "Quality and Outcomes Framework Annual Summary Report 2008/09." Kirklees. _____. 2009. "NHS Nortamptonshire GP Quality and Outcomes Framework validation policy 2009/10." Northamptonshire. _____. 2009. "Primary Care Performance Quality and Outcomes Framework 2008/09: Post Payment Verification/Random Counter Fraud Check Report." Western Cheshire Primary Care Trust Board Meeting, Western Cheshire, 2009. _____. 2010. "Annual Report on Qualit and Outcomes Framework in Primary Care." City and Hackney. _____. 2010. QOF Management Gude: Volume 4. U.K. _____. 2010. "Prescribing Incentive Scheme 2010/11." West Sussex. _____. 2011. Pay and Contracts. Employers. http://www.nhsemployers.org/PayAndContracts/GeneralMedicalServicesContract/QOF/P ages/ChangesToQOF200910.aspx (Accessed November 2011). _____. 2012. "Personal Communication." Oldham. NHS PCC (National Health Service Primary Care Contracting). 2009. QOF Management Guide: Volume 1. London: U.K. . _____. 2009a. Quality and Outcomes Framework: Guidance for GMS Contract 2009/10. U.K. _____. 2009b. QOF Management Guide; volume 2. London: U.K. NICE. 2010. National Institute for Health and Clinical Excellence. http://www.nice.org.uk/aboutnice/qof/indicatorsindevelopment.jsp (Accessed May 22, 2010). Oberoi Consulting. 2011. Disease Register Validation. http://www.oberoi-consulting.com/drv/ (Accessed August 2011). OECD. 2010. OECD Health Data 2010. http://www.oecd.org/document/11/0,3343,en_21571361_44315115_45549771_1_1_1_1, 00.html (Accessed November 2010). Pulse. 2007. "PCTs to Save Millions in QOF Pay." Pulse. Rawlins, Michael, and Val Moore. 2009 "Helping to Provide High Quality Care in Primary Care." Ulster Medical Journal 78, no. 2: 82-83. Serumaga, Brian., et al. 2011. "Effect of Pay for Performance on the Management and Outcomes of Hypertension in the United Kingdom: Interrupted Time Series Study." BMJ 342:d108. 27 Sutton, Matt, and Gary McLean. 2006. "Determinants of Primary Medical Care Quality Measured under the New UK Contract: Cross Sectional StudY." BMJ 332: 389–390. The NHS Information Centre, Prescribing Support Unit. 2009. Quality and Outcomes Framework Achievement Data 2008/09. The Health and Social Care Information Centre. _____. 2009. Quality and Outcomes Framework Exception Data 2008/09. The Health and Social Care Information Centre. U.K. Department of Health. 2003. Delivering Investment in General Practice: Implementing the New GMS Contract. London, UK. U.K. National Audit Office. 2008. NHS Pay Modernisation: New Contracts for General Practice Services in England. London: Report by the Comptroller and Auditor. U.K. NHS (National Health Service). 2009. "Quality and Outcomes Framework: Guidance for GMS Contract 2009/10." 28 ANNEX 1. TEMPLATE FOR VERIFICATION VISITS IN NHS WESTERN CHESHIRE 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 ANNEX 2. NOTTINGHAMSHIRE COUNTY PCT SELF-ASSESSMENT FORM PRACTICE NAME: QUALITY & OUTCOMES FRAMEWORK ORGANISATIONAL INDICATORS 2011/12 Practice Self Declaration of Achievement For 2011/12 all practices are required to submit a self declaration of achievement of the organisational indicators. The Practice should indicate on this form which indicators have been met in full. If any other additional evidence is required along with this form you will be advised accordingly by your Contract Manager. Please note that all policies, procedures and guidelines must have been reviewed during the current year and updated where necessary to reflect current practice. All documents must show the date of last review and the next scheduled review date. Please check carefully the "Declarations by Practice" requirements for each indicator and indicate achievement by stating “Y� in the right hand column. If you are not aspiring to a particular indicator, please indicate NAT (not aspiring to) in the right hand column. Once the practice has reviewed/achieved all the indicators and the form has been completed, please sign and date the declaration at the bottom of the form and return it to your Primary Care Contract Manager. All forms should be returned by 29 February 2012 to allow sufficient time for them to be checked and indicators agreed for sign off before 31 March 2012. st st Any forms received after 31 March or any indicators that have not been completed by 31 March cannot be signed off by the PCT and the practice’s final QOF achievement on QMAS will be adjusted to reflect this. Please contact your Primary Care Contract Manager if you have any queries about completing the form. For full details of indicator requirements please refer to the "Green Book" QOF Guidance which can be downloaded from: http://www.bma.org.uk/images/qofguidancefourthversion2011_v2_tcm41-205262.pdf Please continue to keep your organisational evidence portfolio up to date as the PCT auditors may carry out random audits of organisational domain achievement. This includes keeping a record of which medical records were checked by the practice for Records Management 9, 15, 18, 19 and 20; and Medicines Management 11 and 12. 44 Achieved “Y� or Not Indicator Points Indicator Requirements Declarations by Practice Aspiring to “NAT� Practice confirms that a written procedure is in place, that this has been reviewed, amendments to system A system for transferring and acting on information about documented (if any) and review date noted on Records 3 1 patients seen by other doctors out of hours document A designated place for the recording of drug allergies and adverse reactions in the notes and these must be clearly Practice confirms that drug allergies and adverse Records 8 1 recorded reactions are clearly recorded in patients records The practice has conducted a survey of the drugs by randomly selecting 50 patients' records, listing the eligible drugs from these records, and has identified the For repeat medicines, an indication for the drug can be percentage of these drugs that have an indication in the identified in the records (for drugs added to the repeat records. Please note here the percentage achieved Records 9 4 prescription after 1.4.04. Minimum standard 80% and attach separate survey proforma as evidence. The blood pressure of patients aged 45 and over is No action by the practice. Achievement is recorded on QMAS - PCT to assess recorded in the preceding 5 years for at least (11) 65% QMAS at QOF year end Records 11, 17 15 and (17) 80% of patients A system to alert the out of hours service to patients dying Records 13 2 at home Practice confirms that a system is in place. The practice has conducted a survey of a minimum of 50 patients records recording the percentage that have clinical summaries and the percentage that are up to The practice has up-to-date clinical summaries in at least date. Please note here the percentage achieved and Records 15, 18, 20 45 (15) 60%, (20) 70% and (18) 80% of patient records attach separate survey proforma as evidence. The practice has conducted a survey of records received between 8 and 26 weeks previously (a minimum of 30 records or all the records if less than 30 such registrations) noting if the records have been received and summarised. Please note here the 80% of newly registered patients have had their notes percentage achieved and attach separate survey Records 19 7 summarised within 8 weeks of receipt by practice. proforma as evidence. The percentage of patients aged over 15 years whose No action by the practice - achievement is recorded on QMAS - PCT to assess Records 23 11 notes record smoking ststus in the past 27 months QMAS at QOF year end Practice confirms that a written protocol is in place, that this has been reviewed, amendments documented (if The practice supports smokers in stopping smoking by any), review date noted on document and all Information 5 2 providing literature and offering appropriate therapy appropriate literature is available and is up-to-date Evidence of all staff having attended basic life support training - clinical staff within the last 18 months; all other Practice confirms that ALL staff have attended BLS Education 1, 5 7 staff within last 36 months training within appropriate timescales. 45 Practice confirms that they have documented their patient's suggestions and complaints as they have occurred during the year and have subsequently discussed these at a team meeting, identifying the learning points from these and any The practice conducts annual review of complaints and changes/improvements that have needed to be made suggestions to ascertain general learning points which are and that minutes of the meeting are available for Education 6 3 shared with the team. inspection if required. Practice confirms that a significant event review case report is documented for any event that occurs as listed in the QOF Guidance for these indicators and the format of the review case reports are in line with the suggestions given in the QOF Guidance. Practice confirms that during the last year/three years the A minimum of 3 significant event reviews undertaken in minimum number of significant event reviews have the past year (Ed 10) and 12 reviews to have been been undertaken and the review case reports are Education 7, 10 10 undertaken in the past 3 years (Ed 7) available for inspection if required. Practice confirms that ALL practice nurses have had an Practice employed nurses have personal learning plans appraisal during this QOF year and that this included a Education 8 5 which have been reviewed at annual appraisal personal learning plan. Practice employed non-clinical team members have an Practice confirms that ALL non-clinical staff have had Education 9 3 annual appraisal an appraisal during this QOF year. Practice confirms that an up-to-date local child Individual healthcare professionals have access to protection procedures manual is available and information on local procedures relating to child accessible to all health care professionals and they all Management 1 1 protection know of its whereabouts Practice confirms that a written policy is in place, that this has been reviewed, amendments documented (if There are clearly defined arrangements for backing up any) and review date noted on document. NOTE - any computer data, back-up verification, safe storage of back- practice that has migrated to SystmOne this year up tapes, and authorisation for loading programmes will need to review their policy as back up Management 2 1 where a computer is used arrangements will have changed. Practice confirms that the Hepatitis B status of all The hepatitis status of all doctors and relevant practice relevant staff is known and recorded and available for Management 3 0.5 employed staff is recorded inspection if required. Practice confirms that they offer a range of appointment The practice offers a range of appointment times AT times as required and that these are clearly stated in Management 5 3 LEAST five mornings and four afternoons per week the practice leaflet. The practice has systems in place to ensure regular and appropriate inspection, calibration and replacement of equipment including: • a defined responsible person • Practice confirms that systems are in place as required clear recording • systematic pre-planned schedules • and that a log of inspection and maintenance is kept Management 7 3 reporting of faults and is available for inspection if required. The practice has a protocol for the identification of carers Practice confirms that a written protocol is in place, that and a mechanism for the referral of carers for social this has been reviewed, amendments documented (if Management 9 3 services assessment. any) and review date noted on document. 46 There is a written procedures manual that includes staff Practice confirms that a written procedures manual, employment policies, including equal opportunities, including all appropriate employment policies, is in bullying and harassment and sickness absence (including place and accessible to all staff - that all policies have illegal drugs, alcohol and stress), to which staff have been reviewed, amendments documented (if any) and Management 10 2 access review dates noted on each policy Practice confirms that they possess appropriate Medicines The practice possesses the equipment and in date equipment to treat anaphylaxis and emergency drugs Management 2 2 emergency drugs to treat anaphylaxis are in date Medicines There is a system for checking the expiry dates of Practice confirms that they have a system in place for Management 3 2 emergency drugs on at least an annual basis checking the expiry dates of emergency drugs 4 - Practice confirms availability of prescriptions for The number of hours from requesting a prescription to collection in under or up to 72 hours availability for collection by the patient is (4) 72 hours or Medicines less or (8) 48 hours or less (excluding weekends and 8 - Practice confirms availability of prescriptions for Management 4, 8 9 bank/local holidays) collection in under or up to 48 hours The practice meets the PCT prescribing advisor at least Medicines annually and has agreed up to 3 actions related to Management 6 4 prescribing Verification of this indicator will be obtained by the QOF assessment team from the Medicines Management Team at the PCT Medicines The practice provides evidence of change/improvement in Management 10 4 relation to the 3 agreed prescribing actions A medication review is recorded in the notes in the Medicines preceding 15 months for all patients being prescribed four The practice has conducted a survey of a minimum of Management 11 7 or more repeat medicines. Standard 80%. 50 patients records recording the percentage that have a medication review noted. Please note here the percentage achieved and attach separate survey A medication review is recorded in the notes in the proforma as evidence. Medicines preceding 15 months for all patients being prescribed Management 12 8 repeat medicines. Standard 80%. The practice conducts internal and external peer reviews of prescribing and agrees plans for three areas of Quality and improvement producing reports to the PCT detailing the Practices to submit reports to Medicines Management Team as per separate productivity 1 - 2 13 two reviews and the three improvement areas. guidance. Verification of these indicators will be obtained by the QOF assessment team from the Medicines Management team who will be collating all performance data and reports from practices Quality and The percentage of prescriptions complying with the productivity 3 - 5 15 agreed plans for each of the three improvement areas The practice conducts internal and external peer reviews of practice data on secondary care outpatient referrals Practices to submit reports to Primary Care Contract Managers as per separate and engages with the development of and follows three guidance. Verification of these indicators will be obtained by joint assessment of agreed pathways for improving the management of reports by the QOF assessment team and CCG Commissioning and Development patients to avoid inappropriate outpatient referrals - and Management Team Quality and produces a report to the PCT detailing the two reviews productivity 6 - 8 21 and all action taken in respect of the three agreed 47 pathways The practice conducts internal and external peer reviews of practice data on emergency admissions and engages Practices to submit reports to appropriate CCG Commissioning and Development with the development of and follows three agreed Managers as per separate guidance. Verification of these indicators will be pathways in the management and treatment of patients to obtained by joint assessment of reports by the QOF assessment team and CCG avoid emergency admissions - and produces a report to Commissioning and Development Management Team Quality and the PCT detailing the two reviews and all action taken in productivity 9 - 11 47.5 respect of the three agreed pathways Practice has referred to "Green Book" QOF Guidance pp 149 - 150 and confirms that the practice meets the Patient The length of routine booked appointments with the 75% requirement for consultations and appointments to Experience 1 33 doctors in the practice is not less than 10 minutes. be booked at least at 10 minute intervals The percentage of patients aged 25 to 64 whose notes Please note here the percentage of eligible patients Cervical record that a cervical smear has been performed in the whose notes record that they have had a cervical smear Screening 1 11 last 5 years. Standard 40% - 80% in the last 5 years Practice confirms that a system is in place and that this Cervical The Practice has a system for informing all women of the is documented in the practice's policy for the Screening 5 2 results of cervical smears management of cervical screening. The practice has a policy for auditing its cervical screening service, and performs an audit of inadequate Practice confirms that a written policy is in place and Cervical cervical smears in relation to individual smear-takers at that an audit has been carried out within the last 2 Screening 6 2 least every 2 years years. The practice has a protocol that is in line with national guidance and practice for the management of cervical screening which includes staff training, management of Practice confirms that a written protocol is in place, that Cervical patient call/recall, exception reporting and the regular this has been reviewed, amendments documented (if Screening 7 7 monitoring of inadequate smear rates. any) and review date noted on document Practice confirms that a written child health surveillance Child Health Child development checks are offered at intervals that are programme protocol is in place, that this has been Surveillance 1 6 consistent with national guidance and policy reviewed and updated as necessary. Practice confirms that written guidelines on antenatal care and screening are in place and that these have been reviewed and updated as necessary. The practice confirms that there is a shared care policy with Antenatal care and screening are offered according to midwives and that patients have the choice of midwife Maternity 1 6 current local guidelines and/or GP care The practice can produce a register of women who have Practice confirms that a register has been produced. been prescribed any method of contraception at least Achievement is recorded on QMAS - PCT to assess Contraceptive once in the last year, or other appropriate interval eg last QMAS at QOF year end Services (SH) 1 4 5 years for an IUS 48 The percentage of women prescribed an oral or patch contraceptive method who have also received information from the practice about long acting reversible methods of Contraceptive contraception in the previous 15 months (payment stages Practice has referred to "Green Book" QOF Guidance Services (SH) 2 3 40% - 90%) pp159 - 162 and confirms that written and verbal The percentage of women prescribed emergency information on LARCs is being provided. Achievement hormonal contraception at least once in the year by the is recorded on QMAS - PCT to assess QMAS at QOF practice who have received information from the practice year end about long acting reversible methods of contraception at Contraceptive the time of, or within one month of, the prescription Services (SH) 3 3 (payment stages 40% - 90%) Practice confirms that regular meetings have taken place. PCT will also verify with Gold Standard Framework/End of Life team that standards are being met (if End of Life Team does not have evidence of Regular multidisciplinary meetings (eg with district nurses) recent practice visit the PCT may require the practice to must be held, at least 3 monthly, where all palliative care provide additional evidence that meetings are taking Palliative Care 2 3 patients are discussed. place) DECLARATION: PRACTICE NAME: I declare that the practice has met the requirements of the QOF indicators marked as "Y" in the self declaration document above Signed on behalf of the practice (one partner can sign on behalf of the practice): Print Name: Date: 49 medication review noted. Please note here the percentage achieved and Medicines A medication review is recorded in the notes in the preceding 15 months for all attach separate survey Management 12 8 patients being prescribed repeat medicines. Standard 80%. proforma as evidence. The practice conducts internal and external peer reviews of prescribing and agrees Quality and plans for three areas of improvement producing reports to the PCT detailing the two Practices to submit reports to Medicines Management productivity 1 - 2 13 reviews and the three improvement areas. Team as per separate guidance. Verification of these indicators will be obtained by the QOF assessment team from the Medicines Management team who will be collating all performance data and reports from practices Quality and The percentage of prescriptions complying with the agreed plans for each of the productivity 3 - 5 15 three improvement areas Practices to submit reports to Primary Care Contract The practice conducts internal and external peer reviews of practice data on Managers as per separate guidance. Verification of secondary care outpatient referrals and engages with the development of and these indicators will be obtained by joint assessment of follows three agreed pathways for improving the management of patients to avoid reports by the QOF assessment team and CCG Quality and inappropriate outpatient referrals - and produces a report to the PCT detailing the Commissioning and Development Management Team productivity 6 - 8 21 two reviews and all action taken in respect of the three agreed pathways Practices to submit reports to appropriate CCG The practice conducts internal and external peer reviews of practice data on Commissioning and Development Managers as per emergency admissions and engages with the development of and follows three separate guidance. Verification of these indicators will be agreed pathways in the management and treatment of patients to avoid emergency obtained by joint assessment of reports by the QOF Quality and admissions - and produces a report to the PCT detailing the two reviews and all assessment team and CCG Commissioning and productivity 9 - 11 47.5 action taken in respect of the three agreed pathways Development Management Team Practice has referred to "Green Book" QOF Guidance pp 149 - 150 and confirms that the practice meets the 75% requirement for consultations and Patient The length of routine booked appointments with the doctors in the practice is not appointments to be booked at Experience 1 33 less than 10 minutes. least at 10 minute intervals Please note here the percentage of eligible patients whose notes record Cervical The percentage of patients aged 25 to 64 whose notes record that a cervical smear that they have had a cervical Screening 1 11 has been performed in the last 5 years. Standard 40% - 80% smear in the last 5 years Practice confirms that a system is in place and that this is documented in the practice's policy for the Cervical management of cervical Screening 5 2 The Practice has a system for informing all women of the results of cervical smears screening. Practice confirms that a The practice has a policy for auditing its cervical screening service, and performs an written policy is in place and Cervical audit of inadequate cervical smears in relation to individual smear-takers at least that an audit has been carried Screening 6 2 every 2 years out within the last 2 years. 50 Practice confirms that a written protocol is in place, The practice has a protocol that is in line with national guidance and practice for the that this has been reviewed, management of cervical screening which includes staff training, management of amendments documented (if Cervical patient call/recall, exception reporting and the regular monitoring of inadequate any) and review date noted Screening 7 7 smear rates. on document Practice confirms that a written child health surveillance programme protocol is in place, that this Child Health Child development checks are offered at intervals that are consistent with national has been reviewed and Surveillance 1 6 guidance and policy updated as necessary. Practice confirms that written guidelines on antenatal care and screening are in place and that these have been reviewed and updated as necessary. The practice confirms that there is a shared care policy with midwives and that patients have the choice of midwife Maternity 1 6 Antenatal care and screening are offered according to current local guidelines and/or GP care Practice confirms that a register has been produced. The practice can produce a register of women who have been prescribed any Achievement is recorded on Contraceptive method of contraception at least once in the last year, or other appropriate interval QMAS - PCT to assess Services (SH) 1 4 eg last 5 years for an IUS QMAS at QOF year end Practice has referred to The percentage of women prescribed an oral or patch contraceptive method who "Green Book" QOF Guidance Contraceptive have also received information from the practice about long acting reversible pp159 - 162 and confirms Services (SH) 2 3 methods of contraception in the previous 15 months (payment stages 40% - 90%) that written and verbal information on LARCs is being provided. Achievement The percentage of women prescribed emergency hormonal contraception at least is recorded on QMAS - PCT once in the year by the practice who have received information from the practice to assess QMAS at QOF year Contraceptive about long acting reversible methods of contraception at the time of, or within one end Services (SH) 3 3 month of, the prescription (payment stages 40% - 90%) Practice confirms that regular meetings have taken place. PCT will also verify with Gold Standard Framework/End of Life team that standards are being met (if End of Life Team does not have evidence of recent practice visit the PCT may require the practice to provide additional Regular multidisciplinary meetings (eg with district nurses) must be held, at least 3 evidence that meetings are Palliative Care 2 3 monthly, where all palliative care patients are discussed. taking place) 51 DECLARATION: PRACTICE NAME: I declare that the practice has met the requirements of the QOF indicators marked as "Y" in the self declaration document above Signed on behalf of the practice (one partner can sign on behalf of the practice): Print Name: Date: 52 About this series... This series is produced by the Health, Nutrition, and Population Family (HNP) of the World Bank’s Human Development Network. The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual authors whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@worldbank.org) or HNP Advisory Ser- vice (healthpop@worldbank.org, tel 202 473-2256, fax 202 522-3234). For more information, see also www.worldbank.org/hnppublications. The world bank 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org