54512 H N P D i s c u s s i o N P a P e R Impact Evaluation of the Egyptian Health Sector Reform Project - Pilot Phase Rebekka Grun and Javier Ayala August 2006 Impact Evaluation of the Egyptian Health Sector Reform Project Pilot Phase Rebekka Grun Javier Ayala In co-operation with: Ali Elazeem Gazalah, Amira Tahio, Bassyouni Zaki Salem, Hany Talaat, Isaac El Mankabadi, Laila Mostafa, Marwa El Shorbagy, Mohamed Abdel Rahman, Mohamed Nouh, Nagwan El Sammak, Samir El Nemaki, Tarek Timraz August, 2006 Health, Nutrition and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population Family (HNP) of the World Bank's Human Development Network. The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author(s) whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor, Homira Nassery (HNassery@worldbank.org). Submissions should have been previously reviewed and cleared by the sponsoring department, which will bear the cost of publication. No additional reviews will be undertaken after submission. The sponsoring department and author(s) bear full responsibility for the quality of the technical contents and presentation of material in the series. Since the material will be published as presented, authors should submit an electronic copy in a predefined format (available at www.worldbank.org/hnppublications on the Guide for Authors page). Drafts that do not meet minimum presentational standards may be returned to authors for more work before being accepted. For information regarding this and other World Bank publications, please contact the HNP Advisory Services at healthpop@worldbank.org (email), 202-473-2256 (telephone), or 202-522-3234 (fax). © 2008 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition and Population (HNP) Discussion Paper Impact Evaluation of the Egyptian Health Sector Reform Project Pilot Phase Rebekka Gruna Javier Ayalab a Middle East and North Africa Human Development Department, World Bank, Washington DC, USA b Georgetown University, Washington DC, USA The Impact Evaluation was done for the Bank-funded Health Sector Reform Project, and partially funded by Middle East and North Africa Department, World Bank Abstract: In 1997, the Government of Egypt (GoE) launched the Health Sector Reform Program (HSRP). Under the program, persistent needs in maternal and child health were addressed through an emphasis on primary health care, as well as through the introduction of the family health model as the principle of primary care. The World Bank, in co-operation with the Technical Support Office (TSO) of the HSRP at the MOH, has conducted an impact evaluation of the different interventions under the reform, focusing on the service delivery component. The evaluation has examined the impact of the HSRP on targeting those in need; coverage and utilization of primary health care, its quality of service and maternal and child health. It has also compared costs and benefits of the interventions and derived lessons for policy going forward. Key suggestions include 1) Human Resource development should be extended to include staffing mechanisms, 2) supervision of Quality of Care could be strengthened locally through empowering citizens, 3) a co-payment can support a shift towards primary care ­ if designed carefully, and 4) the lack of impact of the HSRP on antenatal and natal care warrants re-thinking the focus of outreach activities. Keywords: primary health care, health care reform, service delivery, Egypt, Alexandria, Menoufia, Sohag, Qena, Ministry of Health Egypt, Impact Evaluation, Difference in Differences Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Rebekka E. Grun, The World Bank, 1818 H Street N.W., Washington, DC 20433, USA. Tel: 1-202-473-4984. Email: rgrun@worldbank.org, www.worldbank.org/mena-hd iii List of Acronyms CBA Cost-Benefit Analysis DHS Demographic and Health Survey EC European Commission FHF Family Health Fund FHC Family Health Center FHU Family Health Unit GOE Government of Egypt HIO Health Insurance Organization HSRP Health Sector Reform Project IMR Infant mortality rate L.E. Egyptian Pounds (Livres Egypciennes) M&E Monitoring and Evaluation MMR Maternal mortality rate MOH Ministry of Health, Cairo NGO Non-governmental organization PAD Project appraisal document pc per capita (in the catchment area of a health facility) PHC Primary Health Care ppt Percentage points TSO Technical Support Office TST Technical Support Team USAID United States Agency for International Development VfM Value for Money WB World Bank iv TABLE OF CONTENTS Executive Summary ........................................................................................................... ix The Health Sector Reform Program .............................................................................. ix Data & Methodology ...................................................................................................... x Targeting ......................................................................................................................... x Coverage and utilization ................................................................................................. x Quality of Service .......................................................................................................... xi Maternal and Child Health ............................................................................................. xi Future Reform Strategies .............................................................................................. xii 1. Introduction ................................................................................................................. 1 2. The Egyptian HSRP .................................................................................................... 1 The Program.................................................................................................................... 2 Targeting ......................................................................................................................... 7 3. Coverage and Utilization of Health Services ............................................................ 10 Coverage ....................................................................................................................... 10 Utilization ..................................................................................................................... 14 Drivers of utilization ..................................................................................................... 16 4. Quality of Service Delivery ...................................................................................... 22 Quality Supervision ...................................................................................................... 22 Patient Satisfaction........................................................................................................ 26 5. Health of the Population ........................................................................................... 32 Maternal Health ............................................................................................................ 32 Child Health .................................................................................................................. 35 6. Cost Effectiveness ..................................................................................................... 39 Cost-effectiveness of the HSRP Package ..................................................................... 39 Cost-effectiveness of HSRP interventions .................................................................... 41 7. Conclusion and Lessons for Policy Makers .............................................................. 42 The Egyptian HSRP shows some successes ................................................................. 42 These successes can be supported through modifications of the original interventions42 8. Annexes..................................................................................................................... 45 Data Sources ................................................................................................................. 45 Econometric Methods ................................................................................................... 48 Econometric Results ..................................................................................................... 51 References ......................................................................................................................... 58 v FIGURES Figure 1: Infrastructure investment by facility ................................................................... 3 Figure 2: Infrastructure investment by facility, by governorate ......................................... 4 Figure 3: Infrastructure investment by beneficiary, by governorate................................... 5 Figure 4: Cost of General Training per Facility.................................................................. 6 Figure 5: Cost of Technical Training per Facility .............................................................. 7 Figure 6: District level targeting outcome of the infrastructure investment ....................... 9 Figure 7: Estimated share of families on roster (average) ................................................ 11 Figure 8: Physicians per families in catchment ................................................................ 12 Figure 9: Average number of days with drugs out of stock .............................................. 13 Figure 10: Average number of days with drugs out of stock: most problematic facilities 13 Figure 11: Share of generics in prescriptions ................................................................... 14 Figure 12: Number of visits to physicians, by governorate .............................................. 15 Figure 13: Utilization of all primary facilities, 2000-2005 ............................................... 15 Figure 14: Referrals per physician-encounters, by governorate ....................................... 16 Figure 15: Encounters per physician vs. accreditation score ............................................ 17 Figure 16: Encounters per physician vs. customer satisfaction score ............................... 18 Figure 17: Encounters per physician per year vs. share of family-health trained physicians ........................................................................................................................................... 18 Figure 18: Encounters per physician vs. infrastructure investment .................................. 19 Figure 19: Distribution of accreditation scores, 2005 ....................................................... 22 Figure 20: Trend in accreditation scores, by governorate ................................................. 23 Figure 21: Accreditation and share of practice-ready physicians ..................................... 24 Figure 22: Accreditation and infrastructure investment ................................................... 24 Figure 23: Share of facilities that obtain above 90% of patient satisfaction, by governorate ....................................................................................................................... 27 Figure 24: Patient satisfaction and accreditation score ..................................................... 28 Figure 25: Patient satisfaction and share of family-health trained physicians.................. 28 Figure 26: Patient satisfaction and infrastructure investment ........................................... 29 Figure 27: Patient satisfaction and number of prescriptions per physician visit .............. 29 Figure 28: Patient satisfaction and % of visits that results in referrals ............................. 30 Figure 29: Trends in maternal health indicators, 2000-2005 ............................................ 32 Figure 30: Anemia in women and children, 2000-2005 ................................................... 33 Figure 31: Share of institutionalized deliveries, reformed facilities 2003-2005 .............. 33 Figure 32: Distribution of institutionalized deliveries, end 2004 ..................................... 34 Figure 33: Trends in child health indicators, 2000-2005 .................................................. 35 Figure 34: Diarrhea mortality rate, 2003-2005 ................................................................. 36 Figure 35: Diarrhea death rate and coverage .................................................................... 37 Figure 36: Diarrhea death rate and care utilization ........................................................... 37 Figure 37: Diarrhea death rate and family health training ................................................ 38 vi TABLES Table 1: Impact of the HSRP on maternal health indicators ............................................. xi Table 2: Impact of the HSRP on child health indicators .................................................. xii Table 3: Total infrastructure investment by governorate .................................................... 3 Table 4: Incidence of infrastructure investment by poverty decile, district level ............... 8 Table 5: Current and potential coverage of the HSRP (number of people) ...................... 10 Table 6: Impact of the HSRP on primary care utilization ................................................ 20 Table 7: Impact of the HSRP on accreditation scores ...................................................... 25 Table 8: Impact of the HSRP on maternal health indicators ............................................ 35 Table 9: Econometric analysis of the impact of the HSRP on child health ...................... 39 Table 10: Incremental benefits of the HSRP as a package ............................................... 40 Table 11: Incremental investment per facility and per beneficiary (LE) .......................... 40 Table 12: Annual incremental cost of the integrated HSRP package, per facility and per beneficiary......................................................................................................................... 40 Table 13: Benefits per cost of individual HSRP interventions ......................................... 41 Table 14: Econometric analysis of the impact of the HSRP on primary care utilization . 51 Table 15: Econometric analysis of the impact of the HSRP on the diarrhea mortality rate in children ......................................................................................................................... 53 Table 16: Econometric analysis of the impact of the HSRP on the facility accreditation score .................................................................................................................................. 54 Table 17: Econometric analysis of the impact of the HSRP on vaccination coverage, anemia, and contraception ................................................................................................ 56 Table 18: Econometric analysis of the impact of the HSRP on fever treatment of children, and ..................................................................................................................................... 57 CASE STUDIES Case Study Box 1: Al Battanoon, Shibin Al Kum, Menoufia .......................................... 21 Case Study Box 2: San Estefano, FHC, El Raml, Alexandria .......................................... 26 Case Study Box 3: Facilities of El Amreya and El Wakkad, Amriyyah District, Alexandria ......................................................................................................................... 31 vii ACKNOWLEDGEMENTS The authors are grateful to the World Bank for publishing this report as an HNP Discussion Paper. They also thank various inspirers, encouragers and contributors: Akiko Maeda, Alaa Hamed and Sami Ali from the World Bank, for initiating and supporting the impact evaluation of the HSRP, Nagwan El Sammak, Rania Fares, Mohamed Nouh and Marwa El Shorbagy at the Egyptian Ministry of Health (MOH), for several evaluation contributions; Ali Elazeem Gazalah, Amira Tahio, Bassyouni Zaki Salem, Hany Talaat, Laila Mostafa, Mohamed Abdel Rahman, Samir El Nemaki, and Tarek Timraz, at the MOH, for facilitation and discussion of qualitative and quantitative data, Dr Isaac El Mankabadi, formerly at the MOH, for coordination of the Egyptian evaluation team; and Heba El Gazzar, Magnus Lindelow and Paul Geli at the Bank, for proof-reading and valuable feedback. viii EXECUTIVE SUMMARY THE HEALTH SECTOR REFORM PROGRAM In 1997, the Government of Egypt (GoE) launched the Health Sector Reform Program (HSRP). Under the program, persistent needs in maternal and child health were addressed through an emphasis on primary health care, as well as through the introduction of the family health model as the principle of primary care. The HSRP was introduced in a first phase in the pilot governorates of Alexandria, Menoufia and Sohag (1998-2004) and subsequently extended to Qena and Suez (2004-2005). This pilot phase addressed both the delivery and the financing of primary health services and came to a close in 2006. The service delivery component included interventions regarding the renewal of infrastructure and equipment; human resource development, centered around family health training; and quality assurance, through a system of accreditation standards and a regular inspection schedule for health care facilities. The financing component envisaged the re-channeling of funds from direct financing to contracted financing through Family Health Funds (FHF) at the governorate level. This component has not yet been completed. The financing component also envisaged affiliating the uninsured with a non-linear price system at the point of delivery, requiring a one-off payment for opening a file (30 L.E.) and a co-payment for each visit (3 L.E.).1 At the time of this report, the MOH is interested in knowing the impact of the pilot phase of the HSRP. The World Bank, in co-operation with the Technical Support Office (TSO) of the HSRP at the MOH, has therefore conducted an impact evaluation of the different interventions under the reform, focusing on the service delivery component. The evaluators have drawn on a variety of data sources, such as national surveys, the MOHs Monitoring & Evaluation System for the HSRP, and administrative records. Rigorous econometric techniques have been applied wherever the data allowed, and completed by graphical analysis and qualitative case studies. The evaluation has examined the impact of the HSRP on targeting those in need; coverage and utilization of primary health care, its quality of service and maternal and child health. It has also compared costs and benefits of the interventions and derived lessons for policy going forward. We summarize our findings below. 1 Co-payment has been institutionalized with Decree 147. ix DATA & METHODOLOGY This evaluation draws on various quantitative data sources: (i) the monitoring database of the facilities accredited under the HSRP, (ii) administrative data from the MOH, (iii) the national Demographic and Health Surveys 2000 and 2005 and (iv) the Population and Housing Census 1996. The quantitative data has been backed up with (v) qualitative data from field visits to four primary care facilities in Alexandria and Menoufia. The data has been analyzed with descriptive and econometric methods as described in the companion manual (Grun 2006). We would like to underline that the econometric impact evaluation presented here is ,,second best as it has been conducted retrospectively, i.e. after the targeted implementation of the HSRP had already happened. However, panel data with a baseline, both from HSRP facility monitoring, and from the DHS surveys, allow us to match early and later entrants to the HSRP carefully, and thereby establish a comparison. Still, some measurement error, and fuzzy targeting of the intervention, may bias our results and are candidly discussed in the Annexes (section 8), together with a general explanation of data sources and econometric methods used. TARGETING The interventions have been deliberately targeted to poor and under-served districts. This report shows that the health infrastructure investment is fairly pro-poor2, with over 30% going to the poorest decile of districts, and about 50% of the infrastructure value going to the poorest 3 deciles. COVERAGE AND UTILIZATION After the pilot phase, the HSRP covers (i.e. has on roster) 5m people, and this coverage rate is rising. However, we observe regional discrepancies in equipment. On the one hand, some regions report more medical staff per population than generous health systems in Western Europe, while others struggle to recruit personnel. Further, there appear to be localized difficulties with access to basic supplies, such as essential drugs. Four out of five facilities reporting a very problematic development regarding drug stocks reside in Sohag. Various reform interventions can be shown to increase utilization (i.e. people actually visiting the facilities). In general, family health training and renewal of medical equipment drive utilization up, and requiring a co-payment drives it down somewhat. A high utilization rate is also associated with a high quality of service (measured through both accreditation scores and customer satisfaction), and innovative outreach activities of the facility. Further, reassuringly in the light of the reform objectives, the HSRP as an integrated package occasions a shift from secondary to primary care. Also, while patient referral rates between primary and secondary care seem to be increasing in some reformed facilities, the phenomenon appears to cede to the extent that primary facility staff learn 2 Using district-level poverty-deciles from the 1996 Social Fund III poverty map. x from the referral reports and feel equipped to perform recommended treatments themselves. QUALITY OF SERVICE At the end of 2005, 399 facilities are accredited3, with an average accreditation score of 75.5%. While unusually many facilities score just above 80%, the quality trend varies across governorates and is declining at the time of this evaluation. We find that investment in buildings and medical equipment as well as the FHF incentive payments likely improve accreditation outcomes. A qualitative case study suggests that competitive recruitment and retention of staff are key features that influence accreditation outcomes. Patient satisfaction seems to improve with a high accreditation score as well as with family health training and infrastructure investment. Patients are also more satisfied if they receive a referral or prescriptions. Qualitative evidence from the field suggests again that competitive access to human resources is crucial for a facility to satisfy its customers. This may explain why patient satisfaction differs markedly across regions. MATERNAL AND CHILD HEALTH The HSRP as an integrated package improves maternal health regarding nutrition and family planning, but does not have an impact on antenatal and natal care, see summary in Table 1. Table 1: Impact of the HSRP on maternal health indicators Share of Share of couples Share of Anemia rate in women having using modern professionally women* 4 or more ANC contraception attended births visits HSRP integrated +4ppt -7ppt O O package ppt= percentage points. *indicating malnutrition. Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. Note that the DHS allows us to analyze the impact at district level only, which makes it difficult to identify the individual interventions, as their phasing-in does not vary as much at district level as it does at facility level. The programme was also associated with an improvement in child health regarding child vaccination rate, child access to medical treatment for fever or cough, and child diarrhea mortality rate, see Table 2. (Please note that the first two columns draw on different data than the last one.) 3 Of which 62 in Alexandria, 128 in Menoufia, 75 in Qena, 118 in Sohag and 16 in Suez. xi Table 2: Impact of the HSRP on child health indicators Vaccination coverage Medical treatment Diarrhea mortality rate (measles) when fever/cough HSRP integrated +4ppt +22ppt* (-)** package Family health training Impact within Impact within -1ppt package package if all doctors trained -0.01ppt Impact within Impact within per L.E. per Civil works investment package package person Medical Impact within package Impact within package O equipment*** ppt= percentage points. *would require rollout in every single facility. Other impacts require only rollout in some facilities per district, without large sensitivity to extent of rollout. **Sizenot identifiable ue to data availability. ***Note that the impact of medical equipment and furniture cannot be identified separately; the same is true for FHF contracting and co-payment. Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. The successes of the HSRP were obtained at an estimated annual incremental expenditure per beneficiary of 32LE (including co-payment revenue), and per facility of 434,224LE. FUTURE REFORM STRATEGIES In order to expand the successes of the HSRP, we propose chiefly four modifications of the current interventions. 1. Human Resource Development should be extended to include staffing mechanisms While all individual interventions showed an impact, the quantitative and qualitative evidence for Human Resource Development was strongest. In particular, the case studies suggest it should be a priority to replicate the competitive recruitment possibilities of successful urban facilities in remote and rural areas. Among the internationally applied policies that have worked well towards this objective are: merit scholarships to medical students on the condition of a rural placement; military conscripts placed temporarily in health care facilities for menial tasks; a ,,smart subsidy fund to be auctioned competitively to high quality contractors in rural areas. ­ Details see chapter 7. 2. Supervision of Quality of Care could be strengthened locally through empowering citizens The new Quality Supervision is an integral component of the HSRP. Our evaluation suggests that a higher accreditation score is linked to higher customer satisfaction and higher utilization. However, we find evidence of localized supplies shortage in rural areas. This raises the question what contributes to this situation, and whether processes are as supervised in remote areas as they are in more central regions. xii One solution would be to empower citizens locally to participate in monitoring and supervising their services. This has successfully been implemented in various international contexts, e.g. through: publishing supply allocations of facilities via newspaper, radio or the Mosques; equipping staff with cameras that take dated pictures, and require daily pictures to be sent with the monitoring data, in order to control absenteeism; establishing an ,,escalation ladder of options for citizens to complain about service delivery, to local, regional, and national instances. ­ Details see chapter 7. 3. The co-payment can support a shift towards primary care ­ if designed carefully Our evaluation finds that financial incentives under the HSRP, such as the incentive pay and co-payments have strong impacts. In order to take advantage of this dynamic, the co- payment mechanism could be modified to support the observed shift from secondary to primary care: giving secondary care a higher co-payment than primary care; and to support the increasing number of visits per person: restricting the co-payment to a one-off fee, rather than a per-visit fee. However, these findings have to be read in the context that, a priori and in the short term, our evidence showed the new co-payment in primary care to reduce utilization, and to shift it towards non-reformed facilities. 4. The lack of impact of the HSRP on antenatal and natal care warrants re-thinking the focus of outreach activities We have found that the integrated HSRP package failed to have an impact on the take-up of antenatal and natal care. This begs the question whether facilities reach out effectively to mothers-to-be, and what an effective outreach could look like. Our results encourage an enquiry into this direction. xiii 1. INTRODUCTION In 1997, the Ministry of Health (MOH) of the Government of Egypt (GoE) launched a comprehensive Health Sector Reform Program (HSRP) aiming to develop a national health system that would address sector issues in equity, access, efficiency, quality and financial sustainability. The HSRP has been supported by various donors including the World Bank (WB), the European Commission (EC), and the United States Agency for International Development (USAID). The HSRP was introduced in a first phase in the pilot governorates of Alexandria, Menoufia and Sohag (1998-2004) and subsequently extended to Qena and Suez (2004- 2005). This phase concentrated on primary health care (PHC) and addressed both the delivery and the financing of health services in Egypt. The service delivery component included interventions in infrastructure, human resources, management and quality assurance, while the financing component envisaged the re-channeling of funds from direct financing to contracted financing through Family Health Funds (FHF) at the governorate level. It also envisaged affiliating the uninsured with a non-linear price system at the point of delivery, requiring a one-off payment for opening a file (30 L.E.) and a co-payment for each visit (3 L.E.).4 The phase covering the five pilot governorates is called the Family Health Project and will come to a close over the course of 2006. At this stage, the MOH is interested in knowing the impact of the pilot phase of the HSRP, in order to learn for the allocation and implementation of funds under the subsequent phases. This report is the result of a joint effort of the MOH and the WB to answer this request. It evaluates the key interventions under the first phase of the HSRP, concentrating on the service delivery component. The available data allows measuring the interventions in infrastructure and training, as well as the introduction of FHF contracts and patient co- payments. It does not allow measuring the actual realization of management changes. Therefore, the structure of this report reflects the focus on service delivery. The next chapter (2) describes the implementation of the HSRP until 2005, chapter 3 observes trends in coverage and utilization of health services, chapter 4 and 5 analyze the impact of the HSRP on quality of service delivery and health indicators, respectively, chapter 6 compares the cost-effectiveness of the different interventions and chapter 7 derives the lessons learned. A series of Annexes illustrates data sources and methods. 2. THE EGYPTIAN HSRP In 1997, the GoE diagnosed that the countrys health system suffered from various ailments. Both primary and secondary care was delivered through a highly fragmented system of service units, and especially infant, child and maternal mortality rates were still at unacceptable levels.5 Previous programs had addressed specific issues in an isolated 4 Co-payment has been institutionalized with Decree 147. 5 HSRP Strategy Paper (1997) 1 manner, but had failed to make a sustainable impact. In the light of this, the MOH launched the HSRP as an integrated reform effort. THE PROGRAM The HSRP addressed the various observed shortcomings in the Egyptian health system in a comprehensive manner. Under the program, the fragmented delivery structure was consolidated. Further, the needs in maternal and child health were addressed through an emphasis on primary health care, as well as through the introduction of the family health model as the principle of primary care. These objectives required a series of interventions, strengthening the primary care infrastructure, staffs competence in addressing family health needs, and the overall quality of service. Infrastructure In order to strengthen the infrastructure, primary care was consolidated into only two types of provision, Family Health Units (FHU) and Family Health Centers (FHC). FHUs are now the primary contact between the beneficiary and the health system and provide basic outpatient services. FHC offer limited specialist outpatient and inpatient services, and count with a small specialist staff and nurses. Staff levels and equipment generally follow a plan made to international contemporary standards. Both units refer to hospitals in case of need for more specialist or intensive care. In the course of the consolidation, existing delivery units were transformed to meet the criteria for either FHU or FHC. This entailed remodeling through civil works, and in some areas of need, new construction. It could also mean downgrading and leaving some specialties to hospitals, but in most cases it meant upgrading and renewing/adding modern equipment, such as sterilization ovens, delivery chairs and dentist chairs, or in the more specialized FHC, ultrasounds, x-rays, and hematological and cytological labs.6 At the time of the evaluation, 332 facilities have received infrastructure investment, with a total investment amount of round 830m LE, see Table 3. 6 MOH 2003 and authors Field Visits in Alexandria and Menoufia FHUs and FHCs, Nov 17-21, 2005 2 Table 3: Total infrastructure investment by governorate FHU FHC Number Investment (LE) Number Investment (LE) Alexandria 45 139,521,837 19 93,905,753 Menoufia 69 149,448,335 16 78,497,285 Qena 57 68,548,214 8 15,347,513 Sohag 90 235,369,342 7 25,561,996 Suez 19 17,468,133 2 5,862,955 Sum 280 610,355,860 52 219,175,502 Source: MOH, Health Sector Infrastructure Development 2006. Figures concern health facilities only and exclude administrative infrastructure. FHU constitute over 80% of the facilities and claim over 70% of the investment amount. This reflects their role as the key entry point for patients under the new service delivery organization. The average total infrastructure investment per facility is 2.5m L.E. Most of this is spent on civil works, see Figure 1. Figure 1: Infrastructure investment by facility 3,000,000 Total: 2,498,588 L.E. 2,500,000 180,450 432,485 2,000,000 Furniture Medical Equipment 1,500,000 Civil Works 1,000,000 1,885,653 500,000 - Average investment per facility Source: MOH, Health Sector Infrastructure Development 2006; and author's calculations. The average investment by facility in both civil works and medical equipment resulted highest in Alexandria, see Figure 2. 3 Figure 2: Infrastructure investment by facility, by governorate 4,000,000 3,500,000 Furniture Investment 3,000,000 Medical Equipment Civil works Investment Average L.E. by facility 2,500,000 2,000,000 1,500,000 1,000,000 500,000 - Alexandria Sohag Menoufia Quena Suez Source: MOH, Health Sector Infrastructure Development 2006; and author's calculations. However, this is mainly due to bigger catchment areas in Alexandria. The average investment by beneficiary (in the catchment population) is higher in Menoufia and Sohag, the pilot governorates representing rural and Upper Egypt, respectively. 4 Figure 3: Infrastructure investment by beneficiary, by governorate 250 Furniture 200 Medical equipment Civil works by beneficiary in catchment Average L.E. invested 150 100 50 - Menoufia Sohag Alexandria Suez Quena Source: MOH, Health Sector Infrastructure Development 2006; and author's calculations. Please note that average beneficiary numbers per facility are calculated from those facilities that make numbers available only, Note that the relatively low investment registered in Quena and Suez in both graphs reflects partially that these Governorates joined the pilot group later, and partially that the investment needs were initially under-estimated and later adjusted. Human Resources The infrastructure changes were preceded or accompanied by a comprehensive training package for facility staff. The package centered round Family Health practice for both physicians and nurses, but also contained subject-specific training for the various non- medical specialists in the facilities, such as pharmacists, lab technicians, sanitarians and social workers.7 Further, substantial administrative changes were introduced via training, such as reaching out to and rostering families, and keeping medical records electronically and in family folders. Operationally, training was administered in two waves: a wave of general training courses, addressing more than one staff category at once, and a wave of technical courses, addressing each staff category on its own. In the second wave, facilities were trained together, in order to fill training spaces. 7 PAD 1998 and records held by the MOH team for Human Resources development (Dr Samir El Nemaki). 5 The total cost of one full training package for an average facility8 would be just below L.E. 88,000. General training would amount to a total of L.E. 18,000, see Figure 4. Figure 4: Cost of General Training per Facility 18000 500 2000 16000 14000 3500 HSRP Concept Orientation 12000 Housing Enumeration & Family Registration 10000 3500 Ministerial Decree No 147 L.E. 8000 Family Folders 3500 6000 Basic Computing 4000 Quality Concept & Accreditation 5000 2000 0 Cost per facility Source: MOH data (2006), Human Resources Development team. Note: Ministerial Decree 147 regulates co-payment rules ..while technical training for an average facility would amount to nearly L.E. 70,000, see Figure 5 . 8 An average facility counts with 5 physicians, 8 nurses, 1 dentist, 2 pharmacists, 2 lab technicians and 2 social workers. Source: MOH M&E data, field visits. 6 Figure 5: Cost of Technical Training per Facility 70,000 Drug Logistic System (Pharmacists) 1,675 3,150 60,000 Dental Services (Dentists) 50,000 Immunization (Sanitarians) 26,250 40,000 Vital Event Registration (Sanitarians) L.E. Laboratory Services 30,000 (Lab technicians) Family Practice (Head Nurses) 20,000 35,000 Introduction Family Medicine (Physicians) 10,000 Family Practice (Nurses) - Cost for an average facility Source: MOH data (2006), Human Resources Development team, M&E data and field visits. Quality supervision In order to improve and ensure quality in primary care, the MOH developed accreditation standards and a regular inspection schedule for facilities, in line with international standards. Further, facility staff was trained on the requirements for service quality, e.g. in the area of infection control (hand washing and sterilization). Quality was trained across staff categories, see Figure 4 ,,Quality Concepts & Accreditation. Quality supervision was further strengthened through a re-definition of roles between the MOH at the central-, and governorates, districts and facilities at the de-central level. The MOH now establishes quality norms and the accreditation mechanism, while the Family Health Funds at the governorate level pay incentives for high quality performance, and districts decide on the allocation of investment and staffing in the facilities.9 TARGETING The scope and complexity of the interventions described above make clear that the HSRP could not be rolled out universally at once. Rather, the MOH had to make choices and determine regions and beneficiaries of priority. In line with these, the implementation had to be phased. 9 MOH 2003 and authors Field Visits to MOH and Alexandria and Menoufia FHUs and FHCs, Nov 17-21, 2005 7 The pilot governorates for the first phase of rollout (1998-2004) were Alexandria, Menoufia, and Sohag, because they represented the three major regions in Egypt, urban, Lower and Upper Egypt, respectively. In a second phase (2004-2005), Qena and Suez were added. Within all five governorates, priority was reportedly given according to poor and underserved regions, health status, concentration of women and children, commitment to reform, administrative capacity, presence of the HIO and representativeness and replicability.10 In the actual allocation process, household surveys first assessed the need for healthcare, and then informed master-plans which determined the consequent infrastructure- investment and its location. Regional targeting followed a socioeconomic vulnerability index.11 The needs-based master-plans also served to reduce infrastructure in areas of excess capacity; and informed staffing patterns. Benefit incidence analysis: Infrastructure Investment The data available on infrastructure investment allows us to explore which parts of the population benefited most, the poorest 10%, the next poorest 10%, or the richer deciles? Below we present a benefit incidence analysis of the infrastructure investment, by poverty decile, see Table 4.12 Table 4: Incidence of infrastructure investment by poverty decile, district level Infrastructure District poverty deciles investment (L.E.) Share13 (%) 1 257,686,604 31.1% 2 37,516,519 4.5% 3 107,795,194 13.0% 4 137,511,639 16.6% 5 24,977,674 3.0% 6 - 0.0% 7 11,600,497 1.4% 8 119,367,179 14.4% 9 37,222,339 4.5% 10 95,853,716 11.6% Total 829,531,361 100% Source: Population and Housing Census 1996, Social Fund III poverty map; Health Sector Infrastructure Development 2006; and author's calculations. The intervention appears to be fairly pro-poor, with over 30% going to the poorest decile of districts, and about 50% of the infrastructure worth going to the poorest 3 deciles. 10 Antos et al. (2004) 11 Antos et al (2004): The social vulnerability index used 8 socioeconomic indicators: illiteracy ratio, unemployment ratio, income dependency ratio, inaccessibility to electricity, inaccessibility to potable water, average family size, household crowding factor and the population size of the village. 12 Note that our data only allows conducting this analysis at the district level, which is less precise than conducting it at the household level 13 Of overall health infrastructure investment under the HSRP. 8 Plotting the investment share against the share in the population allows discerning how progressive the program is, see Figure 6. An investment-share curve above a 45-degree line against the population-share signifies a progressive benefit incidence (=poorer deciles receive more than their weight in the population), while a line below 45-degree signifies a regressive program. From Figure 6 we see that the infrastructure investment under the HSRP has been allocated in a progressive manner. Figure 6: District level targeting outcome of the infrastructure investment 100% Share of infrastructure investment 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% - 1 2 3 4 5 6 7 8 9 10 District poverty deciles Source: Population and Housing Census 1996, Social Fund III poverty map; Health Sector Infrastructure Development 2006; and author's calculations. 9 3. COVERAGE AND UTILIZATION OF HEALTH SERVICES This chapter examines how many people have access to reformed primary care, and how they make use of the enhanced offer. We analyze the coverage with primary care services, their utilization through beneficiaries and finally, the drivers of utilization. At this stage we only look at the general utilization rate. Take up of specific services such as maternal care and vaccination will be discussed in chapter 5, Health of the Population. COVERAGE At the time of this report, the HSRP covers 5m people At the date, the interventions discussed in the previous chapter cover an estimated total of 4.97m14 people in 414 contracted facilities. If facilities had their entire catchment areas on roster, as envisaged, they would benefit 5.59m people15. Table 5: Current and potential coverage of the HSRP (number of people) Average Governorate Coverage (actual) Coverage (potential) catchment size Alexandria 617,219 1,090,000 17,869 Menoufia* 1,806,978 1,806,978 15,444 Qena* 1,040,000 1,040,000 14,857 Sohag 1,428,622 1,462,597 9,559 Suez 76,231 192,068 14,774 Total 4,969,050 5,591,643 Source: FHF Unit administrative records, and FHF estimates.*Note that Qena and Menoufia have not submitted their most recent data. Our estimates are based on the figures available in December 2005. And coverage is rising The share of families on roster in the facilities seems to go up fairly continuously from the date of first data collection (a rough indicator of rollout). At the time of this report, on average just above 40% of the families in a catchment appear rostered. 14 People on roster in the facilities. 15 Note that these figures partially rely on estimates. Menoufia and Qena have not yet reported their latest figures of actual and targeted coverage. 10 Figure 7: Estimated share of families on roster (average) .5 .4 .3 .2 .1 0 2001:2 2002:3 2003:4 2005:1 Quarter Source: MOH data (2005), Monitoring & Evaluation team. Note that outliers with a share above 1 have been dropped. Please note however, that this indicator suffers from measurement error, and should be considered a rough estimate. Some facilities report inconsistent and varying data on both families on roster and in their catchment area. Our M&E data also reveals that, in 2004 16 , on average 59% of the family files in reformed facilities were active, going down from 66% in 2003. However, more than half of the facilities had between 80% and 100% of active files, whereas only 9% had less than 10% active files. There may be localized over-staffing Regarding the ratio physicians to families in catchment, different governorates show different patterns. While the average has gone down from about 13 per 1000 families in catchment to about 12, at the time of the evaluation, Suez reports about 20 doctors per 1000 families, and Qena about 7. 16 The last year with full M&E data. 11 Figure 8: Physicians per families in catchment .002 Physicians per Families .0015 .001 .0005 2001:2 2002:3 2003:4 2005:1 Quarter All Alexandria Menofia Qena Sohag Suez Source: MOH data (2005), Monitoring & Evaluation team. Note that data reporting on this variable is annual which explains the constant figures across quarters of a year. This average of doctors/population corresponds roughly to the ratio observed in the Netherlands and Norway, but is well above the ratio observed in the United Kingdom, for example. 17 It is therefore worth enquiring to what extent overstaffing exists in some facilities. ..while some facilities experience difficulties with access even to basic supplies In the past, lack of availability of basic equipment and supplies (e.g. soap, needles) in some regions of Egypt has been an area of concern. While our data does not allow us to measure all types of supplies, we can measure the number of days that drugs were out of stock at a facility. This gives us some indication of whether basic supply logistics have been an issue. We observe that, on average, drug logistics have improved. While there have been struggles during the period we measure, the number of days with drugs out of stock is at its lowest since 2003, see Figure 9. 17 WHO Europe (2004) 12 Figure 9: Average number of days with drugs out of stock 60 40 20 0 2003q2 2003q4 2004q2 2004q4 2005q2 Quarter Source: MOH data (2005), Monitoring & Evaluation team. While the overall situation appears to be under control, a few facilities stand out as more problematic. Semoha, Al Ekhedr, Mbany Anibes, Mazata and Ngoa Bany Wasel report sharp and continuous increases in the number of days that drugs were out of stock (figures are above 365 because the data measures ,,drug-days). All of these facilities are in Sohag, with the exception of Semoha which is in Alexandria. Figure 10: Average number of days with drugs out of stock: most problematic facilities 2,000 2820 1,800 Samoha H.C. 1,600 Al Ekeedr 1,400 Mbany Anibes 1318 Drug days out of stock 1,200 Mzata & Elshig Gabr Ngoa Bany Wasel 1,000 900 800 600 432 400 300 200 - 0 2003q1 2003q2 2003q3 2003q4 2004q1 2004q2 2004q3 2004q4 Quarter Source: MOH data (2005), Monitoring & Evaluation team. These developments may point to on-going logistical issues in Sohag. 13 At the same time, the share of generics in prescriptions has risen. However, in the case of the facilities highlighted above, the increased use of generics can obviously not compensate for deeper logistical issues. Figure 11: Share of generics in prescriptions 1 .9 .8 .7 .6 .5 2003:1 2003:3 2004:1 2004:3 2005:1 Quarter All Alexandria Menofia Qena Sohag Suez Source: MOH data (2005), Monitoring & Evaluation team. Note: one outlier (>1) excluded. UTILIZATION Utilization has recently leveled off in reformed facilities Utilization (people actually visiting the facilities) differs markedly by region and over time. While the overall trend in encounters per physician per year has leveled off recently, the governorates of Alexandria, Qena and Suez show increases.­ The same pattern can be observed if we measure encounters per person in catchment rather than per physician. Much of the decrease can be explained by the phased introduction of co- payments from the first quarter 2004. 14 Figure 12: Number of visits to physicians, by governorate 1500 1000 500 0 2002:3 2003:1 2003:3 2004:1 2004:3 2005:1 Quarter All Alexandria Menofia Qena Sohag Suez Source: MOH data (2005), Monitoring & Evaluation team. But for Egypt as a whole, and since 2000, utilization has gone up In Egypt as a whole, for both public and private facilities, utilization went up sharply between 2000 and 2005, see Figure 13. While in 2000 the main reason for non-usage among women was money, the main reason stated is lack of access to female practitioners. Figure 13: Utilization of all primary facilities, 2000-2005 2000 2005 50% 45% 40% 35% 30% 25% 20% 15% 10% 5% 0% % women that used a money distance to facility have to take transport no female health providers facility in last 6 months reasons not to use a facility Source: DHS 2000 and 2005, author's calculations. 15 Referrals in reformed facilities may be temporarily increasing Regarding access to secondary care via primary, it appears that the vertical link between the service levels has temporarily strengthened for the reformed facilities. As Figure 14 shows, referrals per visit to a facility have taken off at the beginning of 2004, but for most governorates (Alexandria, Qena, Sohag, Suez) the rise has been temporary. ­ This finding is consistent with evidence from Eastern Europe where primary care reform resulted in a temporary increase of referrals, but with increased learning and training, fewer cases needed to be referred. Figure 14: Referrals per physician-encounters, by governorate .3 .2 .1 0 2003:1 2003:3 2004:1 2004:3 2005:1 Quarter All Alexandria Menofia Qena Sohag Suez Source: MOH data (2005), Monitoring & Evaluation team. Note: one outlier (>1) excluded. It is also consistent with qualitative evidence from the facilities: "We now get feedback after we referred a patient, (we get) a written report. Then we can learn from it and need not refer again in a similar case." Physician, Alexandria. These developments beg the question which factors drive primary care utilization. DRIVERS OF UTILIZATION In this section, we relate primary care utilization to a series of factors that may potentially drive it. We consider quality, as measured through customer satisfaction and accreditation compliance; and reform interventions, such as training, infrastructure investment and contractual changes.18 18 In order to find out whether a relation exists, we take advantage of the fact that the MOHs M&E system tracks quarterly data and observes facilities before and after they received an intervention. We analyze the data both with cross-plots and regression analyses. In addition, we examine the impact of the reform with the DHS surveys 2000 and 2005. 16 Quality of service and training seem to improve utilization To begin with, we observe that utilization has a clear positive association with quality of service. Facilities with a higher accreditation score (indicating compliance with quality standards) as well as a high customer satisfaction score (above 90%) show significantly higher utilization rates, see Figure 15 and Figure 16. Figure 15: Encounters per physician vs. accreditation score 1500 1000 500 0 50 60 70 80 90 100 % Accreditation Source: MOH data (2005), Monitoring & Evaluation team. Facilities with a high accreditation score, especially above 80%, report a higher number of encounters per physician, see above. Also, facilities reporting a customer satisfaction above 90% are clearly used more intensively by patients, see below. - Reformed facilities continuously evaluate customer satisfaction of patients through a questionnaire and compare actual satisfaction with the potential maximum (100%). (Data records available to us only track satisfaction above (=1) or below (=0) of 90% of the maximum, which is why the horizontal axis only shows 0 and 1.) 17 Figure 16: Encounters per physician vs. customer satisfaction score 1500 1000 500 0 0 1 Customer Satisfaction Source: MOH data (2005), Monitoring & Evaluation team. `1' on the horizontal axis means customer satisfaction has been above 90%. Regarding interventions under the HSRP, Family health training seems to clearly make a difference. Facilities with a higher share of practice-ready trained physicians report a higher number of encounters per physician as well. Figure 17: Encounters per physician per year vs. share of family-health trained physicians 2000 1500 1000 500 0 0 .2 .4 .6 .8 1 Share of Practice-Ready Physicians Source: MOH data (2005), Monitoring & Evaluation team. Infrastructure investment however does not show a conclusive influence on the utilization rate. 18 Figure 18: Encounters per physician vs. infrastructure investment 2000 1500 1000 500 0 0 20 40 60 80 100 Investment / Population Source: MOH data (2005), Monitoring & Evaluation and Infrastructure Development teams. The above graphs have explored a series of potential drivers of utilization. We have to bear in mind however that a graph always only explores two dimensions. In reality, utilization is driven by various factors at once, each to a different extent. It is possible for example, that the co-payment has been implemented earlier in facilities that also received training earlier and thereby attract patients. In order to explore this properly, we need to apply regression analysis. Econometric regressions test the influence of various factors at once and are able to attribute the contribution of each one to changes in the target variable (e.g. utilization). Below we report our results from regression analysis. Multivariate regression results: Family health training and medical equipment drive utilization up, and co-payment drives it down. We conducted various regression analyses and found that family health training for physicians drives the utilization rate of reformed facilities up, and co-payment drives it down, see Table 6. This is when co-payment is phased in facility by facility rather than at once. Family health training is measured as the share of trained physicians. Unfortunately we are not able to measure other kinds of training. A high share of trained physicians could also proxy a high capacity of the facility to retain staff; so that trained people do not leave. The finding is statistically significant and robust to different ways of measurement. We also find that investment in medical equipment has a positive impact on the utilization rate when the latter is measured as visits/ person. 19 Further, the HSRP as an integrated package occasions a shift from secondary to primary care. We further observe a shift from secondary to primary care in the medical treatment of fever/cough in children. This corresponds to the objectives of the reform and is reassuring. It is triggered by the HSRP as a package, especially by the renewal of medical equipment. It can however be slowed down by major civil works components. This observation is consistent with the views expressed by facility staff19: "Patients have shifted from the nearby hospital to our facility, after the interventions started. They are happy, because it is cleaner and better organized and they have more privacy after the remodeling." (Physician, Menoufia.) Table 6 shows our results. Note that the first two columns feed on facility-level records from the MOH, and column three feeds on the Egyptian Demographic and Health Survey (DHS) at the district level. This is why we cannot analyze the integrated HSRP package in the first two columns: the MOHs Monitoring &Evaluation (M&E) data contains only facilities that will eventually be reformed; we can only identify the ,,phasing in of the separate interventions over time. In contrast, the DHS allows comparison across all the districts of Egypt, reformed and un-reformed. Table 6: Impact of the HSRP on primary care utilization Visits/ Visits/ Shift to primary care Person Physician from secondary care HSRP integrated ni** ni + package impact within Family health training + + package Civil works investment O O - Medical equipment* (+) O + impact within Co-payment - - package Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. *Note that the impact of medical equipment and furniture cannot be identified separately; the same is true for FHF contracting and co-payment. **Not identifiable. Unfortunately there are some interventions which our data does not trace. An important one for increasing and maintaining utilization is the use of outreach towards patients under the HSRP. Facilities are encouraged to contact potential patients through a variety of channels and activities, to raise awareness of primary care. It is likely that outreach contributed to the favorable secondary-to-primary shift. As we cannot examine this quantitatively, we present a qualitative case study, see Case Study Box 1. 19 For a more detailed description of the field visits carried out for this evaluation, see Annex: Data sources. 20 Outreach activity can be key to maintaining high utilization in the face of co- payment Case Study Box 1: Al Battanoon, Shibin Al Kum, Menoufia Al Battanoon is a FHU in rural Menoufia, at the date looking after 1,900 families of the 2,300 in its catchment area. Although it has implemented the co-payment already, and therefore lost some visitors, it has managed to keep a fairly high utilization rate, estimated by staff at 70%, or 30-40 patients a day. Under the HSRP, the FHU received minor remodeling, as waiting areas already existed, and a thorough renovation of buildings and medical equipment. It also received the full package of training courses, on family health, management and quality. Further, FHU management went conscientiously about reaching out to patients, in two ways: it made announcements about the facility's services at the Mosque, after Friday prayer, and it started to follow up patients who stopped coming. Staff report that the new equipment and hygiene standards made a difference. The FHU changed reportedly from `unbearable' to a clean facility attractive to patients. The renovation also helped improve the staff situation. Recruitment and retention have traditionally been difficult in Al Battanoon, due to its rural and somewhat remote location. Doctors reside during the week and return home on weekends. However, the infrastructure investments under the HSRP included renovating the flats for resident doctors, and this reportedly improved the retention substantially. Finally, the committed outreach appears to have borne fruits. Given that in the villages served by Al Battanoon, the institutions with the highest (probably universal) coverage are the Mosques, the Friday announcements have resulted very effective. - In line with staff reports, we discern three factors that may have allowed Al Battanoon to keep a significant utilization rate in the presence of co-payments: 1. the improvement in staff quality through increased retention of doctors; 2. the visible improvements in equipment and hygiene standards and: 3. the innovative outreach activities of the facility. Al Battanoon needs to be commended for turning the constraint of being in a small village into a virtue: it successfully reached many patients through traditionally influential institutions, and drew on the closely-knit community to follow up patients personally. The case of Al Battanoon confirms the findings of our quantitative analysis regarding the influence of quality and co-payment. It provides additional evidence, because it allows us to scope the impact of outreach activities, unfortunately not registered in our quantitative data. Source: Field visits 17 November 2005 After having examined the coverage and utilization of health services, we now turn to the quality of their provision. 21 4. QUALITY OF SERVICE DELIVERY We can evaluate quality of service delivery under two perspectives: compliance with the new quality standards under the HSRP, and patient satisfaction. Both are measured under the HSRP, but only quality supervision data is held centrally. Patient satisfaction data has been gathered from the governorates as far as available. QUALITY SUPERVISION A new quality supervision system was an integral part of the HSRP. Before they received any interventions, the facilities were inspected regarding their compliance with a set of quality standards oriented in international Best Practice. They were accredited for one year if they met 50%-80% of the standards and for two years if they met more than 80%. While many facilities score above 80%... At the end of 2005, 399 facilities are accredited20, with an average accreditation score of 75.5%. Scores range from 50% to 98%, with most facilities in the 80%-90% range, see Figure 19. Figure 19: Distribution of accreditation scores, 2005 6 4 Density 2 0 .5 .6 .7 .8 .9 1 Accreditation Score Source: HSRP Quality and Standardization Team, 2005; author's calculations 20 Of which 62 in Alexandria, 128 in Menoufia, 75 in Qena, 118 in Sohag and 16 in Suez. 22 ..the quality trend varies across governorates and is declining Accreditation compliance varies across governorates and is highest in Alexandria with an average of 80.4% and lowest in Qena with 66.1%. The general trend across all five pilot governorates has been rising until 2003, but declining since, see Figure 20. Figure 20: Trend in accreditation scores, by governorate Average accreditation score by governorate 100% 95% 90% 85% 80% Alexandria Menoufia 75% Qena Sohag 70% Suez Average 65% 60% 55% 50% 2000 2001 2002 2003 2004 2005 Source: HSRP Quality and Standardization Team, 2005; author's calculations This begs the question of what factors drive the quality of service. Below we explore how accreditation relates to the interventions under the HSRP. A priori, there seem to be no clear drivers of accreditation compliance.. It does not seem to be related to the share of doctors trained in Family health, see Figure 21. 23 Figure 21: Accreditation and share of practice-ready physicians 100 90 % Accreditation 80 70 60 50 0 .2 .4 .6 .8 1 Share of Practice-Ready Physicians Source: HSRP M&E and Quality and Standardization Teams, 2005; author's calculations However, we do observe an association between accreditation score and infrastructure investment, see Figure 22. Figure 22: Accreditation and infrastructure investment 100 80 Investment / Population 60 40 20 0 50 60 70 80 90 100 % Accreditation Source: HSRP Infrastructure Development and Quality and Standardization Teams, 2006; author's calculations Interestingly, a higher investment per person in the catchment area seems to be associated with a lower accreditation score. (We measure investment per person, because catchment areas vary in size. Alexandria for example has bigger catchment areas throughout and therefore a higher investment per facility. However note that our accounting for people in the catchment area is not without measurement error.) The reason that facilities with a lower score receive higher investment may reflect the pro-poor targeting of investment under the HSRP. As we saw before, investment went 24 mainly to the areas in highest need. It is conceivable that facilities in these areas had more catching up to do in terms of quality than facilities in areas with less need. The relationship above should therefore not be given any causal interpretation. This argument reflects that our graphs examine only two dimensions at a time. But we know that the situation in the facilities depends on many factors. The best way to control for all these factors at the same time is to employ statistical regression analysis. But regression analysis shows that infrastructure investment as well as FHF incentives may improve accreditation We have analyzed accreditation econometrically and found that infrastructure investment as well as contracting with the FHF fund appear to have a positive impact on accreditation scores. The impact of training is unfortunately not measurable with our data, as allocation of training seems to depend in part on low accreditation scores. Table 7: Impact of the HSRP on accreditation scores Intervention Impact Extent (estimate) Family health training Not identifiable 9-16 %points Civil works investment (+) (per 10 LE pc) 10-13 %points Medical equipment* (+) (per 10 LE pc) FHF contracting (+) 3 % points Source: Econometric analysis of HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. *Note that the impact of medical equipment and furniture cannot be identified separately; the same is true for FHF contracting and co-payment. . These figures are to be taken with caution as it is not fully clear to what extent the allocation of interventions depends on accreditation scores in practice. And qualitative evidence shows that staffing and process integration can make a big difference.. Therefore, in order to delve further into the reasons behind high performance, we have prepared a qualitative case study of an outstanding facility. San Estefano, a FHC in Alexandria, shows consistently accreditation compliance above 90%, see Case Study Box 2. 25 Case Study Box 2: San Estefano, FHC, El Raml, Alexandria San Estefano in Alexandria is a newly created FHC. An existing building was renovated and equipped to provide the services expected from a FHC, i.e. General Practice with limited specialities. In accordance with this remit, end of 2005 San Estefano counts with 8 family physicians, 6 specialists (pediatry (2), psychiatry, internal medicine and gynecology (2)) and 2 dentists. The Center covers roughly 60% of its catchment area, or 6,400 families; and operates at about 80% utilization. It has implemented all training- and management changes foreseen in the HSRP. San Estefano stands out among the accredited facilities for consistently showing compliance above 90%. Information gathered from various field visits in November 2005, and expert interviews within the MOH, reveal that service delivery at San Estefano may be different for chiefly three reasons: 1. Staff commitment and quality. San Estefano is located within an urban, rather densely populated and attractive region. This enables the facility to conduct competitive recruitment and select the best staff, because willingness to live and work near San Estefano is generally high. Further, as well as being very qualified, the selected staff also shows a high commitment to performance. 2. Adoption of integrated processes in health care. Staff are convinced of the usefulness of the new integrated concept of service delivery and apply it thoroughly. They make sure to conduct integrated diagnostics of a patient, involving his/her health, socio- economic background, domestic issues and skill level. At the same time they are aware to respond in an integrated manner, confidently referring if needed, or offering parenting orientations and certified basic skills training, as well as other empowering and educating activities of a `family club', on-site, in case of need. 3. Mobilization of sponsors. Many activities in the facility are sponsored by business angels, such as the small business training of the family club and diverse outreach activities, catering to different age- and interest groups, from school children to women of reproductive age. The sponsor money frees up resources for further improvements. Source: Field visit 21November 2005, interviews with Dr Ali Elazeem Gazalah and Dr BassyouniZaki Salem The experience of San Estefano could serve as a best practice example and should, at least in part, be replicable elsewhere. If competitive recruitment makes a huge difference, there may be mechanisms to replicate such a situation even in rural areas. Also, mobilization of sponsor money could probably be an activity that deserves encouragement. Chapter 0 will examine lessons going forward in more detail. As remarked in the beginning, accreditation compliance is only one perspective on quality. The second one, patient satisfaction, is no less important. PATIENT SATISFACTION The second perspective on quality of service delivery is the satisfaction of the patient. In most reformed facilities, patient satisfaction is measured continuously through a comprehensive questionnaire covering all major areas of interaction of the patient with the unit. The responses are compiled into a ,,customer satisfaction score with a maximum of 100%. Data is collected de-centrally, which means immediate data availability is 26 restricted. We therefore have to limit our discussion to the situation in Alexandria and Sohag. Patient satisfaction differs across regions 87 facilities have reported their patient satisfaction in 2005, and 47 (54%) 21 of these obtain a level above 90%. Over time, the share of facilities at above 90% has developed differently in Alexandria and Sohag, see Figure 23. Figure 23: Share of facilities that obtain above 90% of patient satisfaction, by governorate 1 .8 .6 Share .4 .2 0 2002:3 2003:4 2005:1 2006:2 Quarter Alexandria Sohag Source: TSO Alexandria and Sohag, 2006; author's calculations While in Alexandria more than 80% of the facilities manage to score above 90% most of the time (with the exception of the first quarter 2004), Sohag struggles to reach that share. In order to explore the background behind this, we consider potential drivers of patient satisfaction. A high accreditation score as well as family health training and infrastructure investment seem to satisfy patients To begin with, a high accreditation score and high customer satisfaction seem to be linked, which is to be expected, see Figure 24. (Note that we can only measure whether customer satisfaction is above or below 90%, not a sliding scale. That is why we have 0 and 1 only on the horizontal axis.) 21 Using latest available data for each facility. 27 Figure 24: Patient satisfaction and accreditation score 100 90 % Accreditation 80 70 60 0 1 Customer Satisfaction Source: Source: HSRP Quality and Standardization Team; TSO Alexandria and Sohag, 2006; author's calculations There may also be a mild positive relationship between the share of practice-ready physicians and patient satisfaction. Figure 25: Patient satisfaction and share of family-health trained physicians 1 .8 .6 .4 .2 0 0 1 Customer satisfaction Source: Source: HSRP Monitoring & Evaluation Team; TSO Alexandria and Sohag, 2006; author's calculations The same is true for investment. Patient satisfaction appears to be clearly higher in facilities that received a higher investment per capita. 28 Figure 26: Patient satisfaction and infrastructure investment 100 80 Investment / Population 60 40 20 0 0 1 Satisfaction Source: Source: HSRP Infrastructure Development Team; TSO Alexandria and Sohag, 2006; author's calculations After having considered how the different interventions under the reform are linked to patient satisfaction, we now examine some features of treatment such as prescriptions and referrals. More prescriptions and a higher referral rate also play a role More prescriptions in a facility could be weakly linked to higher patient satisfaction, see Figure 27. Figure 27: Patient satisfaction and number of prescriptions per physician visit 5 4 No of Prescriptions 3 2 1 0 0 1 Customer Satisfaction Source: HSRP M&E Team;TSO Alexandria and Sohag, 2006; author's calculations 29 This was actually confirmed in field visits to facilities in Menoufia and Alexandria. Staff remarked "To attract more people, it is essential to have more drugs available on the list of prescriptions." (Pediatrician, Alexandria). Further, there may be a weak positive connection between referrals and satisfaction; facilities with a higher referral rate appear to report higher customer satisfaction as well. ­ A finding that is not desirable under the objectives of the HSRP, which aims at promoting primary care. Figure 28: Patient satisfaction and % of visits that results in referrals 1 .8 .6 % Referrals .4 .2 0 0 1 Customer Satisfaction Source: Source: HSRP Quality and Standardization Team; TSO Alexandria and Sohag, 2006; author's calculations Limitations in our data sample do not allow us to conduct econometric tests of the drivers of patient satisfaction. Satisfaction figures are only available for Alexandria and Sohag, which would reduce the sample size severely. Therefore, in order to shed light on the mechanism behind our graphical analysis, we have conducted qualitative case studies. Qualitative evidence suggests that staffing is important for patient satisfaction, as is investment size and timing The facility of Amreya, Alexandria stood out because of its consistently high customer satisfaction scores, while El Wakkad, Alexandria appeared to face challenges. Case Study Box 3 explains the background, which also offers some clues for the performance of Alexandria and Sohag in general, as discussed earlier on. 30 Case Study Box 3: Facilities of El Amreya and El Wakkad, Amriyyah District, Alexandria El Amreyah and El Wakkad are both reformed facilities in El Amriyyah, Alexandria, but report strikingly different outcomes. While prescription- and referral practices are roughly the same, Amreyah is far more successful attracting patients. Families on roster are close to 100%, and the utilization rate is relatively high. Customer satisfaction scores are consistently outstanding. On the other hand, El Wakkad reports barely 50% of families on roster, and a declining utilization rate. Patient satisfaction is well below El Amreyah's. Reasons for this may both relate to the environment of the facilities, and the interventions they received under the HSRP. First, and probably most centrally, the surroundings of the two facilities differ considerably. El Amreyah is located in an urban, densely populated area, attractive to staff and easily accessible for patients. Consequently, there are relatively few recruitment issues and staff retention is high. On-the-job learning can be put to use immediately, and knowledge that staff has gained over time, such as familiarity with patients, is not lost through high turnover. At the same time, patients access and use the facility frequently. El Wakkad, on the other hand, is placed in a remote area, which is less accessible and less apt to recruitment. At times, the facility has found it difficult to fill even key positions on the staffing pattern. Also, the interventions under the HSRP differed in extent and timing. El Amreyah received total 22 investments of about 420 L.E. per capita and 47 L.E. pc in medical equipment, while El Wakkad has so far received 350 L.E. pc in total and 25 L.E. pc in medical equipment. For a long time, the facility had to reside in a temporary location without a waiting area, and investments appear to have kicked in late. But the facility received early and comprehensive training. Quality training started beginning 2003 and on family health for both nurses and physicians beginning 2004. On the other hand, El Amreyah received quality and family health training from mid 2004, and reports not to have received any family health training for physicians. The cases of El Amreyah and El Wakkad confirm once more the importance of staff recruitment and retention for the success of a facility. Further, they confirm the results of our previous graphical analysis, where the size and timing of investment per person had a clear influence on customer satisfaction, but training was less important. Source: HSRP M&E Team, HRSP Infrastructure Development team, HSRP Technical Support Office Alexandria, Alexandria Family Health Fund and MOH Family Health Fund Unit. After having examined coverage and quality of the reformed primary care service, we now turn to the impact of the service on the health of the population. 22 pc= per capita, per person in catchment area 31 5. HEALTH OF THE POPULATION This chapter is the core chapter of the evaluation report. It examines key health indicators in Egypt before and after the reform, and attempts to attribute a specific impact to each of the different HSRP interventions. We explore the health situation under two broad headings: maternal health and child health, given that these two were the areas of focus for the reform. Under each heading we first examine the current health trends in Egypt, then observe possible correlations between health indicators and underlying factors, and finally undertake statistical analysis to measure the impact of the various interventions under the reform on the trends. MATERNAL HEALTH Overall, the maternal health situation has improved in Egypt Recent trends show that the maternal health situation has improved in Egypt as a whole. The most striking improvement has happened in the use of antenatal care. In 2005, nearly 60% of women had the recommended minimum of four antenatal health visits, compared to 40% in 2000. Also, substantially more women receive the assistance of qualified staff during delivery. 78% compared to 56% now confide in the services of a doctor, mainly at the expense of traditional services such as days and relatives. Figure 29: Trends in maternal health indicators, 2000-2005 80% 70% 2000 60% 2005 50% 40% 30% 20% 10% 0% % couples women who doctor nurse/midwife daya other none using had alone contraception four or more antenatal visits assistance at birth Source: DHS Final reports 2000 and 2005 However, the overall nutrition situation of neither women nor their children appears to have improved. While there are fewer issues of undernourishment, malnourishment as measured through anemia, is on the rise, see Figure 30. 32 Figure 30: Anemia in women and children, 2000-2005 Anemia incidence 60% 2000 2005 50% 40% 30% 20% 10% 0% Women Children Source: DHS Final reports 2000 and 2005 While Egypt as a whole shows a trend to more qualified deliveries, this appears not to be the case for the HSRP facilities In reformed facilities, the reported share of institutionalized deliveries is very low, and never passes 30%. Figure 31: Share of institutionalized deliveries, reformed facilities 2003-2005 .3 .25 .2 .15 .1 .05 2003q1 2003q3 2004q1 2004q3 2005q1 Quarter Source: MOH Monitoring & Evaluation team, 2005, and author's calculations Most facilities do not report any institutionalized deliveries at all. 33 Figure 32: Distribution of institutionalized deliveries, end 2004 8 6 Density 4 2 0 0 .2 .4 .6 .8 1 Share of qualified deliveries Source: MOH Monitoring & Evaluation team, 2005, and author's calculations Given these findings, it will be interesting to examine what difference the HSRP made for maternal health in Egypt. The HSRP improves maternal health regarding nutrition and family planning, but does not have an impact on antenatal and natal care As in the previous chapter, we have used regression analysis to test for the attributable impact of the HSRP on maternal health. The indicators available from our data concern contraception, the anemia rate (proxying for malnourishment), antenatal care visits and professionally attended births. The results, summarized in Table 8 confirm some of our graphical analysis. The HSRP marginally increases the share of couples using modern contraception and significantly decreases the incidence of anemia in women. While there is a general trend in Egypt towards higher anemia incidence, the HSRP slowed this down in the reform districts. (Detailed results see Annex: Econometric results.) On the other hand, in line with our graphical analysis, we do not find that the HSRP improves the use of antenatal care or professional deliveries. While there is a general trend in Egypt towards increased use of these services, the HSRP has not been able to add to it. - This finding is consistent with a previous case study carried out by the FHF in the MOH, which raised concerns about the efficient utilization of delivery rooms. 34 Table 8: Impact of the HSRP on maternal health indicators Share of Share of couples Share of Anemia rate in women having using modern professionally women 4 or more ANC contraception attended births visits HSRP integrated (+) - O O package Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. Note that the DHS allows us to analyze the impact at district level only, which makes it difficult to identify the individual interventions, as their phasing-in does not vary as much at district level as it does at facility level. Note that all of our maternal health indicators come from the DHS, which permits comparisons across Egypt at the district level. This means we can gauge the impact of the overall reform package very well, but not so easily for the different interventions within the package, as there is not too much variation at the district level. We next examine the impact on child health. CHILD HEALTH Child health indicators show improvement in Egypt In the whole of Egypt, vaccination coverage and use of medical treatment has improved for children. The most noticeable improvement can be noted in the treatment of respiratory infections, where 73% of children now receive qualified treatment, compared to 67% before. Figure 33: Trends in child health indicators, 2000-2005 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% vaccination coverage children receiving medical treatment children receiving medical treatment (measles 12 month) for diarrhea for respiratory infection 2000 2005 Source: DHS Final reports 2000 and 200 and author's calculations 35 Also in the HSRP areas, an improvement can be registered. From the MOH data, we can analyze the diarrhea cases and mortality. In the beginning of 2005, the average pilot facility had to treat 23 diarrhea cases in young children, but less than 1 diarrhea death (0.4 deaths on average). The trend in the diarrhea mortality rate, while time varying, is at its lowest since 2003, at the time of this evaluation. Figure 34: Diarrhea mortality rate, 2003-2005 .03 .02 % Diarrhea Deaths .01 0 2003:1 2003:3 2004:1 2004:3 2005:1 Quarter Source: MOH Monitoring & Evaluation team, 2005, and author's calculations The diarrhea mortality rate of children appears to improve with higher care utilization and family health training of physicians A lower death rate from diarrhea in young children could be associated with a higher coverage of the catchment area in reformed facilities (as measured by % families rostered), see Figure 35. Figure 35Figure 35: Diarrhea death rate and coverage 36 .3 % Diarrhea Deaths .2 .1 0 0 .2 .4 .6 .8 1 % Families Rostered Source: MOH Monitoring & Evaluation team, 2005, and author's calculations Further, a higher utilization rate is clearly associated with a lower mortality rate from diarrhea in children. Figure 36: Diarrhea death rate and care utilization .3 % Diarrhea Deaths .2 .1 0 0 500 1000 1500 2000 Encounters per Physician Source: MOH Monitoring & Evaluation team, 2005, and author's calculations Most interestingly, the family health training for physicians might make a difference. Figure 37 shows that a higher share of trained physicians in a facility is weakly associated with a lower death rate from diarrhea. (Note however, that our sample size for this comparison is fairly low.) Figure 37: Diarrhea death rate and family health training 37 .15 % Diarrhea Deaths .1 .05 0 0 .2 .4 .6 .8 1 Share of Practice-Ready Physicians Source: MOH Monitoring & Evaluation team, 2005, and author's calculations. Note that one outlier has been taken out. We have also analyzed the relationship between the diarrhea death rate and further interventions such as infrastructure investment and the accreditation score, but have found no association. However, as mentioned variously, a two-dimensional analysis serves mainly to scope the data, but does not derive a causal relationship. Therefore, we have explored the impact of the HSRP interventions in a series of econometric analyses. The results are summarized below. The HSRP improves the child vaccination rate, the child diarrhea mortality rate and some child access to medical treatment Table 9 summarizes the results of our econometric investigation into child health. First, we observe that the HSRP integrated package has a significant positive impact on the measles vaccination rate. Our detailed results (see Annex: Econometric results) show that there was a general trend towards improvement in Egypt, but even so, the HSRP made a difference. Second, we observe the same impact on the access to medical treatment for children with fever/cough. Finally, we notice an improvement in the child diarrhea mortality rate, resulting from both family health training and civil works (although the training impact is just below significance). Table 9: Econometric analysis of the impact of the HSRP on child health Vaccination Medical treatment Diarrhea mortality coverage (measles) when fever/cough rate HSRP integrated + + Not identifiable** package Impact within Impact within Family health training (-) package package Civil works Impact within Impact within - investment package package Medical equipment* Impact within Impact within O 38 package package Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. *Note that the impact of medical equipment and furniture cannot be identified separately; the same is true for FHF contracting and co-payment. . **due to data availability. Note that the results in the first two columns draw on data from the DHS, at the district level, and the last column on MOH data at the facility level, due to the availability of indicators. The first dataset allows cross-Egypt comparisons at the district level, which enables us to get a good picture of the overall reform impact. The second dataset traces the phasing-in of different interventions over time in all eventually reformed facilities. Therefore, it allows identifying the impact of these, but not of the overall package. The next chapter compares the various results obtained from the HSRP and tries to discern, which interventions provided the best Value for Money. 6. COST EFFECTIVENESS This chapter summarizes the impacts presented in the previous chapters, and contrasts them with the costs incurred to bring them about. Given the two datasets available to us, we present some cost-effectiveness measures that involve the integrated HSRP-package and some that refer to its separate components. COST-EFFECTIVENESS OF THE HSRP PACKAGE We first list the benefits attributable to the integrated HSRP package, and then the costs of an average integrated HSRP intervention per facility and per beneficiary. The benefits below show the different impacts (in percentage points) resulting from the rollout of the HSRP as an integrated package. 39 Table 10: Incremental benefits of the HSRP as a package Increase in % Increase in share of Increase in Decrease in of children couples using measles female anemia treated modern vaccination rate medically for contraception coverage fever/ cough Impact of HSRP integrated +4ppt -7ppt +22ppt* +4ppt package ppt= percentage points. *would require rollout in every single facility. Other impacts require only rollout in some facilities per district, without large sensitivity to extent of rollout. Source: Econometric analysis of DHS surveys, HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. We contrast this with the costs of the integrated package. Table 11 shows the investment for an average facility, and per beneficiary. Table 11: Incremental investment per facility and per beneficiary (LE) Investments Per facility Per beneficiary All training 87,860 7 Civil works 1,885,653 140 Furniture 180,450 13 Medical equipment 432,485 32 Total 2,586,448 191 Source: HSRP Infrastructure Development Team, Human Resources Development Team and FHF Unit. We count facilities only, not administrative buildings. Beneficiary numbers are per full catchment size. Note that the above expenditure flow represents investments, not on-going costs. In order to convert these into yearly costs, we need to account for yearly depreciation and interest, and add any new on-going revenue. Training, medical equipment and furniture can be expected to have a lifetime of five years each, whereas the lifetime of civil works is probably about twenty years. The annual interest rate for property of the public hand in Egypt is 8.25% at the date. In addition, a facility now receives estimated average annual co-payments of 13,600L.E.23 Taking all this into account, the annual on-going costs per facility and beneficiary are as listed below. Table 12: Annual incremental cost of the integrated HSRP package, per facility and per beneficiary Investments Per facility Per beneficiary All training 24,820 2 Civil works 249,849 18 Furniture 50,977 4 Medical equipment 122,177 9 Co-payments (13,600) (1) Total on-going cost 434,224 32 Source: HSRP Infrastructure Development Team, Human Resources Development Team and FHF Unit. We count facilities only, not administrative buildings. Beneficiary numbers are from FHF. From the quarterly MOH data, we can also make some predictions about individual interventions. 23 FHF Unit 2006. 40 COST-EFFECTIVENESS OF HSRP INTERVENTIONS Our econometric analysis had discerned a noticeable impact of family health training and new medical equipment. The following table summarizes average benefits of all measurable HSRP interventions. Table 13: Benefits per cost of individual HSRP interventions Deaths averted (% reduction) Interventions Visits/ person per cost per cost +130 visits/ 1,000 people -1ppt Family health training if train all doctors = 2.20 LE if you train all doctors (physicians) pc -0.01ppt Civil works 0 per 1 L.E. spent per person Furniture* Not identifiable Not identifiable +2 visits/ 1,000 people Medical equipment* 0 per 1 LE spent per person -40 visits/ 1,000 people Co-payment 0 as soon as implemented** Source: Econometric analysis of HSRP M&E, Infrastructure Development and FHF data, as well as 1996 census poverty data. Details see Annex: Econometric Results. *Note that furniture investment is too correlated with medical equipment to be identifiable separately. **Note that this is the effect when people have the choice not to pay in another facility. The impact is likely lower if implemented in parallel. From this comparison, family health training for physicians emerges as the most cost- effective intervention, since the incremental cost-effectiveness ratio is the largest for a given cost. Note, however, that this is the intervention whose timing we can measure most precisely. The lower impact of the investment interventions may in part be due to our inability to measure precise timing. The share of trained physicians is also our only measure to indicate staff quality, and a high sustained share may thereby proxy for a facilitys ability to retain good staff. This was a theme that came up various times in our qualitative case studies and warrants further attention. Our final chapter summarizes the findings of this report and draws out some lessons going forward. 41 7. CONCLUSION AND LESSONS FOR POLICY MAKERS THE EGYPTIAN HSRP SHOWS SOME SUCCESSES Our evaluation results confirm that the integrated HSRP package has multiple positive impacts. In the districts implementing the reform, there has been a noticeable shift from secondary to primary care in child treatment for fever/cough, an increase in the child vaccination rate, an increased use of modern family planning and a reduction in female malnutrition (measured through the anemia rate). All these changes can rigorously be attributed to the reform. However, we did not find any improvement in antenatal or natal care usage under the HSRP. Further, in addition to the integrated package, our results support the different interventions under the reform one by one. We find that family health training significantly increases the utilization rate of facilities, as well as decreases child diarrhea mortality. This latter result is also enhanced by a civil works component, as is a high accreditation score. Medical equipment renewal is likely to increase the utilization rate as well as the accreditation score. All investment components together are progressively targeted towards the needy. On the other hand, the request of co-payments lowers the utilization rate (at least if no-payment options are still available), as can be expected. Qualitative case studies of six individual facilities support the quantitative findings and highlight particularly the importance of staffing for quality and performance. THESE SUCCESSES CAN BE SUPPORTED THROUGH MODIFICATIONS OF THE ORIGINAL INTERVENTIONS 1. Human Resource Development should be extended to include staffing mechanisms While all individual interventions showed an impact, the evidence for Human Resource Development was particularly strong. Family health training of physicians significantly and cost-effectively increased utilization and decreased child diarrhea mortality. Further, the qualitative case studies highlighted the importance of staff quality for utilization, accreditation and patient satisfaction. The studies also showed that facilities operating in an attractive labor market were able to recruit and retain staff competitively, and thereby achieved a high staff quality and motivation. The opposite was true for facilities operating in remote and rural locations. It would therefore be a priority to replicate the competitive staffing possibilities of urban, successful facilities in rural areas, especially Upper Egypt. International experience shows a few options how this could be done. In Colombia high profile scholarships are awarded with conditions, such as working in a sector or region of national priority after graduating. In Egypt, it may be an option to award a few national merit scholarships to medical students and nurses with the condition that they do their internships and their residency in Upper Egypt. Funds would be disbursed conditional on compliance. Such a 42 mechanism would solve some of the rural staffing issues, while competitively selecting high quality staff. Another example comes from Germany, where young conscripts may refuse conscription if they take up a one-year care job. In hospitals and old peoples homes, the ,,civil servicers ease the workload of nurses and doctors and have now become an indispensable part of the service infrastructure. ­ While it may not be adequate in Egypt to allow an alternative to conscription, a modified version of this model is thinkable, where a part of the military service is spent in rural public health facilities, relieving the professional care staff of menial and admin jobs. Under a complete contracting model as envisaged by the FHF, where facilities are free to hire, and reimbursed per visit, direct central influence on staffing patterns may be difficult. However, even under this model the difficulties of rural facilities are likely to warrant specific treatment. If direct influence on staffing is not possible, a solution may be to raise a `smart subsidy fund' for remote areas, the proceeds of which are allocated in a competitive bid to the highest quality contractor, for a limited time. This mechanism has been used successfully in various Latin American countries to promote infrastructure in rural areas24. 2. Quality Supervision could be strengthened locally through empowering citizens The new Quality Supervision is an integral component of the HSRP. Our evaluation suggests that a higher accreditation score is linked to higher customer satisfaction and higher utilization. However, Figure 10 shows that four of the five facilities experiencing extreme drug logistics issues reside in Sohag. This raises the question what contributes to this situation, and whether processes are as supervised in remote areas as they are in more central regions. One solution would be to empower citizens locally to monitor and supervise their services. International experience suggests some options. Schools in Kenya had an issue comparable to facilities not having drugs in stock. In many remote schools, funds for books did not arrive, and service was underprovided. In order to remedy this, the central government published all funds and resources allocated to local schools in newspapers circulating in the relevant areas. In this way, local parents were aware of their rights, and could complain. The service situation changed dramatically. ­ While newspaper announcements of drug and other supply allocations may not be enough to empower citizens, especially if there are illiteracy issues, the announcement can be made through wider reaching channels, e.g. the radio, or the Mosque (e.g. alerting the Imam via religious publications). Southern Indian schools had an issue of teacher absenteeism that was very difficult to control centrally. The government equipped the teachers with cameras that took dated pictures and asked teachers to take a picture of themselves with 24 Examples are the Peruvian telecoms fund, and the Colombian rural telecoms and postal services funds. 43 their class every day and submit the picture as part of monitoring evidence. A rigorous evaluation of the project found it was successful.25 The United Kingdom had long implemented its childcare delivery in a fairly de- central manner, leaving most responsibility with local authorities. However, there was evidence that not all local authorities performed to the expectation of local citizens. When childcare delivery was reformed in 2005, the government therefore paid special attention to giving local citizens a ,,Voice over local service delivery, and considered various options, for example obliging customer satisfaction surveys in the entire villages, not just to service users, and establishing an ,,escalation ladder of options for citizens to complain about service delivery and sue the providers in cases of failure, at local, regional, and national instances. 3. The co-payment can support the success of the HSRP service delivery ­ if designed carefully Our evaluation finds that financial incentives under the HSRP have strong impacts. On the one hand, there is evidence that the incentive payments from the FHF prompt higher accreditation performance; on the other hand the co-payment significantly drives down utilization. In the light of these impacts, special care should be given to ensure that the design of any future incentives supports the successes of the HSRP service delivery. For example, the co-payment mechanism could be modified: to support the observed shift from secondary to primary care: giving secondary care a higher co-payment than primary care, unless the person was referred; and to support the increasing number of visits per person: restricting the co-payment to a one-off fee, rather than a per-visit fee. 4. The lack of impact of the HSRP on antenatal and natal care warrants re-thinking the focus of outreach activities We have found that the integrated HSRP package failed to have an impact on the take-up of antenatal and natal care. While there is a positive trend in this regard in Egypt as a whole, the HSRP did not make a difference. This begs the question whether facilities reach out effectively to mothers-to-be, and what an effective outreach could look like. Our results encourage an enquiry in this direction. 25 Duflo and Hanna (2005) 44 8. ANNEXES DATA SOURCES The evaluation report draws on various data sources; from the MOH, other official national statistics and surveys, and direct field visits of Bank staff. Below we describe the main content, advantages and shortcomings of each. Data from the Ministry of Health, Cairo; and associated agencies Monitoring & Evaluation Data (M&E Data) The Department of Monitoring & Evaluation of the HSRP in the MOH monitors the performance of each health facility, district and governorate in quarterly and annual intervals. The M&E system measures indicators of outcome (e.g. diarrhea death rate, vaccination rate), output (e.g. visits, utilization rate) and process (e.g. number of family folders completed). Measurement starts at the facility level with the first accreditation. On the one hand, this means that typically, a facility has already been monitored before receiving any interventions under the HSRP. The M&E data therefore constitutes de facto baseline data. On the other hand it means that the time span of monitoring does not fully overlap for all facilities. Most reformed facilities report data for all quarters 2003 and 2004, and a few for quarters before and afterwards. The database contains 335 facilities, reporting between 1 and 12 quarters, thereby forming an overlapping panel. Chief contact: Director of M&E Department, MOH Infrastructure Investment Register The Engineering Unit of the HSRP in the MOH tracks all infrastructure investment in the reformed facilities, by investment type, with disbursement, contractor, donor and date of finalization. From this register, we have extracted all investment in civil works, furniture and medical equipment, by facility, under the HSRP. Finalization dates vary considerably by facility and range from January 2003 to June 2007. This enabled us to convert the dataset into a quarterly panel dataset, tracking investment finalization by quarter. Chief contact: Dr Mohamed Abdel-Rahman and team, Head of the TSO, MOH Training Data While training programs are procured centrally, they are implemented de-centrally at the District level. The Human Resource Development Team at the central level registers the courses delivered under the HSRP, their planned number of participants, their duration and their cost. The Technical Support Offices in the governorates record when which course was administered to which facility, and which share of the eligible staff received it. At the time of this report, the detailed data from the governorates is only available for 45 Alexandria and Sohag. The governorate data can be converted in a quarterly panel by facility. Chief contacts: Director of Human Resource Developmen for the HSRP, MOH; Technical Support Office Director in the Pilot Phase, Alexandria Family Health Fund Data Facilities receive two new forms of direct payments under the HSRP, co-payments provided by the patients for opening a file and at each visit; and incentive payments for good performance by Family Health Funds at the governorate-level. The Family Health Unit at the MOH tracks the implementation for both kinds of payment regimes for all reformed facilities. Chief contact: Family Health Fund Unit, MOH Accreditation Register The Quality and Regulation Unit at the MOH regularly accredits and inspects the reformed facilities. A register tracks the accreditation scores (in percent) for each facility at the date of the first and each subsequent accreditation. This file can be converted into a quarterly or yearly panel. Chief contact: Quality and Regulation Unit, MOH Customer Satisfaction Customer satisfaction is measured in each facility on a continuous basis, through a questionnaire covering all points of interaction between patient and facility. The data is collected at the governorate level as scores below or above 90% of satisfaction. At the time of this report, only Alexandria and Sohag make their data available. We have converted it into a quarterly panel and merged with data of similar structure. Chief contact: Technical Support Office Director, Alexandria, and Family Health Fund Unit for other governorates Other National Statistics Demographic and Health Survey 2000 and 2005(DHS) The DHS is a regular survey of around 20,000 women and households, covering demographics, anthropometry, health care usage, health behavior, disease awareness and health-relevant traditions such as female circumcision. As most variables are repeated over the years, the two years of the survey can be combined in a panel, and merged with all the datasets on interventions described above. Chief contact: Dr Fatma El Zanaty, El Zanaty & Associates 46 Population and Housing Census 1996 This dataset is the 1996 wave of the ten-year Egyptian household census. It covers the universe of households, and registers data on household demographics, income, assets, access to public services and basic health indicators. We use a set of district level summary indicators on poverty and basic needs that reflect the selection criteria for the targeting of the HSRP. We include these indicators as control variables in all econometric analyses to ensure an adequate matching of treatment and control groups. Chief contact: CAPMAS- Statistical Agency Field Visits Between 16 and 23 November 2005, Rebekka Grun and Sami Ali of the World Bank conducted field visits to four facilities in Menoufia and Alexandria, covering rural and urban areas. The facilities visited were Quesna, Al Battanoon, San Estefano and El Amrawy. The visits included a tour of the facility, staff interviews and mini-focus groups with management and junior staff of the facilities. The data gathered covers basic data on each facility (staffing, catchment, environment, utilization rate), the HSRP intervention history, additional activities, and, most importantly, qualitative assessments of the impact of the reform on processes, staff and patients. It also collected staffs suggestions for improvements and next steps. Chief contact: Rebekka Grun, Senior Economist, World Bank 47 ECONOMETRIC METHODS We have combined the different data sources described above into two generic panel data sets: 1. a quarterly, revolving panel at the facility level, and 2. a two-year full panel at the district level. The first data set mainly feeds on outcome indicators from the MOHs M&E data from 2002 to 2005, and the second one on the DHS panel 2000-2005. Both datasets contain measures of the interventions under the reform, such as family health training, infrastructure investment and implementation of co-payment. They also contain the accreditation and customer satisfaction scores. Further, they comprise 1996 measures of all social indicators that were used to target the HSRP: illiteracy ratio, unemployment ratio, income dependency ratio, inaccessibility to electricity (measured through access to a washing machine ­ the household appliance with the with the widest coverage), inaccessibility to potable water, average family size, household crowding factor and the population size. Our data only allows us to use the targeting indicators at the district level, rather than village level. However, this may be the best option as the main targeting under the reform was done at district level, while leaving a lot of de-central autonomy regarding the allocation within district. The two data sets are apt towards different econometric models. We have conducted clustered random-effects GLS regressions on the quarterly MOH dataset at the facility level, and a Difference-in-differences estimation on the two-year DHS dataset at the district level. Details are described below. With the Quarterly MOH panel The quarterly panel contains consistent data for about 200 reformed facilities across 4 to 8 quarters, with a total of above 1000 observations. In order to take advantage of both the cross-sectional and the time-series dimension, we run a GLS random effects model: Yitl FH itl c INVcitl ditl l X l itl co pay (1) c l with itl = f il g lt itl . The subscript i stands for the facility, t for a quarter and l for a district. Yitl denotes any outcome variable, such as the diarrhea mortality rate or the accreditation score. FH itl denotes the share of family-health trained physicians in the facility (to approximate training under the reform), INVcitl denotes the c different types of investment administered under the reform, civil works, medical equipment and furniture, and 48 co pay d itl is a dummy for the implementation of the co-pay regime in a facility. X l consists of the district-level social indicators that were used to target the reform interventions, notably the illiteracy ratio of household heads, unemployment ratio, income dependency ratio, accessibility to electricity (measured through access to a washing machine ­ the household appliance with the with the widest coverage), inaccessibility to potable water, average family size, household crowding factor and the population size. Note that there may be additional targeting happening within districts, at the facility level, but we cannot control for this. The allocation of funds within districts lies within the discretion of the district health management, and may be random, or follow criteria we do not measure. We will interpret our findings in the light of this constraint. f il is a facility-level random effect, g tl is a district-level (cluster) random effect and itl is a facility-level independent error term. Our assumptions are: E itl X 0 (strict exogeneity); E f il X 0 and E g lt X 0 (covariance between observables and unobservables is zero); X'X p lim N M XX is positive definite for all T. (enough variation in the data); as T N well as E itl f il 0 ; E itl g tl 0 ; Cov fil , gtl 0 ; E itlitl 2 g 2 2 f E itl isl f ; (random effect: facility-level correlation over time) and 2 E itl jtl g ; (peer effect: clustering at the district level, inter-facility correlation in 2 one district). Note that these assumptions imply that any time-invariant idiosyncrasies of facilities within a district have nothing to do with each other; an assumption that may be challenged in reality. However, our dataset controls amply for time-invariant characteristics of the districts, which affect all facilities, notably poverty and access to public services. We run the regression (1) for all outcome variables of interest available from the MOH M&E data with enough data points: diarrhea mortality rate of children, accreditation score and utilization rate (encounters per person). The coefficients , and give us 49 the impact of the training interventions, investments and co-payment, respectively, while the coefficients do not have a causal interpretation; they only serve to match treatment and control groups. With the DHS panel We conduct an econometric impact evaluation after Attanasio and Mesnard (2006) 26 , where a Difference-in Difference estimator is formulated as Y 1 2d HSRP*2005 3d HSRP 4d 2005 5 X (2) Where Y is the outcome, d HSRP*2005 an interaction term between treatment district and HSRP second year of data, d a dummy for a treatment district, d 2005 a dummy for the second year and X a set of controls. Equation (2) can be extended to account for different interventions under the reform, by adding separate treatment dummies, and 2nd year interaction terms for training, investment and co-pay, respectively. We attempt to control for the selection of treatment districts with the social indicators described above. When the MOH targeted the HSRP, they used these indicators to form a selection index at the district level. They should therefore match treatment and control groups fairly precisely. Moreover, we are confident using a parametric approach as our sample can be shown to have very broad common-support. This likely results from the fact, that before targeting, the pilot governorates were selected to represent Egypt. Also, given the relatively moderate overall sample size, a parametric approach is adequate to improve efficiency. Our evaluation relies on the assumption that no other policies have specifically targeted the pilot districts in the relevant time frame. Other policies can be in place, as long as they are evenly distributed across the country. Our understanding is that this does indeed reflect the situation. We will run the above regression for all outcome variables available in our dataset. The coefficient on d HSRP*2005 , or the same interaction term for other reform interventions, will return the impact of the project on the respective outcome variable, at the district level. 26 Attanasio O, Mesnard A.: The Impact of a Conditional Cash Transfer Programme on Consumption in Colombia, EDePo, Institute of Fiscal Studies Working Paper, Report Summary Familias 02, April 2006 50 ECONOMETRIC RESULTS From the Quarterly MOH panel (facility level) We have analyzed the impact of the reform interventions on the primary care utilization rate, the infant diarrhea mortality rate and the accreditation score. We consider the interventions of family health training (measured as share of practice ready trained physicians in a facility), infrastructure investment (we test both the total investment amount per person in the catchment, and the civil works- and medical/furniture investment pc as sensitivity). Table 14: Econometric analysis of the impact of the HSRP on primary care utilization Dependent Outpatient encounters per physician Outpatient encounters per person in Variable catchment Coefficient z-stat Coefficient z-stat Coefficient z-stat Coefficient z-stat Interventions share family- health physicians 390 3.0 390 3.0 0.13 2.0 0.13 2.0 infrastructure inv (total pc) 0.05 0.1 0.001 2.3 civil works investment pc 0.1 0.1 -6.17E-06 -0.2 Medical/furn investment pc 0.03 0.5 0.002 2.4 co-payment -115 -2.8 -115 -2.8 -0.04 -1.7 -0.04 -1.7 Controls district size 0.0001 0.7 0.0001 0.7 -5.85E-08 -0.3 -5.86E-08 -0.3 pc consumption -0.3 -1.8 -0.3 -1.8 4.74E-05 0.5 4.83E-05 0.5 av hh size -58 -0.8 -58 -0.8 -0.10 -2.7 -0.10 -2.6 % illit hh heads -7 -1.3 -7 -1.3 -6.98E-04 -0.3 -6.27E-04 -0.3 dependency -90 -1.1 -90 -1.1 0.01 0.1 0.00 0.1 unemployment -1645 -2.4 -1649 -2.4 -0.35 -1.3 -0.35 -1.3 crowding 11 0.0 10 0.0 -0.01 0.0 0.00 0.0 access water -2.9 -0.3 -3 -0.3 -1.83E-03 -0.5 -1.75E-03 -0.4 washing mach 1.1 0.2 1 0.2 0.0004 0.1 0.000 0.1 constant 1,806 1.4 1811 1.35 0.64 1.4 0.74 1.6 Obs 883 883 881 881 Group variable facility facility facility facility Number of groups 186 186 186 186 R-sq: within 0.18 0.18 0.05 0.05 between 0.14 0.14 0.13 0.12 overall 0.13 0.13 0.08 0.07 Gaussian 88.16 Wald 294.5 305.6 76.71 chi2(12/13) Note: model is Random-effects GLS regression, Std Err adjusted for clustering in district, assumption corr(u_i, X) ==0 51 We find that a higher share of family health trained physicians drives the utilization rate up, see Table 14. This finding is robust to measuring investment as a total or as its components, and to measuring utilization as encounters per person or encounters per physician. A 1% increase in the share would trigger roughly an additional 4 visits per physician per year, or just an additional visit per 1000 people in the catchment. We also find that the implementation of the co-payment drives utilization down. Implementation reduces the visits per physician by about 114 a year, and visits per 10,000 people by 4. The impact of infrastructure investment is not robust across the two utilization measures. It seems to make a difference for person visits only, however a substantial one. 1 L.E. spent per person in the catchment area triggers one additional visit per 1,000 people. If we differentiate by civil and medical/furniture investment (furniture is so correlated with medical we cannot identify it separately), we see that the increase in utilization is likely triggered by the investment in medical/furniture equipment, while civil works do not make a difference. If we assume that medical equipment is the main driver between medical and furniture, 1 L.E. spent on medical equipment per person in the catchment area triggers 2 additional visits per 1,000 people. Our control variables serve only to match treatment and control districts; they do not have a causal interpretation. Although these results look highly likely, we need to bear in mind that allocation to facilities within districts may not be random. While we control well for allocation to districts, we do not have data to do this within districts. Also, our data is neither as complete as desirable nor free from measurement error. We had to drop a few observations where facilities reported more family health trained physicians than total physicians. Also, we cannot measure all training provided under the HSRP in full detail because data on all courses is only available for Alexandria and Sohag. The next table, Table 15, summarizes the regression results of the diarrhea mortality rate. 52 Table 15: Econometric analysis of the impact of the HSRP on the diarrhea mortality rate in children Dependent Variable Diarrhea mortality rate Coefficient z-stat Coefficient z-stat Interventions Share family-health physicians -0.01 -1.2 -0.01 -1.2 Infrastructure inv (total pc) 2.16E-05 0.3 Civil works investment pc -1.06E-05 -2.0 Medical investment pc 6.11E-05 0.4 Co-payment -0.003 -0.4 -0.003 -0.4 Controls district size -3.40E-08 -1.6 -3.40E-08 -1.6 pc consumption 1.81E-05 0.7 1.83E-05 0.7 av hh size -0.003 -0.3 -0.003 -0.3 % illit hh heads 0.001 1.2 0.001 1.3 dependency 0.010 1.0 0.010 1.0 unemployment 0.012 0.1 0.013 0.1 crowding 0.020 0.6 0.021 0.7 access water 1.15E-03 1.3 1.15E-03 1.3 washing mach 0.001 1.6 0.001 1.6 constant -0.259 -1.8 -0.261 -1.9 Obs 740 740 Group variable facility facility Number of groups 178 178 R-sq: within 0.002 0.004 between 0.07 0.05 overall 0.02 0.01 Gaussian Wald chi2 (12/13) 28.5 137.94 Note: model is Random-effects GLS regression, Std Err adjusted for clustering in district, assumption corr(u_i, X) ==0 The results for the diarrhea mortality rate are less conclusive. The share of family health trained physicians is not really significant although it bears the expected negative sign (driving the mortality rate down). Investment gives different results when considered as a bulk, or as its elements. In the latter case, civil works investments (renovation and waiting areas) seem to drive the mortality rate down. The implementation of co-payment is insignificant. However, we are reluctant to take these results as significant, as the R-sq is never above 7%. Further all caveats mentioned previously apply here as well. 53 The next table, Table 16, shows the regression results for the facility accreditation scores. Table 16: Econometric analysis of the impact of the HSRP on the facility accreditation score Dependent Variable Accreditation score Coefficient z-stat Coefficient z-stat Interventions Share family-health physicians -3.12 -2.9 -3.45 -3.0 Infrastructure inv (total pc) 0.09 7.5 Civil works investment pc 0.16 1.4 Medical investment pc 0.10 2.1 FHF contract/ Co- payment 2.27 2.4 2.33 2.4 Controls district size -1.28E-08 0.0 2.78E-07 0.1 pc consumption -0.01 -1.3 -0.005 -1.1 av hh size -5.81 -1.4 -5.67 -1.4 % illit hh heads -0.15 -0.8 -0.14 -0.7 dependency -3.14 -0.9 -2.96 -0.8 unemployment -23.74 -1.2 -23.49 -1.2 crowding -9.52 -1.2 -9.22 -1.2 access water -0.05 -0.2 -0.05 -0.2 washing mach 0.03 0.2 0.03 0.2 constant 149 4.5 146.78 4.4 Obs 788 788 Group variable facility facility Number of groups 153 153 R-sq: within 0.19 0.20 between 0.25 0.24 overall 0.21 0.21 Gaussian Wald chi2 (12/13) 172.71 117.45 Note: model is Random-effects GLS regression, Std Err adjusted for clustering in district, assumption corr(u_i, X) ==0 The results for the accreditation score are significant, but not all as expected. The share of family health trained physicians appears to drive the accreditation score down, albeit by very little.27 This relationship likely results from an endogeneity. It is quite possible that training is allocated earlier, and with greater intensity, to facilities that appear to need it given their accreditation score. As said in the beginning, the new quality supervision system under the HSRP contains an improvement plan for failing facilities. Unfortunately, 27 Note that the accreditation score is measured in full percent, like 10 for 10% and 20 for 20%, whereas the share of family health physicians is reported as a ratio, 0.1 for 10%, and 0.2 for 20%. It is only a question of dimensions. 54 our data does not allow us to track intervention targeting within the districts, therefore we cannot control for this. Given this likely endogeneity, we have to assume that the coefficients on the investment variables are biased upwards. Both civil and medical investment seem to increase the accreditation score, 10 L.E. per capita of civil works entailing a 16%point increase, and 10 L.E. of medical equipment entailing a 10%point increase in accreditation scores. In spite of the bias, there is some confidence in this impact, because the significance of the investment coefficients is robust to excluding the endogenous variable. The coefficients however change. Also, contracting with the FHF seems to make a difference, as expected. As soon as a facility contracts with the FHF, the accreditation score appears to go up 3% points. This result is robust to excluding the training variable. Apart from GLS regressions with the quarterly data, we have also run Difference in differences estimates with the DHS panel data. The results are displayed below. From the DHS panel (district level) We have access to two years of the DHS which straddle the HSRP intervention years, 2000 and 2005. This two-year panel lends itself to an examination of the HSRP impact with a difference-in-differences (DiD) estimator. We have collapsed the DHS at the district level, and merged it with the socio-economic data from the 1996 census, and MOH data on interventions for: contracting with the FHF (which measures participation in the HSRP at all), family health training and the different kinds of investment. For all potential outcome variables available to us from the DHS, we have run three types of regressions: 1. a straight DiD measuring the impact of the reform as a package with just a dummy for treatment (,,HSRP participation); 2. a DiD testing for different independent components of the HSRP, with dummies for training, investment, and contracting; and 3. a DiD measuring interventions in their degrees, i.e. the share of facilities in a district affected by family health training, the average civil investment per person in a district and the average medical investment per person in a district, as well as the share of facilities in a district that are contracted. Note that as DiDs, all regressions contain a time-invariant dummy for HSRP participation, and a dummy indicating the second year, 2005. They also contain the most complete set of district-level control variables available to us, i.e. the components of the socio-economic vulnerability index used for targeting, and regional dummies for upper/lower, and rural/urban Egypt, respectively. Given that we work with about 500 observations, we choose a parametric approach. Significant results are presented below. 55 Table 17: Econometric analysis of the impact of the HSRP on vaccination coverage, anemia, and contraception Dependent Variable Vacc coverage % couples using Anemia rate (measles) contraception (women) Coefficient z-stat Coefficient z-stat Coefficient z- stat Interventions HSRP participation (dummy) -0.05 -2.3 -0.06 -3.34 0.01 0.35 year 2005 0.02 1.9 0.05 4.74 0.12 6.93 HSRP in 2005 (dummy) 0.04 1.5 0.04 1.43 -0.07 -1.53 Controls lower urban -0.04 -1.6 -0.01 -0.6 -0.07 -1.9 lower rural -0.004 -0.2 -0.022 -1.1 -0.004 -0.1 upper urban -0.03 -1.0 -0.02 -0.9 -0.07 -1.8 upper rural -0.06 -2.4 -0.11 -4.7 0.11 2.6 frontier governorates -0.04 -1.4 -0.13 -5.7 0.02 0.6 district size -7.54E-09 -0.2 8.11E-08 2.7 -3.33E-08 -0.6 pc consumption 1.83E-05 1.2 -2.4E-05 -1.9 -1.1E-05 -0.5 av hh size 0.005 0.4 0.027 2.3 0.007 0.3 % illit hh heads -0.0002 -0.3 -0.0002 -0.3 0.0021 1.8 dependency 0.009 0.5 -0.057 -3.4 -0.115 -4.0 unemployment -0.020 -0.1 0.145 1.2 -0.271 -1.3 crowding -0.019 -0.6 -0.133 -4.5 0.108 2.1 access water 0.0002 0.5 -0.0015 -3.6 -0.0012 -1.6 washing mach -0.0002 -0.3 0.0025 5.0 0.0012 1.4 constant 0.73 6.8 0.94 10 0.94 10 Observations 435 449 449 left censored 1 0 18 right censored 13 4 2 LR chi2(17) 46.73 395.46 104.04 Prob chi2 0 0 0 Default region urban urban urban Note: model is a Difference in differences Tobit at the district level We see that the HSRP as a package has the intended result on vaccination coverage (measured as measles coverage), use of family planning and the anemia rate (indicating nutritional status). It also appears to have a moderate impact on the use of medical treatment for child fever/cough cases, when measured as the share of participating facilities in a district, see Table 18. Further, both the reform package and separately medical investment seem to have occasioned a shift from secondary to primary care. 56 Table 18: Econometric analysis of the impact of the HSRP on fever treatment of children, and demand for primary and secondary care in fever treatment Dependent Variable Demand difference primary/ Fever medically treated (%) secondary (fever) Coefficient z-stat Coefficient z-stat Interventions HSRP participation (dummy) -0.18 -1.8 -0.06 -0.9 year 2005 0.08 1.5 0.02 0.59 HSRP in 2005 (share in district) 0.43 2.2 0.22 1.57 share of facilities with training -0.21 -1.0 -0.09 -0.63 civil investment pc -0.01 -1.9 -1.88E-05 -0.01 medical investment pc 0.02 1.5 0.002 0.19 Controls lower urban -0.40 -2.9 -0.07 -0.9 lower rural -0.22 -1.9 -0.160 -2.1 upper urban -0.40 -2.6 -0.01 -0.1 upper rural -0.08 -0.6 -0.17 -1.9 frontier governorates -0.21 -1.4 -0.14 -1.6 district size 2.19E-07 1.4 5.12E-08 0.5 pc consumption -1.22E-04 -1.3 -0.0001 -1.2 av hh size 0.12 1.9 -0.015 -0.4 % illit hh heads 0.01 1.3 -0.0011 -0.4 dependency -0.16 -1.7 -0.102 -1.7 unemployment 0.24 0.4 1.308 2.8 crowding -0.04 -0.3 -0.211 -1.9 access water 0.001 0.5 0.0005 0.3 washing mach 0.001 0.3 -0.001 -0.3 constant -0.43 -0.8 1.35 3.8 Observations 423 423 left censored 294 32 right censored 6 65 LR chi2(17) 47.98 56.08 Prob chi2 0 0 Default region urban urban Note: model is a Difference in differences Tobit at the district level We did not find any impact of the HSRP at the district level on the share of women having at least 4 antenatal visits, share of professionally attended births and the share of diarrhea cases in children receiving medical treatment. 57 REFERENCES Antos Joseph, Hafez-Afifi Nihal and Sameh El-Saharty: Egypts Health Sector Reform and Financing Review; World Bank, February 2004 Attanasio Orazio, Mesnard Alice: The Impact of a Conditional Cash Transfer Programme on Consumption in Colombia, EDePo, Institute of Fiscal Studies Working Paper, Report Summary Familias 02, April 2006 Duflo Esther and Rema Hanna: "Monitoring Works: Getting Teachers to Come to School", MIT 2005 Grun Rebekka: The Quantitative Evaluation of Projects, A Step-by-step Primer on Benchmarking and Impact Evaluation, The World Bank, HNP Discussion Paper, 2006 Ministry of Health (MOH) Egypt: Egypt Health Sector Analysis and Future Strategies, Health Sector Reform Program, Version 2, January 2003 Ministry of Health (MOH) Egypt, World Bank, the European Commission and USAID: "Egypt Health Sector Reform Program Strategy Paper", December 1997 Schieber, George: Project Appraisal Document for a Proposed Credit in the Amount of SDR 66.8m to the Arab Republic of Egypt for a Health Sector Reform Program, The World Bank, 1998 (PAD) WHO Regional Office for Europe: "Health for all/ Gesundheit fuer Alle", Data Base, June 2004 58 About this series... This series is produced by the Health, Nutrition, and Population Family (HNP) of the World Bank's Human Development Network. The papers in this series aim to provide a vehicle for publishing preliminary and unpolished results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual authors whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Homira Nassery (hnassery@worldbank.org) or HNP Advisory Service (healthpop@worldbank.org, tel 202 473-2256, fax 202 522-3234). For more information, see also www.worldbank.org/ hnppublications. THE WORLD BANK 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org