Publication: Which Doctor? Combining Vignettes and Item Response to Measure Doctor Quality
Loading...
Date
2004-05
ISSN
Published
2004-05
Author(s)
Hammer, Jeffrey
Editor(s)
Abstract
The authors develop a method in which vignettes-a battery of questions for hypothetical cases-are evaluated with item response theory to create a metric for doctor quality. The method allows a simultaneous estimation of quality and validation of the test instrument that can be used for further refinements. The authors apply the method to a sample of medical practitioners in Delhi, India. The method gives plausible results, rationalizes different perceptions of quality in the public and private sectors, and pinpoints several serious problems with health care delivery in urban India. The findings confirm, for instance, that the quality of private providers located in poorer areas of the city is significantly lower than those in richer neighborhoods. Surprisingly, similar results hold for providers in the public sector, with important implications for inequities in the availability of health care.
Link to Data Set
Citation
“Hammer, Jeffrey; Das, Jishnu. 2004. Which Doctor? Combining Vignettes and Item Response to Measure Doctor Quality. Policy Research Working Paper;No.3301. © World Bank. http://hdl.handle.net/10986/14032 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication Geopolitical Fragmentation and Friendshoring(Washington, DC: World Bank, 2025-06-26)This paper examines the relationship between geopolitical fragmentation and friendshoring of foreign investments over time, countries, and sectors. The analysis uses comprehensive data on foreign direct investments covering greenfield projects, mergers and acquisitions, and stocks of affiliates, as well as data on four alternative measures of geopolitical distance between countries. The gravity estimations suggest that, first, geopolitical differences have a negative effect on foreign investments and the magnitude has heightened in the post-pandemic period compared to a decade ago. Second, it is primarily the companies from advanced Western economies whose foreign investment decisions are increasingly shaped by friendshoring forces. Finally, the paper shows that friendshoring is not only confined to strategic industries, implying that allocations of foreign direct investments may not solely reflect national security or resilience considerations.Publication Soaring Food Prices Threaten Recent Economic Gains in the EU(Washington, DC: World Bank, 2025-07-02)The surge in food prices following the 2021 economic rebound has become a significant concern for households, particularly low-income ones, in Bulgaria, Croatia, Poland, and Romania. Food price inflation, which surpasses general inflation rates, risks worsening poverty and food insecurity in these countries. This paper explores the distributional impacts of rising food prices and the effectiveness of government response measures. Low-income households, who allocate a larger share of their income to food, are disproportionately affected and are struggling to cope with unexpected expenses, leading to increased difficulties in accessing proper nutrition. Simulations indicate that rising food prices contribute to higher poverty rates and greater income inequality, especially among vulnerable populations. They also suggest that the main poverty-targeted social assistance schemes offer critical support for the extreme poor, but expanding both coverage and benefits is vital to shield all at-risk individuals. Targeted policies that balance immediate relief with long-term resilience-building are essential to addressing the challenges posed by escalating food prices.Publication Disentangling the Key Economic Channels through Which Infrastructure Affects Jobs(Washington, DC: World Bank, 2025-04-03)This paper takes stock of the literature on infrastructure and jobs published since the early 2000s, using a conceptual framework to identify the key channels through which different types of infrastructure impact jobs. Where relevant, it highlights the different approaches and findings in the cases of energy, digital, and transport infrastructure. Overall, the literature review provides strong evidence of infrastructure’s positive impact on employment, particularly for women. In the case of electricity, this impact arises from freeing time that would otherwise be spent on household tasks. Similarly, digital infrastructure, particularly mobile phone coverage, has demonstrated positive labor market effects, often driven by private sector investments rather than large public expenditures, which are typically required for other large-scale infrastructure projects. The evidence on structural transformation is also positive, with some notable exceptions, such as studies that find no significant impact on structural transformation in rural India in the cases of electricity and roads. Even with better market connections, remote areas may continue to lack economic opportunities, due to the absence of agglomeration economies and complementary inputs such as human capital. Accordingly, reducing transport costs alone may not be sufficient to drive economic transformation in rural areas. The spatial dimension of transformation is particularly relevant for transport, both internationally—by enhancing trade integration—and within countries, where economic development tends to drive firms and jobs toward urban centers, benefitting from economies scale and network effects. Turning to organizational transformation, evidence on skill bias in developing countries is more mixed than in developed countries and may vary considerably by context. Further research, especially on the possible reasons explaining the differences between developed and developing economies, is needed.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Money for Nothing : The Dire Straits of Medical Practice in Delhi, India(World Bank, Washington, DC, 2005-07)The quality of medical care received by patients varies for two reasons: differences in doctors' competence or differences in doctors' incentives. Using medical vignettes, the authors evaluated competence for a sample of doctors in Delhi. One month later, they observed the same doctors in their practice. The authors find three patterns in the data. First, what doctors do is less than what they know they should do-doctors operate well inside their knowledge frontier. Second, competence and effort are complementary so that doctors who know more also do more. Third, the gap between what doctors do and what they know responds to incentives: doctors in the fee-for-service private sector are closer in practice to their knowledge frontier than those in the fixed-salary public sector. Under-qualified private sector doctors, even though they know less, provide better care on average than their better-qualified counterparts in the public sector. These results indicate that to improve medical services, at least for poor people, there should be greater emphasis on changing the incentives of public providers rather than increasing provider competence through training.Publication Strained Mercy: The Quality of Medical Care in Delhi(World Bank, Washington, D.C., 2004-03)The quality of medical care is a potentially important determinant of health outcomes. Nevertheless, it remains an understudied area. The limited research that exists defines quality either on the basis of drug availability or facility characteristics, but little is known about how provider quality affects the provision of health care. The authors address this gap through a survey in Delhi with two related components. They evaluate "competence" (what providers know) through vignettes and practice (what providers do) through direct clinical observation. Overall quality as measured by the competence necessary to recognize and handle common and dangerous conditions is quite low, albeit with tremendous variation. While there is some correlation with simple observed characteristics, there is still an enormous amount of variation within such categories. Further, even when providers know what to do they often do not do it in practice. This appears to be true in both the public and private sectors though for very different, and systematic, reasons. In the public sector providers are more likely to commit errors of omission-they are less likely to exert effort compared with their private counterparts. In the private sector, providers are prone to errors of commission-they are more likely to behave according to the patient's expectations, resulting in the inappropriate use of medications, the overuse of antibiotics, and increased expenditures. This has important policy implications for our understanding of how market failures and failures of regulation in the health sector affect the poor.Publication The Quality of Medical Advice in Low-Income Countries(World Bank, Washington, DC, 2008-01)This paper provides an overview of recent work on quality measurement of medical care and its correlates in four low and middle-income countries-India, Indonesia, Tanzania, and Paraguay. The authors describe two methods-testing doctors and watching doctors-that are relatively easy to implement and yield important insights about the nature of medical care in these countries. The paper discusses the properties of these measures, their correlates, and how they may be used to evaluate policy changes. Finally, the authors outline an agenda for further research and measurement.Publication Ghost Doctors : Absenteeism in Rural Bangladeshi Health Facilities(Washington, DC: World Bank, 2004-09)Unannounced visits were made to health clinics in Bangladesh to determine what proportion of medical professionals were at their assigned post. Averaged over all job categories and types of facility, the absentee rate was 35 percent. The absentee rate for physicians was 40 percent at the larger clinics and 74 percent at the smaller sub-centers with a single physician. Whether the medical provider lives near the health facility, the opportunity cost of the provider's time, road access, and rural electrification are highly correlated with the rate and pattern of absenteeism.Publication Indonesia's Doctors, Midwives and Nurses : Current Stock, Increasing Needs, Future Challenges and Options(World Bank, 2009)This paper is one of several inputs prepared for a comprehensive Health Sector Review that the Government of Indonesia is currently conducting. It compiles, analyzes and interprets available information on Indonesia's health service providers; doctors, midwives and nurses. Within the limitations imposed by questions about the accuracy and timeliness of current workforce data, the paper describes the stock and distribution of health workers. The paper draws attention to weaknesses in the workforce planning methods in use, and then reviews the human resource policies, including governance structures and the regulatory framework, that affect health workers. It concludes by describing future challenges and some suggested ways of addressing these challenges. In brief, this paper identifies a number of main challenges in the health workforce in Indonesia. These are: (i) there is a shortage and inequitable distribution of medical doctors and specialists; (ii) the education of health professionals is of poor quality and the accreditation and certification system is weak; (iii) health workforce policy development and planning are not based on evidence or demand, but rather on standard norms that do not reflect real need or take into account the contribution of the private health sector; nor have they adapted to a decentralized paradigm, and finally; (iv) the growing and changing demand for health care due to demographic and epidemiological changes will increase the burden on the already ineffective health system. The final section of this paper makes nine suggestions designed to address these main challenges.
Users also downloaded
Showing related downloaded files
Publication Crime and Violence in Central America : A Development Challenge - Main Report(World Bank, 2011-01-01)Crime and violence are now a key development issue for Central American countries. In three nations El Salvador, Guatemala, and Honduras crime rates are among the top five in Latin America. This report argues that successful strategies require actions along multiple fronts, combining prevention and criminal justice reform, together with regional approaches in the areas of drug trafficking and firearms. It also argues that interventions should be evidence based, starting with a clear understanding of the risk factors involved and ending with a careful evaluation of how any planned action might affect future options. In addition, the design of national crime reduction plans and the establishment of national cross-sectoral crime commissions are important steps to coordinate the actions of different government branches, ease cross-sectoral collaboration and prioritize resource allocation. Of equal importance is the fact that national plans offer a vehicle for the involvement of civil society organizations, in which much of the expertise in violence prevention and rehabilitation resides. Prevention efforts need to be complemented by effective law enforcement. The required reforms are no longer primarily legislative in nature because all six countries have advanced toward more transparent adversarial criminal procedures. The second-generation reforms should instead help deliver on the promises of previous reforms by: (i) strengthening key institutions and improving the quality and timeliness of the services they provide to citizens; (ii) improving efficiency and effectiveness while respecting due process and human rights; (iii) ensuring accountability and addressing corruption; (iv) increasing inter-agency collaboration; and (v) improving access to justice, especially for poor and disenfranchised groups. Specific interventions reviewed in the report include: information systems and performance indicators as a prerequisite to improve inter-institutional coordination and information sharing mechanisms; an internal overhaul of court administration and case management to create rapid reaction, one-stop shops; the strengthening of entities that provide legal counseling to the poor and to women; and the promotion of alternative dispute-resolution mechanisms and the implementation of community policing programs.Publication The Mexican Social Protection System in Health(World Bank, Washington DC, 2013-01)With a population of 113 million and a per-capita Gross Domestic Product, or GDP of US$10,064 (current U.S. dollars), Mexico is one of the largest and highest-income countries in Latin America and the Caribbean (LAC). The country has benefited from sustained economic growth during the last decade, which was temporarily interrupted by the financial and economic crisis. Real GDP is projected to grow 3.8 percent and 3.6 percent in 2012 and 2013, respectively (International Monetary Fund, or IMF 2012). Despite this growth, poverty in the country remains high; with half of the population living below the national poverty line. The country is also highly heterogeneous, with large socioeconomic differences across states and across urban and rural areas. In 2010, while the extreme poverty ratio in the Federal District and the states of Colima and Nuevo Leon was below 3 percent, in Chiapas, Guerrero, and Oaxaca it was 25 percent or higher. These large regional differences are also found in other indicators of well-being, such as years of schooling, housing conditions, and access to social services. This case study assesses key features and achievements of the Social Protection System in Health (Sistema de Proteccion Social en Salud) in Mexico, and particularly of its main pillar, Popular Health Insurance (Seguro Popular, PHI). It analyzes the contribution of this policy to the establishment and implementation of universal health coverage in Mexico. In 2003, with the reform of the General Health Law, the PHI was institutionalized as a subsidized health insurance scheme open to the population not covered by the social security schemes. Today, the PHI covers all of its intended affiliates, about 52 million peoplePublication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.Publication Guide to the Debt Management Performance Assessment Tool(Washington, DC, 2008-02-05)The purpose of this document is to provide guidance and supplemental information to assist with country assessments of debt management performance, using the Debt Management Performance Assessment (DeMPA) tool. The DeMPA is a methodology used for assessing public debt management performance through a comprehensive set of 15 performance indicators spanning the full range of government Debt Management (DeM) functions. It is based on the principles set out in the International Monetary Fund (IMF) and World Bank guidelines for public debt management, initially published in 2001 and updated in 2003. It is modeled after the Public Expenditure and Financial Accountability (PEFA) framework for performance measurement of public financial management. The DeMPA has been designed to be a user-friendly tool to undertake an assessment of the strengths and weaknesses in government DeM practices. This guide provides additional background and supporting information so that a no specialist in the area of debt management may undertake a country assessment effectively. The guide can be used by assessors in preparing for and undertaking an assessment. It is particularly useful for understanding the rationale for the inclusion of the indicators, the scoring methodology, and the list of supporting documents or evidence required, and the questions that could be asked for the assessment.Publication Argentina Country Climate and Development Report(World Bank, Washington, DC, 2022-11)The Argentina Country Climate and Development Report (CCDR) explores opportunities and identifies trade-offs for aligning Argentina’s growth and poverty reduction policies with its commitments on, and its ability to withstand, climate change. It assesses how the country can: reduce its vulnerability to climate shocks through targeted public and private investments and adequation of social protection. The report also shows how Argentina can seize the benefits of a global decarbonization path to sustain a more robust economic growth through further development of Argentina’s potential for renewable energy, energy efficiency actions, the lithium value chain, as well as climate-smart agriculture (and land use) options. Given Argentina’s context, this CCDR focuses on win-win policies and investments, which have large co-benefits or can contribute to raising the country’s growth while helping to adapt the economy, also considering how human capital actions can accompany a just transition.