Publication: Which Doctor? Combining Vignettes and Item Response to Measure Doctor Quality
Loading...
Date
2004-05
ISSN
Published
2004-05
Author(s)
Hammer, Jeffrey
Editor(s)
Abstract
The authors develop a method in which vignettes-a battery of questions for hypothetical cases-are evaluated with item response theory to create a metric for doctor quality. The method allows a simultaneous estimation of quality and validation of the test instrument that can be used for further refinements. The authors apply the method to a sample of medical practitioners in Delhi, India. The method gives plausible results, rationalizes different perceptions of quality in the public and private sectors, and pinpoints several serious problems with health care delivery in urban India. The findings confirm, for instance, that the quality of private providers located in poorer areas of the city is significantly lower than those in richer neighborhoods. Surprisingly, similar results hold for providers in the public sector, with important implications for inequities in the availability of health care.
Link to Data Set
Citation
“Hammer, Jeffrey; Das, Jishnu. 2004. Which Doctor? Combining Vignettes and Item Response to Measure Doctor Quality. Policy Research Working Paper;No.3301. © World Bank. http://hdl.handle.net/10986/14032 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication The Future of Poverty(Washington, DC: World Bank, 2025-07-15)Climate change is increasingly acknowledged as a critical issue with far-reaching socioeconomic implications that extend well beyond environmental concerns. Among the most pressing challenges is its impact on global poverty. This paper projects the potential impacts of unmitigated climate change on global poverty rates between 2023 and 2050. Building on a study that provided a detailed analysis of how temperature changes affect economic productivity, this paper integrates those findings with binned data from 217 countries, sourced from the World Bank’s Poverty and Inequality Platform. By simulating poverty rates and the number of poor under two climate change scenarios, the paper uncovers some alarming trends. One of the primary findings is that the number of people living in extreme poverty worldwide could be nearly doubled due to climate change. In all scenarios, Sub-Saharan Africa is projected to bear the brunt, contributing the largest number of poor people, with estimates ranging between 40.5 million and 73.5 million by 2050. Another significant finding is the disproportionate impact of inequality on poverty. Even small increases in inequality can lead to substantial rises in poverty levels. For instance, if every country’s Gini coefficient increases by just 1 percent between 2022 and 2050, an additional 8.8 million people could be pushed below the international poverty line by 2050. In a more extreme scenario, where every country’s Gini coefficient increases by 10 percent between 2022 and 2050, the number of people falling into poverty could rise by an additional 148.8 million relative to the baseline scenario. These findings underscore the urgent need for comprehensive climate policies that not only mitigate environmental impacts but also address socioeconomic vulnerabilities.Publication Exports, Labor Markets, and the Environment(Washington, DC: World Bank, 2025-07-14)What is the environmental impact of exports? Focusing on 2000–20, this paper combines customs, administrative, and census microdata to estimate employment elasticities with respect to exports. The findings show that municipalities that faced increased exports experienced faster growth in formal employment. The elasticities were 0.25 on impact, peaked at 0.4, and remained positive and significant even 10 years after the shock, pointing to a long and protracted labor market adjustment. In the long run, informal employment responds negatively to export shocks. Using a granular taxonomy for economic activities based on their environmental impact, the paper documents that environmentally risky activities have a larger share of employment than environmentally sustainable ones, and that the relationship between these activities and exports is nuanced. Over the short run, environmentally risky employment responds more strongly to exports relative to environmentally sustainable employment. However, over the long run, this pattern reverses, as the impact of exports on environmentally sustainable employment is more persistent.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication The Asymmetric Bank Distress Amplifier of Recessions(Washington, DC: World Bank, 2025-07-11)One defining feature of financial crises, evident in U.S. and international data, is asymmetric bank distress—concentrated losses on a subset of banks. This paper proposes a model in which shocks to borrowers’ productivity dispersion lead to asymmetric bank losses. The framework exhibits a “bank distress amplifier,” exacerbating economic downturns by causing costly bank failures and raising uncertainty about the solvency of banks, thereby pushing banks to deleverage. Quantitative analysis shows that the bank distress amplifier doubles investment decline and increases the spread by 2.5 times during the Great Recession compared to a standard financial accelerator model. The mechanism helps explain how a seemingly small shock can sometimes trigger a large crisis.Publication Impact of Heat Waves on Learning Outcomes and the Role of Conditional Cash Transfers(Washington, DC: World Bank, 2025-07-14)This paper evaluates the impact of higher temperatures on learning outcomes in Peru. The results suggest that 1 degree above 20°C is equivalent to 7 and 6 percent of a standard deviation of what a student learns in a year for math and reading tests, respectively. These results hold true when the main specification is changed, splitting the sample, collapsing the data at school level, and using other climate specifications. The paper aims to improve understanding of how to deal with the impacts of climate change on learning outcomes in developing countries. The evidence suggests that conditional cash transfer programs can mitigate the negative effects of higher temperatures on students’ learning outcomes in math and reading.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Money for Nothing : The Dire Straits of Medical Practice in Delhi, India(World Bank, Washington, DC, 2005-07)The quality of medical care received by patients varies for two reasons: differences in doctors' competence or differences in doctors' incentives. Using medical vignettes, the authors evaluated competence for a sample of doctors in Delhi. One month later, they observed the same doctors in their practice. The authors find three patterns in the data. First, what doctors do is less than what they know they should do-doctors operate well inside their knowledge frontier. Second, competence and effort are complementary so that doctors who know more also do more. Third, the gap between what doctors do and what they know responds to incentives: doctors in the fee-for-service private sector are closer in practice to their knowledge frontier than those in the fixed-salary public sector. Under-qualified private sector doctors, even though they know less, provide better care on average than their better-qualified counterparts in the public sector. These results indicate that to improve medical services, at least for poor people, there should be greater emphasis on changing the incentives of public providers rather than increasing provider competence through training.Publication Strained Mercy: The Quality of Medical Care in Delhi(World Bank, Washington, D.C., 2004-03)The quality of medical care is a potentially important determinant of health outcomes. Nevertheless, it remains an understudied area. The limited research that exists defines quality either on the basis of drug availability or facility characteristics, but little is known about how provider quality affects the provision of health care. The authors address this gap through a survey in Delhi with two related components. They evaluate "competence" (what providers know) through vignettes and practice (what providers do) through direct clinical observation. Overall quality as measured by the competence necessary to recognize and handle common and dangerous conditions is quite low, albeit with tremendous variation. While there is some correlation with simple observed characteristics, there is still an enormous amount of variation within such categories. Further, even when providers know what to do they often do not do it in practice. This appears to be true in both the public and private sectors though for very different, and systematic, reasons. In the public sector providers are more likely to commit errors of omission-they are less likely to exert effort compared with their private counterparts. In the private sector, providers are prone to errors of commission-they are more likely to behave according to the patient's expectations, resulting in the inappropriate use of medications, the overuse of antibiotics, and increased expenditures. This has important policy implications for our understanding of how market failures and failures of regulation in the health sector affect the poor.Publication The Quality of Medical Advice in Low-Income Countries(World Bank, Washington, DC, 2008-01)This paper provides an overview of recent work on quality measurement of medical care and its correlates in four low and middle-income countries-India, Indonesia, Tanzania, and Paraguay. The authors describe two methods-testing doctors and watching doctors-that are relatively easy to implement and yield important insights about the nature of medical care in these countries. The paper discusses the properties of these measures, their correlates, and how they may be used to evaluate policy changes. Finally, the authors outline an agenda for further research and measurement.Publication Ghost Doctors : Absenteeism in Rural Bangladeshi Health Facilities(Washington, DC: World Bank, 2004-09)Unannounced visits were made to health clinics in Bangladesh to determine what proportion of medical professionals were at their assigned post. Averaged over all job categories and types of facility, the absentee rate was 35 percent. The absentee rate for physicians was 40 percent at the larger clinics and 74 percent at the smaller sub-centers with a single physician. Whether the medical provider lives near the health facility, the opportunity cost of the provider's time, road access, and rural electrification are highly correlated with the rate and pattern of absenteeism.Publication Indonesia's Doctors, Midwives and Nurses : Current Stock, Increasing Needs, Future Challenges and Options(World Bank, 2009)This paper is one of several inputs prepared for a comprehensive Health Sector Review that the Government of Indonesia is currently conducting. It compiles, analyzes and interprets available information on Indonesia's health service providers; doctors, midwives and nurses. Within the limitations imposed by questions about the accuracy and timeliness of current workforce data, the paper describes the stock and distribution of health workers. The paper draws attention to weaknesses in the workforce planning methods in use, and then reviews the human resource policies, including governance structures and the regulatory framework, that affect health workers. It concludes by describing future challenges and some suggested ways of addressing these challenges. In brief, this paper identifies a number of main challenges in the health workforce in Indonesia. These are: (i) there is a shortage and inequitable distribution of medical doctors and specialists; (ii) the education of health professionals is of poor quality and the accreditation and certification system is weak; (iii) health workforce policy development and planning are not based on evidence or demand, but rather on standard norms that do not reflect real need or take into account the contribution of the private health sector; nor have they adapted to a decentralized paradigm, and finally; (iv) the growing and changing demand for health care due to demographic and epidemiological changes will increase the burden on the already ineffective health system. The final section of this paper makes nine suggestions designed to address these main challenges.
Users also downloaded
Showing related downloaded files
Publication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.Publication World Development Report 2008(Washington, DC, 2007)The world's demand for food is expected to double within the next 50 years, while the natural resources that sustain agriculture will become increasingly scarce, degraded, and vulnerable to the effects of climate change. In many poor countries, agriculture accounts for at least 40 percent of GDP and 80 percent of employment. At the same time, about 70 percent of the world's poor live in rural areas and most depend on agriculture for their livelihoods. World Development Report 2008 seeks to assess where, when, and how agriculture can be an effective instrument for economic development, especially development that favors the poor. It examines several broad questions: How has agriculture changed in developing countries in the past 20 years? What are the important new challenges and opportunities for agriculture? Which new sources of agricultural growth can be captured cost effectively in particular in poor countries with large agricultural sectors as in Africa? How can agricultural growth be made more effective for poverty reduction? How can governments facilitate the transition of large populations out of agriculture, without simply transferring the burden of rural poverty to urban areas? How can the natural resource endowment for agriculture be protected? How can agriculture's negative environmental effects be contained? This year's report marks the 30th year the World Bank has been publishing the World Development Report.Publication World Development Report 1987(New York: Oxford University Press, 1987)This report, consisting of two parts, is the tenth in the annual series assessing development issues. Part I reviews recent trends in the world economy and their implications for the future prospects of developing countries. It stresses that better economic performance is possible in both industrial and developing countries, provided the commitment to economic policy reforms is maintained and reinforced. In regard to the external debt issues, the report argues for strengthened cooperation among industrial countries in the sphere of macroeconomic policy to promote smooth adjustment to the imbalances caused by external payments (in developing countries). Part II reviews and evaluates the varied experience with government policies in support of industrialization. Emphasis is placed on policies which affect both the efficiency and sustainability of industrial transformation, especially in the sphere of foreign trade. The report finds that developing countries which followed policies that promoted the integration of their industrial sector into the international economy through trade have fared better than those which insulated themselves from international competition.Publication World Development Report 2005(World Bank, 2004)Firms and entrepreneurs of all types from microenterprises to multinationals play a central role in growth and poverty reduction. Their investment decisions drive job creation, the availability and affordability of goods and services for consumers, and the tax revenues governments can draw on to fund health, education, and other services. Their contribution depends largely on the way governments shape the investment climate in each location through the protection of property rights, regulation and taxation, strategies for providing infrastructure, interventions in finance and labor markets, and broader governance features such as corruption. The World Development Report 2005 argues that improving the investment climates of their societies should be a top priority for governments. Drawing on surveys of nearly 30,000 firms in 53 developing countries, country case studies, and other new research, the Report explores questions such as: What are the key features of a good investment climate, and how do they influence growth and poverty? What can governments do to improve their investment climates, and how can they go about tackling such a broad agenda? What has been learned about good practice in each of the main areas of the investment climate? What role might selective interventions and international arrangements play in improving the investment climate? What can the international community do to help developing countries improve the investment climates of their societies? In addition to detailed chapters exploring these and related issues, the Report contains selected data from the World Bank's new program of Investment Climate Surveys, the Bank's Doing Business Project, and World Development Indicators 2004‹an appendix of economic and social data for over 200 countries. This Report offers practical insights for policymakers, executives, scholars, and all those with an interest in economic development.Publication World Development Report 1994(New York: Oxford University Press, 1994)World Development Report 1994, the seventeenth in this annual series, examines the link between infrastructure and development and explores ways in which developing countries can improve both the provision and the quality of infrastructure services. In recent decades, developing countries have made substantial investments in infrastructure, achieving dramatic gains for households and producers by expanding their access to services such as safe water, sanitation, electric power, telecommunications, and transport. Even more infrastructure investment and expansion are needed in order to extend the reach of services - especially to people living in rural areas and to the poor. But as this report shows, the quantity of investment cannot be the exclusive focus of policy. Improving the quality of infrastructure service also is vital. Both quantity and quality improvements are essential to modernize and diversify production, help countries compete internationally, and accommodate rapid urbanization. The report identifies the basic cause of poor past performance as inadequate institutional incentives for improving the provision of infrastructure. To promote more efficient and responsive service delivery, incentives need to be changed through commercial management, competition, and user involvement. Several trends are helping to improve the performance of infrastructure. First, innovation in technology and in the regulatory management of markets makes more diversity possible in the supply of services. Second, an evaluation of the role of government is leading to a shift from direct government provision of services to increasing private sector provision and recent experience in many countries with public-private partnerships is highlighting new ways to increase efficiency and expand services. Third, increased concern about social and environmental sustainability has heightened public interest in infrastructure design and performance. This report includes the World Development Indicators, which offer selected social and economic statistics for 132 countries.