Publication: Do Value-Added Estimates Add Value? Accounting for Learning Dynamics
Loading...
Date
2009-09-01
ISSN
Published
2009-09-01
Author(s)
Editor(s)
Abstract
Evaluations of educational programs commonly assume that what children learn persists over time. The authors compare learning in Pakistani public and private schools using dynamic panel methods that account for three key empirical challenges to widely used value-added models: imperfect persistence, unobserved student heterogeneity, and measurement error. Their estimates suggest that only a fifth to a half of learning persists between grades and that private schools increase average achievement by 0.25 standard deviations each year. In contrast, estimates from commonly used value-added models significantly understate the impact of private schools on student achievement and/or overstate persistence. These results have implications for program evaluation and value-added accountability system design.
Link to Data Set
Citation
“Andrabi, Tahir; Das, Jishnu; Khwaja, Asim Ijaz; Zajonc, Tristan. 2009. Do Value-Added Estimates Add Value? Accounting for Learning Dynamics. Policy Research working paper ; no. WPS 5066,Paper is funded by the Knowledge for Change
Program (KCP). © World Bank. http://hdl.handle.net/10986/4257 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Geopolitics and the World Trading System(Washington, DC: World Bank, 2024-12-23)Until the beginning of this century, the GATT/WTO system worked. Economic research provided a compelling explanation. It showed that if governments maximize the well-being of their own countries broadly defined, GATT/WTO principles would facilitate mutually beneficial cooperation over their trade policy choices. Now heightened geopolitical rivalry seems to have undermined the WTO. A simple transposition of the previous rationalization suggests that geopolitics and trade cooperation are not compatible. The paper shows that this is only true if rivalry eclipses any consideration of own-country well-being. In all other circumstances, there are gains from trade cooperation even with geopolitics. Furthermore, the WTO’s relevance is in question only if it adheres too rigidly to its existing rules and norms. Through measured adaptation to the geopolitical imperative, the WTO can continue to thrive as a forum for multilateral trade cooperation in the age of geopolitics.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication Global Socio-economic Resilience to Natural Disasters(Washington, DC: World Bank, 2025-05-22)Most disaster risk assessments use damages to physical assets as their central metric, often neglecting distributional impacts and the coping and recovery capacity of affected people. To address this shortcoming, the concepts of well-being losses and socio-economic resilience—the ability to experience asset losses without a decline in well-being—have been proposed. This paper uses microsimulations to produce a global estimate of well-being losses from, and socio-economic resilience to, natural disasters, covering 132 countries. On average, each $1 in disaster-related asset losses results in well-being losses equivalent to a $2 uniform national drop in consumption, with significant variation within and across countries. The poorest income quintile within each country incurs only 9% of national asset losses but accounts for 33% of well-being losses. Compared to high-income countries, low-income countries experience 67% greater well-being losses per dollar of asset losses and require 56% more time to recover. Socio-economic resilience is uncorrelated with exposure or vulnerability to natural hazards. However, a 10 percent increase in GDP per capita is associated with a 0.9 percentage point gain in resilience, but this benefit arises indirectly—such as through higher rate of formal employment, better financial inclusion, and broader social protection coverage—rather than from higher income itself. This paper assess ten policy options and finds that socio-economic and financial interventions (such as insurance and social protection) can effectively complement asset-focused measures (e.g., construction standards) and that interventions targeting low-income populations usually have higher returns in terms of avoided well-being losses per dollar invested.Publication From Patriarchy to Policy(Washington, DC: World Bank, 2025-05-29)Legal institutions play an important role in shaping gender equality in economic domains, from inheritance to labor markets. But where do gender equal laws come from? Using cross-country data on social norms and legal equality, this paper investigates the socio-cultural roots of gender inequity in the legal system and its implications for female labor force participation. To identify the impact of social norms, the analysis uses an empirical strategy that exploits pre-modern differences in ancestral patriarchal culture as an instrument for present-day gender norms. The findings show that ancestral patriarchal culture is a strong predictor of contemporary norms, and conservative social norms are associated with more gender inequality in the de jure legal framework, the de facto implementation of laws, and the labor market. The paper presents evidence for a political selection mechanism linking norms to laws: countries with more conservative norms elect political leaders who are more hostile to gender equality, who then pass less progressive legislation. The results highlight the cultural roots and political drivers of legalized gender inequality.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Intelligence, Personality, and Creativity : Unleashing the Power of Intelligence and Personality Traits to Build a Creative and Innovative Economy(World Bank, Washington, DC, 2014-11-04)Cognitive ability, as measured by IQ and background factors such as socioeconomic status and demographics have historically been seen as the principal determinants of a student s academic success. However, a growing body of research from psychology, education, behavioral economics and neuroscience is showing that personality traits also predict academic and work performance. This change in paradigm suggests that education systems face a more complex challenge than traditionally recognized: to work not only with the different types of intelligence possessed by students but also with their different personality traits in order to produce academic success measured by cognitive and non-cognitive skills. This paper reviews the research findings from the different literatures (psychology, education, behavioral economics, and neuroscience) that relate to these questions. Several good reviews summarize the findings on aspects of these questions, but rarely address all of the questions above. In particular, those reviews do not shed light on how education can improve both cognitive and non-cognitive skills and how such skills promote creativity and labor market outcomes. The scientific literatures on human intelligence and personality are large, but our focus is on the subset of research findings that relate intelligence and personality to academic performance. Likewise, the literature on creativity, innovation, and productivity is extensive, but our focus will be on the research findings that relate academic performance to creativity and, ultimately, to productivity in the workplace.Publication Teaching Mathematics Effectively to Primary Students in Developing Countries(World Bank, Washington, DC, 2008-12-30)This paper uses research from neuroscience and the psychology of mathematics to arrive at useful recommendations for teaching mathematics at primary level to poor students in developing countries. The enrollment rates of the poorer students have improved tremendously in the last decade. And the global Net Enrollment Ratio (NER) has improved since 2001 from 83.2 percent to 90-95 percent except in Sub-Saharan Africa and South Asia. Making teaching of math and other subjects efficient for the poor in developing countries is a great challenge, particularly in south Asia and Sub-Saharan Africa. Many developing countries have explored new means of teaching math and other subjects. Mongolia changed its mathematics education, aiming to build a new set of priorities and practices, given the abandonment of earlier traditions. Similar to international trends of the time, South Africa in the 1990s extensively applied the constructivist learning philosophy which relied on exploration and discovery, with little emphasis on memorization, drill, In conformity with a belief that teachers could develop their own learning programs, there was virtual absence of a national or provincial syllabus or textbooks. Students were expected to develop their own methods for arithmetic operations, but most found it impossible to progress on their own from counting to actual calculating. This study integrates pertinent research from neuroscience and the psychology of mathematics to arrive at recommendations for curricular and efficient means of mathematics instruction particularly for developing countries and poor students at primary level. Specifically, the latest research in neuroscience, cognitive science, and discussions of national benchmarks for primary school mathematics learning, form the basis of our recommendations. These recommendations have a reasonable chance of working in the situational contexts of developing countries, with their traditions and resources.Publication Absenteeism and Beyond : Instructional Time Loss and Consequences(World Bank, Washington, DC, 2007-10)Studies have shown that learning outcomes are related to the amount of time students engage in learning tasks. However, visits to schools have revealed that students are often taught for only a fraction of the intended time, particularly in lower-income countries. Losses are due to informal school closures, teacher absenteeism, delays, early departures, and sub-optimal use of time in the classroom. A study was undertaken to develop an efficient methodology for measuring instructional time loss. Thus, instructional time use was measured in sampled schools in Tunisia, Morocco, Ghana, and the Brazilian state of Pernambuco. The percentage of time that students were engaged in learning vis-à-vis government expectations was approximately 39 percent in Ghana, 63 percent in Pernambuco, 71 percent in Morocco, and 78 percent in Tunisia. Instructional time use is a mediator variable that is challenging to measure, so it often escapes scrutiny. Research suggests that merely financing the ingredients of instruction is not enough to produce learning outcomes; students must also get sufficient time to process the information. The quantity-quality tradeoff that often accompanies large-scale enrollments may be partly due to instructional time restrictions. Time wastage also distorts budgetary outlays and teacher salary rates. To achieve the Millennium Development Goals students must get more of the time that governments, donors, and parents pay for.Publication Learning from Local Practices : Improving Student Performance in West Bank and Gaza(World Bank, Washington, DC, 2014-06-12)The motivation for this study is to contribute to the preparation of the new Palestinian Education Strategy by shedding light on the school and classroom level factors that influence student learning, and to identify good practices that can be generalized from high-performing classrooms to those that need improvement. While most Palestinian children are in school, performance on assessments indicates that many of them are not learning as much as they could. This represents not only inefficiency in the use of public resources, but also a lost opportunity for individual students and the society as a whole. The current study was carried out by the Assessment and Evaluation Department (AED) of the Palestinian Ministry of Education and Higher Education (MoEHE) with technical and financial support from the World Bank. The analysis presented in this paper was prepared by the World Bank team as a complement to a previous paper prepared by a team of experts from AED. For the purposes of this study, schools were classified by student performance in TIMSS 2011 and the 2012 Palestinian national exams. Classroom and school-based tools were then used to gather information from both high and low-performing schools. A total of 122 public, private and United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA) schools were surveyed using four different instruments: (i) stalling's classroom observations; (ii) school leadership survey; (iii) teacher survey; and, (iv) school facilities survey, which are all provided in annex one.Publication Quality of education in Colombia - Achievements and challenges ahead : analysis of the results of TIMSS 1995-2007(World Bank, 2010-08-06)The main objective of this report is to analyze the effect of time changes and factors associated with student achievement in Colombia in order to foster policies to improve education quality that are grounded in research and the Colombian context. In 2007, Colombia participated for the second time in the Trends in International Mathematics and Science Study (TIMSS), providing a first-time opportunity to analyze the effects of time changes on student achievement over a period (1995-2007) during which a number of education reforms were made. Using the TIMSS 2007 results offers a chance to deepen the study on the factors associated with learning in Colombia and to benchmark Colombia's education system against that of other countries. This effort began during the first phase of the Colombia Programmatic Quality and Relevance of Education Analytic and Advisory Activities (AAA), in which an analysis of Colombia's debut in the 2006 Program for International Student Assessment (PISA), resulted in publication of, 'the quality of education in Colombia: an analysis and options for a policy agenda' report (hereafter, 'PISA report'). The present report builds on this work through an analysis of Colombia's participation in TIMSS 2007 in relation to its performance on TIMSS 1995, and reaffirms the urgent need for improved student learning outcomes in Colombia, further confirming a number of the policy options put forward in the PISA report to inform a future agenda for system design and reform.
Users also downloaded
Showing related downloaded files
Publication Impact Evaluation in Practice, Second Edition(Washington, DC: Inter-American Development Bank and World Bank, 2016-09-13)The second edition of the Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. First published in 2011, it has been used widely across the development and academic communities. The book incorporates real-world examples to present practical guidelines for designing and implementing impact evaluations. Readers will gain an understanding of impact evaluations and the best ways to use them to design evidence-based policies and programs. The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation. The handbook is divided into four sections: Part One discusses what to evaluate and why; Part Two presents the main impact evaluation methods; Part Three addresses how to manage impact evaluations; Part Four reviews impact evaluation sampling and data collection. Case studies illustrate different applications of impact evaluations. The book links to complementary instructional material available online, including an applied case as well as questions and answers. The updated second edition will be a valuable resource for the international development community, universities, and policy makers looking to build better evidence around what works in development.Publication World Development Report 2009(World Bank, 2009)Places do well when they promote transformations along the dimensions of economic geography: higher densities as cities grow; shorter distances as workers and businesses migrate closer to density; and fewer divisions as nations lower their economic borders and enter world markets to take advantage of scale and trade in specialized products. World Development Report 2009 concludes that the transformations along these three dimensions density, distance, and division are essential for development and should be encouraged. The conclusion is controversial. Slum-dwellers now number a billion, but the rush to cities continues. A billion people live in lagging areas of developing nations, remote from globalizations many benefits. And poverty and high mortality persist among the world’s bottom billion, trapped without access to global markets, even as others grow more prosperous and live ever longer lives. Concern for these three intersecting billions often comes with the prescription that growth must be spatially balanced. This report has a different message: economic growth will be unbalanced. To try to spread it out is to discourage it to fight prosperity, not poverty. But development can still be inclusive, even for people who start their lives distant from dense economic activity. For growth to be rapid and shared, governments must promote economic integration, the pivotal concept, as this report argues, in the policy debates on urbanization, territorial development, and regional integration. Instead, all three debates overemphasize place-based interventions. Reshaping Economic Geography reframes these debates to include all the instruments of integration spatially blind institutions, spatially connective infrastructure, and spatially targeted interventions. By calibrating the blend of these instruments, today’s developers can reshape their economic geography. If they do this well, their growth will still be unbalanced, but their development will be inclusive.Publication Poverty Reduction in Indonesia : Constructing a New Strategy(Washington, DC, 2001-10-29)The objective of the report is to point at the need for a new poverty strategy, and the areas of action it should cover, where each area should be specifically discussed, addressing the lives of Indonesia's poor, and the tradeoffs policymakers will need to consider, based on the belief that this poverty strategy should emerge from a broad dialogue among stakeholders. First, in broadening poverty, the report looks at the facts of the late 1990s crisis, which revealed the precariousness of Indonesia's gains in reducing expenditure-based poverty. Thus to extend those gains, the poverty strategy needs to be defined, and then redeveloped by acknowledging the multidimensional reality of poverty, and, it is this notion which will lead to making the strategic choices. Second, within the country's political transition to a democratic, decentralized mode of governance, a poverty strategy needs to be consistent with an empowered populace, and democratic policymaking mechanisms. In creating a policy environment for raising the incomes of the poor, the report identifies the resumption of rapid sustainable growth, with rising real wages, employment opportunities, and, limited inflation, including the economic empowerment of the poor, enhanced by poverty-focused public expenditures. Inevitably, the provision of core public services is an area which should address the people's will in local governance policies, focusing on education and health, while providing appropriate infrastructure, and developing safety nets.Publication Boom, Bust and Up Again? Evolution, Drivers and Impact of Commodity Prices: Implications for Indonesia(World Bank, Jakarta, 2010-12)Indonesia is one of the largest commodity exporters in the world, and given its mineral potential and expected commodity price trends, it could and should expand its leading position. Commodities accounted for one fourth of Indonesia's Gross Domestic Product (GDP) and more than one fifth of total government revenue in 2007. The potential for further commodity growth is considerable. Indonesia is the largest producer of palm oil in the world (export earnings totaled almost US$9 billion in 2007 and employment 3.8 million full-time jobs) and the sector has good growth prospects. It is also one of the countries with the largest mining potential in view of its second-largest copper reserves and third-largest coal and nickel reserves in the world. This report consists of seven chapters. The first six chapters present an examination and an analysis of the factors driving increased commodity prices, price forecasts, economic impact of commodity price increases, effective price stabilization policies, and insights from Indonesia's past growth experience. The final chapter draws on the findings of the previous chapters and suggests a development strategy for Indonesia in the context of high commodity prices. This section summarizes the contents of the chapters and their main findings.Publication World Development Report 2004(World Bank, 2003)Too often, services fail poor people in access, in quality, and in affordability. But the fact that there are striking examples where basic services such as water, sanitation, health, education, and electricity do work for poor people means that governments and citizens can do a better job of providing them. Learning from success and understanding the sources of failure, this year’s World Development Report, argues that services can be improved by putting poor people at the center of service provision. How? By enabling the poor to monitor and discipline service providers, by amplifying their voice in policymaking, and by strengthening the incentives for providers to serve the poor. Freedom from illness and freedom from illiteracy are two of the most important ways poor people can escape from poverty. To achieve these goals, economic growth and financial resources are of course necessary, but they are not enough. The World Development Report provides a practical framework for making the services that contribute to human development work for poor people. With this framework, citizens, governments, and donors can take action and accelerate progress toward the common objective of poverty reduction, as specified in the Millennium Development Goals.