Publication: How Much Teachers Know and How Much It Matters in Class: Analyzing Three Rounds of Subject-Specific Test Score Data of Indonesian Students and Teachers
Loading...
Published
2016-02
ISSN
Date
2016-02-03
Author(s)
Editor(s)
Abstract
Improving the quality of education is one of today's main challenges for governments in the developing world. Based on a unique matched student-to-teacher panel data set on test scores this paper presents two empirical results for Indonesia. First, through detailed inspection of teacher-level responses to test questions, the paper concludes that subject matter knowledge of primary school teachers in Indonesia is low on average and that a 1.0, but also a 2.0 standard deviation increase in teachers' subject matter knowledge seem to be achievable medium-term goals for education policy making in Indonesia. Second, the paper presents the results of three types of value-added regressions, a (standard) level specification, a school fixed-effects specification, and a flexible student-teacher fixed-effects specification. The student-teacher fixed-effects approach estimates the parameters of a value-added model using test score variation within each student-teacher pair across three different subjects, mathematics, science and Indonesian language. The results suggest that a 1.0 (and 2.0) standard deviation increase in teachers' subject matter knowledge across-the-board can yield increases in student achievement by 0.25 (and 0.50) student-level standard deviations by the time students complete the six-year primary school cycle.
Link to Data Set
Citation
“de Ree, Joppe. 2016. How Much Teachers Know and How Much It Matters in Class: Analyzing Three Rounds of Subject-Specific Test Score Data of Indonesian Students and Teachers. Policy Research Working Paper;No. 7556. © World Bank. http://hdl.handle.net/10986/23731 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Strengthening Mathematics Education in Sri Lanka(Washington, DC, 2011-07)A well-informed and knowledgeable community is of vital importance for the economic and social development of a modem society. The knowledge and skills required for present day activities are much more complex than those required in the past. Today, many jobs require expert thinking and non-routine analytical skills, to identify and solve problems. Mathematics education focuses on developing a person's analytical and problem solving abilities. Thus a high quality mathematics education win ensure that students develop the skills that are essential not only in science and technology, but also in everyday life and the workplace. The government of Sri Lanka, recognizing the need for a high quality mathematics education has implemented several reforms in the recent past. Although the progress achieved is commendable, there is still room for improvement in certain important aspects of the learning teaching process. The main aim of the mathematics curriculum under these reforms is to create individuals who are able to think mathematically and apply mathematical knowledge effectively and responsibly in problem solving and decision making.Publication Inside Indonesia's Mathematics Classrooms : A TIMSS Video Study of Teaching Practices and Student Achievement(World Bank, 2010-10-01)The following report is the first of a two-stage video study to examine teaching practices and activities in Indonesian classrooms. The first stage is linked to results of the 2007 Trends in International Mathematics and Science Study (TIMSS) where 100 of the 150 classes that participated in the TIMSS examination also participated in this additional video study component. The second stage will also involve 100 classes that will participate in the 2011 exam. The second phase will follow the same methodology, allowing for comparison across years, but will also involve more in-depth analysis of the links between teaching practices and student outcomes and how teaching practices are influenced by teachers' belief systems and subject content knowledge. Indonesia has been a committed participant in the TIMSS, Program for International Student Assessment (PISA) and the Progress in International Reading Literacy Study (PIRLS) international standardized student examinations for many years and is one of the few non-OECD (Organization for Economic Cooperation and Development) countries to participate so fully. Indonesian student performance in these examinations has been relatively low, even when taking socio-economic levels into account. For example, for mathematics Indonesia ranked 36th out of 48 participating countries in TIMSS 2007, and its score of 397 was more than one standard deviation below the international average (Mullis et al, 2008). The results have been useful in providing an indication of Indonesia's relative standing in student achievement and its progress over time, but the real challenge is to take the next step and translate the results into an understanding of the factors leading to the test scores and what might be done to enhance student achievement in Indonesia.Publication School Choice and Cognitive Achievement in Rural Bangladesh(World Bank, Washington, DC, 2006-06)This paper presents new evidence on the impact of school characteristics on secondary student achievement using a rich data set from rural Bangladesh. The authors deal with a potentially important selectivity issue in the South Asian context: the non-random sorting of children into religious schools. The authors do so by employing a combination of fixed effects and instrumental variable estimation techniques. Additionally, the authors use the variation between two classrooms of the same grade within individual schools to identify causal class-size effects. The empirical results do not reveal any difference in test scores between religious and secular schools when selection into religious school is taken into account. Net of school fixed-effects, the authors do not find evidence in support of smaller class size. However, they document significant learning deficit by gender and primary school type: girls and graduates of primary madrasas have a lower test score even after controlling for school and classroom-specific unobservable correlates of learning.Publication Quality of education in Colombia - Achievements and challenges ahead : analysis of the results of TIMSS 1995-2007(World Bank, 2010-08-06)The main objective of this report is to analyze the effect of time changes and factors associated with student achievement in Colombia in order to foster policies to improve education quality that are grounded in research and the Colombian context. In 2007, Colombia participated for the second time in the Trends in International Mathematics and Science Study (TIMSS), providing a first-time opportunity to analyze the effects of time changes on student achievement over a period (1995-2007) during which a number of education reforms were made. Using the TIMSS 2007 results offers a chance to deepen the study on the factors associated with learning in Colombia and to benchmark Colombia's education system against that of other countries. This effort began during the first phase of the Colombia Programmatic Quality and Relevance of Education Analytic and Advisory Activities (AAA), in which an analysis of Colombia's debut in the 2006 Program for International Student Assessment (PISA), resulted in publication of, 'the quality of education in Colombia: an analysis and options for a policy agenda' report (hereafter, 'PISA report'). The present report builds on this work through an analysis of Colombia's participation in TIMSS 2007 in relation to its performance on TIMSS 1995, and reaffirms the urgent need for improved student learning outcomes in Colombia, further confirming a number of the policy options put forward in the PISA report to inform a future agenda for system design and reform.Publication Findings from the Bhutan Learning Quality Survey(Washington, DC, 2009-01)The education sector in Bhutan has been growing steadily since the 1960s and concomitantly the literacy rates of the population have also been steadily going up over time. The mostly mountainous country regards education as central to its national development. Every cohort has seen an increasing share of children going to school and the education system now strains to keep up with the speed with which enrolment has expanded over the last ten years, in-line with Bhutan's commitment to meet the education Millennium Development Goals (MDGs). This report is structured as follows: section two presents the background and context of Bhutan's education system; section three discusses previous theoretical and empirical literature on education quality; section four describes the sampling design methodology, the sample and empirical methodology used in this study; section five presents findings on students' actual knowledge in three subjects and their corresponding scaled scores; section six presents the results of multivariate regression analysis for estimating school, teacher and child related correlates of learning outcomes; section seven profiles teachers in grades two and four and the education process; and section eight concludes with brief summary, discussion of policy implications, and recommendation for future research.
Users also downloaded
Showing related downloaded files
Publication The Economics of Teacher Supply in Indonesia(2009-06-01)This paper examines the phenomenon of the over-supply of teachers but shortage of qualified teachers in Indonesia. Using a theoretical framework of government-dominated market with government-set wage rate and demand for teachers, the analysis explores how teacher supply, particularly the composition of the teaching force with low or high qualification, would be determined by current and future public policies. Using 2001 to 2008 Indonesian Labor Force Survey data, the paper further estimates the potential effect of the most recent teacher law, which could give college educated teachers a significant pay increase, on the composition of the Indonesian teaching force with differentiated education backgrounds. Using a sample of workers with college education, the author finds that the relative wage rate of teachers and that of alternative occupations significantly influence the decision of college educated workers to become teachers. It is also found that the wage rate set by the most recent teacher law would increase the share of teachers approximately from 16 to 30 percent of the college-educated labor force. This increase that is due to the new government-set wage rate, would result in a pupil-teacher ratio of 24 to 25 pupils per teacher with college education, but will require a more than 31 percent increase in the wage bill for teacher salaries. The empirical approach of this paper is derived from a structural model that takes into account the endogeneity of the wage rate and corrects for sample-selection bias due to occupational choice.Publication Education Quality and Opportunities for Skills Development in Albania : An Analysis of PISA 2000-2012 Results(Washington, DC, 2012)Albanian students improved their performance on PIS A1 reading, math, and science assessments between 2000 and 2012, a period over which access to upper secondary education expanded dramatically. The improvements are equivalent to a quarter to half a year of schooling in math and science and more than one year of schooling in reading. The improvements were realized despite substantial increases in enrollment and the attendant changes in the PISA sample composition. The gross enrollment rate in upper secondary education was only about 40 percent in 2000, but increased markedly to 72 percent in 2009 and 83 percent in 2012. A spike in enrollment usually results in lower average student performance due to the inclusion of poorer and traditionally excluded students. In the context of a rapidly expanding education system, it is remarkable that Albania managed to improve its average PISA scores. It is even more notable that the improvement came from significant advancements made by low-achieving students. Albania s improved PIS A scores coincide with the launch of intensive reform efforts in its education sector. The Government of Albania initiated the education sector strategy formulation process in 2002, which resulted in the 2004 passage of the National Education Strategy (NES) and the first attempt to develop a long-term roadmap for the sector. The NES served as the catalyst for a range of reforms, including: improved teacher recruitment, compensation, and management; a revised curriculum for basic and general upper secondary education; enhanced transparency and accountability through reform of the Matura, the national student assessment; reduced price and improved textbook quality through a reformed procurement process; and provision of textbook subsidies to the poorest households. While it is beyond the scope of this report to determine a causal relationship between the range of policy reforms and improved PISA scores, this report shows that socioeconomic variables played an important role. Explanatory analyses show that the effect of education variables cannot be disentangled from the effect of socioeconomic variables. Furthermore, with economic growth averaging 4.9 percent per year from 2000-2012, a viable explanation is that the general improvement in economic conditions helped to raise the tide on all social indicators, including PISA performance.Publication Who Learns What in Basic Education? Evidence from Indonesia(World Bank, Washington, DC, 2018-04-30)Indonesia consistently fares poorly on the international tests like PISA and TIMSS. To help diagnosing the causes of poor learning we track student achievement across 9 years in basic education. We find that 40 percent of students do not learn the basics in the early grades of primary school (recognizing 2-digit numbers by 2nd grade, and ordering 4-digit numbers by 4th grade). The authors also find that schools do not cover the complete Indonesian curriculum. Only few students learn how to calculate the surface area of a triangle by 5th grade, the rules about the order of operations, and to complete exercises embedded in stories. Poor and incomplete coverage of the primary curriculum helps explain the low levels of student achievement authors observed in secondary school. Our analysis also provides directions for future research. We observed a large catching-up effect in learning in 6th grade. A plausible explanation for this pattern is the increased pressure on schools, teachers and students to perform well on the high-stakes national exams. The fact that the system can produce learning once (all) actors are sufficiently motivated, suggests that 1.) performance pressure might help, and 2.) that low levels of teacher’s knowledge and skills are currently not a major binding constraint to learning in Indonesia.Publication Using the Results of a National Assessment of Educational Achievement(World Bank, 2009)This book, the fifth and final volume of the National Assessments of Educational Achievement series, draws on the experiences of over forty countries that have implemented a national assessment. It considers the role of contextual factors which impinge on the use of assessment findings. The attachment of sanctions to performance on a national assessment and use of assessment results for accountability are discussed. Key components of a national assessment report are specified . Other instruments to communicate findings are described. Uses of national assessment findings for policy, management, teaching, and raising public awareness are described. A number of ways in which the use and value of national assessments could be optimized are proposed. This volume is intended primarily for teams who are responsible for conducting national assessments and policy makers responsible for the dissemination and use of national assessment results.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.