Publication: Preparation, Practice, and Beliefs: A Machine Learning Approach to Understanding Teacher Effectiveness
Loading...
Date
2021-11
ISSN
Published
2021-11
Author(s)
Editor(s)
Abstract
This paper uses machine learning methods to identify key predictors of teacher effectiveness, proxied by student learning gains linked to a teacher over an academic year. Conditional inference forests and the least absolute shrinkage and selection operator are applied to matched student-teacher data for math and Kiswahili from grades 2 and 3 in 392 schools across Tanzania. These two machine learning methods produce consistent results and outperform standard ordinary least squares in out-of-sample prediction by 14–24 percent. As in previous research, commonly used teacher covariates like teacher gender, education, experience, and so forth are not good predictors of teacher effectiveness. Instead, teacher practice (what teachers do, measured through classroom observations and student surveys) and teacher beliefs (measured through teacher surveys) emerge as much more important. Overall, teacher covariates are stronger predictors of teacher effectiveness in math than in Kiswahili. Teacher beliefs that they can help disadvantaged and struggling students learn (for math) and they have good relationships within schools (for Kiswahili), teacher practice of providing written feedback and reviewing key concepts at the end of class (for math), and spending extra time with struggling students (for Kiswahili) are highly predictive of teacher effectiveness. As is teacher preparation on how to teach foundational topics (for both Math and Kiswahili). These results demonstrate the need to pay more systematic attention to teacher preparation, practice, and beliefs in teacher research and policy.
Link to Data Set
Citation
“Nahata, Vatsal; Filmer, Deon; Sabarwal, Shwetlena. 2021. Preparation, Practice, and Beliefs: A Machine Learning Approach to Understanding Teacher Effectiveness. Policy Research Working Paper;No. 9847. © World Bank. http://hdl.handle.net/10986/36600 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Geopolitics and the World Trading System(Washington, DC: World Bank, 2024-12-23)Until the beginning of this century, the GATT/WTO system worked. Economic research provided a compelling explanation. It showed that if governments maximize the well-being of their own countries broadly defined, GATT/WTO principles would facilitate mutually beneficial cooperation over their trade policy choices. Now heightened geopolitical rivalry seems to have undermined the WTO. A simple transposition of the previous rationalization suggests that geopolitics and trade cooperation are not compatible. The paper shows that this is only true if rivalry eclipses any consideration of own-country well-being. In all other circumstances, there are gains from trade cooperation even with geopolitics. Furthermore, the WTO’s relevance is in question only if it adheres too rigidly to its existing rules and norms. Through measured adaptation to the geopolitical imperative, the WTO can continue to thrive as a forum for multilateral trade cooperation in the age of geopolitics.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Global Poverty Revisited Using 2021 PPPs and New Data on Consumption(Washington, DC: World Bank, 2025-06-05)Recent improvements in survey methodologies have increased measured consumption in many low- and lower-middle-income countries that now collect a more comprehensive measure of household consumption. Faced with such methodological changes, countries have frequently revised upward their national poverty lines to make them appropriate for the new measures of consumption. This in turn affects the World Bank’s global poverty lines when they are periodically revised. The international poverty line, which is based on the typical poverty line in low-income countries, increases by around 40 percent to $3.00 when the more recent national poverty lines as well as the 2021 purchasing power parities are incorporated. The net impact of the changes in international prices, the poverty line, and new survey data (including new data for India) is an increase in global extreme poverty by some 125 million people in 2022, and a significant shift of poverty away from South Asia and toward Sub-Saharan Africa. The changes at higher poverty lines, which are more relevant to middle-income countries, are mixed.Publication From Patriarchy to Policy(Washington, DC: World Bank, 2025-05-29)Legal institutions play an important role in shaping gender equality in economic domains, from inheritance to labor markets. But where do gender equal laws come from? Using cross-country data on social norms and legal equality, this paper investigates the socio-cultural roots of gender inequity in the legal system and its implications for female labor force participation. To identify the impact of social norms, the analysis uses an empirical strategy that exploits pre-modern differences in ancestral patriarchal culture as an instrument for present-day gender norms. The findings show that ancestral patriarchal culture is a strong predictor of contemporary norms, and conservative social norms are associated with more gender inequality in the de jure legal framework, the de facto implementation of laws, and the labor market. The paper presents evidence for a political selection mechanism linking norms to laws: countries with more conservative norms elect political leaders who are more hostile to gender equality, who then pass less progressive legislation. The results highlight the cultural roots and political drivers of legalized gender inequality.Publication Global Socio-economic Resilience to Natural Disasters(Washington, DC: World Bank, 2025-05-22)Most disaster risk assessments use damages to physical assets as their central metric, often neglecting distributional impacts and the coping and recovery capacity of affected people. To address this shortcoming, the concepts of well-being losses and socio-economic resilience—the ability to experience asset losses without a decline in well-being—have been proposed. This paper uses microsimulations to produce a global estimate of well-being losses from, and socio-economic resilience to, natural disasters, covering 132 countries. On average, each $1 in disaster-related asset losses results in well-being losses equivalent to a $2 uniform national drop in consumption, with significant variation within and across countries. The poorest income quintile within each country incurs only 9% of national asset losses but accounts for 33% of well-being losses. Compared to high-income countries, low-income countries experience 67% greater well-being losses per dollar of asset losses and require 56% more time to recover. Socio-economic resilience is uncorrelated with exposure or vulnerability to natural hazards. However, a 10 percent increase in GDP per capita is associated with a 0.9 percentage point gain in resilience, but this benefit arises indirectly—such as through higher rate of formal employment, better financial inclusion, and broader social protection coverage—rather than from higher income itself. This paper assess ten policy options and finds that socio-economic and financial interventions (such as insurance and social protection) can effectively complement asset-focused measures (e.g., construction standards) and that interventions targeting low-income populations usually have higher returns in terms of avoided well-being losses per dollar invested.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Teacher Performance-Based Incentives and Learning Inequality(World Bank, Washington, DC, 2020-09)This study evaluates the impacts of low-cost, performance-based incentives in Tanzanian secondary schools. Results from a two-phase randomized trial show that incentives for teachers led to modest average improvements in student achievement across different subjects. Further, withdrawing incentives did not lead to a "discouragement effect" (once incentives were withdrawn, student performance did not fall below pre-baseline levels). Rather, impacts on learning were sustained beyond the intervention period. However, these incentives may have exacerbated learning inequality within and across schools. Increases in learning were concentrated among initially better-performing schools and students. At the same time, learning outcomes may have decreased for schools and students that were lower performing at baseline. Finally, the study finds that incentivizing students without simultaneously incentivizing teachers did not produce observable learning gains.Publication Learning-Adjusted Years of Schooling(World Bank, Washington, DC, 2018-09)The standard summary metric of education-based human capital used in macro analyses—the average number of years of schooling in a population—is based only on quantity. But ignoring schooling quality turns out to be a major omission. As recent research shows, students in different countries who have completed the same number of years of school often have vastly different learning outcomes. This paper therefore proposes a new summary measure, Learning-Adjusted Years of Schooling (LAYS), that combines quantity and quality of schooling into a single easy-to-understand metric of progress. The cross-country comparisons produced by this measure are robust to different ways of adjusting for learning (for example, by using different international assessments or different summary learning indicators), and the assumptions and implications of LAYS are consistent with other evidence, including other approaches to quality adjustment. The paper argues that (1) LAYS improves on the standard metric, because it is a better predictor of important outcomes, and it improves incentives for policymakers; and (2) its virtues of simplicity and transparency make it a good candidate summary measure of education.Publication Incentivizing Schooling for Learning : Evidence on the Impact of Alternative Targeting Approaches(World Bank, Washington, DC, 2013-07)This paper evaluates a primary school scholarship program in Cambodia with two different targeting mechanisms, one based on poverty level and the other on baseline test scores ("merit"). Both targeting mechanisms increased enrollment and attendance. However, only the merit-based targeting induced positive effects on test scores. The paper shows that the asymmetry of response is unlikely to have been driven by differences between recipients' characteristics. Higher student and family effort among beneficiaries of the merit-based scholarships suggest that the framing of the scholarship mattered for impact. The results suggest that in order to balance equity and efficiency, a two-step targeting approach might be preferable: first, target low-income individuals, and then, among them, target based on merit.Publication School Enrollment, Selection and Test Scores(2009-07-01)There is a strong association between schooling attained and test scores in many settings. If this association is causal, one might expect that programs that increase school enrollment and attainment would also improve test scores. However, if there is self-selection into school based on expected gains, marginal children brought into school by such programs may be drawn disproportionately from the left-hand side of the ability distribution, which could limit the extent to which additional schooling translates into more learning. To test this proposition, this paper uses data from Cambodia. The results show that a program that provides scholarships to poor students had a large effect on school enrollment and attendance, which increased by approximately 25 percentage points. However, there is no evidence that, 18 months after the scholarships were awarded, recipient children did any better on mathematics and vocabulary tests than they would have in the absence of the program. The paper discusses results that suggest that the self-selection of lower-ability students into school in response to the program is an important part of the explanation. The analysis also shows minimal program effects on other outcomes, including knowledge of health practices, expectations about the future, and adolescent mental health.Publication How to Improve Education Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of Schooling Metric(World Bank, Washington, DC, 2020-10)Many low- and middle-income countries lag far behind high-income countries in educational access and student learning. Limited resources mean that policymakers must make tough choices about which investments to make to improve education. Although hundreds of education interventions have been rigorously evaluated, making comparisons between the results is challenging. Some studies report changes in years of schooling; others report changes in learning. Standard deviations, the metric typically used to report learning gains, measure gains relative to a local distribution of test scores. This metric makes it hard to judge if the gain is worth the cost in absolute terms. This paper proposes using learning-adjusted years of schooling (LAYS) -- which combines access and quality and compares gains to an absolute, cross-country standard -- as a new metric for reporting gains from education interventions. The paper applies LAYS to compare the effectiveness (and cost-effectiveness, where cost is available) of interventions from 150 impact evaluations across 46 countries. The results show that some of the most cost-effective programs deliver the equivalent of three additional years of high-quality schooling (that is, schooling at quality comparable to the highest-performing education systems) for just $100 per child -- compared with zero years for other classes of interventions.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication Digital Progress and Trends Report 2023(Washington, DC: World Bank, 2024-03-05)Digitalization is the transformational opportunity of our time. The digital sector has become a powerhouse of innovation, economic growth, and job creation. Value added in the IT services sector grew at 8 percent annually during 2000–22, nearly twice as fast as the global economy. Employment growth in IT services reached 7 percent annually, six times higher than total employment growth. The diffusion and adoption of digital technologies are just as critical as their invention. Digital uptake has accelerated since the COVID-19 pandemic, with 1.5 billion new internet users added from 2018 to 2022. The share of firms investing in digital solutions around the world has more than doubled from 2020 to 2022. Low-income countries, vulnerable populations, and small firms, however, have been falling behind, while transformative digital innovations such as artificial intelligence (AI) have been accelerating in higher-income countries. Although more than 90 percent of the population in high-income countries was online in 2022, only one in four people in low-income countries used the internet, and the speed of their connection was typically only a small fraction of that in wealthier countries. As businesses in technologically advanced countries integrate generative AI into their products and services, less than half of the businesses in many low- and middle-income countries have an internet connection. The growing digital divide is exacerbating the poverty and productivity gaps between richer and poorer economies. The Digital Progress and Trends Report series will track global digitalization progress and highlight policy trends, debates, and implications for low- and middle-income countries. The series adds to the global efforts to study the progress and trends of digitalization in two main ways: · By compiling, curating, and analyzing data from diverse sources to present a comprehensive picture of digitalization in low- and middle-income countries, including in-depth analyses on understudied topics. · By developing insights on policy opportunities, challenges, and debates and reflecting the perspectives of various stakeholders and the World Bank’s operational experiences. This report, the first in the series, aims to inform evidence-based policy making and motivate action among internal and external audiences and stakeholders. The report will bring global attention to high-performing countries that have valuable experience to share as well as to areas where efforts will need to be redoubled.Publication China Economic Update, June 2024(Washington, DC: World Bank, 2024-06-14)Economic activity picked up in China in early 2024, buoyed by stronger exports. Meanwhile, growth in domestic demand moderated. Manufacturing and infrastructure investment and consumer spending on services remained robust, while the property market correction continued. In the long term, China’s rapidly aging population will have wide-ranging economic impacts, but with the right policies the demographic transition is manageable. The economic challenges from an aging population can be overcome with policies that increase labor force participation and extend productive working lives. Affordable childcare, better work-life balance, elimination of gender bias in hiring, a higher retirement age, skills upgrading, and lifelong learning are measures that could expand China’s workforce and make it more productive.Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.