Publication: Psychometric Considerations for Implementing Phone-Based Learning Assessments
Loading...
Date
2022
ISSN
Published
2022
Author(s)
Editor(s)
Abstract
The Coronavirus disease 2019 (COVID-19) pandemic has had a significant impact on education worldwide, affecting whether and how students learn. World Bank estimates and initial analysis at the country level indicate that school closures and interruptions to in-person education are likely to negatively impact student learning. Moreover, these estimates predict that students in low- and middle-income countries will suffer from more significant learning losses than students in high-income countries, worsening the existing global learning crisis. As part of their education response to the pandemic, many countries have provided learning content to students at home through various means, including through the internet via devices such as computers, tablets, and smartphones, as well as through television, radio, printed materials, and feature (basic) phones. With schools closed, teachers have also had to adapt to using these different media to continue their teaching. So, phone-based assessment solutions can be used for at least two purposes. The first purpose is to conduct formative assessment, for example to gauge how well students have absorbed the learning content, identify any misconceptions in understanding, provide constructive feedback to students or caregivers, and offer additional learning resources and activities to support learning. The second purpose is to conduct impact evaluations, for example analyzing the effect on student learning outcomes of interventions introduced in response to the pandemic.
Link to Data Set
Citation
“Liberman, Julia; Luna-Bazaldua, Diego; Levin, Victoria; Khurana, Aishwarya. 2022. Psychometric Considerations for Implementing Phone-Based Learning Assessments. © World Bank. http://hdl.handle.net/10986/37533 License: CC BY 3.0 IGO.”
Associated URLs
Associated content
Other publications in this report series
Publication Environmental Implications of a Central Bank Digital Currency (CBDC)(Washington, DC : World Bank, 2022-07)Two-thirds of central banks in the East Asia and Pacific (EAP) region have started researching or testing the implementation of a Central Bank Digital Currency (CBDC). At the same time, the region accounts for one-third of world CO2 emissions and is vulnerable to climate risks. As the Group of 7 (G7), European Central Bank (ECB), and Bank of England (BoE) have stated in their public statements, it is increasingly important to consider environmental impact when designing CBDC. However, only a few brief studies have been done on this subject, which will be crucial for the region. This Note explores the environmental implications of CBDC by comparing technical mechanisms and energy consumption within its distributed structure. It also illustrates differences in ecological footprint between CBDC and other payment methods (cryptocurrency, cash, and card networks). As the legitimacy of CBDC is backed by the trust of central banks, CBDC does not need to prove its legitimacy through its technological structure. Therefore, CBDC does not require the energy-intensive consensus or mining mechanisms used by a cryptocurrency, so its energy consumption is lower (comparable to that of a credit card system). CBDC can be designed to use various systems, such as Real Time Gross Settlement (RTGS), Distributed Ledger Technology (DLT), or a mixture of both. Careful deliberation to meet the objectives and implications will be important as CBDC can be a catalyst for financial innovation.Publication Assessing Incentives to Increase Digital Payment Acceptance and Usage(World Bank, Washington, DC, 2022-01-18)An important step to achieve greater financial inclusion is to increase the acceptance and usage of digital payments. Although consumer adoption of digital payments has improved dramatically globally, the acceptance and usage of digital payments for micro, small, and medium-sized retailers (MSMRs) remain challenging. Using random forest estimation, The authors identify 14 key predictors out of 190 variables with the largest predictive power for MSMR adoption and usage of digital payments. Using conditional inference trees, they study the importance of sequencing and interactions of various factors such as public policy initiatives, technological advancements, and private sector incentives. The authors find that in countries with low point of sale (POS) terminal adoption, killer applications such as mobile phone payment apps increase the likelihood of P2B digital transactions. They also find the likelihood of digital P2B payments at MSMRs increases when MSMRs pay their employees and suppliers digitally. The level of ownership of basic financial accounts by consumers and the size of the shadow economy are also important predictors of greater adoption and usage of digital payments. Using causal forest estimation, they find a positive and economically significant marginal effect for merchant and consumer fiscal incentives on POS terminal adoption on average. When countries implement financial inclusion initiatives, POS terminal adoption increases significantly and MSMRs’ share of person-to-business (P2B) digital payments also increases. Merchant and consumer fiscal incentives also increase MSMRs’ share of P2B electronic payments.Publication A Novel Tobacco Market Diversification(World Bank, Washington, DC, 2022-04-27)In this working paper, an exploration of available data and information is conducted and findings presented, to support the view that the dichotomous business model and related harm reduction narrative promoted nowadays by the tobacco industry, merits scrutiny by the international community. The promotion of e-cigarettes as welfare enhancing in rich countries, particularly because they are posited to help adult smokers quit, tends to obfuscate a dire reality. The same tobacco industry that promotes (e-cigarettes as harm reduction in rich countries, derives the bulk of its profits by selling cigarettes in lower income countries.Publication Structured Lesson Plans for Literacy Instruction(World Bank, Washington, DC, 2022-03-31)Literacy is the cornerstone of education, and a driver of human economic, social, and civic wellbeing. Despite its importance, far too many children fail to become literate. The World Bank uses a measure called learning poverty to indicate when a child cannot read and understand an age-appropriate text by age ten. The best available data showed that more than two-thirds of children in low- and middle-income countries suffer learning poverty. The World Bank is committed to helping countries achieve the learning target: to cut learning poverty by at least half by 2030. Achieving better outcomes in literacy requires a comprehensive effort in many domains. One of the most important is ensuring that students and teachers have and use high-quality instructional materials, especially textbooks, for reading instruction. As countries and systems review their literacy teaching and learning materials, they will want to compare them to the materials from other countries and systems. The purpose of the compendium is to allow such reviews and comparisons by grouping a critical mass of structured pedagogy lesson plans and related materials in one place.Publication WBG COVID-19 Crisis Response Operational Update(Washington, DC, 2022-03-31)This note provides an update on the WBG’s COVID-19 Crisis Response, outlined in June 2020 to help developing countries address the impacts of the pandemic while maintaining a line of sight to long-term development goals. It comprises five short sections: (I) the impacts of COVID-19 and compounding crises on developing countries, (II) an update on the WBG’s operational crisis response and priorities moving forward, (III) the critical role of international coordination, (IV) WBG financing framework for GRID, and (V) concluding remarks.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication A Synthesis Report on Piloting of Remote Phone-Based Formative Assessment Solutions in Ghana, Nepal, and Pakistan(World Bank, Washington, DC, 2022)School closures related to Coronavirus disease 2019 (COVID-19) not only disrupted education but also impacted teachers’ ability to know whether and what their students were learning. This information gap was most challenging in contexts with limited internet connectivity and low access to smart digital devices. In such circumstances, conducting formative assessments remotely using basic phones was seen as a means to provide timely information to teachers, parents, and students and support learning continuity outside the classroom. As students return to school, the same solutions can be used to complement in-person instruction to accelerate learning recovery, expand the use of formative assessment in hard-to-reach schools, and improve the resilience of education systems when confronted by future shocks. This report describes the three pilot studies that aimed to test the logistical and technological feasibility of using short message service (SMS), interactive voice recording (IVR), and live phone calls (LPC) to conduct formative assessment of foundational math and literacy skills for primary-grade students. These pilot studies were conducted in Ghana, Nepal, and Pakistan between October 2020 and January 2022.Publication Guidance Note on Phone-based Formative Assessment(World Bank, Washington, DC, 2022)Phone-Based Formative Assessment refers to the use of mobile phone technology to engage students outside the classroom, assess their learning in real time, and provide timely, constructive feedback. Such assessment is particularly critical to promote learning continuity in the context of COVID-19-related school closures, which have led to the expansion and availability of remote teaching and learning resources.Publication Classroom Assessment to Support Foundational Literacy(Washington, DC: World Bank, 2025-03-21)This document focuses primarily on how classroom assessment activities can measure students’ literacy skills as they progress along a learning trajectory towards reading fluently and with comprehension by the end of primary school grades. The document addresses considerations regarding the design and implementation of early grade reading classroom assessment, provides examples of assessment activities from a variety of countries and contexts, and discusses the importance of incorporating classroom assessment practices into teacher training and professional development opportunities for teachers. The structure of the document is as follows. The first section presents definitions and addresses basic questions on classroom assessment. Section 2 covers the intersection between assessment and early grade reading by discussing how learning assessment can measure early grade reading skills following the reading learning trajectory. Section 3 compares some of the most common early grade literacy assessment tools with respect to the early grade reading skills and developmental phases. Section 4 of the document addresses teacher training considerations in developing, scoring, and using early grade reading assessment. Additional issues in assessing reading skills in the classroom and using assessment results to improve teaching and learning are reviewed in section 5. Throughout the document, country cases are presented to demonstrate how assessment activities can be implemented in the classroom in different contexts.Publication Guidance Note on Using Learning Assessment in the Process of School Reopening(World Bank, Washington, DC, 2020-11-21)As countries consider how to reopen schools safely in the context of COVID-19 (coronavirus), one key question is how to assess students' learning to support learning recovery. The expected magnitude of learning losses, particularly among students with the highest needs, makes it essential for key stakeholders in the education process — policymakers, teachers, school principals, students, and their parents — to determine where students are in their learning trajectory relative to what had been expected prior to the pandemic, so they can adjust instruction and allocate resources accordingly. To collect this information, stakeholders can rely on student learning assessment, which is an essential feedback mechanism in the education system. This note provides key steps that countries with different availability of resources should consider in developing their plans for learning assessment activities to support learning recovery in the context of school reopening. Throughout this note, assessment of student learning is defined as gathering and evaluating information on what students know, understand, and can do to make informed decisions about the next steps in the educational process. In addition, some considerations and country examples for the implementation of high-stakes examinations are discussed. This note concludes with examples of learning assessment activities that countries around the world are planning or implementing during the COVID-19 pandemic. Likewise, this note highlights important lessons that can support resilience to future emergencies and crises.Publication Primer on Large-Scale Assessments of Educational Achievement(Washington, DC: World Bank, 2021-04-26)To improve their education systems, countries around the world have increasingly initiated national largescale assessment programs or participated in international or regional large-scale assessment studies for the first time. Well-constructed large-scale assessments can provide credible information on student achievement levels, which, in turn, can promote better resource allocation to schools, stronger education service delivery, and improved learning outcomes. The World Bank developed this Primer on Large-Scale Assessments of Educational Achievement as a firststop resource for those wanting to understand how to design, administer, analyze, and use the results from these assessments of student achievement. The book addresses frequently asked questions from people working on large-scale assessment projects and those interested in making informed decisions about them. Each chapter introduces a stage in the assessment process and offers advice, guidelines, and country examples. This book also reports on emerging trends in large-scale assessment and provides updated information on regional and international large-scale assessment programs.
Users also downloaded
Showing related downloaded files
Publication Applying the Degree of Urbanisation(European Union/FAO/UN-Habitat/OECD/The World Bank, 2025-01-31)This manual develops a harmonised methodology to facilitate international statistical comparisons and to classify the entire territory of a country along an urban-rural continuum. The degree of urbanisation classification defines cities, towns and semi-dense areas, and rural areas. This first level of the classification may be complemented by a range of more detailed concepts, such as: metropolitan areas, commuting zones, dense towns, semi-dense towns, suburban or peri-urban areas, villages, dispersed rural areas and mostly uninhabited areas. The manual is intended to complement and not replace the definitions used by national statistical offices (NSOs) and ministries. It has been designed principally as a guide for data producers, suppliers and statisticians so that they have the necessary information to implement the methodology and ensure coherency within their data collections. It may also be of interest to users of subnational statistics so they may better understand, interpret and use official subnational statistics for taking informed decisions and policymaking. This report has been produced in close collaboration by six organisations, the European Commission, the Food and Agriculture Organization of the United Nations (FAO), the United Nations Human Settlements Programme (UNHabitat), the International Labour Organization (ILO), the Organisation for Economic Co-operation and Development (OECD) and The World Bank.Publication Making Procurement Work Better – An Evaluation of the World Bank’s Procurement System(Washington, DC: World Bank, 2024-12-06)This evaluation assesses the results, successes, and challenges of the World Bank 2016 procurement reform. Procurements acquire the works, goods, and services necessary to achieve the World Bank’s project development outcomes. The World Bank’s procurement processes must ensure that clients get the best value for every development dollar. In 2016, the World Bank reformed its procurement system for Investment Project Financing and launched a new procurement framework aimed at enhancing the Bank’s development effectiveness through better procurement. The reform sought to reduce procurement bottlenecks impeding project performance and modernize procurement systems. It emphasized cutting edge international good practice principles and was intended to be accompanied by procurement capacity strengthening to help client countries. This evaluation offers three recommendations to scale up reform implementation and enhance portfolio and project performance: (i) Improve change management support for the reform’s implementation. (ii) Strategically strengthen country-level procurement capacity. (iii) Consistently manage the full spectrum of procurement risks to maximize project success.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.Publication Inequalities in Sub-Saharan Africa(Washington, DC: World Bank, 2025-01-17)The growing disparity between the rich and poor remains a critical challenge, affecting countries across all continents, irrespective of their per capita gross domestic product. This widening gap not only impedes efforts to eradicate extreme poverty but also hinders progress toward social justice and resilience-building. Rising inequalities pose substantial barriers to sustainable development, and it is within this context that this book, 'Inequalities in Sub-Saharan Africa: Multidimensional Perspectives and Future Challenges', contributes to ongoing debates, offering a comprehensive analysis of the current challenges and future perspectives of inequality on the African continent. Despite the intensification of calls for wealth taxation and inequality reduction, progress has been slow. A key challenge lies in creating a viable political path for implementing progressive taxation policies. Resistance from those benefiting from the current system often stalls efforts, making progress difficult. Moreover, reducing inequality requires mechanisms that address inequality at its roots. Policies targeting education, competition, financial market regulation, and industrial development all hold the potential to create equitable economic opportunities, ensuring access to credit, job creation, and more-balanced economic growth. Despite facing unique, profound challenges, Africa is often overlooked in these global discussions. This book seeks to place the continent’s issues of income inequality, unequal access to education and health care, climate vulnerability, and inclusive growth at the center of the conversation. The book further advocates for innovative policies, including competition reforms and bargaining frameworks that shift the balance between capital and labor. Given that inequality in Africa is deeply rooted in historical, economic, and institutional factors, a stronger focus on pre-distribution policies is necessary. These systemic changes can help reshape the conditions under which inequality emerges and persists. In addition to policy reforms, it is vital to strengthen the research and academic infrastructure that underpins the understanding of inequality. Equity concerns must be addressed within the scientific field, and African research capabilities must be bolstered. This volume, written in collaboration with the African Center of Excellence for Inequality Research, calls for a greater focus on empowering African researchers as part of a broader development strategy. By doing so, it aligns with the World Bank’s and the Agence Française de Développement’s commitment to supporting research as a critical tool for sustainable development.Publication Services Unbound(Washington, DC: World Bank, 2024-12-09)Services are a new force for innovation, trade, and growth in East Asia and Pacific. The dramatic diffusion of digital technologies and partial policy reforms in services--from finance, communication, and transport to retail, health, and education--is transforming these economies. The result is higher productivity and changing jobs in the services sector, as well as in the manufacturing sectors that use these services. A region that has thrived through openness to trade and investment in manufacturing still maintains innovation-inhibiting barriers to entry and competition in key services sectors. 'Services Unbound: Digital Technologies and Policy Reform in East Asia and Pacific' makes the case for deeper domestic reforms and greater international cooperation to unleash a virtuous cycle of increased economic opportunity and enhanced human capacity that would power development in the region.