Publication: A Framework for the Economic Evaluation of Digital Health Interventions
Loading...
Published
2023-04-19
ISSN
Date
2023-04-19
Author(s)
Editor(s)
Abstract
Numerous digital health interventions have been piloted in response to the health care challenges low- and middle-income countries face. Because the opportunity cost of investing in digital health interventions can be large, countries must make choices about which interventions to scale up. To make good investment decisions about digital health interventions, there is a need to define and establish their value, and to use economic evaluation to make informed decisions, however DHIs present methodological challenges for economic evaluation. To address these challenges, this paper first creates a ‘gap map’ of digital evidence which reveals a dearth of economic evaluation evidence about digital health interventions; this lack can limit decisions on policy, programming, and appropriate scale-up of digital health interventions. To advance work in this field, this paper then develops an economic evaluation framework that can be used when determining the economic value of digital health interventions. Such a standardized approach, alongside guidance to assist the conduct and use of economic evidence, can improve decision making and investments in DHI under constrained health budgets. The resulting digital health intervention economic evaluation framework consists of 5 steps: (1) determine the context, (2) determine the intervention type, (3) establish the level of complexity, (4) set the analytic principles, and (5) represent the value proposition. Users of the framework should attempt to adhere to its steps and principles, but where this is not feasible or appropriate, they should provide justification for the methodological choice. The framework should facilitate methodological transparency, thereby improving the overall usefulness of economic evaluations of digital health interventions.
Link to Data Set
Citation
“Wilkinson, Thomas; Wang, Mengxiao; Friedman, Jed; Prestidge, Marelize. 2023. A Framework for the Economic Evaluation of Digital Health Interventions. Policy Research Working Papers; 10407. © World Bank. http://hdl.handle.net/10986/39713 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Integration of Health Systems and Priority Health, Nutrition, and Population Interventions : A Framework for Analysis and Policy Choices(World Bank, Washington, DC, 2008-08)A longstanding debate on health system organization relates to the benefits of integrating programs that emphasize specific interventions into mainstream health systems to increase access and improve health outcomes. This debate has long been characterized by polarization of views and ideologies, with protagonists for and against integration arguing relative merits of each approach. Recently, the debate has been rekindled due to substantial rises in externally funded programs for priority health, nutrition, and population (HNP) interventions and an increase in international efforts aimed at health system strengthening. In this paper the author present a conceptual framework and an analytical approach to define and assess the nature and extent of integration in relation to critical health system functions. The framework can also be used to systematically compare and contrast programs in different settings to generate meaningful evidence to inform policy. In this framework, the author defines integration as the extent, pattern, and rate of adoption and eventual assimilation of priority health interventions into each of the critical functions of a health system, which include, inter alia: (i) governance, (ii) financing, (iii) planning, (iv) service delivery, (v) monitoring and evaluation, and (vi) demand generation. The framework and the analytical approach are intended for evaluative and formative studies of policies, systems, and programs.Publication Costa Rica Case Study : Primary Health Care Achievements and Challenges within the Framework of the Social Health Insurance(World Bank, Washington, DC, 2013-01)The objective of this paper is to assess the key interventions Costa Rica has developed to expand health coverage for the poor and other vulnerable groups, with an emphasis on its approach to primary health care. Universal health coverage in Costa Rica is provided through a single national health insurance program. This program, which protects the poor without the pitfalls of a fragmented system, and the sustained policies that have enabled the building of a solid primary health care system, is broadly recognized as a success story. At the same, time new challenges are emerging to sustaining the success of Costa Rica's universal health coverage. Social Security of Costa Rica (Caja Costarricense de Seguridad Social, CCSS) faces increased production costs and demographic and epidemiological changes in a rapidly aging population. This report is divided into three broad sections: (1) objective of the case study and health system overview; (2) primary health care and the organization of health services within CCSS; and (3) agenda of key policy decisions for a renewed primary health care approach as part of a more responsive and sustainable health insurance system.Publication Impact Evaluation of Trade Interventions : Paving the Way(2011-11-01)The focus of trade policy has shifted in recent years from economy-wide reductions in tariffs and trade restrictions toward targeted interventions to facilitate trade and promote exports. Most of these latter interventions are based on the new mantra of "aid-for-trade" rather than on hard evidence on what works and what does not. On the one hand, rigorous impact-evaluation is needed to justify these interventions and to improve their design. On the other hand, rigorous evaluation is feasible because unlike traditional trade policy, these interventions tend to be targeted and so it is possible to construct treatment and control groups. When interventions are not targeted, such as in the case of customs reforms, some techniques, such as randomized control trials, may not be feasible but meaningful evaluation may still be possible. Theis paper discusses examples of impact evaluations using a range of methods (experimental and non-experimental), highlighting the particular issues and caveats arising in a trade context, and the valuable lessons that are already being learned. The authors argue that systematically building impact evaluation into trade projects could lead to better policy design and a more credible case for "aid-for-trade."Publication Health in All Policies as a Strategic Policy Response to NCDs(World Bank, Washington, DC, 2014-02)The aim of Health in All Policies (HiAP) is to bring diverse sectors together to find shared solutions; it focuses on identifying 'win-win' or 'cobenefits' such that policy measures contribute not only to improved health outcomes but also to outcomes desired by other sectors, such as education, environment, welfare, agriculture, and transportation. HiAP can be applied to NCD prevention and control from three different entry points: risk factors or conditions, population groups (including life course), and sectors. (HiAP) is a relatively new concept and policy practice that attempts to incorporate consideration into the policy decision-making process of how public policies and programs affect community health and well-being. It represents a way of working across sectors that aim to find solutions for complex, interrelated, and persistent problems. With the global epidemic in non-communicable diseases (NCDs), HiAP offers a potential approach and a pathway to secure coordinated action on social determinants of health that relate to NCDs and result in health inequalities. Promising examples can be seen globally for action on both specific NCD risk factors as well as in a more systemic approach to policy decision making.Publication Using Allocative Efficiency Analysis to Inform Health Benefits Package Design for Progressing towards Universal Health Coverage(PLoS, 2021-11-29)Countries are increasingly defining health benefits packages (HBPs) as a way of progressing towards Universal Health Coverage (UHC). Resources for health are commonly constrained, so it is imperative to allocate funds as efficiently as possible. We conducted allocative efficiency analyses using the Health Interventions Prioritization tool (HIPtool) to estimate the cost and impact of potential HBPs in three countries. These analyses explore the usefulness of allocative efficiency analysis and HIPtool in particular, in contributing to priority setting discussions.
Users also downloaded
Showing related downloaded files
Publication Global Economic Prospects, June 2025(Washington, DC: World Bank, 2025-06-10)The global economy is facing another substantial headwind, emanating largely from an increase in trade tensions and heightened global policy uncertainty. For emerging market and developing economies (EMDEs), the ability to boost job creation and reduce extreme poverty has declined. Key downside risks include a further escalation of trade barriers and continued policy uncertainty. These challenges are exacerbated by subdued foreign direct investment into EMDEs. Global cooperation is needed to restore a more stable international trade environment and scale up support for vulnerable countries grappling with conflict, debt burdens, and climate change. Domestic policy action is also critical to contain inflation risks and strengthen fiscal resilience. To accelerate job creation and long-term growth, structural reforms must focus on raising institutional quality, attracting private investment, and strengthening human capital and labor markets. Countries in fragile and conflict situations face daunting development challenges that will require tailored domestic policy reforms and well-coordinated multilateral support.Publication The Container Port Performance Index 2023(Washington, DC: World Bank, 2024-07-18)The Container Port Performance Index (CPPI) measures the time container ships spend in port, making it an important point of reference for stakeholders in the global economy. These stakeholders include port authorities and operators, national governments, supranational organizations, development agencies, and other public and private players in trade and logistics. The index highlights where vessel time in container ports could be improved. Streamlining these processes would benefit all parties involved, including shipping lines, national governments, and consumers. This fourth edition of the CPPI relies on data from 405 container ports with at least 24 container ship port calls in the calendar year 2023. As in earlier editions of the CPPI, the ranking employs two different methodological approaches: an administrative (technical) approach and a statistical approach (using matrix factorization). Combining these two approaches ensures that the overall ranking of container ports reflects actual port performance as closely as possible while also being statistically robust. The CPPI methodology assesses the sequential steps of a container ship port call. ‘Total port hours’ refers to the total time elapsed from the moment a ship arrives at the port until the vessel leaves the berth after completing its cargo operations. The CPPI uses time as an indicator because time is very important to shipping lines, ports, and the entire logistics chain. However, time, as captured by the CPPI, is not the only way to measure port efficiency, so it does not tell the entire story of a port’s performance. Factors that can influence the time vessels spend in ports can be location-specific and under the port’s control (endogenous) or external and beyond the control of the port (exogenous). The CPPI measures time spent in container ports, strictly based on quantitative data only, which do not reveal the underlying factors or root causes of extended port times. A detailed port-specific diagnostic would be required to assess the contribution of underlying factors to the time a vessel spends in port. A very low ranking or a significant change in ranking may warrant special attention, for which the World Bank generally recommends a detailed diagnostic.Publication Digital Progress and Trends Report 2023(Washington, DC: World Bank, 2024-03-05)Digitalization is the transformational opportunity of our time. The digital sector has become a powerhouse of innovation, economic growth, and job creation. Value added in the IT services sector grew at 8 percent annually during 2000–22, nearly twice as fast as the global economy. Employment growth in IT services reached 7 percent annually, six times higher than total employment growth. The diffusion and adoption of digital technologies are just as critical as their invention. Digital uptake has accelerated since the COVID-19 pandemic, with 1.5 billion new internet users added from 2018 to 2022. The share of firms investing in digital solutions around the world has more than doubled from 2020 to 2022. Low-income countries, vulnerable populations, and small firms, however, have been falling behind, while transformative digital innovations such as artificial intelligence (AI) have been accelerating in higher-income countries. Although more than 90 percent of the population in high-income countries was online in 2022, only one in four people in low-income countries used the internet, and the speed of their connection was typically only a small fraction of that in wealthier countries. As businesses in technologically advanced countries integrate generative AI into their products and services, less than half of the businesses in many low- and middle-income countries have an internet connection. The growing digital divide is exacerbating the poverty and productivity gaps between richer and poorer economies. The Digital Progress and Trends Report series will track global digitalization progress and highlight policy trends, debates, and implications for low- and middle-income countries. The series adds to the global efforts to study the progress and trends of digitalization in two main ways: · By compiling, curating, and analyzing data from diverse sources to present a comprehensive picture of digitalization in low- and middle-income countries, including in-depth analyses on understudied topics. · By developing insights on policy opportunities, challenges, and debates and reflecting the perspectives of various stakeholders and the World Bank’s operational experiences. This report, the first in the series, aims to inform evidence-based policy making and motivate action among internal and external audiences and stakeholders. The report will bring global attention to high-performing countries that have valuable experience to share as well as to areas where efforts will need to be redoubled.Publication The Container Port Performance Index 2020 to 2024: Trends and Lessons Learned(Washington, DC: World Bank, 2025-09-22)The Container Port Performance Index (CPPI) provides a global benchmark of how container ports perform in handling vessel calls. Developed jointly by the World Bank and S&P Global Market Intelligence, it measures the time ships spend in port and relates this to the number of containers moved during that time. This approach makes the CPPI a unique diagnostic tool that can highlight patterns in port operations and shed light on global and regional supply chain dynamics. Now in its fifth edition, the CPPI report covers the period from 2020 to 2024. It builds on a well-established methodology to generate scores for more than 400 container ports worldwide. Over time, the CPPI has become a trusted reference point for policymakers, industry stakeholders, and researchers who seek to understand how ports adapt to shocks, recover from disruptions, and identify opportunities for investments, reform and modernization. A major innovation in this edition is the introduction of multi-year trend analysis. Rather than presenting annual snapshots, the report now tracks how CPPI scores have changed across five years. This longitudinal perspective reveals shifts in port performance, showing where scores have risen, fallen, or remained stable. By linking these movements to external factors, the CPPI offers insights into how global and regional supply chains evolve under pressure. The results clearly mirror the crises that have shaken global trade. During the COVID-19 pandemic, CPPI scores in different regions declined sharply as congestion, equipment shortages, and delays overwhelmed many ports. By 2023, global averages rebounded in parallel with easing freight markets and reduced congestion. Yet 2024 brought new challenges: the Red Sea crisis disrupted major trade lanes, while climate-related constraints at the Panama Canal added further stress. These shocks were reflected in lower global and several regional average scores, underscoring the vulnerability of maritime transport to geopolitical and environmental events. The CPPI is not about comparing one port against another, but about understanding changes in performance over time. Ports that improved their scores often did so by reducing time at anchor, optimizing berth operations, investing in digital tools, and strengthening coordination across logistics partners. The evidence confirms that improvements are possible across ports of all sizes, and that rising scores are linked to deliberate actions to minimize time in port relative to containers moved. By consolidating five years of results, this edition transforms the CPPI into a long-term reference point. It shows how global crises have affected shipping, how different regions have adapted, and what lessons can be drawn for future resilience. The World Bank and S&P Global Market Intelligence remain committed to maintaining the CPPI as a global public good, providing transparency, comparability, and practical insights to support more reliable and sustainable maritime supply chains.Publication Global Economic Prospects, January 2025(Washington, DC: World Bank, 2025-01-16)Global growth is expected to hold steady at 2.7 percent in 2025-26. However, the global economy appears to be settling at a low growth rate that will be insufficient to foster sustained economic development—with the possibility of further headwinds from heightened policy uncertainty and adverse trade policy shifts, geopolitical tensions, persistent inflation, and climate-related natural disasters. Against this backdrop, emerging market and developing economies are set to enter the second quarter of the twenty-first century with per capita incomes on a trajectory that implies substantially slower catch-up toward advanced-economy living standards than they previously experienced. Without course corrections, most low-income countries are unlikely to graduate to middle-income status by the middle of the century. Policy action at both global and national levels is needed to foster a more favorable external environment, enhance macroeconomic stability, reduce structural constraints, address the effects of climate change, and thus accelerate long-term growth and development.