Publication: Measuring the Unmeasured: Combining Technology and Behavioral Insights to Improve Measurement of Business Outcomes
Loading...
Files in English
271 downloads
Published
2019-04
ISSN
Date
2019-05-02
Author(s)
Editor(s)
Abstract
Business survey outcomes for micro and small firms are notoriously noisy, with multiple sources of measurement and recall error. This paper introduces a new survey methodology that combines automatic consistency checks of electronic data collection with triangulation and dynamic adjustment to arrive at more precise estimates of business performance. The methodology uses insights from behavioral science to lower the cognitive cost of initial recall and establishes salient and relevant anchors to allow for dynamic triangulation and adjustment toward a final estimate. The validity of this method is field tested against traditional performance measures as well as administrative data across three emerging markets: Ghana, Rwanda, and Uganda. The results show significant upward adjustment from traditional measures for both sales and profits, a lower coefficient of variation in the cross-section, and higher autocorrelation in panel data. Comparisons with administrative data further confirm a higher correlation and closer magnitude relative to traditional measures. This research reconciles recommendations for increased attention to survey design with a method to leverage electronic survey technology beyond consistency checks.
Link to Data Set
Citation
“Anderson, Stephen J.; Lazicky, Christy; Zia, Bilal. 2019. Measuring the Unmeasured: Combining Technology and Behavioral Insights to Improve Measurement of Business Outcomes. Policy Research Working Paper;No. 8836. © World Bank. http://hdl.handle.net/10986/31601 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication Climate and Social Sustainability in Fragility, Conflict, and Violence Contexts(Washington, DC: World Bank, 2026-01-07)Climate change is widely recognized as a driver of violent conflict, but its broader social effects remain less understood. Ignoring these dimensions risks a vicious cycle where climate policies might undermine socially just adaptation. Evidence is still limited on how climate shocks influence political participation, trust, or migration. This paper helps fill that gap by examining links between climate change, conflict, and social sustainability, with a focus on inclusion, resilience, cohesion, and legitimacy. Using secondary data from 2019–24, the study applies simple correlation-based methods to test three hypotheses on the nature, severity, and composition of these associations. The analysis combines multiple climate impact measures, new conflict classifications, recent social sustainability frameworks, and controls for population and geography. The results reveal strong correlations—not causation—between climate events and contexts of fragility, conflict, and violence. Climate impacts are most pronounced in both national and subnational conflict settings. The study also finds robust links between fragility, conflict, and violence and low levels of social sustainability, reflecting its role as both a driver and consequence of conflict. Some dimensions—such as violent events and insecurity—appear weaker in areas most affected by climate shocks. Two of the hypotheses are supported, and one remains inconclusive.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Institutional Capacity for Policy Implementation: An Analytical Framework(Washington, DC: World Bank, 2026-01-07)State capacity is an important prerequisite for policy implementation, yet at the country level it is difficult to measure, assess, and reform. This paper proposes a focus on institutional capacity: the ability of public institutions to implement the specific policy mandates for which they are responsible. Based on a review of existing literature, the paper defines the different dimensions that compose institutional capacity and groups them into two cross-cutting categories: organizational dimensions (personnel, financial resources, information systems, and management practices) and governance dimensions (transparency, independence, and accountability). The paper proposes measures for organizational and governance dimensions using existing data, shows intra-institutional variation of these measures within countries, and discusses how new data could be collected for better measurement of these concepts. Finally, the paper illustrates how the framework can be used to diagnose the sources of common problems related to weak policy implementation.Publication South Africa’s Fragmented Cities: The Unequal Burden of Labor Market Frictions(Washington, DC: World Bank, 2026-01-08)Using high-resolution administrative, census, and satellite data, this paper shows that South African cities are characterized by spatial mismatches between where people live and where jobs are located, relative to 20 global peers. Areas within 5 kilometers of commercial centers have 9,300 fewer residents per square kilometer than expected, which is 60 percent below the global median. Poor, dense neighborhoods are most affected. In Johannesburg, a 10-percentile increase in distance from the nearest business hub corresponds to a 3.7-percentile drop in asset wealth (a proxy of household wellbeing) and 4.9-percentile drop in employment. In Cape Town, the declines are 4.0 and 3.7 percentiles, respectively. Employment is 87 percent lower in the poorest decile than the richest in Johannesburg and 61 percent lower in Cape Town. These findings suggest that South Africa’s spatial organization of people and economic activity constrains agglomeration and reinforces inequality. This methodology provides a scalable and standardized data-driven framework to analyze spatial accessibility and agglomeration frictions in complex, data-constrained urban systems.Publication Investment in Emerging and Developing Economies(Washington, DC: World Bank, 2026-01-07)The world faces a pressing challenge to meet key development objectives amid slowing growth and rising macroeconomic and geopolitical risks. With the number of job seekers rising rapidly, infrastructure shortfalls continuing to be large, and climate costs mounting, the case for a significant investment push has never been stronger. Yet the capacity to respond in many emerging markets and developing economies has eroded. Since the global financial crisis, investment growth has slowed to about half its pace in the 2000s, with both public and private investment weakening. Foreign direct investment inflows—a critical source of capital, technology, and managerial know-how—have also fallen sharply and become increasingly concentrated, leaving low-income countries with only a marginal share. The risks of further retrenchment are significant, as trade tensions, policy uncertainty, and elevated debt levels continue to weigh on investment. Reigniting momentum will require ambitious domestic reforms to strengthen institutions, rebuild macro-fiscal stability, and deepen trade and investment integration—the foundations of a supportive business climate. At the same time, international cooperation is indispensable. A renewed commitment to a predictable system of cross-border trade and investment flows, combined with scaled-up financial support and sustained technical assistance, is essential to help emerging markets and developing economies—especially low-income countries and economies in fragile and conflict situations—bridge financing gaps and implement the domestic reforms needed to restore investment as an engine of growth, jobs, and development.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Investment Climate Reforms and Job Creation in Developing Countries : What Do We Know and What Should We Do?(World Bank Group, Washington, DC, 2014-09)This paper reviews the literature on the role of the investment climate reforms in job creation. It finds that the current landscape of employment and private sector activity in developing countries indicates a number of potential channels through which investment climate reforms can positively affect job creation. However, rigorous empirical evidence is scarce and most of the relevant studies focus on business entry reforms with a few focusing on business taxation and investment promotion activities. Overall, there is evidence of job creation through business entry, tax reforms, and investment promotion activity in developing countries. Almost all of these evidences are from quasi-experimental studies that are significant improvements over conventional cross-country or cross-section panel data analysis. Still, various endogeneity concerns in these studies cannot be ruled out completely. In assessing job effects, future research should provide deeper insights on the gross versus net and short-run versus long-run job effects and general equilibrium effects of various investment climate reforms related to jobs, productivity, competition, and other developmental outcomes. Another critical agenda for future research is to shed light on which investment climate reforms matter most for spurring the employment and productivity growth of firms in developing countries. The World Bank Group, in partnership with development partners and client government countries, can play a significant role in bridging the current knowledge gap by integrating rigorous evaluation as an integral part of project design and implementation, and improving data quality, particularly through its information and communication technologies-led private sector development reform initiatives.Publication Ecuador : The Faces of Informality (Las Caras de La Informalidad)(Washington, DC, 2012)Informality hampers economic performance at both macro and micro levels. High degree of informality and low economic growth tend to go hand-in-hand, although evidence shows that the relationship likely comes from common determinants of both. Informality in Ecuador remains high compared to other countries. This study offers a novel look at informality in Ecuador by directly asking firm owners and workers about their views and behavior. The analysis of the causes and costs of informality for firms in Ecuador in this study is based on data from a 2011 enterprise survey commissioned specifically for this study and complemented by focus groups and in-depth interviews. The survey captures the aspects of the regulatory framework which are most problematic for compliance, and the specific reasons for non-compliance. The study focuses on micro and small firms in the urban areas of Ecuador. The sectors of activity, geographic locations, and sizes of firms covered by this study were agreed upon with the Government during the initial consultations on the report. The study finds that many micro and small firms in Ecuador have limited growth potential, due to low entrepreneurial ability and lack of access to and poor quality of credit.Publication Performance of Manufacturing Firms in Africa : An Empirical Analysis(Washington, DC: World Bank, 2012-08-07)This book sheds light on the characteristics of formal and informal manufacturing firms in Africa by comparing these firms with firms in other regions. Drawing on two data sources, the authors find that there is a very low share of manufacturing in GDP in Africa and in African exports. Most African manufacturing firms are informal. These firms are also smaller than firms in other regions and few export. Labor productivity is low in Africa relative to other regions, but this may be because of the more challenging environment - with the lack of physical infrastructure, the heavy burden of business regulation, and other issues. However, after accounting for these differences, the authors find that firms in Sub-Saharan Africa appear more, not less, productive than firms elsewhere. This analysis suggests that improving the business environment might allow firms to enhance their performance. However, given the pervasive distortions in the business environment and the limited resources at the disposal of most African countries, Africa cannot and should not wait until the business environment becomes healthier before growing a more viable manufacturing sector. The book shows that binding constraints vary by country, by sector, and by firm size. Therefore, countries should identify the constraints in the most promising sectors and adopt policies designed specifically to remove these constraints. The evidence in this book overwhelmingly dispels the false notion of Africa's inability to compete globally in manufacturing goods.Publication What Prevents More Small Firms from Using Professional Business Services? An Information and Quality-Rating Experiment in Nigeria(World Bank, Washington, DC, 2021-04)Why do more small firms in developing countries not use the market for professional business services like accounting, marketing, and human resource specialists? Two key reasons may be that firms lack information about the availability of these services, and that they struggle to distinguish the quality of good versus bad providers. A brand recognition exercise finds that most small firms are unaware of most providers in this market, and a survey of service providers reveals that they largely rely on word-of-mouth and informal reputation mechanisms for acquiring customers. This study set up a business services marketplace that contains information about the different providers present in the market and used mystery shopper visits to develop a quality ratings system. A randomized experiment with more than 1,000 firms provided access to this marketplace to the treatment group and randomized whether firms received just information or also quality ratings. The provision of quality ratings information shifts small firms’ preferences over which provider they would like to use, increasing the average quality rating of their preferred providers by 0.2 to 0.4 ratings points out of 5. However, neither the provision of information nor these quality ratings had any significant impact on the likelihood that small firms go on to hire a business service provider over the subsequent six months. The results suggest that alleviating information frictions alone is insufficient to increase usage of professional business services.Publication Mozambique - Investment Climate Assessment - 2009 : Sustaining and Broadening Growth(World Bank, 2009-10-01)Mozambique's recent history is a rare example of a successful post-conflict recovery and economic takeoff. Emerging from decades of economic stagnation and decline, a consequence first of a failed socialist economic experience and then of a vicious civil war that only ended in 1992, the country has achieved a commendable degree of political stability. This has been accompanied by prudent and stable economic policy continuity, as well as coordinated and ever more efficient use of substantial international aid. These factors have contributed to sustained economic growth that averaged 7.8 percent between 1992 and 2006. Furthermore, this growth has been 'pro-poor': increasing output has been accompanied by real and significant decreases in poverty levels, with the poverty headcount index declining from 69 percent in 1997 to 54 percent in 2003. Based on the enterprise survey results for Mozambique, this report assesses the main obstacles to achieving an investment climate that supports private sector growth and provides policy options for improving the business environment and increasing competitiveness with the goal of achieving sustained and broad-based growth. The focus is on microeconomic constraints and reforms where, according to a recent World Bank report, most of the challenges for sustainable growth are concentrated. Therefore, the analysis presented in this report should be of interest to policy makers, academics, non-governmental organizations and representatives of the private sector involved in the policy dialogue in the country. Improvements to the business environment and increased access to finance are the most critical aspects to firm growth in Mozambique identified in this report. Despite recent progress, the business environment for the Mozambican enterprise sector is still in many ways problematic. Based on econometric evidence as well as on business perceptions and quantitative data, this study indicates that while all aspects of the investment climate are important, reform priorities should focus on increasing access to finance and improving the business environment.
Users also downloaded
Showing related downloaded files
Publication Housing Subsidies for Refugees(Washington, DC: World Bank, 2025-01-22)Refugees require assistance for basic needs like housing but local host communities may feel excluded from that assistance, potentially affecting community relations. This study experimentally evaluates the effect of a housing assistance program for Syrian refugees in Jordan on both the recipients and their neighbors. The program offered full rental subsidies and landlord incentives for housing improvements, but saw only moderate uptake, in part due to landlord reluctance. The program improved short-run housing quality and lowered housing expenditures, but did not yield sustained economic benefits, partly due to redistribution of aid. The program unexpectedly led to a deterioration in child socio-emotional well-being, and also strained relations between Jordanian neighbors and refugees. In all, housing subsidies had limited measurable benefits for refugee well-being while worsening social cohesion, highlighting the possible need for alternative forms of aid.Publication Ukraine Country Environmental Analysis(World Bank, Washington, DC, 2016-01)The objective of the Country Environmental Analysis (CEA) is to assess the adequacy and performance of the policy, legal, and institutional framework for environmental management in Ukraine, in light of the decentralization process of environmental governance and wider reform objectives, and to provide recommendations to government to address the key gaps identified. Ukraine is the second largest country in Europe and has a population of 43 million, the majority of whom live in urban areas. It is a lower middle income country, with the services, industry and agriculture sectors being main contributors to the country’s Gross Domestic Product (GDP). Ukraine faces a number of environmental challenges, as identified in its National Environmental Strategy 2020 (NES). Key among these are: air pollution; quality of water resources and land degradation; solid waste management; biodiversity loss; human health issues associated with environmental risk factors; in addition to climate change. The scope of Ukrainian environmental legislation is quite broad and comprehensive (more than 300 legal acts) and covers most areas of environmental protection and natural resources management. However, the environmental legislation faces a number of weaknesses:The environmental legislation is largely declaratory in nature and does not have all the essential enforcement mechanisms for the implementation of legal acts and international agreements; Many of the acts are not coordinated with each other; and Legislation undergoes limited analysis of its impact—for example, no in-depth analysis such as Regulatory Impact Analysis is conducted for proposed pieces of legislation.Publication Regional Poverty and Inequality Update: Latin America and the Caribbean, October 2025(Washington, DC: World Bank, 2025-10-23)This brief summarizes recent facts related to poverty and inequality in Latin America and the Caribbean (LAC) using the latest wave of harmonized household surveys from the Socio-Economic Database for LAC (SEDLAC). This brief was produced by the Poverty Global Practice in the LAC Region of the World Bank.Publication Digital Africa(Washington, DC: World Bank, 2023-03-13)All African countries need better and more jobs for their growing populations. "Digital Africa: Technological Transformation for Jobs" shows that broader use of productivity-enhancing, digital technologies by enterprises and households is imperative to generate such jobs, including for lower-skilled people. At the same time, it can support not only countries’ short-term objective of postpandemic economic recovery but also their vision of economic transformation with more inclusive growth. These outcomes are not automatic, however. Mobile internet availability has increased throughout the continent in recent years, but Africa’s uptake gap is the highest in the world. Areas with at least 3G mobile internet service now cover 84 percent of Africa’s population, but only 22 percent uses such services. And the average African business lags in the use of smartphones and computers as well as more sophisticated digital technologies that catalyze further productivity gains. Two issues explain the usage gap: affordability of these new technologies and willingness to use them. For the 40 percent of Africans below the extreme poverty line, mobile data plans alone would cost one-third of their incomes—in addition to the price of access devices, apps, and electricity. Data plans for small- and medium-size businesses are also more expensive than in other regions. Moreover, shortcomings in the quality of internet services—and in the supply of attractive, skills-appropriate apps that promote entrepreneurship and raise earnings—dampen people’s willingness to use them. For those countries already using these technologies, the development payoffs are significant. New empirical studies for this report add to the rapidly growing evidence that mobile internet availability directly raises enterprise productivity, increases jobs, and reduces poverty throughout Africa. To realize these and other benefits more widely, Africa’s countries must implement complementary and mutually reinforcing policies to strengthen both consumers’ ability to pay and willingness to use digital technologies. These interventions must prioritize productive use to generate large numbers of inclusive jobs in a region poised to benefit from a massive, youthful workforce—one projected to become the world’s largest by the end of this century.Publication Thailand Monthly Economic Monitor, October 2025(Washington, DC: World Bank, 2025-10-22)Fiscal conditions remained stable, with a modest widening of the deficit to 3.1 percent of GDP. New stimulus measures are expected to support short-term demand without breaching the public debt ceiling. Inflation stayed negative, reflecting lower energy and food prices amid subdued domestic demand. The central bank kept the policy rate unchanged, citing limited policy space. Thailand’s growth momentum has slowed further as manufacturing activity and services weakened as projected. Tourism remained subdued, largely due to fewer Chinese visitors. Goods exports also slowed as earlier front-loaded orders faded, particularly in agriculture and industrial goods. The Thai baht depreciated in early October as the US dollar appreciated and the current account turned negative.