Publication: Comparing Apples with....Apples : How to Make (More) Sense of Subjective Rankings of Constraints to Business
Loading...
Published
2009-09-01
ISSN
Date
2012-03-19
Author(s)
Aterido, Reyes
Editor(s)
Abstract
The use of expert or qualitative surveys to rank countries business investment conditions is widespread. However, within the economic literature there are concerns about measurement error and endogeneity based on characteristics of the respondents, raising questions about how well the data reflect the underlying reality they are trying to measure. This paper examines these concerns using data from 79,000 firms in 105 countries. The findings show that first, qualitative rankings correlate well with quantitative measures of the business environment, using both quantitative measures from within the survey and from external sources. Second, there are systematic variations in perceptions based on firm characteristics - focusing in particular on size and growth performance. However, it is not that an optimistic view of the business environment is simply the expression of a firm s own performance. Rather, firm size and performance affect the relative importance of certain constraints, particularly in areas such as finance, time with officials/inspectors, corruption, and access to reliable electricity. The results also show that much of the variation in subjective responses by firm types is largely due to differences in the objective conditions across firm types. There is little evidence that size and performance have non-linear effects in how constraining a given objective condition is reported to be. Overall, concerns about endogeneity remain in using business environment indicators to explain firm performance, but this stems primarily from the fact that who you are and how well you are doing can affect the conditions you face rather than whether the indicator used is qualitative or quantitative.
Link to Data Set
Citation
“Aterido, Reyes; Hallward-Driemeier, Mary. 2009. Comparing Apples with....Apples : How to Make (More) Sense of Subjective Rankings of Constraints to Business. Policy Research working paper ; no. WPS 5054. © World Bank. http://hdl.handle.net/10986/4246 License: CC BY 3.0 IGO.”
Digital Object Identifier
Associated URLs
Associated content
Other publications in this report series
Publication The Economic Value of Weather Forecasts: A Quantitative Systematic Literature Review(Washington, DC: World Bank, 2025-09-10)This study systematically reviews the literature that quantifies the economic benefits of weather observations and forecasts in four weather-dependent economic sectors: agriculture, energy, transport, and disaster-risk management. The review covers 175 peer-reviewed journal articles and 15 policy reports. Findings show that the literature is concentrated in high-income countries and most studies use theoretical models, followed by observational and then experimental research designs. Forecast horizons studied, meteorological variables and services, and monetization techniques vary markedly by sector. Estimated benefits even within specific subsectors span several orders of magnitude and broad uncertainty ranges. An econometric meta-analysis suggests that theoretical studies and studies in richer countries tend to report significantly larger values. Barriers that hinder value realization are identified on both the provider and user sides, with inadequate relevance, weak dissemination, and limited ability to act recurring across sectors. Policy reports rely heavily on back-of-the-envelope or recursive benefit-transfer estimates, rather than on the methods and results of the peer-reviewed literature, revealing a science-to-policy gap. These findings suggest substantial socioeconomic potential of hydrometeorological services around the world, but also knowledge gaps that require more valuation studies focusing on low- and middle-income countries, addressing provider- and user-side barriers and employing rigorous empirical valuation methods to complement and validate theoretical models.Publication The Macroeconomic Implications of Climate Change Impacts and Adaptation Options(Washington, DC: World Bank, 2025-05-29)Estimating the macroeconomic implications of climate change impacts and adaptation options is a topic of intense research. This paper presents a framework in the World Bank's macrostructural model to assess climate-related damages. This approach has been used in many Country Climate and Development Reports, a World Bank diagnostic that identifies priorities to ensure continued development in spite of climate change and climate policy objectives. The methodology captures a set of impact channels through which climate change affects the economy by (1) connecting a set of biophysical models to the macroeconomic model and (2) exploring a set of development and climate scenarios. The paper summarizes the results for five countries, highlighting the sources and magnitudes of their vulnerability --- with estimated gross domestic product losses in 2050 exceeding 10 percent of gross domestic product in some countries and scenarios, although only a small set of impact channels is included. The paper also presents estimates of the macroeconomic gains from sector-level adaptation interventions, considering their upfront costs and avoided climate impacts and finding significant net gross domestic product gains from adaptation opportunities identified in the Country Climate and Development Reports. Finally, the paper discusses the limits of current modeling approaches, and their complementarity with empirical approaches based on historical data series. The integrated modeling approach proposed in this paper can inform policymakers as they make proactive decisions on climate change adaptation and resilience.Publication Labor Demand in the Age of Generative AI: Early Evidence from the U.S. Job Posting Data(Washington, DC: World Bank, 2025-11-18)This paper examines the causal impact of generative artificial intelligence on U.S. labor demand using online job posting data. Exploiting ChatGPT’s release in November 2022 as an exogenous shock, the paper applies difference-in-differences and event study designs to estimate the job displacement effects of generative artificial intelligence. The identification strategy compares labor demand for occupations with high versus low artificial intelligence substitution vulnerability following ChatGPT’s launch, conditioning on similar generative artificial intelligence exposure levels to isolate substitution effects from complementary uses. The analysis uses 285 million job postings collected by Lightcast from the first quarter of 2018 to the second quarter of 2025Q2. The findings show that the number of postings for occupations with above-median artificial intelligence substitution scores fell by an average of 12 percent relative to those with below-median scores. The effect increased from 6 percent in the first year after the launch to 18 percent by the third year. Losses were particularly acute for entry-level positions that require neither advanced degrees (18 percent) nor extensive experience (20 percent), as well as those in administrative support (40 percent) and professional services (30 percent). Although generative artificial intelligence generates new occupations and enhances productivity, which may increase labor demand, early evidence suggests that some occupations may be less likely to be complemented by generative artificial intelligence than others.Publication External Finance in Emerging Markets and Developing Economies: A Tale of Differences in Vulnerabilities(Washington, DC: World Bank, 2025-12-04)Over the past two decades, many emerging markets and developing economies have been viewed as increasingly resilient to external financial shocks. This paper assesses whether such resilience is broadly shared across emerging markets and developing economies by classifying them into three tiers based on economic size, income level, institutional strength, and financial integration. The analysis shows that first-tier emerging markets and developing economies have improved their external balance sheets and reduced dependence on official support. However, second- and third-tier emerging markets and developing economies have experienced growing external vulnerabilities since the global financial crisis, marked by rising external debt liabilities and declining foreign exchange reserves. Using a range of indicators, including sovereign defaults, arrears, partial defaults, and International Monetary Fund lending, the paper identifies episodes of external financial distress and shows that distress remains widespread among second- and third-tier emerging markets and developing economies. The empirical analysis confirms that key components of the net international investment position—especially external debt and foreign exchange reserves—predict the onset of external financial distress, with institutional quality shaping the impact. Weak institutions amplify risks, while strong institutions mitigate them. These findings highlight the importance of recognizing heterogeneity across emerging markets and developing economies, strengthening institutional quality alongside external balance-sheet management, and rebuilding buffers to safeguard against renewed global financial stress.Publication Rigging the Scores: Corruption through Scoring Rule Manipulation in Public Procurement Auctions(Washington, DC: World Bank, 2025-12-02)Public procurement is highly susceptible to corruption, especially in developing countries. Although open auctions are widely adopted to curb it, this paper finds that corruption remains prevalent even within this procurement format. Procurement officers can collaborate with firms to manipulate scoring rules, ensuring predetermined winners, while corrupt firms submit noncompetitive bids to meet minimum bidder requirements. Using extensive data from Chinese public procurement auctions, the paper introduces model-driven statistical tools to detect such corruption, identifying a corruption rate of 65 percent. A procurement expert audit survey confirms the tools’ reliability, with a 91 percent probability that experts recognize suspicious scoring rules when flagged. Firm-level analysis reveals that local, state-owned, and less productive firms are favored in corrupt auctions. Lastly, the paper explores policy implications. Analysis of the national anti-corruption campaign since 2012 suggests that general investigations may be insufficient to address deeply ingrained corrupt practices. Using counterfactuals based on an estimated structural model, the paper shows that implementing anonymous call-for-tender evaluations could improve social welfare by 10 percent by eliminating suspicious rules and encouraging broader participation.
Journal
Journal Volume
Journal Issue
Collections
Related items
Showing items related by metadata.
Publication Big Constraints to Small Firms’ Growth? Business Environment and Employment Growth across Firms(2009-08-01)Using data on more than 56,000 enterprises in 90 countries, this paper finds that objective conditions in the business environment vary substantially across firms of different sizes and that there are important non-linearities in their impact on employment growth. The paper focuses on four areas: access to finance, business regulations, corruption, and infrastructure. The results, particularly on the impacts of finance and corruption on growth, depend on whether and how the analysis accounts for the possible endogeneity of the business environment. Controlling for endogeneity revises the finding that small firms benefit most from access to finance, particularly for sources of finance associated with investment and growth. The findings are also sensitive to how small is defined. Differentiating micro (less than 10 employees) from other small firms shows that, while small firms can be disadvantaged in such an environment, micro firms tend to be proportionally less affected by a weak business climate and, on occasion, it can help them to grow. Overall, allowing different size classifications provides insights into the impact of the business environment that are lost in more aggregate analyses.Publication The Impact of the Investment Climate on Employment Growth : Does Sub-Saharan Africa Mirror Other Low-Income Regions?(2010-02-01)Using survey data from 86,000 enterprises in 104 countries, including 17,000 enterprises in 31 Sub-Saharan African countries, this paper finds that average enterprise-level employment growth rates are remarkably similar across regions. This is true despite significant differences in the quality of the investment climate in which these enterprises operate. Objective measures of investment climate conditions (including the number of outages, the share of firms with bank loans, and others) indicate that conditions are most challenging within Sub-Saharan Africa, as well as for smaller enterprises. However, enterprises employment in Sub-Saharan Africa is less sensitive to changes in access to infrastructure and finance relative to other low-income regions. This can be understood by looking at non-linear effects by firm size -- and the finding that these size effects are particularly strong within Sub-Saharan Africa. Although unreliable infrastructure services and inadequate access to finance generally hamper growth, in Sub-Saharan Africa they are actually associated with higher employment growth rates among micro enterprises. Although employment growth is good news in Sub-Saharan Africa, that much of the expanded employment is in small, labor-intensive, less productive enterprises raises longer-run concerns about the efficiency of the allocation of resources and aggregate productivity growth in the region.Publication Who Survives? The Impact of Corruption, Competition and Property Rights across Firms(2009-10-01)Size, age, sector, and productivity are commonly cited as factors determining a firm s survival. However, there are several dimensions of the investment climate in which the firm operates that affect whether it continues in business or exits. This paper uses new panel data from 27 Eastern European and Central Asian countries to test the importance of five areas of the business climate on firm exit: the efficiency of government services, access to finance, the extent of corruption or cronyism, the strength of property rights, and the degree of competition. The paper finds that weaknesses in these areas do affect the probability of firm exit largely in ways that undermine the Schumpeterian cleansing role of exit in raising overall productivity. Greater costs and regulatory burdens raise the probability that more productive firms exit, while less developed financial and legal institutions mitigate forces that would otherwise push less productive firms to exit. Thus, the more productive firms stand to gain the most from improvements in the investment climate, whether that is lowering transaction costs or improving market mechanisms. This holds both within countries and across countries. The impact of a particular investment climate measure can also differ significantly by type of firm, with the focus given to firm size. The differential impact on size can be significant at a size cutoff of 10 or more employees. As these are the firms that are near the threshold of many regulatory requirements, the implications are not just with regard to whether a firm remains in operation, but whether it does so in the formal sector.Publication Deals versus Rules : Policy Implementation Uncertainty and Why Firms Hate It(2010-05-01)Firms in Africa report "regulatory and economic policy uncertainty" as a top constraint to their growth. This paper argues that often firms in Africa do not cope with policy rules, rather they face deals: firm-specific policy actions that can be influenced by firm actions (such as bribes) and characteristics (such as political connections). Using Enterprise Survey data, the paper demonstrates huge variability in reported policy actions across firms notionally facing the same policy. The within-country dispersion in firm-specific policy actions is larger than the cross-national differences in average policy. The analysis shows that variability in this policy implementation uncertainty within location-sector-size cells is correlated with firm growth rates. These measures of implementation variability are more strongly related to lower firm employment growth than are measures of "average" policy action. The paper shows that the de jure measures such as Doing Business indicators are virtually uncorrelated with ex-post firm-level responses, further evidence that deals rather than rules prevail in Africa. Strikingly, the gap between de jure and de facto conditions grows with the formal regulatory burden. The evidence also shows more burdensome processes open up more space for making deals; firms may not incur the official costs of compliance, but they still pay to avoid them. Finally, measures of institutional capacity and better governance are closely associated with perceived consistency in implementation.Publication Deals and Delays : Firm-level Evidence on Corruption and Policy Implementation Times(World Bank, Washington, DC, 2014-06)This paper examines whether demands for bribes for particular government services are associated with expedited or delayed policy implementation. The "grease the wheels" hypothesis, which contends that bribes act as speed money, implies three testable predictions. First, on average, bribe requests should be negatively correlated with wait times. Second, this relationship should vary across firms, with those with the highest opportunity cost of waiting being more likely to pay and face shorter delays. Third, the role of grease should vary across countries, with benefits larger where regulatory burdens are greatest. The data are inconsistent with all three predictions. According to the preferred specifications, ceteris paribus, firms confronted with demands for bribes take approximately 1.5 times longer to get a construction permit, operating license, or electrical connection than firms that did not have to pay bribes and, respectively, 1.2 and 1.4 times longer to clear customs when exporting and importing. The results are robust to controlling for firm fixed effects and at odds with the notion that corruption enhances efficiency.
Users also downloaded
Showing related downloaded files
Publication Governance Matters IV : Governance Indicators for 1996-2004(World Bank, Washington, DC, 2005-06)The authors present the latest update of their aggregate governance indicators, together with new analysis of several issues related to the use of these measures. The governance indicators measure the following six dimensions of governance: (1) voice and accountability; (2) political instability and violence; (3) government effectiveness; (4) regulatory quality; (5) rule of law, and (6) control of corruption. They cover 209 countries and territories for 1996, 1998, 2000, 2002, and 2004. They are based on several hundred individual variables measuring perceptions of governance, drawn from 37 separate data sources constructed by 31 organizations. The authors present estimates of the six dimensions of governance for each period, as well as margins of error capturing the range of likely values for each country. These margins of error are not unique to perceptions-based measures of governance, but are an important feature of all efforts to measure governance, including objective indicators. In fact, the authors give examples of how individual objective measures provide an incomplete picture of even the quite particular dimensions of governance that they are intended to measure. The authors also analyze in detail changes over time in their estimates of governance; provide a framework for assessing the statistical significance of changes in governance; and suggest a simple rule of thumb for identifying statistically significant changes in country governance over time. The ability to identify significant changes in governance over time is much higher for aggregate indicators than for any individual indicator. While the authors find that the quality of governance in a number of countries has changed significantly (in both directions), they also provide evidence suggesting that there are no trends, for better or worse, in global averages of governance. Finally, they interpret the strong observed correlation between income and governance, and argue against recent efforts to apply a discount to governance performance in low-income countries.Publication Governance Matters VIII : Aggregate and Individual Governance Indicators 1996–2008(2009-06-01)This paper reports on the 2009 update of the Worldwide Governance Indicators (WGI) research project, covering 212 countries and territories and measuring six dimensions of governance between 1996 and 2008: Voice and Accountability, Political Stability and Absence of Violence/Terrorism, Government Effectiveness, Regulatory Quality, Rule of Law, and Control of Corruption. These aggregate indicators are based on hundreds of specific and disaggregated individual variables measuring various dimensions of governance, taken from 35 data sources provided by 33 different organizations. The data reflect the views on governance of public sector, private sector and NGO experts, as well as thousands of citizen and firm survey respondents worldwide. The authors also explicitly report the margins of error accompanying each country estimate. These reflect the inherent difficulties in measuring governance using any kind of data. They find that even after taking margins of error into account, the WGI permit meaningful cross-country comparisons as well as monitoring progress over time. The aggregate indicators, together with the disaggregated underlying indicators, are available at www.govindicators.org.Publication Design Thinking for Social Innovation(2010-07)Designers have traditionally focused on enchancing the look and functionality of products.Publication Breaking the Conflict Trap : Civil War and Development Policy(Washington, DC: World Bank and Oxford University Press, 2003)Most wars are now civil wars. Even though international wars attract enormous global attention, they have become infrequent and brief. Civil wars usually attract less attention, but they have become increasingly common and typically go on for years. This report argues that civil war is now an important issue for development. War retards development, but conversely, development retards war. This double causation gives rise to virtuous and vicious circles. Where development succeeds, countries become progressively safer from violent conflict, making subsequent development easier. Where development fails, countries are at high risk of becoming caught in a conflict trap in which war wrecks the economy and increases the risk of further war. The global incidence of civil war is high because the international community has done little to avert it. Inertia is rooted in two beliefs: that we can safely 'let them fight it out among themselves' and that 'nothing can be done' because civil war is driven by ancestral ethnic and religious hatreds. The purpose of this report is to challenge these beliefs.Publication Government Matters III : Governance Indicators for 1996-2002(World Bank, Washington, DC, 2003-08)The authors present estimates of six dimensions of governance covering 199 countries and territories for four time periods: 1996, 1998, 2000, and 2002. These indicators are based on several hundred individual variables measuring perceptions of governance, drawn from 25 separate data sources constructed by 18 different organizations. The authors assign these individual measures of governance to categories capturing key dimensions of governance and use an unobserved components model to construct six aggregate governance indicators in each of the four periods. They present the point estimates of the dimensions of governance as well as the margins of errors for each country for the four periods. The governance indicators reported here are an update and expansion of previous research work on indicators initiated in 1998 (Kaufmann, Kraay, and Zoido-Lobat 1999a,b and 2002). The authors also address various methodological issues, including the interpretation and use of the data given the estimated margins of errors.