POVERTY & EQUITY NOTES JUNE 2020 · NUMBER 22 Improving the Productivity of National Offices for Statistics (IPNOS) i Alejandro Medina Giopp, Jose Montes, and Jorge Martinez For decades, policy experts’ and practitioner consensus has been growing about the crucial role quality data plays in informing policy making. This has led to investment and projects to increase quality data availability. But progress has been slow, as reflected by slow improvement in country statistical capacity—many countries remain data deprived. The IPNOS toolkit is based on the notion that producing more and better statistics, while requiring adequate resources, should also be efficient. Various initiatives assess national statistical systems’ (NSS) and data production. The World Bank’s IPNOS initiative complements existing kits, providing in-depth analytical tools to evaluate the efficiency of national statistical offices (NSOs), including costs, data quality, and NSO management. IPNOS application in 3 countries has identified bottlenecks and areas for improvement to inform data policies. Producing more and better statistics requires 2018, scores have increased slightly in some regions, by resources, but it is also about efficiency. Accurate, less than 1 percent in Europe and Central Asia (ECA) and timely, granular, and accessible data is crucial for a bit over 1 percent in Latin America and the Caribbean informed policy making, and to allocate resources (LAC), for instance. SCI scores in some regions efficiently. The World Bank assists in development of a deteriorated, such as the Middle East and North African set of statistics required for monitoring and evaluation (MENA). In addition, lack of poverty data is widespread (M&E) of public interventions, including household and and persistent. Despite some advancements, as many as enterprise surveys, agricultural and price data, and 67 countries are still poverty data deprived; that is, they administrative records, among others. NSOs need have no data, or only up to two data points over 6-year strengthening for better data production and intervals (see Figure 1). Other key data also remain management. scarce. Figure 1: Poverty data deprivation Improving availability of adequate data has become a priority during the last decades. As an example, governments, donors, and multilateral organizations, including the World Bank, have substantially increased the number of projects and funding dedicated to improving the statistical capacity of NSOs and NSSs in all regions of the world over the last 15 years. From 2006 to 2015, for instance, the World Bank invested US$919.4 million in 225 data-for-development projects. Despite these efforts, progress has been limited across regions. Regional values of the statistical capacity index (SCI) iihave barely changed. Between 2004 and Low correlation between investments and statistical Assessments in all 3 areas in turn inform development of results is largely due to poor management of data an action plan to strengthen the NSO functioning. production and dissemination. Initiatives to assess the capacity of NSSs include the United Nations Economic Cost and budget analysis (IPNOS-Cost): An excel- Commission for Africa’s (UNECA) self-assessment QUALITY based tool analyzes the NSO budget trend (budget guidance questionnaire, the European Commission’s allocation, percent budget execution, budget structure), Assessment Questionnaires, or the IMF’s Data Quality and estimates the real costs, direct and indirect, of Assessment Framework for National Accounts producing statistics. The calculation includes accrual Statistics.iii However, these do not measure efficiency, adjustments (depreciation and others), and other public productivity, or costs of creating and disseminating data, sector costs, which, divided by the sample size, provides nor the quality of the data. As a result, statistical the total cost per interview in a given statistical product production has frequently been poor quality, expensive, (such as HHS). Unit cost analysis identifies the proportion or does not meet the information needs of policy makers. of administrative costs, such as rent, accountant, insurance, offices supplies, among others. A high IPNOS: A new tool to assess NSO efficiency proportion of administrative costs could be a sign of inefficiency, with 20 to 25 percent considered an IPNOS addresses the gap in data assessment adequate proportion. initiatives. IPNOS focuses on NSO costs, quality, and managerial processes, as well as on the promotion of Quality analysis (IPNOS-Qual): A second excel- data usage. NSO’s require productivity analysis to based tool, along with other tools and software, assess their efficiency. Productivity is a consequence of assesses the quality of statistical operations and many factors, including institutional context, inputs, estimates underlying quality drivers in the life-cycle processes, outputs, and dissemination. Improvement in process. The tool identifies specific quality thresholds NSO´s productivity therefore implies one or more for the different products (Census, HHS, simultaneous upgrades in the quantity, quality, administrative records) for statistical production timeliness, or unit costs of 3main statistical products: processes, outputs, and user satisfaction. The different Census, Household Surveys (HHS), and administrative measures and indicators used for this exercise (see records. Figure 3) depend on the specific product and aspect Figure 2: The 3 main IPNOS pillars assessed. BUDGET-COST: Statistical production & dissemination costing estimation Figure 3: Measuring quality Product QUALITY: Statistical production processes, statistical products & user's satisfaction diagnostic Aspect HHS Census Administrative Assessed data ORGANIZATIONAL: Organizational & institutional diagnostic, to identify managerial bottlenecks Statistics ISO 9000, GSBPM, GAMSO, production ACTION PLAN and productivity indicators process Data quality Coverage HECRA tool Assessments consider 3 main pillars, leading to the indicators e.g Whipple to assess the non- index, quality of completion of an action plan. The IPNOS package respondent Mayer processes Output offers 3 main assessment tools for: coverage rate, index and and products coefficient of UN gender • Budget and cost-efficiency of production. variation index • Quality of processes, products, and user´s Designed satisfaction. Effect • Institutional and organizational aspects. User Users focus groups and satisfaction surveys satisfaction June 2020 · Number 22 2 Organizational and institutional analysis (IPNOS- improvement plan from 2016-2020 to substantially Org): IPNOS uses questionnaires, focus groups, and improve its statistical capacity, as the WB SCI shows. performance indicators to identify productivity Application of the IPNOS tool in Costa Rica’s INEC indicates that the quantity of good quality data bottlenecks. For institutional aspects, the main produced increased with no increase in costs. However, categories are: it also shows that quality of certain datasets may be • Demand for statistics decreasing, and that some institutional bottlenecks • NSO staff constrain productivity, including the inadequacy of the • Legal framework amount and/or training of staff, the lack of recognition • Financial situation of INEC´s work in budgetary terms, and IT and • Coordination managerial challenges. Figure 3 shows that high staff • Autonomy turnover rates, for instance, affect the quality of Costa Rica’s HHS. For organizational diagnosis, the main categories are: • Strategic plan Figure 3: High staff turnover rate affects quality • Organizational structure Costa Rica’s household survey • M&E 1 7% • Marketing and dissemination 6% • Information technology 0.8 5% • Process management 0.6 4% The Action Plan: Information from assessment of 0.4 3% quality, costs, and bottlenecks provides a base on which 2% 0.2 to create a strategy—or “Action Plan”. Based on 1% agreement reached with the client, the Plan then defines 0 0% 2010 2011 2012 2013 2014 2015 a specific set of actions across all areas assessed, evaluating the effects of the measures (high, medium, Quality Turnover rate and low), the cost, and the period of execution (long, medium, and short term). The Plan also makes a El Salvador Dirección General de Estadística distinction between actions that are strictly internal and (DIGESTYC): The Technical Secretariat of the Presidency those that require intra-institutional management. Office and the Ministry of Economy of El Salvador used the IPNOS report to define its strategy and create a new, independent statistical coordination agency. El Salvador IPNOS in practice used IPNOS to assess the productivity of DIGESTYC. The exercise highlighted the need to define a legal, NSOs in Costa Rica, El Salvador, and the Seychelles institutional, and strategic framework for development applied the IPNOS package to assess capacity. The of an adequate NSS. It also identified some DIGESTYC exercise identified the most important areas for work in weaknesses to produce higher quality data, including these countries, and NSOs are using assessment results high administrative costs, financing structure, and lack of to improve productivity and data quality. staff (Figure 4). The quality of data produced has been Costa Rica Instituto Nacional de Estadistica y Censos irregular, especially in the case of surveys and (INEC). Costa Rica’s NSO implemented the IPNOS administrative records. June 2020 · Number 22 3 Figure 4: Strategic planning, data quality, and IT lag Figure 5: NBS budget allocated to administration has in El Salvador increased in the Seychelles 700,000 600,000 2016 2017 2018 500,000 400,000 300,000 200,000 100,000 USD 0 Programme 1: Programme 2: SP1: SP2: Census, SP3: Social Governance, Statistical Economics Survey and GIS Statistics Management Production Statistics and Administration Seychelles Bureau of Statistics: The Government of the Seychelles is using the IPNOS report to help inform its national strategy for development of statistics (NSDS), as Conclusion: From results to actions well as a guide to improve statistics production management. In the Seychelles, mechanisms for coordination exist even in the absence of a legal IPNOS can help governments around the world identify framework for a NSS and NSDS. The budget of the main bottlenecks and specific areas to improve the Bureau of Statistics (NBS) has been growing since 2014, quality and productivity of statistical data production. but the ratio of expenditures allocated to governance, Through an action plan derived from IPNOS management, and administration is high (Figure 5). NBS implementation, NSOs can produce more and better staff regularly work overtime, but the workforce is stable. statistics at lower cost, and also promote data use with Although using quality control practices and tools to key governmental analysis units or agencies. produce statistics, NBS has not established a proper quality model. The quality of the last two population Importantly, to promote data usage and dissemination, censuses (2002 and 2010) is high, while that of surveys country NSOs must allow access to their data, as well as and administrative data appears lower. While the facilitate interviews with NSO personnel, statistical users, statistics production costs have decreased, the model and allow access to NSO accounting books. needs improvement to decrease unitary cost of production. i IPNOS final case studies, presentations, guidelines, and the IPNOS toolkit will be released soon through the website. ii The World Bank’s SCI. a composite score assessing capacity of a country’s statistical system, is based on a diagnostic framework assessing data methodology, sources, periodicity, and timeliness. iii Others include for instance the Snapshot tool or the or the Pan-African Statistics Program of EUROSTAT, the Tool for Assessing Statistical Capacity of the US Census Bureau, UNECA´s Africa Statistical Development Indicators and Framework, or the AFDB Tool for Assessing Statistical Capacity. ABOUT THE AUTHORS Alejandro Medina Giopp is a Senior Monitoring and Evaluation Specialist in World Bank’s Poverty and Equity Global Practice. Jose Montes is a Data Scientist in the World Bank’s Poverty and Equity Global Practice. Jorge Martinez is a Consultant for the World Bank’s Poverty and Equity Global Practice. This note series is intended to summarize good practices and key policy findings on Poverty-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank, its board or its member countries. Available for download at the World Bank Publications, Documents & Reports site. June 2020 · Number 22 4