Report No: AUS0003356 . Slovakia Understanding the Productivity of Slovakia’s Local Governments Key findings from an empirical study . June 2023 . . The work underlying this report was financed by the EU in collaboration with the EC's DG REGIO. \ © 2023 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2023. Understanding the Productivity of Slovakia’s Local Governments: Key findings from an empirical study. © World Bank.” All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Acronyms BL Bureaucracy Lab DIME Development Impact Evaluation EC European Commission EQI European Quality of Governance Index EU European Union HR Human Resource IPSA Institute for Public and Security analyses MoI Ministry of Interior QoG The Quality of Government Institute SOSR Statistical Office of the Slovak Republic WB World Bank WGI Worldwide Governance Indicators WMS World Management Survey Contents Contents ........................................................................................................................................................ 4 Acknowledgements........................................................................................................................................ i Executive summary ....................................................................................................................................... 2 Chapter 1 Strengthening Government Institutions in Slovakia .................................................................... 5 1.1 The context ......................................................................................................................................... 6 1.2 District offices in Slovakia ................................................................................................................... 9 Chapter 2 Empirical approach..................................................................................................................... 11 2.1 Administrative data sources ............................................................................................................. 12 2.2 Survey of district office public officials ............................................................................................. 16 2.3 Data constraints ................................................................................................................................ 19 Chapter 3 Public Sector Productivity Across Slovakia and Its Determinants .............................................. 20 3.1 Productivity ....................................................................................................................................... 20 3.2 Management quality ......................................................................................................................... 31 3.3 The relationship between measures of management and productivity .......................................... 35 3.4 Labor markets ................................................................................................................................... 38 3.5 Bringing It All Together ..................................................................................................................... 42 Chapter 4 Frontiers of Government Analytics ............................................................................................ 43 Annex 1 Areas to explore and develop ....................................................................................................... 46 Annex 1.1 Fabasoft ................................................................................................................................. 46 Annex 1.2 Cezir ....................................................................................................................................... 49 Annex 1.3 Employment data................................................................................................................... 52 Annex 1.4 Survey of district office public officials .................................................................................. 54 Annex 2 Implementation timeline .............................................................................................................. 57 Annex 3 Questionnaire ............................................................................................................................... 59 List of figures, boxes and tables Figure 1. Government Effectiveness in EU Member States ........................................................................ 7 Figure 2. A Production Function for Public Administration ...................................................................... 11 Figure 3. Distribution of Fabasoft Cases Over Time .................................................................................. 13 Figure 4. Distribution of Business Licensing Cases Across Districts .......................................................... 14 Figure 5. Percentage Share of Successful Hires Across Districts ............................................................... 15 Figure 6. Responses from District Officials Across Districts ...................................................................... 18 Figure 7. Individual Monthly productivity of the Business License Certification Task in 2019 ............... 22 Figure 8. Individual Monthly Productivity of the “Environmental Department” in 2020 ....................... 25 Figure 9. Productivity Across all Core Agenda Cases in the Environment Department in 2020 .............. 28 Figure 10. Productivity of the Task “Statement on Environmental Impact Assessment (EIA)” in the Environment Department in 2020 ............................................................................................................. 30 Figure 11. Management Quality Across District Offices ........................................................................... 33 Figure 12. Non-managerial Perceptions Across District Offices ............................................................... 34 Figure 13. Association Between Management Quality and Non-managerial Perceptions of Management .................................................................................................................................................................... 35 Figure 14. Correlation Between Management Quality and Cases Per Official (Left), and Duration of Cases (Right) ......................................................................................................................................................... 36 Figure 15. Correlation Between Non-managerial Perceptions of Management and Cases Per Official . 37 Figure 16. Number of External Applicants Per Permanent Job Opening Across Districts (2017-2021) ... 39 Figure 17. Correlation Between Public Sector Wages and Competition for Managerial Positions ......... 40 Figure 18. Nominal Wages Across Districts (2020) Fitted into the Wage Tariffs (WT) of District Offices .................................................................................................................................................................... 42 Table 1. Implementation timeline .............................................................................................................. 57 Table 2. Questionnaire for survey of district office public officials ............................................................ 59 Acknowledgements This report is the product of the Bureaucracy Lab, a partnership between the Governance Global Practice and the Development Impact Evaluation Department (DIME) Research Group. It is part of a multi-country collaboration with the European Commission (EC), funded via the Part II Europe 2020 Programmatic Single-Donor Trust Fund with the EC (TF073353), the objective of which is to empirically understand the personnel determinants of, and mechanisms influencing, productivity in public administration and service delivery units in EU Member States. The principal investigators of this project are Daniel Rogger, Development Impact Evaluation Department (DIME), World Bank; Zahid Hasnain, Governance Global Practice, World Bank; Christian Schuster, University College London (UCL); and Gianmarco León-Ciliotta, Pompeu Fabra University (UPF). Patrik Jankovic (DIME) and Ayesha Khurshid (DIME) provided research assistance and field management of the project. The report was prepared under the supervision of Arianna Legovini (Director, DIME) and Fabian Seiderer (Practice Manager, Governance). The research team would like to thank the Ministry of Interior (MoI) and within that in particular the Institute for Public and Security analyses (IPSA), the director Tomáš Černěnko, department manager Marek Mathias, and former counterparts Markéta Tomaga, Alfonz Aczel and Veronika Ferčíková. Finally, we are grateful for the close collaboration and continuous support of officials from DG REGIO and DG REFORM of the EC, particularly Lewis Dijkstra, Philippe Monfort, and Mina Shoylekova. i Executive summary Using a wide range of productivity indicators, we see that there is a substantial regional aspect to productivity in Slovakia. Being a citizen in one district rather than another substantially shifts the quality of service you receive from your district government. However, there is more variation within a district than across districts, implying that citizens within a specific district are served differentially depending on the official that treats their case. Main findings  Slovakia has high-quality and granular administrative data on public sector productivity that it could repurpose towards the use of analytics for improved management of the public service. This report showcases some of the insights that can be gleaned from repurposing government administrative data for analytical purposes. It also points out that Slovakia is only a step away from truly frontier government analytics. The government has not built the analytical infrastructure to capitalize on its data assets, though this could be done at very low cost. The report therefore ends with recommendations as to how the government could move to frontier government analytics at low cost.  This report employs a diverse range of data sources to examine district-level variations in public sector productivity in Slovakia. It leverages administrative data to measure productivity from the Fabasoft and Cezir data systems that provide detailed insights into case management and business licensing processes. Employment data, sourced from the government's job portal, sheds light on competition for public sector jobs. Additionally, a survey of district office public officials captures their management practices and attitudes. By incorporating these varied data sources, the report offers a comprehensive understanding of productivity factors and management dynamics within district offices, enabling a complex and informed assessment of the determinants of public sector productivity.  Our focus is on the variation in productivity across government district offices. This report utilizes administrative microdata to showcases a wide variation in productivity measures (for example, the number of administrative cases processed) across district offices. More than twice as many cases were completed by each official in the most productive offices compared to the least productive over the period we study. Such variation does not exhibit clear geographic patterns but rather across the country there are more and less productive districts on a range of margins.  Different districts are better at some tasks than others, and the ranking of districts varies substantially depending on the specific task. Thus, all districts can be said to have strengths and weaknesses and thus lessons for the wider government. Targeting a specific district as ‘productive’ or otherwise likely doesn’t take a granular view of what aspects of government work they are particularly productive at, and where there is room for improvements. The government can support its local governments broadly with stimulating reform based on analytics of its administrative data. 2  There is more variation in public sector productivity within district offices than between them. In any single office, there are individuals as productive as the most productive office, and those as productive as the least productive office. The composition of the team within a district office is what determines the average productivity of that office. Thus, there is a huge amount of productivity determined by individuals and their capabilities. Using administrative data to identify these differences across individuals is a rich foundation for performance discussions grounded in objective indicators of performance.  The fact that individuals play a key role in productivity differences across districts implies that recruitment, personnel management, and reduced turnover, are key levers for better productivity at the district level. We find a limited relationship between the nature of management practices as measured by the World Management Survey and the productivity measures outlined above. One interpretation of this finding is that management in Slovakia’s public service is relatively passive, and its characteristics have limited impacts on productivity. This is an important finding in itself but would need to be substantiated by further evidence.  By sourcing micro-data on the competitiveness of local government employment, we identify that many open positions have limited competition and thus small increases in competitiveness could have a substantial impact on the quality of hires. Similarly, we observe from the productivity data that there is a substantial productivity loss in the first six to eight months of an official’s tenure compared with productivity levels of officials who have longer tenure at the district office as they become used to the new work environment. This implies a substantial gain from greater orientation, coaching and mentoring in the early months of careers in new roles in Slovakia’s district governments. It also indicates the secondary cost of turnover, and thus highlights the benefits of management practices that keep staff in post. The government’s administrative data allows the analyst to identify those managers most successful in competitive recruitment, orientation, and limiting turnover. Learning from these managers seems a low cost and effective means of reducing these productivity costs.  An interesting finding is that competition for managerial jobs seems to increase the productivity of districts. A natural interpretation of this finding is that increased competition for managerial positions increases the raw talent of managers and/or their effort given competition for their jobs. Recommendations Capitalize on the Insights from Administrative Data to Target Areas of Productivity Loss  Present the variation in productivity to district managers and facilitate discussions within and across district offices as to where bottlenecks to improved service delivery might be. Use survey data to showcase where good-practice management is implemented in specific departmental teams. 3  Facilitate further systematic analysis of the productivity data sources used in the report i.e., Cezir and Fabasoft data. Where possible, embed the possibility to undertake analytics such as those outlined in this report into administrative systems with an aim that the data will be used for internal management. Build a Stronger Architecture for Government Analytics  Design future administrative data systems with the aim that the data will be used for internal analysis and reporting goals. Ensure data processing protocols are clearly documented to ensure a clarity of what data is in administrative systems. Fabasoft and Cezir data sources provide rich estimates of productivity, but the quality of this data requires further validation (see notes in Annexes).  Work on the interconnection of multiple datasets into the management information systems . Connect data from administrative systems to HR systems based on personal identifiers and make anonymized data available to analytics teams.  Undertake a regular and harmonized public servants survey. For public servants surveys to gain effective response rates, they need to be embedded within an ecosystem of political commitment and reform credibility (Wells et al. 2022). Such surveys should be run by a specialized team at a central institution with an overarching mandate for personnel management in district governments.  Embed randomized control trials in the rollout of new reforms to the public administration. Impact evaluations can support future policy reforms by mapping out the causal mechanism behind the impact of policies, and help explain the process through which policies improve outputs such as productivity. 4 Chapter 1 Strengthening Government Institutions in Slovakia Improving government capability is one of the key challenges of economic development. Such capabilities are the critical mediator in the state's ability to effectively implement policy, and efficiently provide regulation, infrastructure provision, and service delivery. As such, measuring those capabilities and redesigning public sector institutions to be responsive to related findings is at the heart of a better public service. Effective management of the public sector has shown to be strongly related with improved service delivery.1 In Italy, Fenizia (2021) shows that better public sector management raises productivity among public officials in the Social Security Agency by 10 percent. 2 Similarly, effective public administration leads to fiscal savings and increased productivity . In Russia, Best et al. (2021) show how if the worst-performing 20 percent of bureaucrats can be made as effective as the median bureaucrat, the Russian government would save 10 percent of its procurement costs, or 0.9 percent of non-resource GDP.3 In Nigeria, a one standard deviation increase in the quality of management can lead to a 32 percent increase in project completion rates (Rasul and Rogger, 2018). Each of these studies shows the power of using micro-data for investigating the quality of government functioning individual-by-individual, work unit-by-work unit. Documenting the successes and weaknesses in a particular government administration can therefore lead to substantial gains in service delivery and financial savings for the government. However, such evidence-based management of the public sector requires data at a granular level and appropriate analysis linked to management decisions. As will be seen in this report, Slovakia has excellent micro-data on government functioning, and thus the opportunity for substantial impacts from government analytics. Its weakness is in capitalizing on that opportunity and undertaking the relevant analysis to inform personnel policy. The report continues as follows. This section outlines the context of Slovakia’s government and existing micro-evidence on the quality of its institutions. Section 2 outlines the methodology of the analysis undertaken in the collaboration between the Ministry of Interior and the World Bank’s Bureaucracy Lab. Section 3 outlines the results we have been able to generate within the constraints faced by the project, and Section 4 outlines these constraints and opportunities for strengthening the work with some additional permissions and information. These constraints to 1 World Bank. (2004). World Development Report 2004: Making Services Work for Poor People . Washington D.C. 2 Fenizia, A. (2022). Managers and Productivity in the Public Sector. Econometrica, 90(3), 1063-1084. 3 Best, M. C., Hjort, J., & Szakonyi, D. (2017). Individuals and Organizations as Sources of State Effectiveness. National Bureau of Economic Research. 5 truly frontier analytics can be released by government action. It is thus in the hands of the government to take the final step towards enabling truly frontier analytics to shape the future of service delivery in Slovakia. 1.1 The context The quality of governance in Slovakia is not at par with its European counterparts. Slovakia continues to be ranked at the lower end of the European spectrum on government effectiveness, trust in government and public administration, and implementation capacity. In terms of the perception of citizens, only 4 out of 10 citizens regard the quality of public services to be good or very good relative to 9 out of 10 in the Netherlands and 2 out of 10 in Greece. 4 The World Bank ranks Slovakia 22nd out of 27 EU countries in the Government Effectiveness Index that reflects perceptions on the quality of public and civil services (WGI, 2021). Furthermore, Slovakia was ranked 20th on the European Quality of Governance Index (EQI), which measures the quality of public services, corruption, and impartiality in the treatment of citizens (see Figure 1, Panel A). 5 4 Mackie, I., Moretti, C., & Stimpson, A. (2020). Public Administrations in the EU Member States: 2020 Overview. Brussels: European Commission. doi:10.2887/793815 5 Charron, N., Lapuente, V., & Bauhr, M. (2021). Sub-national Quality of Government in EU Member States: Presenting the 2021 European Quality of Government Index and its Relationship with Covid-19 Indicators . University of Gothenburg. 6 Figure 1. Government Effectiveness in EU Member States Panel A: Variation Across the European Union Source: EQI (2021) Panel B: Variation Across Slovakia 7 Source: EQI (2021) While there exists significant variation in the EQI across European nations, there also exists variation within Slovakia, with Bratislava ranked as the worst performing region on the EQI and Government Effectiveness (WGI, 2021) (see Figure 1, Panel B) in the country. 6 This variation spans a large part of the distribution of European scores, with the best ranked region (Stredné Slovensko ranked 138 out of the 208 NUTS2 regions) scoring at a similar level as the western Poland regions (Slaskie or Wielkopolskie), and the worst ranked region (Bratislava ranked 164 out of 208) scoring at a similar level as the Greek region Peloponnisos or Hungarian region Pest . Thus, the perception of governance in Slovakia is low relative to European levels on average, but also varies significantly within its borders. The question is to what extent this perception is matched by granular measurement of government itself. The quality of governance across Slovakia matters for service delivery, but also for the quality of employment. Slovakia’s public sector is responsible for a significant proportion of the country’s employment. 25 percent of all formal employment in Slovakia is in the public sector, with 39 percent of public employees working in the public administration and security administration alone (Worldwide Bureaucracy Indicators, 2021). 7 As such, understanding the work environment and productivity of government across Slovakia is important for improving the productivity of a significant proportion of the workforce. 6 Charron, N., Lapuente, V., & Bauhr, M. (2021). Sub-national Quality of Government in EU Member States: Presenting the 2021 European Quality of Government Index and its Relationship with Covid-19 Indicators . University of Gothenburg. 7 Worldwide Bureaucracy Indicators Dashboard. worldbank.org. (2021). World Bank. https://www.worldbank.org/en/data/interactive/2019/05/21/worldwide-bureaucracy-indicators-dashboard. 8 Despite the scale and reach of the public administration in Slovakia, there exists little detailed empirical evidence on public sector productivity in Slovakia. Studies have reported on the Slovak government’s performance on a range of general attributes such as accountability, corruption, and rule of law.8,9 While these point to various challenges that the public administration faces, there exists little cross-region evidence on the productivity and work environment of public officials and managers who are responsible for regulating, financing, and monitoring the work of service providers and other business and citizen-facing public officials. To provide greater insight into the characteristics of Slovakia’s public administration, the Government of Slovakia, specifically the Ministry of Interior (MoI), invited the World Bank’s (WB) Bureaucracy Lab (BL) to undertake an empirical diagnosis of public sector productivity in Slovakia. Specifically, the focus of this exercise was to understand the variation in the productivity and characteristics of district offices of Slovakia, given their key role in citizen centric service delivery in the country. 1.2 District offices in Slovakia Existing measures of the quality of public administration, outlined above, do not focus on the study of variation within countries. Commonly used measures of the quality of public administration are based on expert opinions and provide aggregate assessments, typically at the national level. These national aggregates hide the considerable heterogeneity in public administration capabilities within countries— the management environment, and the abilities and motivation of public officials, varies considerably across organizations within countries. Understanding these practices at local levels helps to break down the complex relationships amongst various determinants of governance and gives more latitude to probe these relationships for targeted interventions. Slovakia has 72 district offices playing a key role in the coordination and oversight of local governments. Originally, the country had 50 regional offices, 49 of which became district offices, and 23 new offices were created to improve services for citizens.10 Since 2013, district offices in Slovakia have been performing state administration tasks, such as civil protection, economic 8 Charron, N., Lapuente, V., & Bauhr, M. (2021). Sub-national Quality of Government in EU Member States: Presenting the 2021 European Quality of Government Index and its Relationship with Covid-19 Indicators . University of Gothenburg. 9 Mackie, I., Moretti, C., & Stimpson, A. (2020). Public Administrations in the EU Member States: 2020 Overview. Brussels: European Commission. doi:10.2887/793815 10 ESO - Efektívna, Spoľahlivá a Otvorená verejná správa, Ministerstvo vnútra Slovenskej republiky . Minv.sk. (2022). https://www.minv.sk/?eso-efektivna-spolahliva-otvorena-verejna-sprava. 9 mobilization, and regional development among others11. Given the central role of district offices, the productivity of these offices has an immediate impact on improved service delivery for citizens. With this in mind, the objectives of this report are twofold. Firstly, to assess current data available to identify, understand, and explain district-level variations in public sector productivity across district offices in Slovakia. Secondly, to showcase its use for ‘government analytics’ (Rogger and Schuster (eds.), forthcoming). Slovakia has high-quality and granular administrative data on public sector productivity that it could repurpose towards the use of analytics for improved management of the public service. In addition to measuring productivity, the determinants of productivity such as management quality and personnel will be used to outline areas of differences and potential improvements. By leveraging Slovakia's rich administrative data, it becomes possible to gain nuanced insights and identify strategies for enhancing the efficiency and effectiveness of district offices. The findings from this analysis will have practical implications for optimizing service delivery and fostering evidence-based organizational and personnel management practices in the public sector. This exercise has been narrowed down to assess district offices in Slovakia for two reasons. First, district offices play a crucial role in service delivery in Slovakia. Understanding potential areas for improvement could have an immediate impact on service delivery. Second, this activity is part of a broader collaboration between the European Commission (EC) and the WB’s BL, funded by the Part II Europe 2020 Programmatic Single-Donor Trust Fund with the EC, on measuring organizational and personnel determinants of public administration productivity. The activity seeks to work with five European Union member states (Slovakia being one of them) to generate micro-level data to support evidence-based organizational and personnel management reforms. The report is structured to give a comprehensive picture about the governance, productivity and human resource practices in Slovakia’s district offices. Chapter 2 sets forth the conceptual framework used to study the challenge and to design the survey. Chapter 3 charts the results from the study. Opportunities for further analytics constitute Chapter 4, with examples of possible avenues of analyses. 11 OECD. (2020). Regulatory Policy in the Slovak Republic: Towards Future-Proof Regulation , OECD Reviews of Regulatory Reform, OECD Publishing. Paris. https://doi.org/10.1787/ce95a880-en. 10 Chapter 2 Empirical approach This report uses a production function of public sector productivity (see Figure 2) to outline and help measure key factors that determine productivity.12 A production function is the process by which “inputs,” are converted to “outputs” and ultimately outcomes that citizens care about. Inputs may include human and financial resources available to a public sector organization (such as infrastructure, number of staff, and goods and materials). Inputs are converted into outputs by management practices (such as performance management practices) and public or organizational policies (which define organizational objectives and work procedures). Whether inputs effectively convert into outputs is moderated by exogenous factors (such as the political environment) and mediated by the attitudes and behaviors of public officials. Figure 2. A Production Function for Public Administration Source: Hasnain et al. (2019) Since attitudes and behaviors are antecedents of outputs, they are critical in understanding public sector productivity. These attitudes and behaviors are influenced by both the quality of inputs, such as skills and type of personnel selected, and the technology to combine these inputs such as management practices. Selection mechanisms that attract and screen for intrinsically motivated workers, in addition to those with the necessary skills, and management practices such as goal setting, performance feedback, and providing staff the necessary autonomy to carry out their daily tasks are important. 12 Hasnain, Z., Rogger, D., Walker, D., Kay, K., & Shi, R. (2019). Innovating Bureaucracy for a More Capable Government. Washington D.C.: World Bank. http://documents.worldbank.org/curated/en/249891549999073918/Innovating-Bureaucracy-for-a-More-Capable- Government 11 Measures of these components can be sourced from a number of administrative and primary sources. This report aims to showcase measures of three of the components detailed in the production function to showcase how one can understand and explain the variation in public sector productivity within Slovakia. Our intention is not to produce a complete picture of the production function, but rather illustrate how, in a Slovakian context, analytics of the functioning of government might be done. First, the project uses micro-level administrative data already aggregated into centralized databases to measure public sector productivity. It does so by focusing on key elements of district level production – the processing of administrative cases. Such measures clearly link to the ‘outputs’ pillar of the production function. Second, the project collects specialized primary survey data to measure the attitudes of public officials focused on management. Such measurement showcases how one might collect data on the ‘technology’ and ‘culture’ pillars of the public sector production function. Third, the project returns to administrative data, but this time does not use data already aggregated into a centralized database, but rather which can be collected automatically from independent governments. Specifically, we collect data on recruitment into district government. Such primary data provides further information on the internal and external labor markets of the government, enabling the measurement of elements of the ‘inputs’ pillar. These data sources are discussed in detail next. 2.1 Administrative data sources We focus our analysis of the production function of Slovakia’s public sector on three key sources of secondary data on district offices. Fabasoft and Cezir are two administrative datasets that were shared with the WB by the MoI. To complement these data sources, the WB constructed an employment dataset from the government of Slovakia’s centralized job portal by extracting information on each job posting on the portal. Fabasoft is a central case-management administrative system of the government institutions in Slovakia. The dataset used in this analysis is from the system of district offices, it contains over 6.5 million observations between January 2015 to April 2021. The dataset includes cases related to business procedures across 13 different departments. There are more than 750 “registration marks” (types of cases) in Fabasoft of which the 20 most used on district offices cover 76 percent of all cases in the dataset. In the business licensing department, common cases are related to 12 business license certification and termination. In the environmental department, common cases relate to waste management regulation, nature and landscape protection, and environmental impact assessments. Finally, in the road and transportation department, common cases relate to the technical capability of vehicles, fines, and road permissions. Each observation represents an individual case documented by the public official responsible for that case, allowing for the progress for each case to be tracked from start to finish. The cases are not distinct to any particular agency or department since the Fabasoft system allows all cases from various departments to be systematically entered and tracked. Figure 3 shows an example of the importance of this data source in assessing the progression of cases through time 13. Data collected through Fabasoft is at a very granular level and allows us to assess the productivity of public officials by measuring their efficiency in task completion over time, and across departments and district offices. To capture productivity objectively using this dataset, we take into consideration cases with business license certification cases for 2019 as the last available period. The average monthly cases undertaken per public official in 2019 was used to measure productivity using the Fabasoft dataset. Figure 3. Distribution of Fabasoft Cases Over Time Source: Fabasoft data The Cezir dataset provides information on the progression of business licensing cases, across 49 district offices. It consists of over a million observations that represent business licensing cases being processed from January 2015 to December 2019. 80 percent of these cases can be linked to the Fabasoft IT system. Since the Fabasoft system consists of employee identifiers, the analysis can be extended to the individual public official. Figure 4 depicts the proportion of cases across 13 For more information on Fabasoft, see Annex 1.1. 13 7 key tasks within the Cezir data in 201914. It showcases dispersion in the type of tasks allocated to public officials across district offices. Understanding how this dispersion in tasks is reflected in the productivity of public officials is a key step in identifying avenues to enhance productivity. The average monthly number of closed business licensing cases per public official in 2019 is used as the measure of productivity using this dataset. Figure 4. Distribution of Business Licensing Cases Across Districts Source: Cezir data Although the Fabasoft and Cezir data sets present a rich picture of public sector productivity, there are limitations in their ‘codebook’, and so we are restricted to using those variables for which there is sufficient definitional information in the analysis that underlies this report. Employment data: Public officials in Slovakia are recruited through job postings advertised on the centralized public job portal. To supplement our empirical analysis, we scraped job postings 14 For more information on Cezir, see Annex 1.2. 14 from this portal that were posted between June 2017 and November 2021. While this dataset consisted of 27,766 job postings overall, it was restricted to 4,250 postings for the recruitment of district office officials. Figure 5 shows a statistic calculated from our analysis of hiring procedure, specifically the proportion of adverts that led to a successful hire across district offices.15 Understanding employment practices is critical in explaining productivity as it affects the selection of high ability staff and, in turn, their productivity. To supplement the analysis of recruitment, unemployment figures and average public sector wages across districts was collected.16 The employment data source in conjunction with public sector wages allows as assessment of regional competitiveness of public sector institutions, particularly for comparisons across the western regions of the country and the capital, Bratislava. The measurement of the employment environment of district office public officials is critical for a well-rounded assessment of public sector productivity given its central role in the selection of staff that are ultimately responsible for the delivery of services to citizens. Figure 5. Percentage Share of Successful Hires Across Districts Source: Government job portal17 15 For more information on the employment data, see Annex 1.3. 16 Both sources of data are available publicly. 17 Slovensko.sk 15 2.2 Survey of district office public officials Towards a well-rounded analysis of the administration of district offices, a survey of district office managers and non-managers was implemented to complement this administrative data. District officials play a key role in the coordination and oversight of local governments. In particular, they manage the execution of state administration and represent the national government. It is pertinent to understand how public officials within these offices are managed, and how these practices influence their work. The survey of district officials was designed through a collaborative and iterative process between the WB and the MoI. Using the extensive experience of the WB Bureaucracy Lab in fielding surveys of public officials, the WB proposed two sets of complementary questionnaires, for managers and non-managers. The questionnaires were focused on understanding the management practices of managers and the attitudes of district office public officials to these practices. Following an internal draft of the online survey, the MoI were asked to provide feedback on the questions. Through this collaborative approach, a questionnaire was finalized that included three questions to managers on their management practices, and to non-managers on the influence of these management practices on their work. The questionnaire is presented in Annex 3. All district office public officials were approached by e-mail to participate in the survey. Data collection was adapted in response to challenges posed by COVID-19 and the survey was implemented online. The survey targeted 5,800 public officials from all 72 district offices and 13 departments. Institutional approvals were obtained for this survey from the MoI, including a letter of support from the MoI. Only after the approvals were received did the MoI reach out to respondents via email with the survey link and instructions on how to complete the survey. The survey was made available to public officials from June 3 to July 21. The MoI provided support during the survey by encouraging participation through frequent reminders (see Annex 3 for a detailed implementation timeline). Following extensive efforts to encourage survey participation, 290 responses were received from district office managers and non-managers. Once the survey was launched, multiple reminders were sent to staff to increase participation within and across district offices. Responses from 290 public officials across 69 offices were received, implying a response rate of 5 percent. The distribution of responses by managers and non-managers across districts is displayed 16 Figure 6. The survey team faced challenges in obtaining responses from public officials, particularly managers. While the low response rate may raise concerns about the reliability of the findings, it is important to note that a sufficient number of observations were received at the district level. This allows for the triangulation of the survey data with the administrative sources discussed previously, enabling a comprehensive analysis at the district office level. While we have responses from either managers or non-managers in 69 districts, information from both groups is available for 35 districts. As such, wherever both sets of responses are used, the analysis is restricted to 35 districts. 17 Figure 6. Responses from District Officials Across Districts Panel A. Responses from district office managers across districts Panel B: Responses from district office non-managers across districts Source: WB survey 18 2.3 Data constraints Access to administrative data and the ability to undertake a survey were key steps in undertaking analysis of Slovakia’s public sector productivity. However, in both cases the constraints to fully capitalizing on the opportunities these approaches provide were significant. The limited centralized understanding of features of the key administrative data sets generated by local government, inability to combine them with other administrative data sets, and limited capabilities to raise response rates to the survey through managerial intervention, indicate that the government is ‘one step away’ from a data ecosystem that could be used for effective data- informed management. Despite the challenges in monitoring effort and output in public sector around the world, this report utilizes rich administrative data from the Fabasoft and Cezir data sources to provide estimates of productivity across district offices in Slovakia. However, the quality of this data requires validation. Measuring productivity using the granular data collected through these systems simplifies the measurement problem identified above by allowing the calculation of completed cases per employee and district office and the time spent on each case. However, it remains difficult to solely rely on these administrative systems, especially given the possibility of diverse management practices across 72 district offices and 13 departments. In particular, critical information such as the usage of these systems by public officials across offices, the workload captured by the systems, and the quality and complexity of cases is not fully known. The research team simply could not source this information from within the government and its revelation would substantially strengthen our understanding of the value of these data sets. Since the Fabasoft and Cezir systems were not inherently designed to measure and track productivity, the validation of these data sources and their triangulation with other determinants of productivity is necessary. By better understanding and linking existing administrative data sets, and routinizing and encouraging complementary survey activities, the usefulness of the underlying data would be substantially strengthened. However, given the richness of what was collected, there is still a number of useful insights that can be gained from the data collected, which we turn to now. 19 Chapter 3 Public Sector Productivity Across Slovakia and Its Determinants This chapter lays out the findings from these data as an example of how administrative and survey data can provide a window into government functioning. The first section outlines the productivity of public officials using data from the Fabasoft and Cezir datasets. The second section documents the management quality at the district level—whether management quality varies across districts and how this relates to how non-managers perceive these management practices. Using secondary sources of data, the third section further details the personnel drivers of productivity and management quality. The fourth section introduces the quality of recruitment of public officials across districts as a key driver of motivation and productivity. Together, the section paints a picture of productivity in Slovakia’s district governments and its key determinants, a summary of which is presented in the fifth section. 3.1 Productivity Slovakia’s rich case progression data was used to measure productivity across district offices.18 While measuring productivity of public officials is a difficult task given the dearth of quantifiable outputs, the quality and speed of task completion can be a useful measure (and similar to frontier measures of productivity in the economics literatures; see for example Fenizia, 2021). The Fabasoft and Cezir systems contain rich information on each task that public officials perform across departments and district offices. This allows us to track the performance of a public official by the task assigned to them to understand the extent of their productivity. With the available data, we measure productivity at the district office level using two data sources and measures:  The average monthly number of closed business licensing cases per public official in 2019 using the Cezir dataset. To ensure sufficient comparability across districts, the particular task for which we measure productivity is the processing of business licenses.  The average monthly cases undertaken per public official in 2019 using the Fabasoft dataset. To capture productivity objectively using this dataset, we take into consideration cases with business license certification cases for 2019 as the last available period. The weakness of this approach is that not all positions handle business license certification tasks. As such, we are unable to capture all public officials within a district office or only capture a fraction of a public official’s workload. Furthermore, since this dataset provides information on cases across departments, we present productivity for the Environment department that is common across 49 district offices. 18 For an analysis of productivity using Fabasoft data across departments, please see Annex 1.1. 20 These are useful measures of productivity because of the standardized nature of many of the positions at the district level. All of the employees we study in the case of business licenses work dominantly on the business license certifications.19 Across both measures, a wide variation in productivity across district offices is observed. More than twice as many cases were completed by each official in the most productive offices compared to the least productive. Figure 7 compares the mean monthly productivity (only task business license certification) of individuals in the business licensing department across districts. Panel A shows this variation across offices, highlighting how much variation there is within offices across individuals. Panel B shows this variation geographically across the country. The map indicates there is significant variation across districts. As such, the government’s administrative data is a useful tool to target accountability, coaching, mentoring, capacity building and so on. Figure 7 Panel B shows that there is even more variation within district offices. We see a huge amount of variation in productivity of public officials within district offices. A blue rectangle in each figure represents a public official within a district office. Across most offices, the most productive public officials are significantly more productive than the least productive official. In any single office, there are individuals as productive as the most productive office, and those as productive as the least productive office. As depicted in Figure 7, data from the most productive district within the Business Licensing department, Prešov, outlines that individual productivity varies between 1 registration per month to 77, on average. This variation partly points towards significant differences in the workload of public officials, even within district offices. But it also indicates distinct capabilities. Where productivity is low, backlogs of cases arise and response times lengthen.20 The composition of the team is what determines the average for a district. Thus, there is a huge amount of productivity determined by individuals and their capabilities. 19 One indicator of the importance of these positions to total staffing is that the number of employee IDs from Cezir (Figure 7) is almost equal to the number of positions in the department (excluding unit of control at 8 regional DOs). Most staff are undertaking some processing at least. 20 Access to data on the demographic characteristics of public officials within offices would allow us to assess trends in productivity across different groups of public officials given the extent of variation within offices. 21 Figure 7. Individual Monthly productivity of the Business License Certification Task in 2019 Panel A: Variation Displayed Across Offices and Individuals 22 Panel B: Geographic Variation Across Slovakia Panel C: Temporal Variation Across Time: Excess Duration (in days) for Months in Office controlling for Calendar Month, Year and Registration Mark Source: Cezir dataset 23 The final panel of Figure 7 – Panel C, shows how productivity varies across time for a new staff member. We see that in early months the ‘excess’ duration (in days) from having limited experience in the office is high. A case takes almost 30 days more when processed by a staff member in their first month in office relative to the average for the office. However, by the end of the first year, the staff member is consistently more productive (less duration than the average). This implies that there is a ‘productivity loss’ from having new, inexperienced, staff, as they take longer to process cases. This implies that staff turnover has very identifiable costs – illustrated by the excess duration in the first 8 months of the new staff’s tenure. The mean monthly productivity in Figure 8 captures the all-task workload of employees in the Environmental Department. The task structure of environmental departments across the 72 districts is similar, so it is a good case to display individual participation in team productivity (orange triangles) and it captures the number of employees well. We once again see significant geographic variation across regions (Panel B) but this is dwarfed by the variation within each office. The richness of the micro-data allows us insights into many aspects of what makes districts productive or otherwise. For example, for the business licensing department (described in Figure 7):  The dispersion of individual productivity does not depend on the size of the department, but on the broader team’s productivity.  There is a high correlation between the individual top-scorer in terms of productivity and the team’s productivity (R-squared ~75%). This could be understood as the positive effect of “superstars”.  Regionally, the highest productivity of the business license certifications in 3 neighboring districts are in eastern Slovakia (Prešov, Vranov nad Topľou and Kežmarok) and in the southwest Slovakia around the capital Bratislava, Malacky and Senec and also in the district of Komárno.  The most consistent high-productivity districts are Malacky (MA), Komárno (KN) and Prešov (PO) with 5 times ranked at the top 5 annual rankings. These are followed by Bratislava (BA), Námestovo (NO) with scores and districts Prievidza (PD), Vranov and Topľou (VT) with 2 rankings within the top 5 in the last five years. Similarly, for the environment department (described in Figure 8):  The best district offices in the first 5 places of productivity are very consistent. In the last 5 years in the top 5 have been districts Malacky (MA), Nováky (NO) and Partizánske (PE) every year, and only once have been out of top 5 districts Dunajská streda (DS) and Senec (SC). 24  The dispersion of individual productivity depends on the size of the department (R2 = 13%), implying that scale matters in this sector more than in business licensing.  There is a high correlation between the individual top-scorer productivity and team productivity (R-squared ~53%). This could again be understood as the positive effect of “superstars”. Figure 8. Individual Monthly Productivity of the “Environmental Department” in 2020 25 Panel A: Variation Displayed Across Offices and Individuals Panel B: Geographic Variation Across Slovakia Source: Fabasoft dataset 26 Alternative measures of productivity can be generated from Slovakia’s administrative data, and though not our core variables of interest, we showcase them here. In Figure 9 and Figure 10 we present indicators of productivity as the number of cases that must be processed per position (an indicator of the demands on individual offices) and the duration (in days) of case processing. Panel A of Figure 9 presents the number of core environment department cases that must be dealt with by a district office. The number of cases per position indicates the workload driven by local demand. We find this to only be partially correlated with the productivity indicators described above, implying that some offices are more productive despite having more or less work. Panel B of Figure 9 showcases another way to conceive of productivity – the duration of the cases undertaken by district officials. This variable has a direct relationship with the case completion assessment that we investigated above. The more cases an officer completes in a month, the less time the applicant will have to wait for a resolution of their case. However, these variables are not corollaries of each other, and once again we can see that the variation across districts is distinct to that for the other variables. So once again we find evidence that distinct aspects of service delivery appear to be stronger in some districts than others, but distinct districts are stronger along distinct dimensions. 27 Figure 9. Productivity Across all Core Agenda Cases in the Environment Department in 2020 Panel A: Number of the core agenda cases (registration marks starting “Z”) per employee- positions of the department Panel B: Mean duration of the core agenda cases (registration marks starting “Z”) Source: Fabasoft dataset One might also want to make comparisons using a very homogenous task, to ensure comparability across jurisdictions. We can focus on very specific types of task undertaken at the district offices. Among the Environmental department case types, in Figure 10, we focus on one 28 task named “Statement on Environmental Impact Assessment (EIA)” which we understand should be a highly standardized administrative procedure across districts. We expect the number of EIAs depend on the density of protected nature, the real estate, or industrial projects and the mean duration of statement creation as an impact of the EIA complexity of stakeholders included. Panel A shows the distribution of this case varies across the country, but all districts process at least some of these assessments. Panel B then showcases how many of these cases are processed over a period longer than 100 days. Taken as a professional threshold, the map in Panel B is thus an indicator of service failure for a very specific type of case. 29 Figure 10. Productivity of the Task “Statement on Environmental Impact Assessment (EIA)” in the Environment Department in 2020 Panel A: Number of the cases “Statement on EIA” (registration marks starting “Z”) per employee-positions of the department Panel B: Mean duration of cases “Statement on EIA” (registration marks starting “Z”) Source: WB survey Thus, using a wide range of productivity indicators, we see that there is a substantial regional aspect to productivity in Slovakia. Being a citizen in one district rather than another substantially shifts the quality of service you receive from your district government. However, there is more 30 variation within a district than across districts, implying that citizens within a specific district are served differentially depending on the official that treats their case. The fact that productivity varies at such a granular level in Slovakia indicates that administrative data sets such as the Fabasoft and Cezir data are a vital tool in the effort to identify bottlenecks and good practices. Without them, crude simplifications of district achievements would likely be made, rather than precise diagnostics of what each district is doing well, and what it is not. Systematic analysis of these indicators will lead to communication of the best practices from other districts, and other officials within the same district. But all this requires administrative data is repurposed for analytics. Then such analytics could be put in front of each and every public sector manager in Slovakia. If managers were confronted with indicators ranked across comparable departments, this might lead to increased motivation for better results. And now we turn to their own qualities. 3.2 Management quality As outlined above, the quality of work varies considerably across Slovakia’s districts and within offices, implying that there are features of the individual and their work environment that shape their capacity for effective service delivery. A key element of the public sector work environment is the nature and qualities of management under which a public official works. This section of the report showcases how management quality information can complement the analysis of productivity data outlined above, to provide one route towards improving public sector functioning. Management practices such as goal setting, regular monitoring of the achievement of targets, and performance management are correlated with employee productivity in many settings (World Bank, 2019). Studies suggest that practices that support employee performance result in better commitment and higher trust.21 Better management quality has resulted in improved task completion among Ghanaian civil servants, particularly in relation to staff autonomy.22 Similarly, a study with Nigerian civil servants finds that management quality, and autonomy of civil servants in particular, is positively associated with productivity and task completion rates.23 Furthermore, better quality managers were found to raise productivity among Italian civil servants in the Social 21 Whitener, E. M. (2001). Do ‘High Commitment’ Human Resource Practices Affect Employee Commitment? A Cross- Level Analysis Using Hierarchical Linear Modeling. Journal of Management, 27(5), 515-535. DOI:10.1177/014920630102700502 22 Rasul, I., Rogger, D., & Williams, M. J. (2021). Management, Organizational Performance, and Task Clarity: Evidence from Ghana’s Civil Service. Journal of Public Administration Research and Theory, 31(2), 259-277. 23 Rasul, I., & Rogger, D. (2018). Management of Bureaucrats and Public Service Delivery: Evidence from the Nigerian Civil Service. The Economic Journal, 128(608), 413-446. 31 Security Agency by 10 percent.24 This points towards a significant role that management plays in the productivity of public officials elsewhere. Measuring management quality across district offices is therefore a legitimate avenue for us to explore in attempting to explain the variation in productivity across district offices. Overall, Slovakia’s public officials report that management practices in their offices positively influence their work. Over 7 in 10 public officials reported that feedback they receive from their managers helps to improve performance, and over 50 percent of public officials reported receiving appropriate recognition on their performance and sufficient training to support task completion. To measure management quality among Slovakia’s district governments, an index was constructed from relevant questions in the public officials’ survey described in Chapter 2 to provide a standardized measure of management quality. This index was created using questions from the World Management Survey (WMS). The WMS is a survey-based measure produced by academics and used to measure the quality of management in many private and public sector organizations globally.25 Given limitations on the length of the questionnaire, we were only able to incorporate three questions from the WMS into our survey to measure management quality. While this is a limitation, an internal analysis on the components of management quality suggests that these three questions are some of the key drivers of the index on management quality and have been shown to explain significant variation in the management quality index. This index was constructed using standardized responses from managers on their management of organizational performance, underperformance, and dealing with agency- specific problems.26 The index is constructed by first standardizing each individual response to the overall public administration sample mean by subtracting the mean from each individual response and dividing by the standard deviation. This allows responses of public officials to be compared to other public officials. A composite index is then constructed by summing over questions that measure management quality and is divided by that same number. A higher level of any index indicates a stronger belief or practice. For example, a higher management quality index indicates that the public official has better management skills than the average official. 24 Fenizia, A. (2022). Managers and Productivity in the Public Sector. Econometrica, 90(3), 1063-1084. 25 The WMS is a cross-industry, cross country survey to gauge the quality of management across organizations. It is a systematic and standardized measure of management performance across establishments. More information on: https://worldmanagementsurvey.org/ 26 Please see Annex 3 for the questionnaire. 32 We begin by measuring management through asking managers directly. Figure 11 describes the variation in the quality of overall management across Slovakia’s district offices, as assessed by the World Management Survey questions asked to managers. Information on this index was available for 35 districts. The quality of management displays significant dispersion across districts. District offices in central Slovakia were more likely to score lower on management quality compared to other offices (see Figure 11). However, different districts were strong in different areas of management practice. Specific questions on management quality show dispersion across district offices that vary depending on the specific practice (see Annex 1.4 for the full elaboration of questions and corresponding variation across districts). In particular, districts in central Slovakia were more likely to score lower on questions on the management of organizational performance and problems, compared to the question on the performance of public officials. This showcases the extent to which management is decentralized or the way in which management practices are inconsistently implemented across offices. Figure 11. Management Quality Across District Offices Source: WB survey 33 We also asked public officials in non-managerial roles about the quality of management they experienced.27 This alternative approach to measuring management allows us to reduce any bias from managerial assessment of their own practices. We standardized responses from non- managerial public officials and created an index using the questions directed at them. This index represents the extent to which district level management practices influence the work of public officials, with higher values indicating a more positive influence than lower values. Across these questions, as with management quality as assessed by managers, there was significant dispersion in the responses across district offices (see Annex 1.4 for a breakdown of all questions). Figure 12 showcases significant dispersion in this index across district offices. Figure 12. Non-managerial Perceptions Across District Offices Source: WB survey Management practices assessed by manager and non-managers are positively correlated between management and non-management indices (see Figure 13). This is a useful validation check on the measurements we have made. But it also allows us to assess the nature of management overall in district offices, and identify, by district office, the areas that management is strong and where it is weaker. Overall, we find that Slovakia’s management is weaker on praise, constructive feedback, training and the implementation of a supportive work 27 Please see Annex 3 for the questionnaire. 34 environment. This is evidenced in both the managers and non-managers assessment of managerial practices. Figure 13. Association Between Management Quality and Non-managerial Perceptions of Management Source: WB survey 3.3 The relationship between measures of management and productivity We can assess how our measures of management in Slovakia predict our measures of performance. In other settings, this relationship has been shown to be relatively strong. However, overall we do not find a strong association between measures of management and productivity in this setting. Using manager’s assessments of their own management quality, the management index is weakly and negatively correlated with the number of cases per official (see Figure 14), and positively with the amount of time spent on each case (see Figure 14).28 28 Please see Annex 1.1 for a breakdown of this correlation by department. 35 Figure 14. Correlation Between Management Quality and Cases Per Official (Left), and Duration of Cases (Right) Source: Fabasoft data and WB survey Similarly, using non-managers assessments of management, there is again a marginally negative relationship between management and the productivity of public officials (see Figure 15). However, neither of these associations are statistically significant, and in general the relationship seems to be relatively limited. 36 Figure 15. Correlation Between Non-managerial Perceptions of Management and Cases Per Official Source: WB survey and Fabasoft dataset What does the weak link between measures of management quality and productivity tell us about Slovakia’s public administration? Particularly in light of studies from other settings where management plays an important role in public sector productivity. One interpretation is that management is relatively passive in Slovakia, such that its particular characteristics have limited impacts on workers efforts. Such an interpretation is bolstered by reports from non-managers in our survey. In districts offices with better management quality, public officials were more likely to report a positive influence of management overall on their work. This implies that stronger management of both organizations and individuals in fostering an environment that positively affects the productivity of staff is an area for development. Since this finding is based on relatively limited data on the strength of management (rather than its characteristics), there is a need for further analysis of attitudes towards how impactful managers are across offices that will support these findings and provide key avenues to translate such attitudes into feasible personnel interventions within the Slovakian public service. 37 3.4 Labor markets Beyond management, another potential route to understanding the variation in productivity we observe is variation in the quality of recruitment of public officials across districts. The selection of officials into the public service has direct implications on the quality of service delivery through both their capabilities and the level of effort they provide. The recruitment of skilled public officials through a transparent and open process is necessary to find employees that have the skills to provide effective service delivery.29 The intensity with which recruitment occurs also impacts the motivation of existing public official by encouraging public officials to put in more effort, as it is indicative of the competition for their positions. This effort may arise from the recognition that their roles are highly valued, and that their performance will be now closely scrutinized. Furthermore, the competitive atmosphere may foster a sense of accountability and promotes a merit-based culture. The resulting motivation leads to increased engagement and is in turn a direct determinant of public sector productivity. To understand the strength and quality of the recruitment process, we extended our data collection approach to scrape publicly available data from the government’s centralized job portal. Such data provides a measure of the extent of competition for public service jobs in Slovakia. For each district office, competition is defined as the average number of applicants for each position, with a higher number of applicants indicating more competition. The data we collected indicates variation in competition for jobs across district offices. The district with the highest competition has approximately 7 times as many applicants per position compared to the district with the lowest competition, on average. Competition for non- managerial job openings is consistently higher compared to managerial positions within district offices (see Figure 16). This variation could indicate differences in the quality of potential applicants across and within districts, with managers only able to recruit from a smaller pool having less choice of talent. 29 OECD. (2019). Recommendation of the Council on Public Service Leadership and Capability. OECD. 38 Figure 16. Number of External Applicants Per Permanent Job Opening Across Districts (2017- 2021) Source: Authors calculations based on data from g overnment job portal30 Interestingly, we find that the productivity of district offices increases as competition for managerial positions increases. As competition of managerial positions in district offices increases, the number of cases completed per official increases and the average duration of each case decreases. Public officials in district offices with more competition for managerial positions were completing more than twice as many cases as district offices with lower competition (see Figure 17). Higher competition, and the resulting quality of public sector managers could explain higher productivity of district offices, where staff could be more motivated, incentivized and have higher autonomy and flexibility to perform better. Figure 17. Correlation Between Competition for Managerial Positions and Cases Per Position (Left) and Duration of Cases (Right) 30 Slovensko.sk 39 Source: Government job portal31 and Fabasoft dataset This could be seen as a reinforcement of the interpretation of our earlier management findings. That the average manager in Slovakia’s public service is passive in their management duties. Once competition increases their level of talent and effort, management becomes increasingly important. How can we induce greater competition for management positions? District offices experienced higher competition where public sector wages were higher . Information on average public sectors wages complements these findings as there exists a significant dispersion in public sector wages across district offices. Districts with higher public sector wages were more likely to experience higher competition for managerial positions (see Figure 17). 32 Furthermore, since wages of public officials are standardized across scales, it is useful to observe variation in wages of the larger economy in relation to the public sector. The higher management wages should then be offset against the savings they induce from better management and higher productivity. Figure 17. Correlation Between Public Sector Wages and Competition for Managerial Positions 31 Slovensko.sk 32 As these are average public sector wages, it is important to note that such wages are strongly affected by the distribution of public institutions. However, these can be used to support trends in competition. 40 Source: Government job portal33 and SOSR The variation in public sector wage premia within Slovakia causes variation in the competition for managerial talent across the country. Figure 20 depicts the wage scales that public officials have to earn in to earn the average wage in the overall economy. With the 9 th bracket being the highest paid, public officials in Bratislava will have to be in that highest wage bracket to earn the average wage. Figure 18 further showcases the extent of this variation across districts, with significant relative differences in wages between the public and private sectors across districts. This could explain the variation in competition for jobs, as more qualified staff that could earn in higher wage brackets would be more likely to apply for jobs in the private sector. 33 Slovensko.sk 41 Figure 18. Nominal Wages Across Districts (2020) Fitted into the Wage Tariffs (WT) of District Offices Source: SOSR and Wage tariffs of the Ministry of Interior 3.5 Bringing It All Together This section has shown how powerful the insights can be from integrating the productivity data Slovakia holds with other administrative and survey data sets. We find that the productivity of district government is very diverse, both overall and across individual tasks. Every district office has something to learn from its neighbors and beyond. And within district offices, there is even more diversity in productivity. Every district official has something to learn from their colleague. What the micro-data we have described in this report can do, is show officials where they and their neighbors are strong, where they are weaker, and whom they should be learning from. We find that management quality is a weaker predictor of productivity in this setting than in others, and postulate that this implies management in Slovakia’s district governments is relatively passive. The management data we collect indicates that Slovakia’s district managers are rather procedural in their approach and are weaker on building a supportive work environment. The data presents direct next steps for every district manager. We also find that when competition for their positions increases their level of talent and effort, management becomes increasingly important, and impacts productivity positively. We note that investments in better managers (through higher wage offers) should be offset against the savings they bring in terms of productivity and reduced staff turnover. The analysis above implies that small improvements in recruitment practice and wage setting could be transformative for Slovakia’s district office productivity. 42 Chapter 4 Frontiers of Government Analytics Overall, this report has showcased how Slovakia’s administrative data can be used for powerful government analytics. Using the Cezir and Fabasoft data sets, we determined measures of productivity that indicate the substantial variation in the quality of government functioning across the country. Our analysis allowed us to observe how varied the quality of government work is within a single district office, and that management quality is mediated in its effects through recruitment policies. The analysis implies a range of practical next steps and areas for further investigation. Findings from the study suggest that there exists wide variation in the productivity of public officials and its determinants across district offices in Slovakia. The study was designed to showcase the within-country variation in productivity and its determinants through an analysis of administrative data and a survey of public officials. Using data from four sources, this study was able to triangulate the data sources and show that along with productivity, its drivers – management quality, attitudes of public officials and inputs such as skilled labor – vary significantly across offices. It suggests that management may have a relatively passive role in influencing workers' efforts, and is supported by reports from non-managers in our survey, where district offices with better management quality were more likely to report a positive influence of management on their work. Furthermore, it suggests that competition for managerial jobs could result in higher quality managers that in turn improve productivity of district offices. This analytical work is a case study of how the capabilities of public sector agencies can be measured and linked to provide avenues for improvement and further research. There remains much more to be explored using the extensive database of information within the Slovak government. While this WB study has been able to outline an aspect of the public service, there remains more to be explored on how the public administration of Slovakia, especially at local levels, functions. We have observed the rich micro-level data that the public administration in Slovakia collects and updates. Using the right analytical tools to explore this further can provide the government with deeper, more nuanced, insights into various factors of the administration that can result in tailored reforms for better service delivery. Here we provide three key areas that can be explored further using data that the government currently collects:  The wide variance in productivity needs to be explored further through better information on the public officials responsible for cases. While this study showcases dispersion in the productivity of public officials across district offices, an analysis of cases by the type of public officials responsible could provide focused insights into which category of public officials are more productive than others. This can be analyzed by 43 matching the Fabasoft and Cezir datasets with Human Resources (HR) data, allowing for productivity to be tracked and assessed by characteristics of public officials (such as grade, experience, education levels etc.) for targeted insights and improvements.  The association of productivity data with measures of management quality needs further exploration. This study shows an intricate association between management quality and productivity. We see that a standard measure of management quality is not associated with the number of closed cases per official or with the duration of cases. A detailed analysis of Fabasoft data can provide a clearer picture of the robustness of this relationship and how this varies across different categories of officials (for example, see Annex 1.1 for variation across departments). Furthermore, an assessment of the quality of cases could be an area for future work to explain this correlation and identify bottlenecks to the performance of public officials across offices.  Finally, merging employment data with HR data will pave the way to track the performance of officials throughout the duration of their service. The employment dataset constructed by the WB is only the first step in understanding how public officials work and what drives their productivity. Merging the HR data with employment and productivity data will support the measurement of public officials’ productivity in real time throughout their time in service and will provide a robust assessment of how competition for positions affects the productivity of public officials. This will, in turn, help identify and implement measures that can improve their productivity. A key implication of the analysis is that Slovakia is not capitalizing strongly on the administrative microdata it has. The analysis outlined in this report was unable to resolve uncertainties related to 1) the reliability of existing productivity data; and, 2) an ability to connect multiple datasets (productivity, job postings, survey and system of positions) other than at the district/department/month level. These could be solved through a greater availability of information made to a central analytics team, and through the matching of data anonymously at the individual-level. By keeping human resource data separate from the productivity data we showcase, the government is limiting its ability to provide supportive ‘government analytics’ to its managers and leaders. The extent of administrative data collected by the government can be complemented by surveys to shape the public administration at a more granular level. Information on the attitudes, behaviors and motivations of public officials is also an important determinant of productivity. This is where surveys of public officials can be useful to provide a deeper insight, 44 directly from officials, on how the public service is functioning and can be improved. Public sector managers can then make decisions based on a larger evidence base regarding the environment in which they are operating, and the attitudes and behaviors of those who are affected by their decisions, or their perceptions of existing management practices. For public servants surveys to gain effective response rates, they need to be embedded within an ecosystem of political commitment and reform credibility (Wells et al. 2022). The World Bank is now supporting governments across the world to develop such ecosystems within their public services and such services could be extended to Slovakia. Finally, the frontier of government analytics includes the use of randomized control trials to investigate routes to strengthened public administration. In an OECD context, impact evaluation methods are now used to provide rigorous evidence on optimal reform trajectories. Linos (2018)34 experiments with recruitment strategies in the US to improve the diversity of police departments and Rogger and Sibieta (2016) experiments with the co- production of local government environmental services in the UK. Public administrators require empirical evidence to determine if policies are working as intended and to identify improvements. Impact evaluations can support future policy reforms by mapping out the causal mechanism behind the impact of policies, and help explain the process through which policies improve outputs such as productivity. In general, the use of micro-data provides the deepest insights available into the nature and determinants of government functioning, and thus service delivery, within countries. The results of this study highlight the importance of micro-data because of the wide variation in the quality of government functioning we observe in the data we were able to work with. Despite the clear importance of the quality of governance indicators for a country’s growth, it is acutely difficult to measure at the granularity that was achieved in this study. That puts Slovakia one step from the forefront of countries who are capable of undertaking frontier government analytics. The question is whether the country can now take the next step and truly take the analysis outlined here to the frontier. 34 Linos, E. (2018). More Than Public Service: A Field Experiment on Job Advertisements and Diversity in the Police. Journal of Public Administration Research and Theory, 28(1), 67-85. 45 Annex 1 Areas to explore and develop While we present a network of datasets that link data sources across district offices and departments, there remains significant room for a more robust analysis. The main barrier to a nuanced and robust analysis of productivity of district offices is the absence of HR data. HR data can provide information on a range of demographic attributes, roles, responsibilities and appraisals. This data would allow us to evaluate productivity at a more granular level, increasing scope for more reform oriented findings for improved service delivery. HR data has not yet been shared with the WB team. The sections above showcased key findings on public sector productivity and its drivers from this study. Given the extent of data collected through the Fabasoft and Cezir administrative systems, and through the extraction of job postings, we were able to gather more insights into the district offices of the Slovak government. This section presents these findings, and outlines areas where more data can help with further explanation. Annex 1.1 Fabasoft In addition to the analysis of Fabasoft data, the administrative system can also be used for analyses across departments. Figure 21 shows the distribution of Fabasoft cases across departments and Figure 22 depicts the relationship between the number of cases with the number of open cases per position for each department. Overall, we observe a negative relationship between the number of cases per position and the duration of cases. However, this association varies in magnitude and direction across departments. 46 Figure 21. Fabasoft cases by department Source: Fabasoft data 47 Figure 22. Correlation between cases per position and duration of cases by department Source: Fabasoft data With HR data and more information on the employees who use Fabasoft most frequently, we can better understand productivity. Based on monthly Fabasoft data, by tracking the number of officials that opened or closed cases each month, we observe a dispersion in the frequency with which officials use Fabasoft, and in most departments it is less than 100 percent. In some cases, we do see more employees using Fabasoft that there are officials in the department (see Figure 23). By merging the Fabasoft system with HR data, we will be able to observe and explain these anomalies. 48 Figure 23. Ratio of Fabasoft users to the number of positions by department (2015-2020) Source: Fabasoft data More information on how cases are entered into Fabasoft is required. Since entries into Fabasoft are made manually, there needs to exist a check on the quality of inputs into the system to standardize its use. More information on how the following instances is handled would support this standardization:  To what extent does each case capture the task currently performed for it?  How precisely are registration marks assigned?  How are errors treated for cases?  How are the start and end dates for each case entered and monitored? Annex 1.2 Cezir Business licensing cases can be tracked through time using the Cezir administrative system. This system tracks each step of a business licensing case. Figure 24 depicts when cases are processed, and how productivity varies throughout the day with it declining significantly in the afternoon. Similarly, Figure 25 shows the number of cases processed each month. Finally, the data can also allow for the number of cases per official to be measured and tracked. Figures 26 show that productivity varies across employees significantly. By the combination of Cezir data with HR data, we can provide more detailed findings by employee type and various demographic characteristics to see which groups of officials are more productive. 49 Figure 24. Number of closed business licensing cases by time of day (2019) Source: Cezir data 50 Figure 25. Number of business licensing cases processed by month (2019) Source: Cezir data Figure 26. Number of business licensing cases by public official (2019) Source: Cezir data 51 Annex 1.3 Employment data In addition to tracking the progress of employees throughout their tenure, data on employment – from job advertisement onwards – can allow us to understand the hiring process better. For example, Figure 27 shows the number of advertisements by the type of position from July 2017 to November 2021. It shows a significant spike in hiring between August and November 2019. By monitoring these trends in real time, administrators can try to understand the reasons by such instances and respond to them accordingly. Generally, the total number of job postings across all district offices is high. Taking into consideration all job postings on the portal for the capital Bratislava between since 2017, the ratio of postings to current positions is 1.68 and the average ratio across all district offices is 0.7. Figure 27. Number of job postings by month and type of posting Source: employment data 52 Figure 28. Number of temporary job postings by month and reason for open position Source: employment data Information on job postings for temporary contracts requires further exploration. As Figure 27 shows, the most common reasoning for posting temporary job opportunities on the portal was replacement for staff on maternity or parental leave (77%). This continued to be the most common reason during the spike in all job postings seen towards the end of 2019. More information is needed to continuously monitor, identify and explain such anomalies in the data. 53 Annex 1.4 Survey of district office public officials This section depicts the variation across district offices for each question in the WB survey of district office public officials. Since the maps display standardized scores on the responses of public officials, we can compare responses across district offices and questions. Figure 29. Managers: what is the process for exposing and solving problems in your department? (Higher scores indicate better practices) Source: WB survey Figure 30. Managers: how does your department track or measure how well it is performing? (Higher scores indicate better practices) Source: WB survey 54 Figure 31. Managers: how does your department manage poor performance? (higher scores indicate better practices) Source: WB survey Figure 32. Non-managers: the feedback I receive on my work helps me to improve my performance. (Higher scores indicate higher agreement) Source: WB survey 55 Figure 33. Non-managers: When I perform well, I get praise and recognition at work. (Higher scores indicate higher agreement) Source: WB survey Figure 34. Non-managers: I receive sufficient training at work to be able to complete my work tasks effectively. (Higher scores indicate higher agreement) Source: WB survey 56 Annex 2 Implementation timeline Table 1. Implementation timeline Activity Details Original Randomized  An RTC was planned to measure the impact of a new IT system Control Trial (RCT)  The government decided to continue with the original system  As such, the RCT was not conducted Sharing of  Following negotiations, the Data License Agreement was signed administrative data  The MoI shared anonymized versions of the Fabasoft, Cezir, and Queuing datasets  Following multiple requests and clarifications, HR data has not been shared. The HR department mentioned that sharing this data might violate MoI rules and GDPR  Cadaster data has been shared. This is following meetings with UGKK to clarify the aims of the project. However, since the management of UGKK has changed, more negotiations are required before this data can be used Training camp for  The WB attended the 2-day training camp for 50 district office new district office directors following their appointment directors  The WB implemented a survey of district office directors and shared evaluations with the AMU Online Learning  In-person courses at the MoI’s training centre were paused due Management System to COVID-19 and online courses were planned through an LMS (LMS)  MoI decided on an LMS platform to use  The WB submitted a request to collect data from this LMS platform and helped develop a course outline for the LMS  MoI began procurement but it was cancelled, and trainings are provided through webinars Construction of  The WB team extracted employment data from job portal as a employment dataset replacement for HR data since 2017 Survey of district  A phone survey was the chosen as the most feasible mode for a office public officials survey of district office public officials. The MoI decided to not share phone numbers with the WB team 57  The WB team then split the survey to include a registration survey prior to the phone survey. A questionnaire for both surveys was drafted and shared with the MoI for comments  The MoI did not allow the WB to send unique survey invitations to staff  As such, the WB hired a Slovak survey firm to run the registration survey, following all GDPR protocols  MoI shared the survey link through a general invitation email on June 3 and sent reminders throughout the survey duration  WB called district office directors to boost response rates  The survey closed on July 21 with a response rate of 5.2% 58 Annex 3 Questionnaire Table 2. Questionnaire for survey of district office public officials Module Question Response options General Please select your department [List of all departments] from the list. Please select your district office [List of all districts] from the list. 1. Manager Please select whether you are manager or non-manager. 2. Non-manager Please select your unit from the [List of all units list. Managerial How does your department track 1. The department does not track or measure how well it is performance. performing? 2. Limited number of performance indicators tracked informally. 3. Limited number of performance indicators tracked formally. 4. A good range of performance indicators are tracked formally. 5. The full set of performance indicators KPIs are tracked formally and continuously. What is the process for exposing 1. Ad-hoc, no set process for and solving problems in your improvement. department? 2. Some process for fixing issues when something goes wrong. 3. Existing process to deal with problems. 4. Continuous process focusing on prevention, not just dealing with problems. 5. Process of continuous improvement focuses on improving (preemptive process). How does your department 1. No action is taken to deal with bad manage poor performance? performers. 59 2. Poor performance is rarely addressed, and typically only at the more junior levels. 3. Poor performance is addressed inconsistently across individuals and staff groups. 4. Poor performance is addressed consistently and at all levels of staff. 5. Poor performance is addressed through formal performance improvement plans. Non- When I perform well, I get praise 1. Strongly disagree managerial and recognition at work. 2. Disagree 3. Neutral – neither agree nor disagree 4. Agree 5. Strongly agree 6. Don’t know/prefer not to respond The feedback I receive on my work 1. Strongly disagree helps me to improve my 2. Disagree performance. 3. Neutral – neither agree nor disagree 4. Agree 5. Strongly agree 6. Don’t know/prefer not to respond I receive sufficient training at work 1. Strongly disagree to be able to complete my work 2. Disagree tasks effectively. 3. Neutral – neither agree nor disagree 4. Agree 5. Strongly agree 6. Don’t know/prefer not to respond 60