Improving Data Quality 1 Improving Data Quality for an Effective Social Registry in Indonesia for an Effective Social Registry in Indonesia 01 This work is a product Improving of the Data Quality for staff of The an Effective World Social Bank. Registry in The findings, interpretations, and conclusions Indonesia expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions © 2022 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Attribution Please cite the work as follows: “Hadiwidjaja, Gracia; Williams, Asha; and Giannozzi, Sara. 2022. Improving Data Quality for an Effective Social Registry in Indonesia. © World Bank.” 02 Improving Data Quality for an Effective Social Registry in Indonesia Acknowledgements This report was authored by Gracia Hadiwidjaja, Asha Williams, and Sara Giannozzi. The team thanks Phillippe George Leite and Changqing Sun for valuable technical comments and contributions to the report. We are also grateful for feedback from Australia’s Department of Foreign Affairs and Trade (DFAT), Dinar Kharisma and team at the National Development Planning Agency (Bappenas), and Sudarno Sumarto at the National Team for the Acceleration of Poverty Reduction (TNP2K). The team would like to express sincere gratitude to Yasser El-Gammal for providing guidance, Sheila Ann Town for editing the report, and to Rizky Fitriany for providing far-reaching administrative support. Financial support for this report was provided by the Australian Government under the Australia-World Bank Indonesia Partnership (ABIP). The team thanks the Australian Government for their generous support. The team is also grateful to Megha Kapoor for her support. 03 Improving Data Quality for an Effective Social Registry in Indonesia Contents Acknowledgements 02 1. Introduction 04 2. Integrated Social Welfare Data - Data Terpadu Kesejahteran Sosial (DTKS) 2.1 Coverage 2.2 Data Collection 2.3 Data Management 2.4 Data Verification 2.5 Local Updating 2.6 Eligibility Criteria 09 3. Towards an Effective Social Registry 3.1 Key Strategies in Improving Data Freshness and Quality 3.2 Efforts to Sustain Data Quality 20 References 29 Annexes 30 04 Improving Data Quality for an Effective Social Registry in Indonesia Introduction M 05 Improving Data Quality for an Effective Social Registry in Indonesia ore and more countries around the world are aiming to not only reduce poverty and inequality but also to achieve Universal Social Protection (USP) 1. Globally the expansion of social safety net programs has contributed to poverty reduction and the improved well-being of the poor. Now countries are now offering more comprehensive packages of social benefits and services to serve broader population needs and to facilitate more sustainable poverty reduction outcomes. In many countries these commitments have translated into an expansion of social assistance and social insurance with broader thresholds for eligibility and higher social spending. If not managed well, the expansion of services and beneficiaries can contribute to fragmentation in program processes and systems that will result in higher administrative costs, lack of coordination, and duplication in processes. Fragmentation can also create additional burdens for potential beneficiaries due to lack of information about programs, repeating similar administrative procedures for different programs, experiencing frustration due to complicated bureaucracy, and incurring significant personal costs due to the time and money spent navigating this process. Frequent climate shocks and the COVID-19 pandemic have also further highlighted critical gaps in social protection coverage, exposing the need for more and better data systems to identify and support those still excluded from social protection programs and systems. Investing in better data systems is critical to maximize the impact of social protection programs. A high quality data system is especially important to direct appropriate benefits and services to the groups that need them. Although there is no one size fits all targeting strategy, building a credible social information system (SIS) and improving social protection delivery systems (DS) 2 are key to achieving USP everywhere. SIS is an integration of systems that includes a Social Registry, a Beneficiary Registry and interoperability with other administrative systems. Improving delivery systems entails improvements in the process and implementation of each element in the delivery chain, to close any gaps and reduce fragmentation. Building a credible SIS requires integration that allows for a dynamic, accurate, and simple data flow. Moreover, although advances in technology such as big data or machine learning can improve targeting accuracy, better data is more important than sophistication in data use 3. This means that priority must be given to investments in communication, human centered design, and people, as technology alone does not solve problems. Social Registries are information systems that support the outreach, intake, registration, and determination of potential eligibility for one or more social programs. As a key element of SIS, social registries serve as a “gateway” to reduce private and public transactions costs by simplifying targeting steps and facilitating potential inclusion of intended populations into programs. Social registries ideally gather information on a large share of the population as potential registrants for multiple benefits and services. Their goal is to provide a list of potential beneficiaries with sufficient information to support the determination of eligibility. In some countries, the types of programs 1 Universal Social Protection refers to a nationally defined system of policies and programs that provide equitable access to all people and protect them throughout their lives against poverty and risks to their livelihoods and well-being. 2 Improving delivery systems entail improvements in the process and implementation of each element in the delivery chain to close any gaps and reduce fragmentation. 3 See, Grosh et all, 2022. 06 Improving Data Quality for an Effective Social Registry in Indonesia that use social registries go well beyond social assistance and can include a wide range of benefits and services, some targeted and others universal. Social Registries include household and individual level information. This includes demographic, family composition, housing assets, and other information associated with welfare. Some of these variables are used directly for assessment of needs and conditions that determine eligibility. This can be directly based on geographic region of residence, or characteristics as age and gender for categorical programs, and/ or on welfare, which can be based on observed incomes, a combination of declared income and an algorithm (hybrid means test - HMT) or solely on the algorithm based on socioeconomics to predict welfare of the family.4 Figure 1. Comparison between Brazil’s (Cadastro Unico) and Indonesia’s (DTKS) Social Registries Social Registries (SR) are distinct from Beneficiary Registries (BR). While social registries cover the intended population for participation in social programs, beneficiary registries consolidate information only on enrolled recipients of benefits and services from existing programs. Another distinction is that beneficiary registries serve as a measure of the “supply” of social programs, while social registries focus on the potential “demand”. Beneficiary registries pulls administrative information across programs information systems, usually through interoperability frameworks, with the purpose of assisting program delivery, tracking beneficiaries of multiple programs, identifying synergies or gaps in provision, detecting duplication and 4 In many cases, the socioeconomic ranking algorithm determines a “score” for the household which is then used to determine whether it is eligible for programs (as each program may establish a specific cutoff ). In other cases, like Indonesia, the ranking is relative, in that programs will target the bottom 5,10 or 15 percent of the ranking for example – whenever a new household is included, it can potentially impact ranking and eligibility. 07 Improving Data Quality for an Effective Social Registry in Indonesia fraud, and facilitating joint monitoring and reporting.5 Regardless of the criteria for individual programs, the most important feature of an SR is that must be built from day one is the interoperability that allows for a two-way flow between the SR and any BR for the programs that use SR data for eligibility determination. The effectiveness and usefulness of any Social Registry for determination of eligibility is determined by three key elements: (i) coverage, (ii) quality and “freshness” of the data, and (iii) the assessment tool. Working on each of these elements yields possible gains for the quality  of data and, ultimately, the effectiveness and efficiency of allocation of benefits. Coverage of social registries is expressed as share of total population or sometimes, as share of the population of interest (in some cases the poor/vulnerable population). Having good coverage is important to ensure the inclusivity of the population in programs. Increasing the share of the population covered also makes it easier for more programs – and for different kinds of program – to be able to use the Social Registry to target their desired population.6 Very often, coverage is positively correlated with the number of programs that use the registry – as covering larger shares of the total population becomes increasingly important in order to support a broader set of programs. This coverage also requires sufficient information about households or individuals to allow for proper assessment, as programs have different criteria and functions. Up to date (see next paragraph) and inclusive social registries are also key to accurate targeting, as statistical models in Proxy Means Test (PMT), for example, only predict households that are in the database. Source: https://unsplash.com/photos/xcmTPq_36Yw 5 Sometimes social registries can also have information about access to benefits and services of the individuals and households in the data- bases. In these cases, we talk about “integrated social registry”. For a full discussion of terminology, see Leite et. Al, 2017 6 Sometimes social registries can also have information about access to benefits and services of the individuals and households in the data- bases. In these cases, we talk about “integrated social registry”. For a full discussion of terminology, see Leite et. Al, 2017 08 Improving Data Quality for an Effective Social Registry in Indonesia Quality and freshness, and assessment tools, relate to how accurate and up-to-date information inside the registry is and the method chosen to select beneficiaries. A significant share of errors in targeted programs are due to under-investment on the delivery systems needed to collect data accurately and regularly.7 Inaccurate information due to lack of verification, capacity, and duplication will result in inaccurate ranking assigned to a particular household. At the same time, social registries require up-to- date data to ensure the inclusion of newcomers due to financial shocks (job loss, disability) and covariate shocks (e.g., natural disasters, pandemics), and exclusion of those who are no longer eligible, or have died. The assessment tool involves selecting the right method to assess the needs and eligibility of beneficiaries. The assessment decision will depend on the goals of social registries and the information available in the database. In the Indonesian context, the greatest gains are more likely to come from investment in coverage and quality of the data through better systems and expertise. Previously Indonesia used proxy means tests (PMT) as the assessment tool to select the poorest population eligible for certain social assistance benefits. Although the Government of Indonesia has revamped algorithms used in welfare prediction to improve their prediction accuracy, there have been only marginal improvements in data quality over time. Having good coverage is important to make sure that the chosen eligibility criteria can be applied to a large enough set of socioeconomic household data, as PMT or other criteria can only accurately select households that are in the database in the first place.8 The aim of building SR however goes beyond building a database for PMT or any other specific targeting methods. The type and depth of information collected in SR must be linked to the government’s goal of building SR which in other countries goes beyond poverty targeting. Arguably, improving data quality is a much more challenging process, as it requires a greater level of collaboration and system interoperability among different institutions. Funding and regulations are another challenge that impede regular updating. Given the importance of data quality in social registries and targeting, this note will focus mostly on the current challenges in maintaining data quality and attempt to provide some options for improvements going forward. 7 For example, data collection exercise must incorporate good data collection techniques and training of interviewers, questionnaires should not be too short or too long, or be too complex. Continuous data collection effort and capacity building is needed, and sometimes by providing better data collection instruments and better training, time spent on field work – and data input – can be improved. 8 “Relative ranking” as used in DTKS may not ensure that the inclusion of all households that need assistance, while threshold type of eligibility will cover all households below certain thresholds. 09 Improving Data Quality for an Effective Social Registry in Indonesia Integrated Social Welfare Data - Data Terpadu Kesejahteran Sosial (DTKS) 9 9 Previously called the Unified Data Base for Social Protection Programs (UDB) 10 Improving Data Quality for an Effective Social Registry in Indonesia Source: shutterstock/Aljofoto/1856875813 D ata Terpadu Kesejahteran Sosial (DTKS) or the Integrated Social Welfare Data is Indonesia’s social registry that was designed to cover the bottom 40 percent of households. It was first developed to support processes to compensate the poor and vulnerable population for the elimination of fuel subsidy in 2005. Since then, DTKS has been updated every three to four years through a census sweep of selected households that were pre-identified in pre-lists.10 The government subsequently named this the Socioeconomic Data Collection (Pendataan Sosial Ekonomi 2005 – PSE 2005), and Data Collection for Social Protection Programs (Pendataan Program Perlindungan Sosial 2008 and 2011 - PPLS 2008 and PPLS 2011). Yet the database was only named in 2011 and referred to as the Unified Data Base for Social Protection Programs (UDB) until 2016 before the Ministry of Social Affairs (MoSA) changed it to DTKS. DTKS contains information on households and individuals, including name, address, and detailed socioeconomic information that can be used among other things to estimate household welfare through a Proxy Means Test (PMT). This welfare estimate, combined with program criteria, has been used to target five national programs at the household level,11 as well as additional programs during the COVID-19 pandemic. 10 In 2005, pre-lists were built based on recommendations of village leaders. In 2011, BPS recreated the pre-lists through small area poverty estimation techniques on the Population Census of 2010. In addition to the pre-lists, in 2011, enumerators were required to add households through sweeps, consulting with poor households, and in 2015 this was done through consultations with community leaders (Forum Kon- sultasi Publik-FKP). 11 These programs include BPNT-Sembako (Food Assistance Program - Bantuan Pangan Non-Tunai); PKH Family Hope Program con- ditional cash transfer (Program Keluarga Harapan); PBI-JKN Subsidized Health Insurance (Penerima Bantuan Iuran - Jaminan Kesehatan Nasional); the PIP cash transfer for poor and vulnerable students (Program Indonesia Pintar), and BST Cash Transfer (Bantuan Sosial Tunai). 11 Improving Data Quality for an Effective Social Registry in Indonesia 2.1 Coverage As of late 2021, DTKS covered the bottom 51.8 percent of Indonesia’s population. 12 Prior to the pandemic in January 2020, approximately 97.4 million individuals or 36 percent of the population were included in DTKS,13 serving as the primary target group for at least five national programs and various local initiatives.14. Throughout 2020 and 2021, the COVID-19 pandemic response efforts led to new program distribution and local level data collection efforts, and resulted in 43.3 million additional people being added to DTKS, which now covers a total of 140.7 million people. The experience of COVID-19 re-emphasized the need to expand DTKS to cover more households beyond the bottom 40 percent and to improve the open registration process for the quick response and distribution of different types of support. As a result, a decision was made to further expand the coverage of DTKS up to 60 percent first and to eventually reach the entire population.15 This would bring improvements in the quality of the ranking and allow DTKS to be used for a broader set of purposes beyond poverty targeting. Recent expansion of DTKS has focused on adding more people into the database without including much more individual information. As part of its comprehensive social response to mitigate the impacts of the COVID-19 induced economic shocks, the government of Indonesia decided to distribute unconditional cash for families that were not yet covered by DTKS through a new program funded from the Village Fund (Dana Desa). Eligible families were selected through village consultations and village officials were responsible to record names and addresses of beneficiaries. These names were then included in DTKS, but the collection of their socioeconomic data has not been carried out. Most recently, there has also been a broader shift in policy direction. Registration in DTKS is now open to the public either by registration from a website for those living in Jakarta (called DTKS Jakarta) or by contacting local officials, for those living outside Jakarta. The information collected however no longer includes a set of socioeconomic variables. For example, in the case of Jakarta, registrants are required to enter detailed identity information that would have been found in Indonesia’s ID Card (KTP) and family card (KK) with additional questions on car and land ownership, source of drinking water, and income. 16 Though this information is adequate for specific programs, more information will be needed to build a comprehensive SR that is able to assess the needs and eligibilities of individuals for most programs. Compared to other developing countries, DTKS contains one of highest number of households. Nonetheless it compares less favorably with other countries in the share of population covered. Although the coverage of DTKS has increased during the pandemic, as described above, information on these additional households have not included all variables that were previously collected and cover only names, addresses, 12 Based on the Decree of the Social Minister of the Republic of Indonesia Number: 145 / HUK / 2021 issued on 26 November 2021 and the total population of 271,584,774 people from Susenas 2021. The number of families and individuals recorded in DTKS was updated twice a year since 2017 through the Decree. 13 Based on the Decree of the Social Minister of the Republic of Indonesia Number: 19 / HUK / 2020 issued on 26 November 2021 and total population of 271,584,774 people from Susenas 2021. 14 350 local governments have requested DTKS data for local poverty reduction programs. See Barca, 2017. 15 Parliamentary meeting, January 2021. Initial conversations started the year before. 16 dtks.jakarta.co.id 12 Improving Data Quality for an Effective Social Registry in Indonesia and the unique national identity number (NIK). The desirable size of social registries depends on the functions it expects to support. An SR that aims to cover the poorest population can have a low coverage while one that aims to distribute social insurance may reach half or more of the population. The larger the share of population covered, the more flexibility an SR offers for shock-responsive programming. Some social registries cover the whole population, such as those in Argentina and Uruguay. Others cover between two third and one half of the population including Pakistan, the Philippines, the Dominican Republic, and Colombia. Others like Chad, Zimbabwe, and Belize operate on a smaller scale by covering less than five percent of the population. Figure 2. Coverage of Social Registries Source: Grosh, et al, 2022 Figure 3. Updating Status of DTKS per Dec 2020 Source: Bappenas, 2021. 13 Improving Data Quality for an Effective Social Registry in Indonesia Increasingly local governments are now using DTKS to distribute local programs. Despite having limited capacity to carry out local updating in many regions, local governments are in fact keen to use DTKS to deliver programs. During the pandemic for example the provincial government of DKI Jakarta relied on DTKS to distribute local programs. In May 2022, DKI Jakarta also announced DTKS Jakarta to highlight the increasing use of DTKS to distribute not only national programs such as BPNT and PKH but also local programs including KLJ, KPDJ, KAJ, KJP Plus and KJMU.17. Source: shutterstock/Aljofoto/1856875774 2.2 Data Collection DTKS has been customarily updated by Statistics Indonesia (BPS) through a census sweep of pre- selected household from pre-lists. As it is quite common for social registries, the initial stages of population of DTKS relied on survey sweep updates to include new households and update information. BPS collected information on a targeted population identified with a pre-list that was built using poverty maps combined with existing data sources and community consultations. 18 This information includes demography, dwelling, education, employment, and assets. BPS recruited enumerators and data entry staff in every district, who in most cases are local residents and longtime partners of BPS. Since 2017, the mandate for data updating has shifted from the national to the local (district) government level.19 After four rounds of census in 2005, 2008, 2011 and 2015, DTKS is now updated by local governments themselves through a process called verification and validation (verval) using their own Local Government Budgets (APBD). According to MoSA guidelines, this process of verval requires updating DTKS variables, removing ineligible households from the beneficiaries list and adding new potential beneficiaries. 17 Kartu Lansia Jakarta Kartu Penyandang Disabilitas Jakarta Kartu Anak Jakarta Kartu Anak Jakarta Kartu Jakarta Mahasiswa Unggul. 18 Poverty maps were developed for every village using the small area poverty estimation technique. 19 According to the Law Number 13 of 2011 on Handling of the Poor, data updating is a tiered process assigned to districts through the Dinas Sosial (Dinsos) and carried out by social welfare unit in the sub-district and villages/kelurahan. MoSa then issued Permensos No. 28 Year 2017 and Permensos No. 5 Year 2019 to implement the regulation. 14 Improving Data Quality for an Effective Social Registry in Indonesia Prior to verval, District Social Agencies (Dinas Sosial -Dinsos) would facilitate village governments to form Social Welfare Centers (Pusat Kesejahteraan Sosial -Puskesos) to help promote the program and reach the poor more easily. Figure 4. DTKS data collection Dinsos was also responsible for organizing training for enumerators consisting of Sub-district Social Welfare Workers (Tenaga Kesejahteraan Sosial Kecamatan -TKSK), Family Hope program (PKH) Facilitators, Youth Organizations and Community Social Workers (Karang Taruna dan Pekerja Sosial Masyarakat -PSM). In this training, staff from the Ministry of Social Affairs (MoSa), District Offices, Regional Planning Agencies (Bappeda), BPS and Dinsos would act as trainers. While this shift in institutional management has been accompanied by a greater role for local governments, it has brought with it a set of distinct new challenges, including local government capacity to update data, establish incentive structures and provision of financing. To manage the verval process, MoSA developed a modular information system (called SIKS-NG) with functions that allow local governments to input and update new beneficiary data directly into DTKS. The system consists of a web based, desktop and Android application. The web-based application was developed to allow local governments, through Dinsos, to update potential beneficiary data for their specific location online to DTKS via a Virtual Private Network (VPN). The desktop application is intended to allow local governments with less stable internet connectivity to send updated potential beneficiary data in batches to DTKS. The Android application is used by SIKS-NG operators to update existing household information and enter data of newly proposed households (the determination of potential eligibility). 20 The database is currently stored in MoSA’s Data Center in Jakarta, with a Data Recovery Center (DRC) located 771 km away in Surabaya. By 2021, MoSA no longer used SIKS-NG and introduced the New DTKS as explained below. 20 More information on SIKS-NG can be found at https://helpdesk.kemensos.go.id/kb/index.php 15 Improving Data Quality for an Effective Social Registry in Indonesia Source: shutterstock/Bastian AS/1973561219 2.3 Data Management DTKS is managed by the Data and Information Center Unit (Pusdatin) within the Ministry of Social Affairs. Between 2012-2015, DTKS (then referred to as UDB) was managed by the National Team for the Acceleration of Poverty Reduction (TNP2K) — an agency under the office of the Vice President. Within TNP2K, DTKS was managed by a unit called Unit for Targeting and Poverty Reduction (UPSPK) that had 20 staff. UPSPK was responsible for (i) providing DTKS data to line ministries, (ii) providing technical assistance on data use to local governments, (iii) generating M&E reports on data utilization, and (iv) maintaining the website portal. To deliver on these functions, UPSPK was staffed by a team consisting of one data and dissemination officer, one infrastructure and security specialist, one GIS specialist, two on-demand application officers, one administrator and two senior programmers. As part of the implementation of Law No. 13 which states that MoSA is responsible for the collection and maintenance of data of the poor, DTKS was gradually transitioned to Pusdatin from 2016 to 2019. Human resource challenges at MoSA restricts the monitoring and quality control of DTKS. The Director of Pusdatin (Echelon 2) currently reports directly to the Minister 21 and, prior to the pandemic, managed 37 staff: nine civil employees, 12 staff technicians, three statisticians, one staff who handles maintenance of the data center, and 14 general staff. In addition to continuing the same tasks as UPSPK, Pusdatin must now work closely with local governments, manage data flow coming from verval, and ensure connectivity between DTKS and Dukcapil’s National ID system (SIAK). In fact, Pusdatin’s performance indicators include the share of DTKS individuals identified in SIAK and targeting accuracy of social programs.22 Data security is also an important dimension. Before the pandemic, MoSA had planned to deploy more expertise to manage Pusdatin to ensure the security and administration of DTKS and provide analysis using DTKS information through a business intelligence feature for stakeholders. This plan was postponed due to the ongoing reorganization and policy shifts at MoSA. 21 Previously to the Secretary General of MoSA. 22 See https://pusdatin.kemensos.go.id/indikator-penilaian 16 Improving Data Quality for an Effective Social Registry in Indonesia 2.4 Data Verification Above verval, another layer of verification is added for NIKs. In 2021, MoSA initiated a series of checks to review the quality of DTKS data, focusing primarily on verification and validation of NIKs, the unique national ID number. In 2011, only 74 percent of DTKS’ predecessor (the UDB) data had a valid NIK. 23 As household members’ NIK was not collected in the 2011 round, the matching process was conducted via an algorithm consisting of names and addresses. Importantly, in the 2015 data collection round, the NIK was collected as a data variable, with over 80 million individuals surveyed (over 80 percent of the total) having a NIK. The entire database was then transferred to SIAK for NIK matching. The NIKs are now collected by MoSA and are supposed to be verified with SIAK. 24 Although DTKS is still not actively linked to SIAK, efforts are now ongoing to link the two systems using web services, and this process has evolved over time. 25 In 2020, MoSA found 10.9 million individuals with invalid NIK, and 86 thousand duplicated NIKs. 26 In an effort to make DTKS a more dynamic registry, MoSA has pursued several parallel strategies and announced the New DTKS in April 2021.27 The proposed dynamism however changes the nature of DTKS through more limited sets of information collected and includes five main strategies. First, the frequency of required beneficiary list updating from local governments increased from every six months to monthly in 2021. Previously MoSA announced the list of program beneficiaries for main social assistance programs every six months. Currently however Dinsos is responsible for assessing the eligibility of beneficiaries and updating the list every month, with criteria that are decided by each region. Second, in 2021, and following a data cleaning exercise, MoSA deactivated 21.2 million duplicated data, consisting of cases of repeated names or receipt of multiple benefits. Third, as of August 2021 as part of the updating process, communities can make “Proposals” and “Rebuttals” online 28 to suggest or object to the inclusion of individuals in the beneficiaries list. Residents who meet the criteria as recipients of social assistance can register using this application by entering their names and addressed to be verified by local government officials. Fourth, in the case of disagreements between community and local governments on the proposals, MoSA may consult universities to validate on the eligibility of applicants. 29. Finally, university students, through the Merdeka Campus program that encourages more independence study in higher education, have been tasked to monitor the quality of DTKS data. 30 23 See Barca, 2017. 24 Population Information Administration System (Sistem Informasi Administrasi Kependudukan). 25 See https://dtks.kemensos.go.id/uploads/ 26 See https://berkas.dpr.go.id/puskajiakn/buku/public-file/buku-public-16.pdf 27 Based on Decree of the Minister of Social Affairs Number 12/HUK/2021 dated 1 April 2021. 28 See http://cekbansos.kemensos.go.id/ 29 See news from the Cabinet Secretariat, 22 April 2021. https://setkab.go.id/en/social-ministry-introduces-new-dtks/ and MOSA Press Conference, accessed at https://kemensos.go.id/en/coordinating-with-law-enforcement-ministry-of-social-affairs-puts-down-21156-mil- lion-duplicate-data 30 About 5000 thousand students were involved in four categories in social programs: DTKS monitoring and analysis, empowerment of the poor, healthy lifestyle and environment campaign, and infrastructure development. See https://koran-jakarta.com/5-140-maha- siswa-ikuti-kampus-merdeka-pejuang-muda?page=all 17 Improving Data Quality for an Effective Social Registry in Indonesia 2.5 Local Updating The freshness of data in DTKS from local updating is highly variable.  The current version of DTKS includes data that has been updated in 19.1 percent districts to varying degrees in terms of quantity of households updated and completeness of socio-economic variables. 31 As of December 2020, only 50 districts had updated more than 50 percent of their data updated and 57.5 percent were updated before 2018 (see figure 2) and these districts tend to have larger reduction in poverty compared with those that were less active in verval. User programs carry out updates for demographic and household composition data on their beneficiaries, as this information affects the calculation of benefit levels. However, these updates are not usually fed back from beneficiary registries to the social registry and there is no regulation or protocol for this. Despite these positive policy shifts and efforts, district governments face significant challenges in maintaining updated data. While local updating has significant political and efficiency-related advantages, it also poses several challenges. First, local governments were given the mandate but not necessarily the allocated resources to update DTKS. Local governments that have more poor and vulnerable households to update likely have lower budgets available to do so. To respond to that, MoSA has begun using a provincial budget (Dana Dekonsentrasi) to provide supplemental budget support; 32 this approach is nascent and should be strengthened. Secondly, local government may not access or implement existing guidelines on how to conduct data collection. Thirdly, local governments may only update sub-sets of the DTKS for their district and may follow different approaches in managing enumeration. This leads to a higher level of potential variation in the quantity and quality of data. Fourth, data fragmentation where local governments have and use their own databases to target local programs can lead to a drop in priority for DTKS verval priority. 33 In this case, the lack of a stronger institutional or legal framework would continue to hinder the effectiveness of all other proposed efforts (e.g., capacity building, funding, interoperability), because incentives for fragmentation and/or “inaction” remain against an integrated, dynamic, and reliable DTKS going forward. Quality concerns are also prevalent. In recent years the media has highlighted concerns as to the quality of DTKS. For example, local government was found to be using the old version of DTKS despite having spent local budget and conducted verval, and some civil servants or even parliament members were found to be included in the database.34 The view that DTKS has to be “perfect” with no room for improvement often leads to mistrust towards the database at multiple levels. One district for example then questions the objectivity of Puskesos team.35 Meanwhile several ministries are hesitant to rely only on DTKS for energy subsidy.36 Instead of pursuing closed distribution to DTKS households for example, the government- subsidized three kilogram gas cylinders are freely available in the market for all households. 31 Bappenas presentation, 2021 and Parliament Assessment on DTKS, 2021. 32 Deconcentrated Funds are funds originating from the APBN implemented by the Governor. 33 For example, Surabaya has Masyarakat Berpenghasilan Rendah (MBR) database and Jakarta has Data Fakir Miskin & Orang Tidak Mampu (FMOTM). 34 See articles on https://fajarcirebon.com/proses-verval-warga-miskin-dinsos-tunggu-anggaran/ and https://curupekspress.rakyatbengku- lu.com/ 35 See https://suaracirebon.com/2022/01/18/ganti-kuwu-jangan-diikuti-pergantian-tenaga-puskesos/ 36 See https://nasional.kontan.co.id/news/sri-mulyani-janji-kebijakan-subsidi-energi-diarahkan-lebih-tepat-sasaran-pada-2022 18 Improving Data Quality for an Effective Social Registry in Indonesia Source: https://unsplash.com/photos/3VLHF9b9Plg The government of Indonesia has piloted several dynamic updating mechanisms to find the right model. The SLRT,37 or the single window referral program that focuses on referring potential beneficiaries to local government social protection programs and collecting social protection grievances is now implemented directly in MoSA and functions in over 140 districts. As of early 2020, SLRT or local government-led updates have not yet led to inclusion of new beneficiaries into Indonesia’s main social program, PKH. Yet the design, implementation procedures, and lessons learned were shared with MoSA, and these have been used in the design of SIKS-NGs updating procedures. Another piloting effort was the On Demand Application (ODA) or Mekanisme Pemutakhiran Mandiri (MPM), run under TNP2K and the Australian-funded Mahkota program pilot that functioned effectively in 11 districts and one province until late 2018 when the pilot closed. 38 In order to support the government’s plan to build a social registry, Bappenas is also piloting a socio-economic national registry (Regsosek) in 95 villages across Indonesia. 39 From these pilots, high level coordination remains a key issue in ensuring agreed implementation and task sharing. 2.6 Eligibility Criteria In general, there are three types of beneficiary assessment mechanisms: geographic, categorical, and socioeconomic/welfare-based selection, each associated with varying levels of accuracy and cost. 40 Geographic targeting means selecting regions or areas for programs to be distributed. This method is quick and cheap with no individual assessment required but generally is less accurate as it ignores differences in welfare and other important characteristics. Categorical approaches select beneficiaries based on groups in the population such as children, elderly, or disabled persons. For welfare-based selection, means testing is acknowledged as the more accurate method when it is supported by a good information system and reliable data on declared income (e.g. income and tax return) that is often not available in developing countries. Alternatively, many developing countries use PMT that applies multiple indicators to predict welfare or HMT that combines declared and verifiable income with predictions such as PMT. To a lesser degree, some countries also rely on community recommendations and self-assessment to select beneficiaries. 37 The SLRT, or the Integrated Referral Service is implemented by the MoSA with support from Bappenas and Mahkota. The SLRT has functioned both as a GRM and as a citizen interface for multiple government services. 38 MPM worked in areas that agreed to allocate budgets for data collection. Through MPM, 215,000 households were added into DTKS. 39 The pilot was part of the Digitalisasi Monografi Desa/Kelurahan (DMD/K) pilots that were conducted in 150 districts. 40 For more detailed information, see Coady et al., 2004; Barrientos & Niño-Zarazúa, 2011. 19 Improving Data Quality for an Effective Social Registry in Indonesia Indonesia has fine-tuned the algorithm used for socio-economic ranking over the years with current model performs well. In absence of reliable income data, Indonesia has used a Proxy Means Tests (PMT) to determine welfare approximations and rank the poorest 40 percent of households. These rankings were developed through district-specific regression models with reasonable level of errors 41. Pusdatin has explored the use of machine learning algorithms to further improve accuracy, yet only marginal improvements were found.42 The largest improvements in targeting accuracy will likely come through increased interoperability of the DTKS with related data that can be used to verify socio-economic data and thus identify inclusion errors (vehicle ownership, tax data) and include entirely new variables (civil service participation, education completion and employment). Although interoperability does not replace the need for data collection, over time this this will support deduplication, reduce the need for fragmented and overlapping data collection, reduce targeting errors and facilitate cost efficiencies in the data collection and targeting processes. 43 Another important improvement is on the delivery systems that ensure good implementation and dynamism. No targeting method can replace the need for good implementation of each step of the delivery chain. Currently MoSA is exploring new welfare parameters. The newly submitted data has not yet been re- ranked via the PMT and there are quality concerns about the updated data that will affect the performance of any PMT model implemented. Most significantly, recent implementations of verval focused only on updating 15 variables on individuals44 to ensure a singular identity, as MoSA plans to replace the PMT assessment. Local governments who need more information on household characteristics were offered the opportunity to use socioeconomic data from the National Population and Family Planning Board (BKKBN) who collected PMT related information in 2021. Source: shutterstock/Dragana Gordic/1692488659 41 Inclusion and exclusion errors for programs that target the bottom 40 percent of the population ranging between 28- 29%. 42 This is consistent with the findings in the literature, as documented in (Grosh, et all, 2022). 43 Targeting accuracy is measured from leakage or inclusion error, when unintended beneficiaries receive program benefits, and under coverage or exclusion error, when intended beneficiaries do not receive program benefits. Another way to measure targeting performance is by looking at the distribution of programs through coverage, incidence, and impact on poverty. 44 The 15 variables are: Province, District, Sub-district, Village, Address, Sub-district, RT, RW, No. KK, NIK, Name, Birthdate, Birthplace, Sex, Name of Mother. 20 Improving Data Quality for an Effective Social Registry in Indonesia Towards an Effective Social Registry 21 Improving Data Quality for an Effective Social Registry in Indonesia 3.1 Key Strategies in Improving Data Freshness and Quality Improving data quality involves efforts to both correct current data, and design future data updating mechanisms. While some issues in data quality were inherited from previous rounds of data collection and must be resolved now, a well-designed updating mechanism can help prevent or minimize issues in the future. These issues include the process in place to collect data, the data collection exercise, and verification of the declared information. Immediate efforts can include data validation and verification, oversight and controls, and interoperability to boost current information quality and accuracy. Achieving an effective local updating mechanism takes time, hence given the pressing need to increase the coverage and update DTKS, it is worth exploring an option to conduct another survey sweep. In the medium and longer term, building a social registry requires the right mindsets and goal setting. Any decisions made on SR must be aligned with the government’s goal of building SR. This includes the choice of information included in SR, enumerator profile, data collection strategy, and other decisions. In the past, eligibility determination for programs in Indonesia has been identified with PMT. PMT however is only one selection method when targeting the poor and the goal of building SR usually goes beyond poverty targeting. SR ideally aims to contain enough information to access the needs of people and match them to the right programs. With different information, the government can then aim to collect the least amount of information possible for efficiency and data privacy protection purposes. Similarly the choice of data updating method can go beyond cost efficiency and ideally focus on providing the opportunities for people to be included and to update their information at any time. A one-time contract with a survey firm can work on data collection with the single aim of “just” getting good data. If the goal of data updating is expected to continue over time, enumerators should ideally have longer contracts, or, as an alternative, data collection can be complemented with an updating point in each region. Currently, local governments play a key role in updating information and may need further support to carry out this role effectively and to maximize their contribution to maintaining an accurate Social Registry. Strengthening the capacity of Pusdatin and local officers is an important investment to improve data quality. Capacity constraints at the central and more importantly at the local level, especially in the areas of beneficiary intake and outreach as well as data collection, restrict improvements to data quality. Continuous training of staff both at the local “front end” and in the “back office” is important to maintaining data quality. Regardless of whether enumerators are recruited through an institution or outsourced to an external firm, this choice should align with the goal of data collection45 and enumerators ideally should live within the same broad community and speak the local language. At the same time, enumerators preferably should not survey their own village to avoid pressures from relatives or community leaders that may affect the accuracy of variables. Mock interviews and in-depth sessions on understanding different definitions in the questionnaire are useful 45 If the goal of data collection was to conduct a survey sweep, data collection can be outsourced to a survey firm. Latin America countries however employ surveyors for at least a year to make sure that the same persons can be responsible for data verification and validation. 22 Improving Data Quality for an Effective Social Registry in Indonesia exercises during training. In the context of Indonesia where communities hold an important role in data updating, it is important to also provide training for local leaders and materials through, for example, pamphlets in strategic places. Two levels of communication strategy will be essential  for both the public and other users of the registry. The first level is policy communication that includes information on updating objectives, eligibility criteria, application requirements, registrant rights and obligations, and contact information. Clear understanding on the policy of DTKS is important to eliminate the association of the registry with specific programs (PKH/Sembako/BPJS) and to clearly communicate the need for updated data, not only as an eligibility requirement but as a shared responsibility of government and citizen. The second level involves a well- planned outreach and communication campaign can help to make sure targeted populations learn about DTKS and can apply to be included in the database. This campaign will involve Dinsos or other local officers visiting villages and organizing meetings with community leaders to distribute program information and discuss the best approach to make sure the most deserving families are reached and assisted to apply for the program. These visits are also useful to develop a culture in which the citizen has a right to know what information the registry has, and their potential eligibility for programs. Standards and guidelines are essential tools for managing a Social Registry but local buy-ins come first. Although regulations can help ensure implementation of guidelines in all regions, willingness to update data comes from buy-ins that can be built through a consultative process, designing guidelines so that local governments can be part of the process. Early in the updating process, there needs to be an agreement on key variables needed, quality protocols, and implications for program duration and eligibility. Any updating mechanism will also be based on the premise that information that feeds into DTKS meets a certain quality threshold. In addition to training, standards and guidelines are key to maintain internal and external consistencies. Standards can be maintained by building common data dictionaries with common definitions of variables, reference units, time reference periods, and other measures. These guidelines should also include setting measurable targets on quality and freshness of data, developing protocols for data updating, establishing the central unit role for data auditing and oversight, and ensuring capacity to provide technical support to local levels on strategies to improve quality. The absence of standards will lead to mistakes piling up and negatively impacting the quality of information stored in DTKS. While regulations can facilitate the allocation of local budgets for data updating, empowerment and knowledge sharing on how local governments can also benefit from maintaining high-quality data in DTKS can yield more sustainable results. Regulations are important to guide local government to allocate resources and to apply guidelines but may not be sufficient to encourage regular data updating. Local governments need to understand and reap the direct benefits from updating and sharing information of their residents. Although incentives can come from programs regularly giving feedback to local government on status of updated data, more tangible incentives such as those coming from results-based budgeting mechanism can also be considered, coupled with the provision of technical assistance for underperforming local governments to avoid increasing the gap between high- and poor performers. 46 Over time, this could 46 Results-based budgeting provides more resources based on the share of households updated. It is important to consider that local gov- ernments with fewer resources (often also those with higher poverty rates) may struggle to achieve targets and risk falling further behind unless appropriate technical assistance is provided. 23 Improving Data Quality for an Effective Social Registry in Indonesia also reduce administrative costs at the local level by improving efficiencies in identification and enrollment processes – even for locally managed benefits and services. With more updated data, households will have a better chance of enrollment and receiving benefits and services. Implementing an on demand-application (ODA) can be an option to increase coverage, update information, and improve the quality of DTKS. Up to now, Indonesia has relied on survey sweep conducted every three to four years to correct errors in data quality. While there are good reasons to conduct a survey sweep at the early stage of building a social registry to ensure an inclusive coverage, it is often challenging to conduct survey sweep data updates with the needed frequency to ensure reliable and inclusive data for targeting purposes. Increasingly, countries are moving towards facilitating regular on-demand registration and updating of information by potential beneficiaries and leveraging technology - through improving interoperability of existing administrative databases. Some Latin American countries for example have built permanent local points of entry where people can come to update their information. 47 This combination can add significant value and increase efficiency for all users of the registry, including both potential beneficiaries and institutions using the data to deliver programs. Mixed data collection methods can leverage the value of self-selection via on-demand applications. 48 A study by Alatas et. al. in 2016 found that on-demand methods result in some degree of self-selection by the non-poor, who make up the bulk of the population. Furthermore, the study found that the on-demand method improved the targeting outcomes of the program, with a higher likelihood of the poor being registered. In contrast with “fixed lists” of registrants and beneficiaries, open and continuous access to registration can help include more poor and vulnerable households and better capture changes in socioeconomic conditions as they happen. On-demand applications combined with active outreach is closely related to the human rights agenda and the progressive realization of universal social protection, whereby anyone who needs social protection can access it at any time. 47 For instance, Chile Atiende provides multiple points of contact (physical offices, call center, website, mobile units etc.) for processes related to a wide range of public benefits and services. https://www.chileatiende.gob.cl/ 48 Alatas et. al. (2016) reports a village-level experiment in Indonesia that compared the outcomes of census-sweep and on-demand methods for a cash transfer program. In the experiment, some areas implemented the registration process via census sweep, whereby mobile teams go to the communities to register everyone in the area. In other areas, an on-demand approach was adopted, whereby households had to physically go to register themselves. 24 Improving Data Quality for an Effective Social Registry in Indonesia Box 1: Incentives for Local Government In order to more closely align incentives and increase accountability, some countries have attempted to transfer the responsibility of both program financing and implementation including data updating to the local level. Results of this policy however have been mixed particularly in Europe and Central Asia. In Serbia for example, the local context in which decentralization was motivated less by the primary ob- jective of improving service delivery, and more by secondary objectives such as economic and financial crises, devolution of fiscal responsibility and contingencies, reactionary rejection of any form of central planning, and the appeasement of ethnic and religious minorities with aspirations for political inde- pendence, affected the performance of local delivery system. Lessons in such implementation show that trade offs in decentralizing the administration and the financing of program delivery system highlight the importance of distinguishing between the type of decentralization (financial or administrative) and the particular aspects to be decentralized and to what extent. On the other hand, local governments in Latin America are in charge of doing data updating. Feder- al government however provides some financing, inputs, and training and creates an incentive based system to encourage more updates. Local governments that meet some indicators will receive larger allocation at the end of the year. In the meantime those that do not achieve the goals receive train- ing only. A specific example of this is that of Brazil’s Bolsa Família, where program financing remains highly centralized, but many aspects of the program’s operations are managed by municipalities. The program faces a typical issue of autonomous local governments executing federal programs. This was being addressed through joint management agreements (Termos de Adesão) between the Ministry of Social Development and municipalities, which formalize roles and responsibilities and establish mini- mum standards for program operation. The Ministry provides a performance based financial incentive to municipalities to promote good implementation. It monitors municipal implementation quality using an index of management capacity based on a four point scale, which covers key indicators of registration quality and verification of compliance with conditionality. Based on the scores, the Ministry pays a pro rated administrative cost subsidy. Poorly performing municipalities do not qualify for this subsidy, but are offered technical assistance to improve performance in addition to a minimum guaranteed level of payment detached from the management index in order to strengthen lower capacity municipalities. Source: Bassett et all, 2012. For data to be regularly updated, incentives must be aligned for local government, citizens, MoSA and user programs. ODA requires that citizens report and update changes, but few have incentives to do so and the shares of households with updated information is usually low whenever citizens are already receiving a benefit or a service. Incentives for updating, however, can arise more naturally when households are required to update their information in order to maintain eligibility in a program. This requires setting and communicating clearly upon entry, the duration of each program, after which a beneficiary is required to “re-validate” social registry data to maintain eligibility. An example of strategy to update information is by matching the duration of the main program with the time expected to have information updated. Bolsa Familia in Brazil for example provide a 24 months transfer before recertification. In this case, transfers would only be continued after information had been updated which created an incentive for beneficiaries to have their information updated. 25 Improving Data Quality for an Effective Social Registry in Indonesia Source: https://unsplash.com/photos/wCnGumGxhDw Finally, each step of data updating from data collection, exchange, validation, verification processes should be safeguarded in several ways. Safeguarding measures should include i) random spot checks during data collection by asking questions and comparing answers to enumeration results, ii) the data received from the primary source should be checked against data standards as well as for internal consistency (including duplicates), iii) the data should be cross verified/validated with other administrative data sources such as national ID and tax databases, iv) all the records failing to pass the above check step(s) should be addressed by specific error or inconsistency rectification protocols, v) protocols should be in place when self-reported information conflicts with information existing in the Social Registry or other information systems, vi) cross-checks on data from other administrative systems should be put in place to ensure that the Social Registry uses the most current data, vii) in case of data conflicts, several strategies may be followed, such as the system shows a red flag, or the system automatically replaces outdated data with the most recent data update from other administrative systems, viii) all the institutional agreements regarding the data exchange and the protocols followed should be in place, and ix) it will be essential for Indonesia to strengthen its currently incomplete data protection and privacy regime. 49 3.2 Efforts to Sustain Data Quality Building a Social Registry requires investments in and continuous improvements to all elements including data collection and management, IT systems, communication strategy, and most importantly human resources. Social Registries need to be actively managed, and therefore require clear protocols for updating household-reported information, interoperability between administrative databases, protocols for correcting data, supervision of data quality, and action plans when potential errors are found in the data. Key ingredients for operating an effective Social Registry include inter-institutional coordination, common eligibility concepts and a shared registration questionnaire, and capabilities for information sharing and data exchange. Even with the most sophisticated system, regularly updated and accurate data is a non-negotiable part of an effective targeting mechanism. 49 These include the passage of the Personal Data Protection Bill which is still pending approval at the time of writing this note. The main disagreement discussed in the parliament relates to the supervisory authority. See, World Bank (2020) Investing in People: Social Protec- tion for Indonesia’s 2045 Vision. Jakarta, Indonesia. 26 Improving Data Quality for an Effective Social Registry in Indonesia For investments in the social registry to be sustainable, it is important to balance immediate needs for new or better data  with long-term  strengthening of  the infrastructure and interface with citizens. There are many different types of costs involved in building a social registry. The first is private costs to individuals and households associated with participating in intake and registration processes. 50 The second cost involves the administrative costs covering both physical and human resource costs relating to staff salaries and training, and method and frequency of data collection. The third set of costs involves the costs of IT systems capabilities (software, database management, IT infrastructure at central and local levels). Studies have found that the costs of social registries that target multiple programs are small compared with total program costs. The cost of social registries ranges between USD1 and USD3 per household in most countries, or less than two percent of the value of benefits channeled through the targeting system.51 Over time social registries need to collect less information from the beneficiaries by relying on interoperability with other administrative databases. This system however may take some investment and time while increasing the frequency of updates throughout the process. Consequently this may increase both administrative cost and the efficiency or productivity of the system. Continuous on-demand methods can help smooth the cost of data updating. When looking only at the registration method, a survey sweep generally involves large “lump sum” budgets that must be financed entirely within a specific period. Continuous on-demand methods on the other hand offer the advantage of smoothing the costs and financing of intake, registration, and updating over time – which may be easier to finance as an on-going operating cost. Moreover, this approach can often take advantage of existing service windows or local municipal offices that provide other functions. Finally, where there is broader use of the social registry across a larger number of programs, governments benefit from efficiency gains in reducing associated administrative and intake costs and consolidating them through a single process. Although on demand systems offer significant benefits, the method poses several risks to exclusion by inadvertently not covering enough population due to information gaps and reduced incentives. Many of the non-poor may opt not to apply, presumably perceiving that the time needed to register is not worth it. Minorities, remote or island communities, persons who are illiterate or have limited education can also potentially be excluded from the register because they will rely on Dinsos and/or other community outreach or on information from others to apply.52 In this situation, active and mixed outreach methods are important to promote awareness and to ensure that everyone has access to registration. Given that DTKS has not been massively updated since 2015, the GoI can consider doing another survey sweep while continuing efforts to improve local updating and ODA. Without clear communication, ODA can offer few incentives for people to update their information, which will lead to people being excluded. Some countries have overcome these challenges by leveraging their social registries as the primary entry point for most or all social benefits and services. Chile, for instance, uses its social registry to determine eligibility for a range 50 These typically include the time and money invested by citizens to gather required documents, travel to citizen interface points, participation in intake and registration interviews, and so forth. 51 For more discussion on cost, see Grosh et all, 2022. 52 See, Lindert et al, 2019. 27 Improving Data Quality for an Effective Social Registry in Indonesia up to 80 different benefits and services including cash transfers, in-kind social assistance, energy subsidies, labor market programs, housing assistance, education and health benefits etc. This, coupled with multiple access points for ODA, has resulted in coverage of close to 75 percent of the population. Source: https://www.pexels.com/photo/aerial-photography-of-village-surrounded-by-trees-1865915/ It is very important to ensure that increasing reliance on interoperability to update information is coupled with reliable – and accessible – grievance mechanisms. Administrative datasets are not error- free, and updating information based on cross-checks may at times lead to errors. This does not outweigh the benefits of leveraging this information overall, but it does require ensuring that people can make complaints or, to put it in other words, that people can initiate requests to update their information and provide supporting documentation to correct an error. These avenues must be described in detail in clear protocols. While a grievance system has been in place through SLRT, it still requires: i) proper assessment of the grievance; ii) opportunity and access for any individual to file a grievance; iii) decisions free from bias, nepotism, patronage; iv) fair and consistent decisions; and v) decisions that are impartial, transparent and capable of review. Interoperability should complement on-demand mechanisms and is important in maintaining data quality. Data exchanges between databases/programs can be operationalized via interoperability or other data sharing arrangements. Interoperability frameworks will facilitate the exchange of data from other administrative information systems to complement “self-reported” information from citizens. 53 53 Examples include linking to administrative information systems such as: civil registration database, national population register, land or property cadasters, vehicle registration, tax system, social security contributions system, pensions payments system, labor and unemploy- ment, education, health, etc. 28 Improving Data Quality for an Effective Social Registry in Indonesia Interoperability can also function as a “built-in” mechanism for data verification and validation. This includes checking data to ensure that the value matches data that may have been provided to other government agencies. Furthermore, data validation will ensure that data collected from citizens are valid (clean, correct, and useful). Integration of databases however can be fully achieved when a unique identifier such as NIK is universal and reliable. Interoperability also requires clear protocols, MOUs, even legal frameworks across agencies and high priority on ensuring quality of data with which DTKS is updated. In the future, emerging technologies such as big data analysis and Artificial Intelligence (AI) can be tapped for beneficiary and data updating. Advancement in data analysis and technology can help solve the inclusion and exclusion errors issues and facilitate more coordination in social protection. Between inclusion and exclusion errors, the priority for improvement should focus on catching those still excluded from the social protection system. For example, geospatial targeting that uses machine learning to create small area estimates of socioeconomic conditions can improve pre-list and highlight ‘clusters’ of the population that are still excluded from the system. 54 Ultimately however maintaining high quality data will be more important than any technology implementation in achieving an effective social registry in Indonesia. 54 See more examples in Grosh, 2022. 29 Improving Data Quality for an Effective Social Registry in Indonesia References Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin A. Olken, Ririn Purnamasari, and Matthew Wai-Poi. 2016. “Self-Targeting: Evidence from a Field Experiment in Indonesia.” Journal of Political Economy 124 (2): 371–427. https://doi.org/10.1086/685299. Alatas, Vivi, Abhijit Banerjee, Rema Hanna, Benjamin A. Olken, and Julia Tobias. 2012. “Targeting the Poor: Evidence from a Field Experiment in Indonesia.” American Economic Review 102 (4): 1206–40. https://www.aeaweb.org /articles?id=10.1257/aer.102.4.1206. Barca, Valentina, and Richard Chirchir. 2017. “Integrating Data and Information Management for Social Protection: Social Registries and Integrated Beneficiary Registries.” Department for Foreign Affairs and Trade, Government of Australia. https://www.dfat.gov.au/sites/default/files/integrating-data-information- management-social-protection-full.pdf. Bassett, Lucy & Giannozzi, Sara & Pop, Lucian & Ringold, Dena, 2012. “Rules, roles, and controls : governance in social protection with an application to social assistance,” Social Protection Discussion Papers and Notes 67612, The World Bank. Grosh, Margaret, Phillippe Leite, Matthew Wai-Poi, and Emil Tesliuc, editors. 2022. Revisiting Targeting in Social Assistance: A New Look at Old Dilemmas. Washington, DC: World Bank. doi:10.1596/978-1-1814-1. License: Creative Commons Attribution CC BY 3.0 IGO Leite, Phillippe, Tina George, Changqing Sun, Theresa Jones, and Kathy Lindert. 2017. “Social Registries for Social Assistance and Beyond: A Guidance Note and Assessment Tool.” Social Protection and Labor Discussion Paper No. 1704, World Bank, Washington, DC. https://openknowledge.worldbank.org/ handle/10986/28284. 30 Improving Data Quality for an Effective Social Registry in Indonesia Annexes Annex 1: Differences between Social Registries and Beneficiary Registries Social Registries: Targeting Beneficiary Registries: Program Delivery What it is: What it is: · It is a registry of beneficiaries across several pro- · It is a registry/database of all people and households grams registered (the percentage of population registered · It supports integrated M&E and planning, and can will depend on the data collection approach and the be designed to support integration of delivery sys- user program needs) tems (e.g. payments and grievances) · Its primary function is to support the initial imple- mentation phases of intake and registration, and as- sessment of needs and conditions for the purposes What it is not: of determining potential eligibility for enrolment in · It only includes existing program beneficiaries selected social programs (‘targeting’) · It cannot be used for ‘targeting’ or determination of · It aims to collect, record and store updated and his- (potential) eligibility for programs, because it only torical information on individual and household contains information on people or households who characteristics and circumstances, and verifies and have already been deemed eligible by existing pro- checks information consistency grams (beneficiaries and not potential beneficiaries · It does not necessarily include data from all social as- What it is not: sistance programs in a country (some programs may not have been integrated) · It includes population covered by the national survey · It is not necessarily ‘national’ since data was collected · It is not just a list of beneficiaries (eligible people who for registration purposes. have been selected for social protection programs) i.e. it includes data on potential eligible households too · It does not necessarily offer a current snapshot of poverty, unless data is kept sufficiently up to date Source: Baca, 2017. 31 Improving Data Quality for an Effective Social Registry in Indonesia Annex 2: Cost of Social Registries in Different Countries Number of Social Total programs Social Total annual registry cost of using the Size of registry Monthly Average expenditure in social information Budget Static/ social social social average benefit Amount of a assistance programs Country system cycle Year dynamic registry registry registry cost key program (ASPIRE) (millions) Number Mil. fam./ Per Average Program (millions) Expenditure people family year Brazil Cadastro Annual 2020 Dynamic US$ 20+ F: 28 US$ US$ Bolsa US$ 2018 Unico 188 P: 75 6.7 46.4 Familia in 6,681.76 2019 Registro Annual 2017 Dynamic US$ 20+ F: 4.9 US$ US$ Subsidio US$ 2017 Social de 30 P: 12.9 6.1 18.25 Unico 813.44 Hogares Familiar (SUF) in 2020 Colombia SISBEN IV 3-year 2017-20 Static US$ 20+ F: 8.5 US$ US$ Familia en US$ 2018 cycle 21.9 P: 39 2.58 51 Accion in 573.93 2019 NEW Social 5-year 2021-26 Dynamic US$ 20+ F: 13 US$ N/A Registrya cycle 52.0 (target) 4 (for P: 75 target) (target) Republic Lisungi 3-year 2017-19 Static US$ 3+ F: 0.12 US$ US$ Lisungi US$ 2019 of Congo Registry cycle 2.5 P: 0.6 20 25 Cash 12.00 Social Transfer in 2018 Jordan Unfied Annual 2020 Dynamic US$ 9 F: 1 US$ US$ National N/A N/A National 2.5 P: 7 2.5 63-200 Aid Fund Register (NAF) in 2020 Malawi UBR N/A 2018 Static US$ 5 F: 0.750 US$ US$ Social Cash N/A N/A (phase2) 1.27 P: 3.5 1.7 8 Transfer Programme (SCTP) in 2018 Mali RSU (only 4-year 2017-20 Static US$ 8 F: 0.092 US$ US$ Jigisemejiri N/A N/A counting cycle 1.68 P: 0.525 18.3 20 Cash those from transfer Jigisemejiri program in program that 2019 represents 24% of RSU) Pakistan National 5-year 2016-21 Static US$ 70+ F: 25.5c US$ US$ Kafaalatd US$ 2016 Socio- cycle 56 P: 167 2.19 14 1,036.62 Economic Registry (NESER)b Philippines Listahanan 3-year 2019-21 Static US$ 20+ F:16 US$ US$ Pantawid US$ 2016 cycle 54 P:75 3.4 11-32 program in 1,080.47 2019 Senegal Registre 4-year 2019/20- Static US$ 3 F:0.56 US$ US$ Programme N/A N/A Social cycle 2023/24 3 P:4.8 5.36 16 National des Bourses de Sécurité Familiale (PNBSF) in 2019 Tunisia Amen 4-year 2016-20 Static US$ 3+(TBC) F:1 US$ US$ Permanent US$ 2015 cycle (still on- 3-4[11] P:3.6 3-4 60 cash 212.69 going (target) transfer program (PNAFN) in 2019 Source: Original data collection based on informal survey of World Bank staff leading projects supporting the deveiopment of these social registries in consultation with government officials. Note: AMEN = safety (in Arabic); N/A= not available a. in terms of municipalities support to SISBEN, there are more than 2,100 people who guarantee the day-to-day operation and implementation of the SISBEN. The costs associated with the staff are under the responsibility of municipalities, and they add to about col$29 billion per year (US8.8 million per year using 2020 conversion factor). b. NSER administrators are already testing new functionalities to move toward a dynamic system, and some of future functionalities as apart of the response of the country to COVID-19 pandemic c. NSER unit is household. On Average, NSER found 1.3 families per household. d. Kafaalat is the main program in Pakistan as part of Ehsaas, which is the broader Social Protection and Poverty Alleviation Program launched in March 2019. Kafaalat was previously called Unconditional Cash Transfer under Benazir Income Support Program-BISP; BISP now one of the programs under Ehsaas) Source: Grosh et all, 2022. 32 Improving Data Quality for an Effective Social Registry in Indonesia Annex 3: Shares of data updates in Verval and changes in poverty rates in selected districts No. District % Verval %Poor 2018 %Poor 2019 Change 1. Pohuwato 71.4 19.40 18.16 -1.24 2. Kota Tasikmalaya 80.3 12.71 11.60 -1.11 3. Pangkajene Kepulauan 62.5 15.10 14.06 -1.04 4. Manokwari Selatan 77.7 30.87 29.94 -0.93 5. Nganjuk 60.3 12.11 11.24 -0.87 6. Samosir 83.2 13.38 12.52 -0.86 7. Kota Samarinda 0.2 4.59 4.59 0.00 8. Bengkalis 2.9 6.22 6.27 0.05 9. Empat Lawang 12.2 12.25 12.3 0.05 10. Halmahera Utara 0.0 4.51 4.55 0.04 11. Mamasa 0.0 13.38 13.42 0.04 12. Kota Banjarmasin 1.9 4.18 4.20 0.02 13. Banjar 0.1 2.70 2.72 0.02 14. Kapuas Hulu 0.0 9.60 9.62 0.02 Source: Bappenas, 2021. Annex 4: Modalities for Intake and Registration in a Social Registry Offsite In-situ via home visits Mobile temporary service Door-to-door mass registra- desk: kiosk; "registration tion or scheduled individual camps"; job fairs; other plac- home visit es where peole congregate Technology assisted In Person Applying online; phone in- At local office terview; chatbots for sched- uling or other simple queries, and so on Source: Grosh et all, 2022