June 2025 Europe Support for Marginalized Households in Bulgaria, Romania, and Slovakia Pilot Evaluation Report P177945 Social Protection & Jobs © 2025 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved. This work is a product of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy, completeness, or currency of the data included in this work and does not assume responsibility for any errors, omissions, or discrepancies in the information, or liability with respect to the use of or failure to use the information, methods, processes, or conclusions set forth. The boundaries, colors, denominations, links/footnotes and other information shown in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. The citation of works authored by others does not mean the World Bank endorses the views expressed by those authors or the content of their works. Nothing herein shall constitute or be construed or considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2025. Support for Marginalized Households in Bulgaria, Romania, and Slovakia. Pilot Evaluation Report. © World Bank.â€? Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Sample credit for a non-WB image: © Ami Vitale / Panos Pictures. Used with the permission of Ami Vitale / Panos Pictures. Further permission required for reuse. Sample credit for a WB image: © Curt Carnemark / World Bank. Further permission required for reuse. Acknowledgments This report was prepared by Lubica Hamarova (consultant, HECSP), Theresa Jones (consultant, HECSP), Lara Lebedinski (consultant, HECSP), Anca Monica Marin (consultant, HECSP), and Vera Borisova Veleva (consultant, HECSP) under the supervision of Lucian Pop (senior social protection specialist, HECSP). Jorge Luis Castaneda Nunez (economist, DIME) provided technical guidance throughout the design and analysis phases of the evaluation. The team is grateful to Ana Maria Munoz Boudet (lead economist, HGNDR) and Valerie Morrica (senior social development specialist, SCASO) for valuable advice and feedback as peer reviewers. The project was financed by the European Commission through the Supporting Bulgaria, Romania and Slovakia to Address Socio-economic Exclusion Trust Fund (TF085534). The opinions, interpretations, and conclusions expressed herein do not reflect the views of the World Bank, its board of executive directors, or the governments they represent. Table of Contents Introduction ............................................................................................................................... 1 1. Pilot Overview ....................................................................................................................3 1.1. Planned Intervention ......................................................................................................3 1.2. Pilot Localities .................................................................................................................5 2. Evaluation Objectives and Methodology .........................................................................13 2.1. Theory of Change and Objectives of the Evaluation .....................................................13 2.2. Methodology.................................................................................................................14 3. Findings ............................................................................................................................ 23 3.1. Beneficiaries’ Profile .....................................................................................................23 3.2. Process Evaluation ........................................................................................................31 3.3. Outcome Evaluation ......................................................................................................49 4. Summary of Findings .......................................................................................................59 Annex 1. Tables and Figures .....................................................................................................62 Annex 2 Trend Analysis and RCT Specifications .......................................................................74 Annex 3 Outcomes Measurement ........................................................................................... 76 Introduction Marginalized households, including marginalized Roma, continue to face entrenched poverty, limited access to public services and employment, and persistent socioeconomic exclusion rooted in intergenerational deprivation. Despite the support provided through targeted social assistance programs, these households often remain disconnected from broader socioeconomic opportunities due to structural barriers and limited engagement with support services. The pilot intervention evaluated in this report was designed to address some of these challenges by testing approaches to enhance financial inclusion, promote self-empowerment, and improve the delivery of social support to extremely poor households, particularly those receiving means-tested benefits. This report presents the evaluation results of the pilot intervention, which aimed to enhance the socioeconomic inclusion of marginalized Roma households in Bulgaria, Romania, and Slovakia. The pilot targeted households in extreme poverty receiving means-tested social assistance with a focus on promoting financial inclusion and personalized casework support. The personalized support was based on beneficiaries’ self-assessments and included the development of individual plans, coaching, financial literacy training, workshops, and referrals to services. The pilot intervention was grounded in a dual-pillar theory of change. The first pillar focused on promoting financial inclusion through tailored financial literacy training. The second pillar emphasized self-empowerment and socioeconomic advancement through personalized casework support, which included self-assessment, goal-setting, coaching, and referrals to services. The underlying assumption was that by equipping beneficiaries with tools to assess their own needs and set personal goals—while simultaneously improving their financial capabilities and access to services—participants would experience intermediate outcomes, such as improved budgeting and goal-setting, increased confidence in using banking tools, and greater engagement with employment services. These, in turn, were expected to lead to longer-term outcomes, such as enhanced financial inclusion, greater labor market participation, and improved overall well-being. The evaluation framework varied across countries due to data access for randomizing participation in the pilot and the availability of survey firms for data collection in Roma communities. In Romania, a difference-in-difference evaluation framework was implemented, while in Bulgaria, the evaluation captures changes in outcomes over time for pilot participants only. For Slovakia, solely qualitative methods were used. The evaluation was designed to answer two overarching questions: 1. Process Evaluation: How was the pilot implemented across the three countries, and to what extent did implementation align with the intended design and theory of change? 2. Outcome Evaluation: To what extent did the pilot achieve its intended short-term outcomes in the areas of financial literacy and financial health, labor market participation, socio- emotional skills, and subjective well-being? The rationale for using both process and outcome evaluations lies in the complexity of the intervention and the diversity of implementation contexts. The process evaluation was essential to understanding the fidelity, quality, and contextual adaptation of the pilot, especially given the varied 1 institutional capacities and community dynamics in each country. It also helped assess whether the assumptions underpinning the theory of change held true in practice. The outcome evaluation, on the other hand, aimed to measure the effectiveness of the intervention in achieving its intended results. Given the constraints in data access and randomization, different approaches were adopted: a difference-in-differences design in Romania (where a control group was available) and trend analysis in Bulgaria. In Slovakia, only the process evaluation was carried out, based on qualitative research methods. This approach of using mixed methods and two types of evaluation was expected to allow for a nuanced understanding of both the implementation process and the observed changes in participant outcomes, while acknowledging the limitations of each method. Evaluation of the pilot interventions faced several limitations that constrain the interpretation and generalizability of the findings. First, only Romania implemented a randomized control trial, while Bulgaria relied on trend analysis and Slovakia on qualitative methods, limiting causal inference. Second, the shortened implementation period (12 instead of 18 months) reduced the time for outcomes to materialize. Third, seasonal effects likely influenced key indicators, such as employment and savings, while small sample sizes reduced statistical power. Finally, low literacy among participants may have affected the accuracy of survey responses; survey timing issues—such as baseline data being collected after pilot activities began in Bulgaria—further complicated analysis. This report is intended for policymakers and practitioners engaged in social protection and inclusion programming, particularly those working with marginalized populations in Europe. Structure of the Report The report begins with an introduction and overview of the pilot’s objectives followed by a description of intervention components. It then outlines the local contexts in Bulgaria, Romania, and Slovakia, highlighting differences in pilot locations and socioeconomic conditions. Next, it proceeds with a timeline of activities and a thorough explanation of the evaluation methodology. The findings are presented in two main parts: process evaluation (focusing on implementation arrangements, implementation fidelity, and participant engagement) and outcome evaluation (analyzing changes in financial behavior, employment, subjective well-being, and socio-emotional skills). The document concludes with a summary of key insights. 2 1. Pilot Overview 1.1. Planned Intervention Intended scope. The target population of the pilot intervention consists of households living in extreme poverty, particularly marginalized Roma, and benefiting from poverty-targeted income support programs (targeted social assistance cash benefits). Initiated by the European Parliament, the intervention (henceforth, pilots) was expected to explore an innovative way to deliver welfare benefits and support social inclusion in three selected countries: Bulgaria, Romania, and Slovakia. The broad objectives of the intervention models were to improve consumption smoothing (including changes in consumption patterns), increase the financial inclusion and agency of marginalized households receiving social assistance, and aid beneficiaries’ empowerment and socioeconomic advancement. More specifically, the pilots were aimed at testing a range of interventions: (i) policy changes to increase the frequency of cash benefits payments (from one payment to two payments per month); (ii) promoting financial inclusion through a shift in the cash benefits delivery mechanism from cash to e-pay or debit cards linked to a free or low-cost bank account; (iii) casework, including family assessments; behavioral change interventions, such as goal-setting and family development plans; accompaniment; coaching; and referrals (including to employment services and skills training). The pilots were expected to introduce a goal-setting and planning tool to build and track the achievement of concrete goals by beneficiaries. Allowances would be provided to beneficiaries for their participation in trainings and other activation measures to compensate for time spent and to incentivize attendance. The pilots were anticipated to be implemented in selected municipalities with high shares of marginalized households, including Roma, in Bulgaria, Romania, and Slovakia. Implementation was carried out by a consortium contracted by the European Commission that included international and local NGOs (implementation partners). Revised scope of the intervention. Following dialogue with the national authorities in Bulgaria, Romania, and Slovakia, and the onboarding of implementation partners, the main elements of the design included two of the three types of interventions considered at the concept stage: (i) promoting financial inclusion by providing financial literacy training/coaching and by assisting willing beneficiaries to shif from receiving benefits in cash to debit cards or basic e-pay cards; and (ii) casework interventions, including family assessment, goal-setting, and family development plans; coaching; accompaniment; and referrals (including to employment services and skills training). The casework approach selected by the implementation partners was the Poverty Stoplight (PSL) intervention model.1 The change in the frequency of payments was not accepted by national authorities. Pilot components. While the intervention design allowed country variations in the extent of implementation and the composition of some activities (including training, coaching, and referrals), in many respects, the intervention followed the same approach across countries. More detail on the 1 https://www.povertystoplight.org/en/; see also Section 3.2. 3 intervention and its implementation, including country differences, can be found in Section 3.2, Process Evaluation Findings. The main components of the intervention were: • PSL self-assessment. Each participating household completed the PSL self-assessment with the support of local facilitators and/or mentors. This process was intended to serve as the entry point for engagement, enabling families to reflect on their current (baseline) situation across key life dimensions. The self-assessment was updated at the end of the pilot using the same set of key life dimensions indicators. • Priority Setting and Individual Development Plans. Following the self-assessment, families worked with facilitators to identify the key indicators they would like to prioritize and work on, which were reflected in individual development plans. These plans outlined specific, achievable goals based on the families' aspirations and needs. • Coaching. Ongoing motivational support was provided by pilot staff through regular check- ins, reminders to attend activities, and encouragement. Coaching was intended to help families maintain focus and make steady progress toward their goals. • Action plan of follow-up activities. Based on the needs identified in the PSL self-assessment, identified priorities, personal development plans, and other considerations, implementers drafted an action plan that listed the planned activities, such as: o Trainings. Participants take part in targeted training sessions covering financial literacy and other topics. o Coaching. Families are offered access to both one-on-one and group coaching on topics such as health, housing, parenting, and employment. These sessions address immediate concerns and support informed decision-making, and sometimes include referrals to specific services. o Referral and accompaniment. When additional support is needed, families are referred to relevant services. Staff (facilitator/ mentor) also provide hands-on assistance—for example, helping beneficiaries obtain documentation, register with general practitioners (GPs), take health checks, and apply for housing support or disability assessment. Delays in implementation. Implementation of activities was delayed compared to the initial plan. Contributing factors included a delay in onboarding the implementation partners and lack of access to social assistance beneficiaries’ data due to data privacy and data protection concerns raised by authorities in the three countries. As a result, the duration of the interventions from initial to the final PSL family assessment decreased to about 12 months from the initial 18 months. Revisions in the scope of evaluation activities. Due to lack of access to the administrative data of beneficiaries of targeted social assistance benefits, it was not possible to randomize the treatment and construct a control group (counterfactual) in Bulgaria and Slovakia. Thus, an impact evaluation was carried out only for Romania. Also, because of challenges in data access and enrollment of beneficiaries in the pilots, the baseline evaluation surveys could not be implemented in advance of the start of pilot activities. Lastly, because of the shorter duration of interventions, the evaluation does not focus exclusively on behavioral change but also includes outcomes that are easier to influence in the short term. Outcomes of interest for the evaluation are financial literacy and financial 4 health, participation in the labor market, expected improvement in income, subjective wellbeing, and socio-emotional skills. 1.2. Pilot Localities The process of selecting pilot locations in each country aimed to balance community needs with the practical conditions required for effective implementation. The process followed these steps: • Selection criteria discussed and agreed upon with national governments in the three countries. • Long list of localities discussed with the implementers to consider their capacity at the local level. • Revised list of localities based on the initial selection criteria and implementation capacity consulted and endorsed by governments and implementers. The final selection criteria were: • Municipalities with at least 10% Roma residents and a total Roma population of no less than 2,000 individuals. • A minimum of 300 households receiving targeted means-tested social assistance benefits. These benefits varied by country and included guaranteed minimum income (GMI) programs, means-tested family benefits, and means-tested housing allowances (e.g., heating benefits). • Roma families residing in legal settlements with valid identity documents. • Proximity to basic infrastructure for financial transactions (e.g., ATMs or banks for debit card payments). • Availability of employment and social assistance services within reach of the community. • Demonstrated interest and support from local authorities and institutional stakeholders. • Local implementing capacity to manage and sustain pilot activities. The following sections describe the pilot locations selected for each country. Bulgaria In Bulgaria, the three locations of Novi Pazar, Kotel, and Gradets were selected. The municipalities of Kotel and Novi Pazar met key demographic and socioeconomic criteria, including a significant number of households receiving means-tested social benefits. In the Kotel municipality, 376 households received the Monthly Social Assistance (MSA) benefit,2 208 received family allowances for raising a child under the age of one, and 1,568 received monthly support for raising a child until completion of secondary education. In Novi Pazar, the corresponding figures were 100, 86, and 996 households, respectively. Novi Pazar had an active network of local NGOs and municipal actors engaged in social inclusion, offering a supportive environment for community outreach. While institutional structures were more limited in the Kotel municipality, the implementer had previous experience and a local presence. The town of Kotel had too few eligible beneficiaries; most potential participants were concentrated in Gradets, a rural Roma village in the Kotel municipality. Given the distribution of eligible Roma households within the Kotel municipality, the implementer proposed the 2 MSA is a GMI type of benefit. 5 inclusion of Gradets as a pragmatic adjustment, although it did not meet several criteria, such as access to banking services, employment offices, or a strong institutional presence. The implementer therefore retained Kotel but with a focus on activities in Gradets. All three Roma communities share strong internal identities but differ in language, religion, and cultural cohesion. In Novi Pazar, Roma officially make up 17.3 percent of the population, though local estimates suggest the actual proportion may be closer to 22 percent. The community is predominantly Turkish speaking. In the town of Kotel, Roma constitute around 28 percent of the population and include diverse sub-groups. Most are Christian and speak Romani. In Gradets, over 80 percent of residents are Roma. This community forms a distinct and insular subgroup with strong internal traditions and its own dialect, making it culturally and socially unique within the region. Across all three locations, Roma communities face entrenched poverty, spatial segregation, and exclusion from formal employment and public services, but the degree of marginalization varies significantly by setting. Informal housing, insecure legal status, and administrative barriers are widespread, but variations in living conditions, service availability, and integration into the urban environment shape the degree and form of marginalization experienced in each setting. Gradets— rural, geographically isolated, and severely underserved— stands out as the most structurally disadvantaged, while Kotel and Novi Pazar, as urban municipal centers,3 offer somewhat better access to infrastructure and services. Infrastructure in Roma areas is uneven across all locations, with rural Gradets facing the most severe deficits. Novi Pazar is a small urban municipality with partly regulated Roma neighborhoods. The central areas have legal housing and infrastructure, but the outskirts lack running water and rely on illegal electricity connections. Public transport has been discontinued, limiting mobility. In the town of Kotel, Roma live in segregated neighborhoods within the urban area. Some zones have regulated housing and basic infrastructure, but marginal areas still lack utilities. Road and transport access to nearby villages is limited but available. Gradets is a large rural settlement with significant infrastructure deficits; many homes lack sewage, clean water, and garbage collection. Located 16 km from Kotel, the nearest urban center, the village is geographically isolated, institutionally peripheral, and underserved. Access to public and social services declines sharply from urban centers to rural peripheries. Novi Pazar offers a relatively good network of public services due to its compact size. ATMs, banks, and health services are centrally located and accessible. Community support centers are well-established and widely used by Roma residents. In Kotel, public services (including banks, ATMs, healthcare) are available in the town center. Social services are more limited and less specialized. Gradets is severely underserved, in terms of public and social services. No kindergarten is available, so children must travel to Kotel for early education. The primary school is fully segregated, and access to health or administrative services is minimal. Seasonal migration is a common coping strategy across all three locations. Roma families in Novi Pazar generally live in nuclear households and often own their homes. Labor migration, particularly to Western Europe for construction work, is common, and many families depend on remittances. In 3 While Kotel and Novi Pazar are officially classified as part of rural municipalities (defined by settlements with fewer than 15,000 inhabitants), both are designated as towns and function as local administrative and economic centers with urban characteristics, such as higher population density, greater access to public services, and diversified employment opportunities. 6 Kotel, households tend to be multi-generational, and women frequently act as de facto heads of household when male family members are away for seasonal work. Internal migration within Bulgaria is widespread and often tied to informal, low-paid jobs. In Gradets, large family units dominate, with a similar pattern of seasonal internal migration. Men typically leave for forestry or construction work, while women remain in the village to care for children and manage the household. Low educational attainment and early school leaving are prevalent and closely linked to poverty and migration. Education levels in Novi Pazar are uneven. While many Roma children attend a segregated primary school, a number continue into vocational education. Illiteracy exists but is less widespread than in other localities. In Kotel, educational attainment is generally low, with high dropout rates and widespread adult illiteracy, particularly among women. Gradets faces even greater educational challenges, with critically low attainment and high rates of early school leaving. Many families prioritize seasonal work over formal education, and girls are at risk of discontinuing school at a young age. Informal and insecure employment dominates all three communities, reinforcing cycles of poverty. The unemployment rate in Novi Pazar is around 6 percent, roughly in line with national averages. Some Roma have access to local employment or migrate seasonally abroad, and youth in particular show potential if supported with training. In contrast, Kotel records a significantly higher unemployment rate of around 29 percent, with most Roma engaged in informal, seasonal work such as forestry and herb collection. In Gradets, economic activity is almost entirely informal, with families relying heavily on seasonal labor across the country. Employment is unstable and underpaid, and contributes to persistent poverty. Despite high bank account coverage among social assistance beneficiaries in both municipalities, financial literacy is limited. In Kotel and Novi Pazar, over 90 percent of social assistance beneficiaries reportedly receive their GMI payments via bank accounts. In June 2023, data showed 347 out of 380 payments in these localities were transferred through banking channels. While the high uptake is notable, focus group discussions in both municipalities revealed that many social assistance recipients have limited knowledge of how banking systems work. Participants showed poor understanding of bank operations, account features, and terms and conditions, and use of debit cards. In Novi Pazar, for instance, older Roma women—especially those who are illiterate—rely heavily on relatives or bank staff to help them withdraw money from ATMs. Younger women in both Kotel and Novi Pazar are more familiar with card use but still struggle with basic functions, such as accessing checking account balances. Across groups, there was little awareness of banking terms and conditions, or distinction between banks and alternative financial institutions. Romania The urban municipalities of FeteÈ™ti and Slobozia were selected. The criteria related to share and size of the Roma population were satisfied by very few localities, so lower thresholds were applied (5 percent share of Roma residents and more than 1,200 Roma residents). Another criterion was to avoid overlap with the project on integrated community services coordinated by the Ministry of Labor and Social Solidarity. FeteÈ™ti met all criteria. Although the share of Roma population is only 2.8 percent, Slobozia was selected based on the suggestion of the implementer. Slobozia, the county capital, and FeteÈ™ti are among the largest municipalities within the county of Ialomita. They follow the national 7 trend of declining population and decreased shares of the Roma population in the total population. Data from the 2021 census show that the share of Roma population declined from 2.8 (2011) to 2.2 percent in the municipality of Slobozia and from 5.4 to 4.9 percent for the municipality of FeteÈ™ti. The number of families receiving means-tested benefits also decreased. In January 2024, during the pilot implementation, a new means-tested benefit was introduced (the Minimum Inclusion Income, or VMI) that consolidated two existing means-tested benefits, the Guaranteed Minimum Income (GMI) and the Family Support Allowance (FSA). The total number of families benefiting from VMI, even if not directly comparable to the data on GMI and FSA, seems to be substantially lower in Slobozia. An increase is registered in the updated data for 2025 (Table 1). The largest banking institutions have offices and ATMs available in the two selected localities. Nonetheless, as at the national level, there is a very low share of means-tested benefits paid in bank accounts. This picture is similar for the uptake of minimum inclusion income by card (in a bank account); see Table 1. Table 1. Uptake of means-tested benefits by card in the selected localities May 2023 March 2024 March 2025 Number of Paid in bank Number of Paid in bank Number of Paid in bank beneficiaries account beneficiaries account beneficiaries account GMI FSA GMI FSA VMI VMI VMI VMI Slobozia 129 158 8 31 176 8 217 20 Fetesti 235 118 1 1 227 0 270 0 Source: County Agency of Payments and Social Inspection of Ialomita County. Both localities include urban marginalized areas, with accumulated deficits in human capital, low levels of formal employment, and poor living conditions. Housing is informal (lack of property deeds) and connections to utilities are inadequate. A large portion of means-tested benefit recipients live in these marginalized areas. While more information is available for Slobozia, the broad trends are likely to be no better in FeteÈ™ti. Key problems drawn from a study conducted in the marginalized communities of Slobozia in October-November 2023 highlight the following: (i) lack of formal employment; (ii) high risk of school dropout among children and young people (the study additionally reveals cases of older siblings taking care of their younger brothers or sisters, or working as day laborers); (iii) low access to health services (hampered additionally by low health literacy), as reflected in a substantial share of unvaccinated children or persons not registered with a family doctor and a low percentage undertaking routine health checkups; (iv) segregation, stigmatization, and discrimination with spatial segregation between ethnic communities; (v) improper and informal housing; and (vi) low access to emergency services due to poor street infrastructure.4 The sewage system seems to be especially problematic in FeteÈ™ti, and there have been cases of informal dumping of waste. In Slobozia, the percentage of active population is slightly higher than the national level (43%), nearly matching the level for urban areas (47%). By comparison, the percentage in FeteÈ™ti is lower than the national average and substantially lower than for urban localities. Additionally, as compared to the national average, FeteÈ™ti has a much higher share of unemployed individuals within its active 4 https://galslobozia.ro./ 8 population and of housewives in its inactive population.5 The most common sectors for employment are the service sector (mostly commerce) and construction, as well as textiles and food. Employment in agriculture is low, despite strong representation of agriculture in the area.6 The supply of education and social services is better developed in Slobozia than in FeteÈ™ti, with a range of social services delivered by public providers. Licensed social services are supplied by private providers in both localities. In Slobozia, religious associations provide community assistance services. In both locations, social services are provided in the community for adults. FeteÈ™ti has a center for counseling and support for children and parents, as well as a daycare center. Education up to upper secondary is available in both locations. The network of early childhood education services is substantially less developed in FeteÈ™ti. The County Employment Agency is represented in both localities, but its capacity is much lower in FeteÈ™ti. Slovakia The Slovak localities selected were Bardejov, SpiÅ¡ský Hrhov, and Doľany.7 The key features of each locality are summarized below. Bardejov is an urban district center offering a strong network of public services yet marked by socioeconomic disparities and localized segregation. As the district capital, Bardejov centralizes public services, including the district office of Labor, Social Affairs, and Family, a free debt consultancy center, healthcare facilities, and banks. The municipality, home to around 30,840 residents, is characterized by limited economic opportunities, mostly in tourism, services, and construction. According to the Atlas of Roma Communities (ARC 2019), Bardejov has an estimated 2,532 Roma residents, concentrated primarily in two localities. The largest settlement, PoÅ¡tárka, in the suburban outskirts, is home to 1,483 people and contains approximately 190 dwellings. Most housing units in PoÅ¡tárka are municipal rental apartments. Rental apartments are connected to water, sewage, and electricity, but many residents rely on solid fuel (wood, paper, waste, etc.) and furnaces for heating. There is a community center and a Catholic primary school. While Romani is the primary spoken language, residents are proficient in Slovak from their formal education and routine interaction with public institutions. SpiÅ¡ský Hrhov is a rural locality with a relatively high proportion of Roma residents and is recognized as a model of successful Roma integration. The village has approximately 1,743 inhabitants. According to the 2021 Census, only 1.7 percent of residents self-identified as Roma; however, estimates from the ARC 2019 suggest that Roma make up around 20 percent of the population. Roma residents are integrated into the village, primarily in self-sufficient flats and houses. The locality benefits from well-developed infrastructure, a local social enterprise, a community center offering educational and leisure activities, and a primary school that actively promotes inclusive education. 5 Census 2021 data. 6 Explained by the high fragmentation of agricultural areas – source: Integrated Urban Development Strategy of the municipality of Slobozia https://municipiulslobozia.ro/wp- content/uploads/documente/2024/proiecte/SIDU-Actualizat-2024.pdf 7 One locality initially selected for the pilot was withdrawn after challenges emerged during the PSL self- assessment. Some of those enrolled refused to participate, were hostile, or did not attend scheduled sessions. A contributing factor to the deterioration in community dynamics may have been the temporary closure of a community center and staff restructuring. 9 SpiÅ¡ský Hrhov has attracted considerable attention in both the media5 and academic literature;6 it is often cited as a case study in effective Roma integration, which is largely attributed to the leadership and long-term vision of its former mayor. Doľany, a village neighboring SpiÅ¡ský Hrhov, is characterized by spatial segregation and severe socioeconomic marginalization. The village has 779 residents, over 70 percent of whom are Roma, according to both the 2021 Census and the ARC (2019). The municipality includes the segregated Roma settlement of RoÅ¡kovce, home to approximately 500 residents. The settlement consists of around 70 dwellings, comprising a mix of municipal flats and poor-quality informal housing, and is geographically segregated from the municipal center. While basic utilities such as water and electricity are formally available, many households struggle to access them consistently due to poverty. The community has an elementary school, a local community center staffed by social workers and community workers, and an office of an NGO. The three localities differ in terms of household structure, socioeconomic profile, and service accessibility. These differences have direct implications for the pilot implementation, particularly concerning financial inclusion, employment support, and household engagement. Bardejov is characterized by smaller, less economically dependent households compared to the rural municipalities of Doľany and SpiÅ¡ský Hrhov (Table 2). The average household size in Doľany and SpiÅ¡ský Hrhov exceeds the national average. These demographic patterns imply a higher dependency burden, greater childcare responsibilities, and increased vulnerability to poverty. This is also reflected in the average value of benefits paid to households in 2023, which correlates broadly with household size and needs. Table 2. Socioeconomic profile of pilot localities in Slovakia Indicator Bardejov Doľany SpiÅ¡ský Hrhov Average household size 1.7 3.9 3.1 Recipients of Material Needs Allowance (2019) 209 30 49 Average benefit received (€) 127 180 248 Unemployment rate (%) 6.5% 29.7% 5.9% % of benefit recipients using debit card 30% 28% 36% Source: Requested from COLSAF (June 2023) Employment conditions vary among the three localities. Doľany has an exceptionally high rate of registered job seekers, reflecting limited economic activity within the municipality and deep labor market exclusion. In contrast, SpiÅ¡ssky Hrhov has a considerably lower unemployment rate of around 5 percent,7 which reflects the successful long-term integration efforts. Bardejov has a similar unemployment rate, thanks to relatively greater opportunities for employment. Regardless, informal work and seasonal migration remain widespread coping strategies for the Roma population in all three localities. While Bardejov centralizes public services within the district, neither Doľany nor SpiÅ¡ský Hrhov has employment, social assistance, or banking services within their municipal boundaries. Residents of these rural municipalities must travel approximately 6 km to the nearest district capital, LevoÄ?a, where they can access public services, banking services, and supermarkets. According to the Central Office of Labor, Social Affairs & Family (COLSAF) data, in 2023, between 28 to 36 percent of 10 benefit recipients received payments via debit card, with the highest share in SpiÅ¡ský Hrhov and the lowest in Doľany. Timeline of Activities Pilot timelines for the three countries are shown below. Overall implementation, from the start of outreach and enrollment until the end of all activities, took place over a longer period in Bulgaria (18 months) and Romania (16 months) than in Slovakia (12 months), in part because it took about six months to complete enrollment, as for the first self-assessment in the case of Romania. In Bulgaria and Romania, the financial literacy training and other interventions followed the preparation of personal development plans. The implementation period in Slovakia was shorter, in part because processes such as enrollment and the self-assessment were each completed within two months. Financial literacy training began the same month as most self-assessments. In Bulgaria, recruitment started in October 2023, the earliest of the three countries. Due to difficulties the target group had in understanding and selecting concrete priorities, priority-setting took longer than expected—several months with multiple follow-ups. This activity was not completed until July 2024. Follow-up pilot interventions continued until March 2025. Thus, some families had more than 12 months to work on their priorities with the support of mentors and facilitators, while others had less than 10 months. Table 3. Timeline of pilot activities in Bulgaria 2023 2024 2025 Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Enrollment Self-assessment 144 8 Priority-setting8 Personal Development Plans Financial literacy training Other interventions Achievements assessment Note: The figures in the table represent the number of families from each month registered on the PSL platform under the self- assessment activity (April 4, 2025 version). In Romania, after carrying out some preparatory activities, pilot implementation spanned from January 2024 to April 2025. The implementer received the lists to be used for enrollment in November 2023 for FeteÈ™ti and in January 2024 in Slobozia. Enrollment was particularly challenging in Slobozia, taking much longer than initially planned. The set of interventions started near the end of 2024 (see the timeline in Table 4). 8 Priority-setting numbers are higher than total households with priorities (107), as this is the count of how many households set at least one priority in the specified month. Each family sets between one and three priorities. 11 Table 4. Timeline of pilot activities in Romania 2024 2025 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr Enrollment Self-assessment 20 36 12 21 16 2 1 Priority-setting Personal Development Plans Financial literacy training Other interventions Achievements assessment Note: The figures in the table represent the number of families from each month registered on the PSL platform under the Self- assessment activity (April 4, 2025 version). In Slovakia, the pilot implementation spanned from April 2024 to March 2025. The launch was delayed primarily because of repeated attempts to access the list of means-tested social assistance beneficiaries for randomization of pilot participants. Enrollment took place from April to May, followed by sessions on financial literacy at the end of May. Table 5. Timeline of pilot activities in Slovakia 2024 2025 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr Enrollment Self-assessment 34 81 2 Priority-setting Personal Development Plans Financial literacy training Other interventions Achievements assessment Note: The figures in the table represent the number of families from each month registered on the PSL platform under the Self- assessment activity (April 4, 2025 version). 12 2. Evaluation Objectives and Methodology 2.1. Theory of Change and Objectives of the Evaluation The intervention was built on two main pillars. The first aimed to promote a shift in the delivery of social benefits from cash to debit or e-pay cards. This pillar included financial literacy training tailored to the beneficiaries' knowledge levels. The second pillar sought to promote self-empowerment and socio-economic advancement. This was to be achieved via the Poverty Stoplight (PSL) approach: self- assessment, priority-setting and individual development plans, coaching, and follow-up activities, such as training and referrals to social and employment services. As a result of these activities, intermediate outcomes, such as improved budgeting, enhanced confidence in banking tools, better goal-setting skills, and increased engagement with employment services, were anticipated. Ultimately, these intermediate outcomes are expected to lead to long-term benefits for the participants, including increased financial inclusion, improved labor market participation, and overall improved well-being. A set of socioemotional skills was also included among the intermediate outcomes. Poverty is often associated with a depletion of socioemotional skills. Poor individuals may feel less able or in control to change their circumstances (affecting their self-efficacy) or not being able to set long-term goals due to short-term needs and demands.9 Interventions like coaching, goal setting and planning, and others, have shown promise in building socioemotional skills and supporting the impact of more traditional anti-poverty interventions.10 In this study, we use a reduced set of socioemotional skills measures: goal-setting capacity, emotional resilience, and feeling of control over one’s own life.11 The success of the intervention relied on several assumptions. These included the participants’ willingness to engage; effective overall implementation, including delivery of personalized support and trainings by implementers; the influence of financial literacy and coaching on behavior; and the 9 For a review see Sheehy-Skeffington and Haushofer 2014, Vakis et al. 2016, and Cassio et al. 2021. 10 Cassio et al. 2021, Baranov et al. 2020 11 See Annex 3 Outcomes Measurement for a detailed description. 13 availability of local services and their responsiveness to referrals. The risks considered included low literacy levels, limited self-awareness, and short implementation timelines. The evaluation aimed to assess both the implementation process and the outcomes of the pilot. The process evaluation aimed to understand how the intervention was implemented across diverse local contexts in Bulgaria, Romania, and Slovakia, and to what extent the theory of change assumptions held. It examined the fidelity of implementation relative to the original design, the quality and consistency of delivery, and the experiences of both implementers and participants. Key areas of interest included outreach and enrollment, the use of the PSL self-assessment tool, the development of individual and family plans, coaching, and the delivery of follow-up services, such as financial literacy training and referrals. The process evaluation also explored the role of local implementing partners, the use of incentives, and the extent to which the intervention fostered trust, engagement, and participant agency. The outcome evaluation aimed to assess the extent to which the intervention achieved its intended short-term effects across five domains: (1) financial literacy and financial health (e.g., budgeting, saving, outstanding bills); (2) labor market participation (e.g., employment status, job-seeking behavior, registration with employment services); (3) socio-emotional skills (e.g., goal- setting, emotional resilience, self-regulation); (4) subjective well-being (e.g., life satisfaction, happiness, perceived marginalization); and (5) expectations for future income. 2.2. Methodology Overview of Data and Instruments The study uses both quantitative and qualitative data to assess the pilot. The main data sources used in the evaluation are grouped into three categories: 1. Two survey rounds 2. Qualitative data from focus groups and in-depth interviews 3. PSL socioeconomic data and family self-assessment modules Survey rounds 1 and 2. The two survey rounds were implemented in Bulgaria and Romania.12 The surveys were administered in person13 in the local language. The same questionnaire was used in both countries across both rounds, with a few questions removed in the second round—specifically, those that were not expected to change as a result of the pilot. The survey included a set of standard household questionnaire modules (household composition and socio-demographics, child grid, housing and living conditions, household income, expenditures, and labor) and additional modules designed specifically for this evaluation (social benefits module; financial inclusion, financial literacy, and financial health module; subjective wellbeing, aspirations, and socio-emotional skills modules). Both survey rounds included a monitoring module with questions about the households’ experience with the pilot implementation. 12 In Bulgaria, the data was collected by the survey company Estat. In Romania, it was collected by the survey company Metro Media Transilvania. The survey was not implemented in Slovakia due to challenges in identifying a suitable survey firm. 13 Due to some tensions in the community, a second round of the survey was conducted partially in person and partially over phone in Bulgaria. 14 Qualitative data. In all three countries, qualitative research methods in the form of in-depth interviews and focus groups were used to collect data on the implementation of the pilot. The qualitative research focused on the main processes to implement the pilot. The main respondents included participants and non-participants in the target population, and pilot implementers (local implementing agency staff, including facilitators and mentors), and representatives of Fundacion Paraguaya. Local firms/researchers in each country carried out the fieldwork, guided by a set of standard questions adapted to the local context and language. PSL data. The PSL data includes (i) a short household module with socioeconomic information (administrative data); (ii) the families’ self-assessment, organized around 20-40 indicators grouped into six dimensions (income and employment, health and environment, housing and infrastructure, education and culture, organization and participation, and self-reflection and motivation); (iii) the priorities selected by beneficiaries; (iv) the beneficiaries’ participation in pilot activities and achievements;14 and (v) facilitators’/mentors’ notes for each family. The family self-assessment on indicators was recorded at baseline and endline, while priorities, participation in activities, and notes were recorded on a rolling basis.15 The PSL data was filled in by facilitators or mentors. Data Sources by Country Quantitative survey data from rounds 1 and 2 is available for Bulgaria and Romania, while qualitative and PSL data was collected in all three countries. Table 6 summarizes the availability of data sources across countries. Table 6. Data sources Bulgaria Romania Slovakia Quantitative surveys Round 1 survey X X Round 2 survey X X Qualitative data Focus group discussions X X X In depth interviews X X X PSL data X X X Timeline of Data Collection In both Bulgaria and Romania, the round 1 survey was implemented after the initial PSL self- assessment. In contrast to Bulgaria, where only participating households were surveyed, the survey in Romania included all households assigned to the treatment and control. In Romania, because a significant number of families dropped out soon after enrolment, the sample had to be completed using a reserve list once the caseload stabilized. In Bulgaria and Slovakia, the qualitative research was 14 In Bulgaria and Romania, the Achievement section was not filled in. 15 In Romania, the participation in activities and facilitators’ notes were introduced in digital format only at the end of the pilot. 15 implemented during the final (endline) PSL self-assessment, while in Romania, the qualitative research took place shortly before the endline PSL self-assessment. Figure 1 provides the timeline of data collection. Figure 1. Timeline data collection Oct.24 Jan.24 Mar.24 May.24 Dec.24 Jan.25 Mar.25 May.25 Nov.24 Apr.24 Apr.25 Feb.24 Jun.24 Jul.24 Aug.24 Sep.24 Feb.25 Jun.25 PSL baseline assessment BG PSL baseline assessment RO PSL baseline assessment SK Round 1 survey BG Round 1 survey RO Round 2 survey BG Round 2 survey RO Qualitative data collection BG Qualitative data collection RO Qualitative data collection SK PSL Endline BG PSL Endline RO PSL Endline SK Evaluation Methods The evaluation includes two main components: A process evaluation and an evaluation of outcomes. The process evaluation is based on findings from qualitative research, PSL data, and, in Bulgaria and Romania, data from the monitoring modules included in the two survey rounds. The outcomes evaluation is based on the survey data available for Bulgaria and Romania. In both countries, we use a trend analysis to compare the outcomes of interest before and after the pilot. In Romania, where data on a control group was collected, an impact evaluation is conducted to estimate the impact of the pilot. Table 7 provides an overview of the methodology used in the three countries. 16 Table 7. Evaluation components and methods Bulgaria Romania Slovakia Process Evaluation: mixed methods Findings from qualitative research X X X PSL data analysis X X X Monitoring modules data analysis X X Outcomes Evaluation: quantitative methods Trend analysis X X Randomized control trial X Quantitative Methods16 Trend Analysis A trend analysis comparing pilot outcomes at round 1 and round 2 is used to assess how the outcomes of interest evolved over time in Bulgaria and Romania. The main weakness of the trend analysis is the absence of a control group. The identification assumes there are no preexisting trends in the outcome variables or any outside events or changes in economic conditions that could, independently of the intervention, affect the outcomes over the observed period. Randomized Control Trial In Romania, the pilot impact is estimated using a difference-in-differences approach. In Romania, the universe of means-tested benefit recipients in the two selected locations was stratified by location and the presence of a child in the family; within the strata, households were randomly assigned to the treatment and control groups. Participation in the pilot was voluntary. To account for non-compliance, the main specification estimates the local average treatment effect, i.e., the treatment on the treated.17 This is done by estimating the pilot's impact in a two-stage procedure, using the assignment to treatment as an instrument. Questionnaire Survey Data Romania: Sample Selection Of the total 456 potential participants, 281 were randomly assigned to the treatment group. Due to differences in local data availability, distinct approaches were used to construct the sampling frames in each locality. In FeteÈ™ti, a consultant and a representative of the mayoralty conducted visits to all beneficiaries of means tested benefits to collect consent forms for personal data sharing, resulting in a list of 176 active beneficiaries with signed consent. An additional 18 active beneficiaries were identified through the County Employment Agency, bringing the total sampling frame in FeteÈ™ti to 194. Of these, 105 were randomly assigned to the treatment group and 89 to the control group. The NGO implementing the pilot contacted those in the treatment group, and 66 families ultimately enrolled. In the other location, Slobozia, a list of means-tested assistance beneficiaries was provided 16 See Annex 2 for technical details 17 An alternative would be to estimate the intent-to-treat effect—that is, to compare the means of individuals offered treatment to the mean of individuals in the control group independently of the actual treatment status. However, the ITT is diluted by noncompliance and underestimates the true effect (Angrist, 2006). 17 by the mayor’s office,18 forming a sampling frame of 262 beneficiaries. From this group, 176 were randomly assigned to the treatment group and 86 to the control group, with 42 families enrolling in the pilot. Table 8 summarizes the number of treated and control in total and by location. Table 8. Sampling frame by location (RO) Total Fetesti Slobozia Assigned to treatment 281 105 176 Accepted treatment 108 66 42 Refused treatment 173 39 134 Control group 175 89 86 Total 456 175 281 Romania: Survey Sample Description The target population was stratified by geographic location and the presence of children in the household, and treatment was randomly assigned within each stratum. The randomization was successful, as there are no significant differences between the assigned to treatment and control groups. Table 9 shows the balance of covariates at baseline.19 While more families were interviewed from the assigned to treatment group (81 percent) than from the control group (67 percent), all covariates are balanced.20 Beneficiaries are more often females than males, and on average, they are in their mid- forties. Households have, on average, roughly 3 household members, 1.3 of them children. Sixty-four percent of households receive the minimum income benefit, and about half (52 percent) receive the family support allowance. 18 Data was sent by the social work department of the Mayoralty of Slobozia (based on the Representation Warranty Letter signed by the local authorities). 19 The split between assigned to treatment and control groups is uneven; there are 61.6% assigned to treatment versus 38.4% control households. This occurred because many households refused to participate and the pool of assigned to treatment had to be increased. 20 The p-values were calculated using regressions with strata fixed effects. 18 Table 9. Baseline covariate balance (RO) Assigned to Total treatment Control p-value Interviewed (rounds 1 and 2) 76% 81% 67% <0.001*** Beneficiary female 59% 62% 53% 0.494 Beneficiary age 44.5 (16.6) 44.9 (16.4) 44.0 (17.1) 0.613 Number of household members 2.9 (1.8) 2.9 (1.8) 2.9 (1.7) 0.693 Number of children in household 1.3 (1.4) 1.3 (1.4) 1.4 (1.4) 0.814 Minimum income benefit 64% 62% 67% 0.948 Family support allowance 52% 52% 50% 0.799 Observations 456 281 175 Notes: Data on beneficiaries and their benefit receipts refer to the period October–November 2023, prior to enrollment in the pilot. Standard errors are in parentheses. *** significant at 1%, ** significant at 5%, and * significant at 1%. Source: Administrative data. For FeteÈ™ti, data was collected for pilot purposes, while for Slobozia, data was obtained from the mayor's office. There were significant differences at baseline between interviewed and not interviewed households that suggest limits to the external validity of the results. As compared to non-interviewed households, interviewed households had both more family members and more children, and they were more likely to be recipients of minimum income benefits, as evidenced in the Appendix (Table A16). These differences suggest that findings from the impact evaluation cannot be generalized for the sampled population. However, when comparing the assigned to treatment and control groups for those interviewed, there are no statistically significant differences (Table A17. ), suggesting that the estimated impacts can be interpreted as causal and have internal validity. In sum, the sample descriptive statistics suggest that randomization was successful. There are no statistically significant differences between those assigned to the treatment and those assigned to the control. When comparing control, assigned to treatment - not treated, and treated households, the findings suggest that they differ only with respect to the minimum income benefit. Those enrolled in the pilot were more likely to be recipients of the minimum income benefit than those assigned to treatment but not participating in the pilot because of refusal or another reason (Appendix, Table A18). Differences between those assigned to treatment and enrolled and those assigned but not enrolled do not pose any problem for the impact evaluation, as the randomization itself worked well and an average treatment of the treated will be estimated. Although there are observable differences between the group that actually received the treatment and the control group, this does not threaten the validity of the causal estimates. This is because the original random assignment to treatment is used as an instrumental variable for actual treatment status. The random assignment is statistically balanced between groups by design, ensuring comparability. By instrumenting treatment with assignment, we can isolate exogenous variation in treatment receipt and obtain an unbiased estimate of the local average treatment effect (LATE) for compliers. 19 Bulgaria: Sample Selection Out of 169 eligible beneficiaries identified and successfully contacted by the implementer, 152 households enrolled in the pilot. The implementing NGO accessed a list of participants in a “hot lunchâ€? intervention, which targeted people receiving the Monthly Social Assistance (MSA) benefit. Health mediators and social service providers supplemented this list with names of young women from marginalized neighborhoods who did not receive the MSA benefit and were not included in the hot lunch intervention but received means-tested family benefits for children. The final list counted 169 benefit recipients in the two localities, including both participants in the hot lunch project and beneficiaries of means-tested family benefits. The implementing NGO reached out to all families on the list and initially enrolled 152 households. However, the dropout rate in the initial phase of the pilot was high: 111 families set their priorities and committed to a development plan, while 41 families dropped out. Bulgaria: Survey Sample Description Interviewed households are not fully representative of all households enrolled in the pilot. Significant differences exist between interviewed and non-interviewed participants in the pilot, as well as between retained participants and dropouts. Interviewed households tended to have lower levels of education and a higher likelihood of receiving the MSA benefit compared to non-interviewed households (Appendix, Table A19. Compared to dropouts, retained participants were generally younger, lived in larger households, had lower educational attainment, and were less likely to receive income from employment. However, they were more likely to receive child allowance (Appendix, Table A20). Qualitative Methods Descriptive and comparative analysis. The qualitative component of the evaluation uses a descriptive and comparative approach to explore how the pilot was implemented and experienced by participants and key actors across the three countries. The analysis draws on desk research, in-depth interviews (IDIs), and focus group discussions (FGs). These methods were used to gather detailed insights from both pilot beneficiaries and representatives of the implementing organizations. Desk research involved a structured review of core pilot documents and methodological materials to establish a common understanding of the pilot’s conceptual foundation and operational design. A combination of IDIs and FGs was employed to collect primary qualitative data from participants and staff. The target groups included several categories: assigned to treatment, not treated cases (for Romania), drop-outs, those who completed the pilot, and key staff members from the implementing organization (mentors, facilitators, trainers, coordinators). FGs and IDIs with beneficiaries were conducted on-site in all pilot locations, while staff interviews were held both in person and online. Interviewers employed multi-step probing to clarify complex terms and ensure shared understanding. The main strength of the qualitative component lies in its depth and ability to capture nuanced implementation dynamics and personal experiences. However, the findings are not statistically representative and depend on the quality of participant engagement and the accuracy of recall. 20 Qualitative Research Data Bulgaria The qualitative fieldwork in Bulgaria consisted of FGs and IDIs with both beneficiaries and implementing staff. The approach aimed to capture a range of perspectives based on pilot engagement levels, demographic characteristics, and geographic distribution across the selected settlements. Beneficiary respondents were drawn from three predefined target groups: (1) individuals who exited the pilot due to employment, (2) those who disengaged during the self-assessment phase, and (3) those who completed the pilot. A total of 6 FGs and 22 IDIs were conducted, including 14 with pilot participants and 8 with representatives of the implementing organization. All key personnel from the implementing organization, including mentors, facilitators, trainers, and coordinators, were interviewed. Recruitment followed a structured procedure. Field supervisors were equipped with settlement- specific targets for the number of IDIs and FGs, alongside detailed selection criteria. These included demographic diversity (age, marital status), participation in trainings, and inclusion of specific subgroups (e.g., employed dropouts in Novi Pazar). Table A21 (Annex 1) includes details regarding the number of IDIs and FGs, and participants’ characteristics. Romania Pilot participant respondents in the qualitative research were selected from the lists of participants in financial literacy training21 and the PSL platform. For pilot non-participants, selection was based on the information provided by the implementing NGO on families assigned to treatment who refused to enroll in the pilot. The selection aimed for a balanced gender distribution. The group discussions included separate discussions with male and female participants in each location. Similarly, individual interviews with pilot participants included both men and women. Table A22 (Annex 1) includes details regarding the number of IDIs and FGs, and participants’ characteristics. Slovakia Fieldwork consisted of IDIs and FGs conducted with pilot participants, members of the target population, and the team of the local NGO implementing the pilot. IDIs and FGDs with community members were carried out in person, while interviews with the implementation team were conducted online. The selection of respondents aimed to ensure diversity across demographic, geographic, and pilot- related criteria. Demographically, efforts were made to include a balance of ages and genders, with a specific focus on individuals receiving material needs assistance (MNA) to mimic the pilot’s beneficiary population. Geographically, only residents of the designated pilot localities were eligible. Individuals who took part in the intervention and those who were targeted but did not participate were included. Participant recruitment was coordinated by the implementer with support from social workers based in local community centers and the research team during field visits. Despite initial instructions to ensure gender diversity, the final sample consisted exclusively of women aged 21 to 66. Attempts to recruit men were unsuccessful, as many delegated participation to their spouses due to work 21 One of the pilot activities. The lists of participants were provided by the implementor. 21 commitments, particularly seasonal employment. Table A23 (Annex 1) includes details regarding the number of IDIs and FGs, and participants’ characteristics. 22 3. Findings 3.1. Beneficiaries’ Profile This section provides an overview of the socio-demographic characteristics of beneficiaries and their households based on information captured in the PSL socioeconomic module and the baseline survey. While both sources provide relevant information for profiling, the analysis primarily relies on the baseline survey due to its greater richness and detail. Socio-demographic Characteristics The pilot was implemented in dissimilar settings, leading to relatively high heterogeneity in beneficiaries’ socio-demographic characteristics across countries. Pilot areas included urban neighborhoods in Romania and mixed settings in Bulgaria and Slovakia. Household size ranged from 3 to 4 members across countries. Most beneficiaries had a primary education or less, except in Bulgaria, where 41 percent had lower secondary education. According to the survey data, 61 percent of participants in Bulgaria were literate, compared to 31 percent in Romania. In all the countries, the main participants22 were predominantly middle-aged women (30–49 years). Table 10 presents key socio-demographic characteristics of main participants and their households. Table 10. Socio-demographic characteristics of pilot participants Bulgaria Romania Slovakia 23 Urban 44% 100% 49% Household size 3.5 (1.9) 3.2 (1.6) 3.6 (2.4) Highest educational level in household Primary education or less 40% 79% 87% Lower secondary education 41% 12% 13% Upper secondary education or higher 19% 9% 0% Main participant female 90% 74% 65% Age main participant Younger than 30 years 17% 24% 21% 30-49 years 54% 44% 27% 50-64 years 24% 16% 25% Older than 64 years 5% 17% 3% Children under 18 in household 68% 64% 60% Observations 111 108 117 Source: PSL socio-economic module of initial self-assessment conducted by implementers. Standard deviation in brackets for household size. Some indicators may include unreported or missing values for some participants. 22 The main pilot participant is defined as the reference person in the household who was responsible for the PSL self-assessment and priority setting. 23 While Kotel and Novi Pazar are officially classified as part of rural municipalities (defined by settlements with fewer than 15,000 inhabitants), both are designated as towns and function as local administrative and economic centers with urban characteristics such as higher population density, greater access to public services, and diversified employment opportunities. The PSL recorded their official administrative classification. 23 In terms of ethnicity, participants in Bulgaria considered themselves Bulgarian, while those in Romania declared themselves Romani (Table 11). In both countries, the majority predominantly spoke Romani at home (Bulgaria: 65 percent, Romania: 85 percent).24 Table 11. Ethnicity and main language spoken at home Bulgaria Romania Ethnicity Romani 0.0% 87% Bulgarian 84% - Romanian - 14% Turkish 16% - Main language spoken at home Romani 65% 85% Romanian - 21% Bulgarian 42% - Turkish 34% - Observations 99 104 Source: Round 1 survey (not implemented in Slovakia) Income sources vary significantly between countries. In Bulgaria, 20 percent of households reported income from wages and 39 percent from self-employment, while in Romania, income from employment was rare (1 percent). In both countries, households relied primarily on income from social benefits (Figure 3), consistent with the target group of the pilot. They were often recipients of child benefits, social assistance or minimum income support, and housing, rent, or heating subsidies. 24 Based on the round 1 survey. The figures for Slovakia are not available because surveys could not be conducted. 24 Figure 2. Household incomes sources Source: Round 1 survey (not implemented in Slovakia) Figure 3. Household income composition Source: Round 1 survey (not implemented in Slovakia) Access to potable water and public sewage in the home is low. In Romania, only one in eight households had access to water in the home, while in Bulgaria, water in the home was available for five out of ten households. In both countries, less than half of the households had access to sewage. Electricity was available in nearly all households, and over 80 percent of households had access to tap water outside the dwelling. Internet access was generally low ( Access and Use of Financial Services The share of households with access to a bank or ATM in their neighborhood is low in both Bulgaria and Romania, but more households had a bank account in Bulgaria than in Romania. In Bulgaria, 20 25 percent of households reported the presence of an ATM and 27 percent a bank. Similarly, in Romania, 38 percent reported having a bank nearby and 40 percent an ATM. However, while most households in Bulgaria reported having a bank account, in Romania, only about one in five households was banked. Almost all households with a bank account also had a card. Confidence in using banking services was high among households with a bank account (Bulgaria: 80 percent, Romania: 90 percent), but significantly lower among the unbanked—particularly in Romania, at 14 percent, as compared to 62 percent in Bulgaria. As a result, the overall level of confidence in using a bank account was also much higher in Bulgaria: 74 percent, as compared to 28 percent in Romania (Figure 4). Figure 4The main reasons unbanked Romanian households cited for not having an account were the distance to financial institutions or ATMs (80 percent) and the high cost of financial services (80 percent).25 Figure 4. Bank account access, confidence using account, and card ownership Source: Round 1 survey (not implemented in Slovakia) Notes: All indicators are at household level Use of bank accounts and bank cards remain modest. In both countries, the majority of households reported receiving money into their account during the previous year; however, few kept any funds in the account over time. In Bulgaria, most households with an account reported receiving at least one social benefit through it, whereas in Romania, no respondents reported receiving means-tested benefits via a bank account (Figure 5).26 The main reasons why households in Romania did not receive benefits27 through a bank account are: not having an account (44 percent), lack of trust in financial 25 Other reasons reported in Romania included: lack of trust (49%), no need for financial services (43%), respondent does not have the necessary documents (42%), existing debt (32%), someone else in family has account (6%) and religious reasons (4%). See Appendix, Table A24. 26 The corresponding table is in the appendix, Table A24. 27 The survey asked only about means-tested benefits received through a bank account, not about all types of social benefits likely received. 26 institutions (16 percent), and the high cost of financial services (15 percent). In Bulgaria, 40 percent of households with a bank account reported opening it specifically to receive a benefit. Many households with a card (63 percent) reported withdrawing money from their account once every three to four weeks. A smaller share withdrew money once every one to two weeks (26 percent), and only 8 percent withdrew once a week.28 Actual usage for payments with card is low in Bulgaria. Among households with accounts, only 23 percent reported using their card for purchases in the past 12 months—compared to 67 percent in Romania. Figure 5. Utilization of accounts Source: Round 1 survey (not implemented in Slovakia) Notes: The higher value for the variable “Receives benefit in accountâ€? compared to “Received money in the last 12 monthsâ€? is explained by missing observations in the former. All indicators are at household level Saving and borrowing are limited, and households generally do not rely on banking services for these activities. Among those who saved, roughly half had a specific savings goal, and most households saved primarily in cash. Borrowing was more common in Bulgaria than in Romania, with households in both countries primarily relying on friends and relatives as their main source for loans. Over the past 12 months, 40 percent of households in Bulgaria and 15 percent in Romania reported having borrowed money (Figure 6). Among those who borrowed, the most common reasons were to cover basic living expenses, purchase clothing and shoes for children, and pay for medical treatments. 28 These figures are for Bulgaria. A similar analysis is not done for Romania because of the small sample. 27 Figure 6. Savings and borrowing Source: Round 1 surveys (not implemented in Slovakia) Note: All indicators are at the household level Outcomes of Interest The evaluation considers five broad categories of outcomes of interest: financial literacy and health, labor market participation, socio-emotional skills, subjective well-being, and future income expectations. Improvements in financial health and financial literacy, and inclusion in financial services were anticipated because of the financial literacy training provided to participants. Employment was identified among priorities by some participants, and in response, workshops aimed at enhancing employment outcomes were conducted in Bulgaria and Slovakia. Consequently, improvements in labor market outcomes were expected. The coaching sessions as well as support to goal setting and achievement were expected to strengthen participants’ socio-emotional skills. These skills include goal-setting capacity and achievement, control over one's life, and emotional resilience. Although the pilot did not directly target subjective well-being or future income expectations, it was expected that helping participants define and pursue their goals could improve their sense of inclusion, personal outlook, and optimism about the future. Each outcome included is described in detail in the Appendix (Outcomes Measurement). This subsection discusses baseline values. A summary table with descriptive statistics is provided in Appendix (Table A25). Related to the financial literacy and health outcomes, outstanding utility payments are rare, but only a small proportion of households reported saving money. Figure 7 presents baseline findings on financial literacy and financial health. Budgeting indicators show that in Bulgaria, most households planned their budgets and allocated funds specifically for bills.29 In contrast, only about one in three respondents in Romania engaged in these budgeting practices. Outstanding utility payments were rare in both countries, with just one in ten households reporting any. 29 The first round of the survey was implemented shortly after the financial literacy training. It is possible that some of these answers were affected by the social desirability bias. 28 Figure 7. Financial literacy and financial health outcomes Source: Round 1 survey (not implemented in Slovakia) Overall, participation in the labor market is low, with Romania showing much weaker outcomes compared to Bulgaria. Figure 8 presents the responses of beneficiaries and their partners regarding labor market participation. Employment levels are low, both for the beneficiary and at the household level (i.e., few households reported at least one member employed). Registration with the public employment services (PES) is mandatory in both countries for all working-age and able-to-work adults from families enrolled in GMI programs who declare themselves unemployed. In Bulgaria, the registration of at least one family member with PES was relatively high (58 percent); in Romania, this indicator was lower (25 percent), reflecting the higher share of those not able to work because of age or other constraints (e.g., caring for dependents, disability). However, the percentage of those actively looking for a job was lower than PES registration, especially in Romania, where most respondents were not searching for a job. Figure 8. Labor market outcomes at Round 1 (baseline) Source: Round 1 survey (not implemented in Slovakia) Notes: Only respondents aged 18 to 64. 29 The majority of employed in Bulgaria reported being wage-employed, whereas in Romania, they declared themselves as self-employed. In Bulgaria, respondents indicated employment in sectors such as manufacturing and services. In contrast, very limited information is available regarding the sectors of employment in Romania. The distributions of socio-emotional skills outcomes in the two countries are not similar. Bulgarian respondents have somewhat higher self-declared values of socio-emotional skills than Romanian respondents (Figure 9). Figure 9. Socio-emotional skills outcomes at baseline Source: Round 1 survey (not implemented in Slovakia) Respondents in both countries do not feel marginalized and have positive expectations about relative future income.30 Table 12 summarizes the subjective well-being31 and future income expectation outcomes. At baseline, respondents in Bulgaria are aligned with the national average in terms of happiness and life satisfaction, while these two indicators are lower among Romanian respondents compared to the national average. In Bulgaria, 68 percent of respondents reported feeling happy, compared to only 40 percent in Romania. A similar pattern emerges for the satisfaction- with-life indicator, with average scores of 5.5 in Bulgaria and 4.1 in Romania. Perceived marginalization yields comparable results in both countries, reflecting a relatively low sense of marginalization. Respondents were also asked to rate their current position and expected future 30 The relative income, or income rank, is defined as the perceived respondent position on the income distribution (see Outcomes Measurement in the Appendix). 31 Subjective wellbeing is measured through three indicators: happiness, life satisfaction, and perceived marginalization. 30 position on a 10-step ladder representing the income distribution from poorest to richest. A significant share of respondents in both countries expected an improvement in their relative income over the next four years, with 43 percent in Bulgaria and 61 percent in Romania expecting to move up the ladder. Table 12: Happiness, life satisfaction, perceived marginalization, and future income expectations, at baseline and at national level Bulgaria Romania Happiness Round 1 Survey 68% 40% National average 60% 78% Life Satisfaction Round 1 Survey 5.5 4.1 National average 5.5 6.5 Not feeling marginalized Round 1 Survey 7.1 6.8 Expected improvement in relative future income 43% 61% Round 1 Survey Source: Round 1 surveys (not implemented in Slovakia), World Value Survey (2022) for happiness national average, Gallup World Poll (2024) for satisfaction with life. Satisfaction with life and perceived marginalization are measured on a scale from 1 to 10, with 1 representing “completely dissatisfied with lifeâ€? / “very much marginalizedâ€? and 10 representing “completely satisfied with lifeâ€? / “not at all marginalized.â€? Notes: Expected improvement in relative future income refers to the expectation of an improved position on the income ladder/distribution over the following four years. 3.2. Process Evaluation This section of the report examines how the pilot was implemented in the three countries. The broad research questions are: Was it possible for the implementers to engage target families effectively in the pilot? To what extent were implementers able to deliver coaching according to the Implementation Guidelines and the Facilitator Guides? What were the factors which influenced pilot delivery? To what extent have implementers been able to link families to other services, to what extent were they aligned with family priorities, and what are the contributing factors? Actual delivery is compared with the pilot guidelines and manuals,32 and the factors that influenced the processes across each country are discussed. After a summary of the implementation arrangements in each country, the section is organized according to the main business processes. These include outreach; enrollment; preparations for the PSL self-assessment (methodology adaptation and training); implementation of the PSL (self-assessment, preparation of development plans, coaching, and update of self-assessment); and follow-up activities organized by the local implementers (direct provision of services, including coaching (one-on-one and group), training, referrals, and accompaniment). The box below provides an overview of the PSL. References to process duration, overlapping of processes, and lapsed time are based on the timeline charts in Section 1 (Pilot overview). The topic of incentives is 32 Guided Implementation of the Poverty Stoplight, October 2023, Fundacion Paraguaya and The Poverty Stoplight Facilitator’s Guide, Fundacion Paraguaya. 31 covered after the business process sections and is followed by a summary of key takeaways. By looking at the quality, consistency, and experience of implementation, the process evaluation intends to provide important context for the evaluation of outcomes. The information in this section is drawn from the qualitative research described earlier, data from the PSL platform, and the monitoring survey. Poverty Stoplight The Poverty Stoplight (PSL) is a multidimensional approach to poverty reduction based on the idea that people living in poverty can be the central agents in improving their quality of life if they receive appropriate support. PSL begins with a self-assessment comprised of a set of indicators organized into six dimensions. Indicators are defined to be easy to understand (even for those who are illiterate or have low levels of education), adaptable to any context, achievable, and actionable. Construction of the self- assessment aims to inspire the motivation and aspiration needed for participants to take ownership of their situation and work with implementing organizations to improving it. Based on the results of the self-assessment, families choose a few indicators to work on improving. At the same time, organizations use the high- level analysis of the results of the self- assessment and priority-setting to build bottom-up solutions to respond to the challenges of the target group. An essential element of the PSL is the provision of orientation, technical, emotional, and social support (coaching) to families by trained workers in either a personalized or group way during the process. Fundacion Paraguaya is the parent organization of the methodology. Source: Methodology Manual Poverty Stoplight (February 2023) Implementation Arrangements The European Commission selected a consortium (implementation partners) of international NGOs, including Fundacion Paraguaya, and three national NGOs, one in each country, to implement the pilot. The World Bank provided technical inputs regarding the main design and implementation parameters of the pilot. Information on each national NGO, referred to in the report as “implementersâ€? and pilot staffing, is summarized below. Staff does not include those responsible for training activities, which 32 were organized differently by country as part of follow-up activities. Staff roles in pilot implementation varied by country. In Bulgaria, the implementer was a Roma-led NGO experienced in community mobilization, engagement in local decision-making, advocacy, and education. The NGO had worked closely with municipal administrations and contributed to the establishment of local action groups, including through its involvement in initiatives such as the ROMACT Program. The implementer had not previously specialized in delivering structured, family-level support. This shift in focus led to the introduction of internal training, supervision, and field-based coordination mechanisms adapted to the demands of the pilot. In Novi Pazar, the implementer partnered with a local NGO (accredited as a social services provider by the government) to facilitate community entry and trust building. Two staff members were responsible for coordinating and overseeing the overall implementation of the pilot, including PSL and other interventions. In addition, another two staff members (facilitators) supported the work of the pilot at the local level. In their respective locations, Novi Pazar and Kotel (including Gradets), the two facilitators led the initial training of mentors, managed the PSL digital platform, and ensured consistency in data entry and documentation. The implementer also recruited three local staff (mentors)33 for Novi Pazar, the town of Kotel, and Gradets, respectively, to be the primary link with families, providing day-to-day support and follow-up. Candidates were selected based on their community trust, communication skills, and digital literacy. In Novi Pazar, the mentor was a health mediator and social worker with 17 years of experience who was employed by the local NGO mentioned above. The Kotel mentor had experience with social services. The Gradets mentor had work experience in the local public administration and had strong personal networks and prior experience in youth engagement. In Romania, the implementer had longstanding experience working with the Roma population. The implementer was an NGO whose mission is the multisectoral development of Roma based on the exercise of civil, economic, social, cultural, and political rights; access to resources; and identity reconstruction. The NGO was established in 2005 and works extensively on initiatives promoting awareness of Roma identity and increasing their self-esteem. The organization has implemented projects to improve Roma children’s access to quality education and has promoted several initiatives for ensuring respect for civil rights. Project coordination activities were conducted by three of the permanent staff. The two facilitators who worked in each pilot locality started at different stages of the pilot and had different experience levels and workloads. The FeteÅŸti facilitators were part of the pilot from January- February 2024. One of them used to be part of the community, had substantial experience working on projects in the Roma community, and benefited from a positive reputation. He had extensive responsibilities in pilot implementation: in addition to working with families on the PSL processes, he was a trainer for the financial literacy course and assisted families to obtain home ownership documents in both localities. The second facilitator in FeteÅŸti was a graduate of higher education but had little related work experience. In Slobozia, there was a turnover of facilitators after enrollment. The new facilitators had experience as health and school mediators. Both knew very well and had worked in the Roma community in Slobozia. Three out of the four facilitators spoke Romani. The facilitators’ responsibilities in Romania included outreach and enrollment, initial and second PSL self- assessment, fostering pilot engagement through mobilizing participation in the training sessions, and, in some cases, providing information on accessing specific social benefits or services. Project coordination, performed by the implementer permanent staff, involved training and coordination of 33 In Bulgaria, staff who worked directly with families were known as mentors, while facilitators focused primarily on supervision and support of field activities. In Romania and Slovakia, staff who worked directly with families were called facilitators. 33 facilitators, communication with local stakeholders, and overall project organization. One of the coordinating staff members delivered the training session on cultural identity and ethnic self-esteem. In Slovakia, the implementer was a nonprofit organization focused on educational and social inclusion of marginalized children and families, particularly from Roma communities. The NGO works primarily in the field of early childhood development, inclusive education, and community engagement, supporting children with special needs, those from migrant backgrounds, and families experiencing social exclusion. Its programs aim to combat discrimination and segregation while promoting active citizenship and access to quality services. The implementer had a strong foundation of prior engagement in the three pilot localities, supported by established networks and community presence. The organization had experience working with Roma communities, specifically children. Through these initiatives, its staff had built a network of contacts within municipal administrations, local schools, social service providers, and other NGOs active in the area. The implementation team consisted of three key staff members (project coordinator and two facilitators). All positions were filled by NGO staff. However, most staff had other work commitments aside from their work for the NGO. One facilitator was responsible for Bardejov and the other for SpiÅ¡ský Hrhov and Doľany. The work, in terms of families attended to, was evenly distributed. Both facilitators lived and worked in the localities (SpiÅ¡ský Hrhov and Bardejov), which fostered regular communication with local stakeholders and helped facilitate implementation and endorsement of local authorities. The facilitators coordinated the timeline and pilot delivery. The facilitator for SpiÅ¡ský Hrhov and Doľany was a well-established figure in the community and had teaching experience. His longstanding relationships with families, particularly parents of schoolchildren, facilitated trust- building and effective outreach. Although the Bardejov facilitator lacked prior experience working with Roma communities, he had strong project management skills and maintained effective working relationships with municipal authorities and field-based social workers. Outreach According to the Implementation Guide for the pilot, the goal of outreach was to inform the targeted families (recipients of means-tested social assistance) about the PSL and what they could expect from their participation. Interested families were encouraged to enroll in the pilot. The Implementation Guide recommends using a variety of communication channels to ensure effective outreach. A script and set of FAQs were developed to communicate with potential participants. In all countries, outreach occurred through a combination of group and door-to-door communication. While the role of facilitators and mentors was most important, other actors also participated, including, for example local or regional authorities/ agencies. Trust and confidence in some of these actors and their knowledge of the community promoted interest in the pilot. However, evidence suggests that the message delivered on the content and objectives of the PSL self- assessment was often garbled. Some provided insufficient information on PSL activities, while others mentioned the opportunity to change one’s status of dependency on social assistance, having “nothing to loseâ€? (Romania), access to vouchers (Bulgaria), and the future provision of services (home ownership documents and remedial education). Written materials were not particularly effective, considering the low literacy of many potential participants. In Bulgaria, potential beneficiaries confused the pilot with activities carried out by other local actors. Local authorities in Romania and Slovakia facilitated the identification of the target group of means- tested social assistance recipients.34 In Romania, outreach occurred mainly as part of the enrollment 34 See also the Sample Selection in Section 2.2 for more details regarding participants selection in the program 34 process in one location (Slobozia), but preceded enrollment in the other location (FeteÅŸti).35 In Slovakia, national-level endorsement by the Ministry of Labor, Social Affairs, and Family facilitated outreach. Local offices of the Ministry invited all social assistance beneficiaries in the selected municipalities to information sessions regarding the pilot at community centers. Social workers employed by either the municipality or the local labor office hand-delivered invitations. Community center staff members reminded potential beneficiaries of the meeting date and time. As a result of these efforts, about two-thirds of families who received the invitation attended meetings. It was not possible to obtain an official list of social assistance beneficiaries in Bulgaria. The local implementer used a combination of strategies to identify which households to invite to participate (see Bulgaria: Sample Selection in Section 2.2). Ultimately, approximately 170 individuals were invited to participate out of an estimated 476 recipients of the GMI benefit and 294 recipients of the means- tested family benefit. Facilitators and mentors contacted households, informed them about the pilot, and enrolled those who expressed interest. The qualitative research findings suggest some perceived that not all those in the target group had an equal opportunity to participate in the pilot. This caused some resentment, particularly related to access to vouchers. There was more variation among localities in Bulgaria than in the other countries: In one locality, outreach was informal, based on personal connections and word of mouth, while in another, outreach was more structured, combining community sessions with one-on-one contact. Where outreach was led by the local community-based partner of the implementer and included group sessions and follow-up visits, focus group participants noted the respectful attitude of staff and the opportunity to ask questions. In sum, findings suggest that while some outreach efforts were well received, others may have fallen short in building trust and transparency. Enrollment According to the Implementation Guide, the enrollment process for beneficiaries follows a structured procedure designed to ensure clarity and voluntary participation. Interested applicants should be sensitized on the PSL self-assessment and other activities so that they understand what will happen and their role. Upon expressing their commitment to the PSL self-assessment process, individuals formalize their enrollment by signing a consent form. Facilitators and mentors play an important role. In all countries, facilitators (in Bulgaria, mentors) enrolled participants. A relationship of trust between the target population and the facilitator, aided by the facilitator’s strong knowledge of the community, promoted the decision to enroll. However, qualitative research results suggest that enrollment did not include much sensitization on the PSL self-assessment and related activities, and it was not the main motivation for deciding to enroll in the pilot. This situation did vary somewhat even across localities within the same country. The common standard was signing a consent form. 35 In Romania, the target population was first randomly assigned to the treatment and control groups. Prior to this, for FeteÈ™ti only, an initial door-to-door outreach to obtain consent to participate in the survey was carried out (in advance to the outreach for participation in the pilot). The survey was supposed to cover both treatment and control. Outreach for participation in the pilot was done only to the assigned to treatment group. Participation in the pilot was voluntary. 35 Uptake rates were about 40 percent in Romania and 90 percent in Bulgaria and Slovakia.36 The duration of the process varied, from two months in Slovakia to 5-6 months in Bulgaria and Romania. The reasons participants enrolled differed by country and even by locality. In Bulgaria, mentors did door-to-door outreach and awareness-raising in marginalized neighborhoods to identify and recruit eligible households. Particularly in one locality (Gradets), vouchers were the main reason for participation, while in another locality, the opportunity to choose training courses was an important factor. In Gradets, non-participant beneficiaries of social assistance saw the enrollment process as opaque and even exclusionary, and they perceived favoritism. People were approached at home and in the neighborhood. They were asked to provide the information needed for enrollment, sometimes without knowing the pilot’s purpose or eligibility criteria. In Slovakia, beneficiary motivations to enroll in the pilot were a combination of being able to learn something new, improve their employment prospects, and, to a lesser extent, gain access to vouchers. In Romania the motivation to enroll was related to expectations of support for a better life, some direct benefits, and trust in facilitators. The enrollment process in Romania was challenging. In one locality, facilitator turnover contributed to an uptake rate of only 24 percent. Additional factors motivating a decision not to enroll in the pilot included previous negative experiences with projects implemented in the Roma community, discussion of receiving social benefits via bank card, and the absence during outreach of mention of the availability of financial incentives. The outreach message and enrollment process in Romania were more complicated than in the other two countries because of the need to accommodate the randomized selection of pilot participants for the impact evaluation. For example, the outreach message had to clarify that enrollment would depend on the “results of a lottery ,â€? and facilitators visited only families who had been assigned to treatment for enrollment. Finally, some families on the treatment list no longer lived in the locality. Preparation for PSL The preparation to implement the PSL self-assessment involved adaptation of the instrument to each country context. The instrument comprises between 20-40 indicators grouped into six dimensions: income and employment, health and environment, housing and infrastructure, education and culture, organization and participation, and self-reflection and motivation. Adaptation in each country included “vulnerability mapping,â€? choosing the indicators, defining their technical aspects, and validation through focus groups and field testing. The Fundación Paraguay (the Foundation), developer of the Poverty Stoplight approach, supported the implementers in the adaptation. The Foundation also trained the staff of the implementers, including the facilitators, or, in the case of 36 An important caveat is that these rates are not comparable, due to differences in identification and outreach: the base for the calculation of uptake is qualitatively dissimilar for each country given the different approaches used to identify families to be contacted for enrollment in the pilot. For example, in Bulgaria only a subgroup of the target population could be identified (i.e., those means-tested social assistance recipients benefitting of the hot-lunch program). In Slovakia about a third of beneficiaries have not attended the information/ awareness meetings. In Romania the outreach was door—to-door, covering all recipients of means-tested benefits assigned to treatment; some refused to be further contacted, others could not be found, while others declined enrollment. 36 Romania, the coordinator of the facilitators. Lastly, the Foundation prepared an Implementation Guide and a Facilitator’s Manual for each country. Adaptation Adaptation took longer than planned and as compared to other experiences of the Foundation. In all countries, the chosen indicators covered all six dimensions, but the number of indicators varied: 22 in Romania, 29 in Slovakia, and 41 in Bulgaria. In Romania, the reasons mentioned by the implementer for dropping indicators include pilot duration, adjusting expectations to the profile of the target group (low educational levels), and the types of activities eligible for funding (no infrastructure or support for improving housing conditions). The largest number of reductions was made in the health and environment dimension and the housing and infrastructure dimension. The implementer viewed the inclusion of self-esteem and motivation dimensions and cultural awareness to be key, supporting achievements in other areas, and innovative, since these indicators were not included in other approaches to poverty analysis. In each country, available social services were mapped against specific indicators in the self-assessment instrument. Depending on the country, the objectives included identifying resources that could serve as referrals and solutions for specific indicators, supporting the coaching of mentors or facilitators (Bulgaria and Slovakia) and the complementary activities of trainers (Romania), and the development of an action plan of follow-up interventions. The adaptation experience was viewed less positively in Bulgaria. While indicators were pretested on the ground in all countries, illustrations were not pretested. Staff of the implementers in both Romania and Slovakia were generally positive about the adaptation work and the support they received, but some thought the process took too long and involved too much translation of the indicators, which created problems. Acceptance of a multidimensional approach was challenging for one implementer. In Bulgaria, the experience was less positive. Due to comprehension difficulties for some indicators in the pretest, the implementer wished to change them but encountered resistance from the Foundation. For all countries, preparation of the illustrations to complement the text to support participants with little education was delayed, so there was no time for pretesting. Training The Foundation trained the staff of the local implementers and either the facilitators or the coordinator of facilitators (Romania) directly using a structured training package consisting of 10 sessions, some delivered in person and others virtually. Topics included the theory of change, theoretical foundations and the technical stages of the PSL processes, adaptation, the development of solutions in the follow-up activities, and how to use the PSL information platform. While implementers generally had a positive assessment of the training, there were some language challenges, since not all facilitators spoke English. In Bulgaria, mentors did not attend training due to the language and format. Instead, the facilitators and the implementer project team trained mentors on how to use the PSL self-assessment and its visual aids and how to communicate with participants, particularly those with limited literacy. They also organized regular team meetings, joint fieldwork, and peer exchange to help mentors engage effectively with families. 37 Delivery of PSL Self-assessment According to the Implementation Guide, the self-assessment is designed to empower families who actively choose their level of deprivation and vulnerability levels. The PSL classifies vulnerability levels as follows: green: no deprivation, and thus strengths; yellow: moderate deprivation, and thus opportunities for improvement; and red: extreme deprivation, and thus weaknesses. The objective is to help families view and reflect on their current situation, empower them to recognize their own potential, and develop the critical agency needed to try to improve their living conditions and communities. The self-assessment explicitly seeks to foster participants’ agency and motivation to improve their circumstances. To ensure that participants understand the purpose of the self- assessment, it should take place after an initial visit by the facilitator or mentor. Qualitative research in all the countries showed a lack of understanding and recognition among the pilot participants of the point of the self-assessment. Some participants recalled the general meaning of the stoplight colors, but others did not recognize the “traffic lightâ€? terminology. A possible explanation is that, instead of being self-guided, the facilitators and mentors had to help most participants complete it, including the selection of indicator colors. The Implementation Guide recommends that participants hold the tablet to foster a sense of ownership, but some participants were apprehensive about using the technology. Although the accompanying pictures were designed to assist those with low education levels and were helpful in some cases (Romania), some participants did not understand them or refused specific questions. Contributing to the lack of understanding of the self-assessment, it appears that the recommended introductory initial meeting was not carried out in any of the three countries. Instead, the self-assessment was explained during enrollment and combined with other information and procedures, which reduced the clarity of the message. Another factor that may have contributed to confusion was the overlap or close timing with the surveys related to pilot evaluation and monitoring. Finally, some participants may have simply forgotten about the exercise. Different shortfalls related to the self-assessment emerged across countries. In Bulgaria, while mentors played a hands-on role in administering the PSL self-assessments, particularly when participants faced literacy or language barriers, the qualitative research identified that gaps in the mentors’ understanding of the methodology contributed to deviations from the recommended approach. Although facilitators supervised the implementation of the self-assessment, provided methodological guidance, and conducted joint field visits, for some participants, completing the exercise felt like responding to a survey, contrary to the principle of the methodology. While participants did not recall the exercise as a particularly meaningful or memorable part of the pilot, in the locality where mentors were more locally embedded, participants were more likely to remember and describe the self-assessment. In one of the pilot localities in Romania, there was some time pressure to complete the self-assessment as the enrollment process took longer and other pilot activities were at risk of delays. In Slovakia, facilitators did view the self-assessment as a key element of the pilot serving as a structured tool for tracking progress and guiding household priorities. However, some beneficiaries refused to answer questions on certain topics, including health, debt, and informal work. In sum, for a variety of reasons, it is doubtful across all countries that the self- assessment enhanced individual agency for many participants. The Foundation did not expect the extent of difficulty on the self-assessment in the pilot: the degree of self-awareness among the target 38 group was low, and it proved hard for them to reflect on their situation, perhaps due to low education levels. Despite these shortfalls, there were some positive aspects related to the role of the facilitators (mentors in Bulgaria). Evidence suggests that the pilot enabled facilitators to interact constructively with participants. Results from the monitoring survey indicate that 70 percent of participants felt comfortable or very comfortable in their initial discussion with the facilitator about family needs and goals (Romania). Evidence is also relatively positive for Bulgaria: 60 percent of participants felt at ease during the self-assessment and were satisfied with the support received from their mentor. Preparation of development plan At the completion of the self-assessment, a dashboard (referred to as a “life mapâ€?) generated in the PSL platform showing the response results for each indicator is intended to enable participants to recognize their situation and to make plans to improve it. The PSL information platform allows the data to be visualized both digitally and in a printed version. In collaboration with a facilitator, the family selects a few priority areas (among those marked red or yellow) they would like to work on until the follow-up self-assessment. Once the family sets priorities, the platform displays a set of questions for each priority indicator to guide preparation of a development plan supported by the facilitator, such as “Why don’t I have it?â€? “What will I do to get it?â€? and “How many months will it take?â€? One of the facilitator’s roles is to refer local social services to participants and provide related information. According to data in the PSL platform, all (Slovakia) or nearly all (Romania) families who completed the self-assessment set priorities for their development plan (Table 14). In Bulgaria, the percentage was lower (70 percent). However, a common finding from the qualitative research is that participants didn’t remember setting priorities and working on a development plan very well. Nor did most understand the link between the results of the assessment and their choice of priorities. It appears that the lack of autonomy for the self-assessment carried over to some extent to the exercise of setting priorities. The Foundation characterized this difficulty as unusual, in their experience. In Romania, older people had difficulty setting priorities (17 percent of pilot families were 65 years old or above) and understanding the notion of objectives. Also, the answers to the questions recorded in the PSL platform were very similar across participants, suggesting that the facilitator may have prompted standard responses. Evidence from the monitoring survey is more positive: Two-thirds of respondents in Romania agreed with the statement that they were able to set priorities on their own. In Bulgaria, participants found it difficult to set priorities. Although mentors helped families articulate their goals, the implementer attributed the fact that beneficiaries could not define their needs and prioritize “This is what I want to do,â€? or understand how their answers related to concrete opportunities for change, to their focus on day-to-day survival. This focus made it difficult for them to set development goals or plan for the future. They had low self-belief and limited confidence in their ability to shape their path going forward. The implementer also found the process administratively demanding, given the need for intensive one-on-one and structured support. As a result, the PSL instrument was simplified, and paper-based discussions were sometimes used. Findings from the monitoring survey are more mixed than in Romania: roughly one-third of respondents reported having autonomy when setting their goals, one-third were neutral, and one-quarter expressed disagreement. Respondents in Novi Pazar were more positive, while those in Gradets reported limited independence. 39 Table 13 lists the indicators most often prioritized by participant families in order of incidence. When making country comparisons, we need to keep in mind that both the list of indicators and the definitions of even the “sameâ€? indicators differ. Employment and income-related indicators were often mentioned by families in Bulgaria and Romania, while “Budgetâ€? was by far the most commonly mentioned indicator in Slovakia. Stable housing emerged as a frequently mentioned indicator37 only in Romania. In all countries, a range of indicators was prioritized by families (including health in Bulgaria and Romania, and nutrition in Slovakia). Financial literacy, a focus of the pilot, was among the top five priorities identified by families in Romania and Slovakia, while financial health indicators (family budget and family savings) were named in Bulgaria. Table 13. Top five indicators prioritized by families % of % of % of Bulgaria Romania Slovakia Priorities Priorities Priorities Employment 24 Income 30 Budget 49 Employability and Professional 16 Stable Housing 17 Career 15 Skills Development Financial Family Budget 13 14 Financial Literacy 10 Literacy Income 9 Health Services 12 Nutrition 6 Health Sufficient 7 10 Employment 6 Services* Clothing *Medical check-ups, family savings, and professional development indicators registered the same prioritization level as health services Source: Poverty Stoplight Results and Findings Reports (Bulgaria, March 2024; Slovakia, July 2024) and Process Evaluation for Romania (May 2025) Figure A19 (Annex 1) presents a gender-disaggregated analysis of the priorities selected by pilot participants during the Poverty Stoplight (PSL) self-assessment process across Bulgaria, Romania, and Slovakia. The figure shows the distribution of the first priority and all selected priorities across key life dimensions such as income and employment, health and nutrition, housing and infrastructure, education and culture, financial literacy and financial health, organization and participation, and self- reflection and motivation. Employment and Financial Literacy and Financial Health subdimensions, which are included in the Income and Employment and Education and Culture dimensions, are also shown separately. Across all three countries, both men and women most frequently prioritized indicators related to income and employment. However, notable gender differences emerge in the types and diversity of priorities selected. In Bulgaria, women put more emphasis on employment and financial literacy, and showed a broader distribution of priorities across multiple dimensions than men. In Romania, women were also more likely than men to select financial literacy and financial health-related indicators. Neither men or women selected employment in large numbers. In Slovakia, the most striking pattern is the overwhelming prioritization of financial literacy and financial health by both men and women, 37 The indicator refers to possession of documents related to property ownership of the home or a rental contract for the same. 40 with women placing even greater emphasis on this area. We cannot exclude the hypothesis of a systematic emphasis on financial literacy and financial health being driven by facilitators. Coaching and Second Self-Assessment According to the Implementation Guide, periodic visits by facilitators/mentors are intended to ensure that pilot participants remain engaged in the PSL process. Visits are used to provide guidance, support, and information to help participants reach “greenâ€? for the priorities set in the development plan. At least one visit/month is recommended. The plan was to update the self-assessment about 10 months after the initial one so that participants could see what progress had been made and select new priorities. The second self-assessment also offered an opportunity for the local implementer to monitor results. Results from both the qualitative research and the monitoring survey reveal positive assessments by families of the facilitators in Romania and Slovakia, though less so in Bulgaria. There is some evidence (Slovakia) that facilitators used the life map as a basis for coaching, including in group sessions. In Slovakia, due to logistical issues and participant preferences, there were few home visits. Instead, most coaching was embedded in group-based training. Nevertheless, it appears that facilitators in Slovakia were able to maintain regular contact with beneficiaries and track engagement through multiple follow-up meetings. Despite a lack of such regular contacts, nearly 90 percent of Romanian respondents said they had high trust in the facilitator they worked with during the pilot, with most giving the highest assessment (complete trust).38 On the other hand, evidence from the monitoring survey suggests that facilitators and mentors were able to establish relationships with families to a lesser extent in Bulgaria. Trust and satisfaction with facilitators and mentors were generally moderate, with notably lower ratings and weaker engagement in Gradets compared to the other locations. Most responses fell into the neutral or moderately positive range, with only 8 percent stating they liked and trusted the facilitator. A similar pattern emerged regarding the mentors: most participants gave mid-range ratings, and only a small number reported high satisfaction. Consistent with the findings of the monitoring survey for Bulgaria, the qualitative research characterizes the work of the mentors as limited. Because of shortcomings in their professional capacity, the work of the mentors consisted mainly of information provision, reminders, motivational encouragement, and informal problem-solving rather than deeper coaching. There was some variation across localities, driven by characteristics of the mentors, including their institutional connections. Overall, however, mentors did not receive the training or support needed to provide structured follow-up; nor did they view the PSL processes as integrated. Rather, they felt that the potential of the approach was underused and created expectations they were not equipped to meet. These gaps may have contributed to the fact that participants perceived the pilot as a series of unrelated components, although, as described in the Implementation Manual, they are closely connected and build on each other. Relatively high shares of participants across all countries updated the self-assessment. The share of participants updating the self-assessment ranged from virtually all in Slovakia to 90 percent in Bulgaria and slightly fewer in Romania (86 percent). The time between the first and second self-assessments varied from 8 months in Slovakia to between 9-12 months in Romania and about 12 months in 38 Ratings were 97 percent in Fetesti and 76 percent in Slobozia. 41 Bulgaria. In Romania, implementer staff were concerned that this was not enough time to show changes on the part of participants. Figure A20 (Annex 1) presents a visual comparison of the first and second PSL self-assessments completed by pilot participants across the three countries. The figure illustrates the distribution of responses across the PSL’s color-coded indicators—green (no deprivation), yellow (moderate deprivation), and red (severe deprivation)—at both baseline and endline. It shows the extent to which participants reported improvements, stagnation, or deterioration in their living conditions across key life dimensions such as income, housing, health, education, and self-reflection. The figure reveals that while some participants showed progress—evidenced by a shift from red or yellow to green in certain indicators—many remained in the same or similar levels of deprivation by the end of the pilot. Interpretation of this result needs to be cautious, given the challenges described earlier that participants had in completing the self-assessment, and differences across localities in the support provided. With those caveats in mind, the results shown in the figure suggest that although the PSL process helped identify priorities and track changes, the short implementation period and structural barriers limited the extent of measurable improvements. The figure underscores the importance of sustained engagement, tailored support, and the availability of relevant social and other services to achieve meaningful change. Level of Participant Engagement in PSL In Romania and Slovakia, once families enrolled in the pilot, most maintained their participation through the second self-assessment (see Table 14). In Romania, 86 percent of enrolled participants completed the PSL processes as compared to 80 percent in Slovakia. Reflecting the difficulties participants had choosing priorities in Bulgaria, drop-outs occurred at this stage. Although 90 percent of those enrolled completed the second PSL self-assessment, many of them had not set priorities. If these participants are excluded, the share of those enrolled who completed the PSL processes drops to only 55 percent. In Slovakia, enrollees dropped out between enrollment and the self-assessment. In Romania, there were a few dropouts between the self-assessment and the setting of priorities (preparation of the development plan). These figures exclude a few discrete events (such as the death of a beneficiary in Romania and a beneficiary in Slovakia who dropped out after completion of the first self-assessment because they found employment). 42 Table 14. Pilot participation trends by country Bulgaria Romania Slovakia Romania Bulgaria Dropped Dropped Slovakia Dropped Out Out Out Enrolled 165 108* 146 Completed first PSL 152 7 108 6 117 29 self-assessment Set priorities 107 45 102 117 Completed second PSL 9139 16 93 9 117 0 self-assessment % of Enrolled Completing PSL 55% 86% 80% processes *reflects number once enrollment stabilized Action Plan and Follow-up Activities According to the Implementation Manual, results from the self-assessment instrument and the priorities chosen in the development plan would guide the action plan and specific interventions. Factors such as the number of indicators classified in the red and yellow categories and the indicators prioritized by the participants, as well as the experience and expertise of the implementer and the other actors involved in the pilot, would guide the choice of intervention. The prioritization of indicators by the organization, as reflected in the choice of intervention, is expected to promote family decisions to take steps to improve their situation. The pilot promoted the formation of a task force in each country to support collaboration with local stakeholders, such as local authorities, decentralized administrations, social services, schools, and local NGOs. In Bulgaria, the two municipalities reactivated their Municipal Working Group for Social Inclusion, originally formed under a previous project and formalized by mayoral order. It served as a coordination body throughout implementation.40 The municipal administrations also met with the community action groups promoted in one of the training interventions to hear community needs; there was some evidence of responses. In Romania, although a task force was formed, it met only once and did not play an effective role during the pilot. Instead, specific task force members were engaged as needed in their areas of competence. In Slovakia, a task force consisting of facilitators, social and community workers, representatives of local government, and stakeholders from the local agencies under the Ministry of Labor, Social Affairs, and Family (such as free debt counselling centers and legal aid centers) was organized and mostly worked on a demand basis. The Foundation prepared a report for each country on the results of the self-assessment and the priority setting in the individual development plans to inform the action plans. In all three countries, the implementer prepared an action plan, which included a list of planned interventions along with timing and proposed arrangements for delivering the activities. Most activities roughly extended over the latter half of 2024 and first quarter of 2025. Table 15 summarizes the interventions outlined in the action plans. The Foundation had a positive view of the action plans, which were diverse and reflected 39 Among participants, 137 completed the second PSL assessment, but only 91 set priorities and were involved in pilot activities. 40 This group included representatives from the Labor Bureau, Social Welfare Directorate, police, schools, kindergartens, and Roma community members. 43 a good balance between activities carried out by the local implementer and the social services in the area. While the selection of activities responds to some extent to participants’ priorities, there are some gaps, particularly regarding the priority given to employment-related indicators. Implementors opted for various arrangements to deliver training and other interventions. In Romania, the implementor used a combination of in-house staff and external consultants while trainers were mainly external consultants in Bulgaria. In the former, one facilitator supported both the delivery of financial literacy training and support to obtain home ownership documents. Reflecting one of the objectives of the pilot, implementers organized financial literacy training in all of the countries. In all the countries, nearly all pilot participants received financial literacy training, often in more than one course. In both Romania and Slovakia, financial literacy was one of the top five priorities mentioned by pilot participants, but not the top priority. It was missing as a priority in Bulgaria, where family budget (financial health) was ranked third. In Slovakia, although commercial banks were approached by pilot implementers, the banks were not interested or willing to cooperate, thus limiting the support to beneficiaries to open or manage bank accounts. Except for Bulgaria, interventions to support activation of pilot participants were relatively few, although employment was a frequently mentioned priority in all countries. Bulgaria was the only country where some participants (14) were able to access vocational training, but even there, relatively few concentrated in one locality received services related to employment. The implementer in Slovakia organized a workshop on practical skills related to job search. The implementer in Romania approached the County Employment Agency on training, but the low education level of participants made collaboration difficult: most courses offered through the agency required at least lower secondary school. Although remedial education was planned, it was not possible to link participants to those services. Interventions in all countries included group coaching on specific topics, referrals, and/or information sessions with other services, such as debt counseling centers in Slovakia. In both Bulgaria and Slovakia, the action plan incorporated individual and group coaching. In the former, families were offered both one-on-one and group coaching on topics such as health, housing, parenting, and employment, and individual coaching was organized for the most vulnerable participants. The implementer in Romania also arranged for trainings on a variety of topics, including childcare, which provided an opportunity for coaching and mentoring. Support in obtaining documents related to property ownership was included as an intervention in both Bulgaria and Romania. In the case of Slovakia, coaching focused on specific topics (family budget, shopping assistance, job interviews, debt management, preschool enrollment, document processing, and local health care services). Although some of these sessions and support focused on narrow topics, in all countries participants generally demonstrated interest in what implementers offered. 44 Table 15. Description of country interventions Intervention Bulgaria Romania Slovakia Category • Introduction to financial • Income and expenses in literacy the household • Planning and budgeting • Financial goals and financial • Financial literacy training • Mitigation of risks of debt Financial Literacy plans and follow-up and strategies for income • Smart shopping • Meetings with banks41 generation • Family savings • Risks associated with loans • Access to credits and loans and debts • Training on job search skills • Income opportunities (job • Vocational training interviews, labor market Employment • Course on small business information, CV and consultations on preparation and use) company registration • Legal advice on housing • Legal assistance related to Housing ownership and property housing documents. • Meeting with local Health • Health information sessions authorities on placement of and consultations with doctor medical doctor • Consultations with medical doctor • Parent’s workshops and individual consultations Parenting and • Nutrition, well-being, • Preparation for • Childcare, nutrition, and socio-emotional resilience, and self- participation in community health education skills confidence action groups and community initiatives Sensitization on • Cultural identity and ethnic Roma History self-esteem and Culture • Civil rights and access to institutions • Provision of clothes and Coaching for 40 most footwear Other vulnerable families, including • Enrollment in second- visits by other social services chance education42 In Bulgaria and Slovakia, facilitators and mentors promoted participation in the follow-up interventions and made referrals. In Bulgaria, facilitators organized follow-up activities, such as 41 Planned but did not take place. 42 Planed but did not take place. 45 vocational orientation and thematic trainings. In a few cases, beneficiaries were supported to enroll in certified professional courses aligned with their interests. In addition, mentors provided referrals and hands-on assistance, such as helping beneficiaries register for courses, obtain documentation, register with GPs and take health checks, and apply for housing support or disability assessment. In one locality, this work was more institutionalized and coordinated, given the existing relationships of the local implementer with other public institutions. In Slovakia, facilitators supported the organization of financial literacy training. In addition, over time, beneficiaries grew more comfortable seeking assistance on a range of issues, including employment, administrative paperwork, and other social services. In Romania, the role of facilitators was more limited: They invited people to participate in training and mobilized and monitored participation. The action plans and the results of the qualitative research indicate that, overall, local implementers were able to deliver financial literacy training and more specialized training and personalized support addressing a range of issues faced by families, despite the relatively short timeline. In all three countries, nearly all participants attended financial literacy training. In Bulgaria, 137 households participated in at least one intervention. Attendance was highest for financial literacy and parents’ workshops. Among all pilot components, participants most positively recalled these supplementary services, including the financial literacy training, as contributing to meeting their aspirations. In Romania, in addition to financial literacy (and follow-up), shorter training sessions covering a variety of topics were well-attended. In Slovakia, all beneficiaries remembered attending the financial literacy training and were positive about the use of practical tools, such as budgeting exercises and support from trainers. As a result of the information sessions, some beneficiaries independently began the debt-resolution process. Overall Assessment The monitoring survey results related to the pilot are positive overall for Romania, less so for Bulgaria. In Bulgaria, participants’ perceptions of the pilot’s impact on their living conditions was mixed. Many gave a neutral (38 percent) or moderately positive answer (27 percent) to the question: To what extent do you agree that the project has helped improve your family’s living conditions? Only a small number (1 percent) strongly agreed that the project had made a difference. In contrast, 13 percent of respondents expressed disagreement, including 6 percent who strongly disagreed. Responses from Gradets were notably more critical, accounting for all strongly disagree responses and most of the neutral or negative ones, while participants from Kotel and Novi Pazar tended to view the project somewhat more positively. These patterns suggest that although some participants saw improvements, many—particularly in Gradets—did not perceive a strong or direct benefit to their household. On the other hand, in Romania, nearly three-quarters of participants agreed or strongly agreed that the project had helped improve their family’s living conditions, with more ratings concentrated in the highest category, and more positive results from participants in FeteÅŸti than Slobozia. 46 Use of Incentives According to the Implementation Guide, the use of vouchers or other incentives is not consistent with the PSL practices and methodology. Providing incentives is not typical, as people should be motivated on their own to participate. It is up to the implementers to generate interest on the part of the participants. As required by the pilot design, in all countries, participants received food vouchers for attending financial literacy and other training, but their use was wider in Bulgaria. The provision of vouchers in Bulgaria extended to participation in the PSL self-assessment and other processes, in addition to training. The qualitative research suggests that receipt of vouchers became the main motive to join the pilot for many participants, thereby overshadowing the intrinsic motivations for learning and changing their situation. In fact, in two of the three localities (Gradets and Kotel), participants perceived the pilot as an additional form of financial assistance, thereby limiting the pilot’s impact on their attitudes and behaviors. The broad use of vouchers, coupled with mistrust in the enrollment process, caused community tensions, particularly in one locality (Gradets). In Romania and Slovakia, the intention to distribute vouchers in connection with one of the pilot activities was not a focus during outreach. In fact, in Romania, according to pilot implementers, the lack of clearly defined incentives was one of the reasons mentioned by non-participants to opt out of the pilot. Likewise, in Slovakia, few participants mention vouchers as a reason for participating in the pilot. In both Romania and Slovakia, implementer staff commented that vouchers were necessary to promote participation in training. Takeaways Despite variations among countries, the findings of the process evaluation suggest several preliminary takeaways. As the preceding discussion suggests, implementation of the pilot was quite varied in several aspects, including among localities in the same country. Thus, identifying general conclusions is difficult. Nevertheless, on a preliminary basis, conclusions emerging from the process evaluation include the following: • Effective outreach was critical, given the nature of the PSL processes. Potential participants needed to know and understand what they were signing up for and opt to enroll voluntarily based on complete and accessible information. Gaps in outreach meant that many beneficiaries entered the process with incomplete or erroneous information. The relatively low take-up in Romania points to the need to consider potential challenges in attracting participation in similar activities. • The difficulties that participants had completing the self-assessment and setting priorities on their own had additive effects on the demands placed on the implementer organizations, especially for mentors and facilitators. The design and theory of change in the PSL process is conditioned on a level of participant autonomy and independence that was largely absent in the pilot. This, is turn, meant that a higher level of structure for the support provided to participants and/or professional training was needed for mentors and facilitators. The same applied in other areas, such as referrals: often, information sessions alone were not sufficient, as beneficiaries needed additional support to follow through. One implication is that similar 47 initiatives with the same target group should consider incorporating some combination of more intensive training, lower caseloads, and a higher level of supervision and support for coaching. • While coaching took place, strategies varied. In Romania, most coaching was provided by trainers during delivery of follow-up activities by the implementer. In Slovakia, families preferred to be coached in a group setting rather than one-on-one in their home. • While the discussion above only touched on some of the differences between localities in the same country, the findings suggest that several factors mattered. One is the quality of coaching, particularly the institutional and community connections of the staff performing that function. A second is the consistency of processes, such as enrollment, and the third relates to the role of incentives. • Despite the inherent challenges of implementing the pilot in highly disadvantaged communities in a relatively short time period, qualitative data suggest that there were notable achievements. First, mentors, facilitators, and trainers were able to establish trusting relationships with many participants. In addition, even without the full support of local authorities in most cases, implementers were able to deliver a range of services to a substantial number of participants, including training, workshops, referrals, and one-on-one or group coaching. Finally, most participants remained engaged in the pilot until its conclusion. Drawing on the takeaways and other findings described, some evidence is available to respond to the research questions that guided the process evaluation mentioned at the beginning of the section. Was it possible for implementers to engage target families effectively in the pilot? The findings suggest that the answer is partially. Not all families opted to participate in the pilot. Many were not able to fully engage in PSL activities. On the other hand, most enrolled families completed the pilot and a high share of them took advantage of follow-up activities organized by implementers. Implementers had challenges in organizing coaching according to pilot guidelines, although there is evidence that except for specific localities, most facilitators and mentors were able to establish a positive relationship with families. Delivery was influenced by the extent to which facilitators and mentors were embedded into the communities and their institutional connections. Given the small size of the pilot other conclusions need to be cautious,43 but it is likely that in the context in which the pilot functioned (highly marginalized communities and population) training on the PSL may not have been intensive enough, the same for other support to the staff delivering coaching. On the last research question, data indicate that implementers were able to link families to other services. While these activities were aligned to some extent with family priorities, there was a gap related to employment between the high priority assigned by families and the few activities organized by implementers, with better performance, but still limited, in the case of Bulgaria. Contributing factors included restrictions related to the low education level of participants, lack of cooperation by the relevant local stakeholders and insufficient local service availability. 43 Trying to identify patterns across countries based on the available data would not be reliable due to the numerous factors influencing the findings and the limited number of observations (number of localities and size of the treatment groups). Factors such as country context, local context, NGO implementation approach, NGO capacity, facilitator influence, and cultural differences within the target population interact in complex ways. This intricacy, combined with the heterogeneity between and within countries, renders determining patterns that shape outcomes across countries challenging. 48 3.3. Outcome Evaluation This section presents findings on temporal changes (trend analysis) for survey outcomes from round 1 to round 2 for Bulgaria and Romania, as well as impact results (difference in differences, or diff- in-diff) for Romania. The observed trends reflect changes in outcomes over time. In contrast, the presence of a control group in Romania allows for a more rigorous attribution of the observed changes to the pilot itself. In this section, we first discuss the limitations of the quantitative analysis, then provide results of the trend analysis for Bulgaria and Romania, and the results of the impact analysis for Romania. Limitations of the Quantitative Analysis The analysis presented in this section has several important limitations that must be considered when interpreting the findings. These limitations include the existence of a control group in only one country, a shorter-than-planned exposure period to pilot activities, seasonality effects in trend estimates, small sample sizes, and potential measurement errors. First, a control group was available only in Romania, which limited the ability to make causal inferences for Bulgaria and Slovakia. In Bulgaria, data were collected only for pilot participants, while in Slovakia, no survey data were collected at all. Second, the exposure period to the pilot was shorter than originally planned, potentially limiting the extent of observable pilot effects; participants were engaged for approximately 12 months instead of the intended 18. Third, trend estimates are subject to seasonal variation and other unobserved factors, with the timing of the two survey rounds potentially influencing outcomes such as employment or income independently of the pilot. Fourth, the sample size of participants was relatively small, with approximately 100 participants per country, which limited the statistical power of the methods used.44 Finally, the potential for measurement error due to the low educational level of respondents may have affected the precision and reliability of the estimates. Trend analysis: Bulgaria and Romania In Bulgaria, the data show a decline in both saving and budgeting behaviors, alongside a decrease in the share of households with outstanding utility debt from the first to the second round of survey collection.45 These results are not surprising, being largely attributed to seasonality. The round 1 survey was conducted in September 2024, at the end of summer, when seasonal employment opportunities are more abundant, and household income tends to be higher in the pilot localities. In contrast, round 2 took place in March 2025, at the end of winter, which is typically associated with fewer seasonal job opportunities, lower household income, and increased energy bills.46 Therefore, households find it difficult to save during the winter. However, we cannot exclude the alternative hypothesis that survey responses were influenced by social desirability bias: In Bulgaria, the round 1 44 Power calculations indicated that a sample size of at least 300 would have been needed to detect minimum detectable effect of 0.30 standard deviations with sufficient statistical power. In other words, sample sizes smaller than 300 result in a lower likelihood of observing effects meaningful enough. See Annex 2 Trend Analysis and RCT Specifications. 45 Utilities include water, electricity, and heating. In Bulgaria, debts were reported for water and electricity only. 46 In Bulgaria, electricity suppliers enforce timely payments rigorously. If payments are not made within a brief grace period, electricity supply is discontinued – an unaffordable situation for households during the cold season. 49 survey followed soon after the financial literacy training, which introduced respondents to savings and budgeting techniques. It is possible that in round 1, more so than in round 2, participants overstated their use of such practices, possibly based on their genuine intention to use them as a result of the training. In Romania, trend estimates of financial health suggest an improvement in savings behavior and budgeting practices, and worsening of debts for utilities. Similar to Bulgaria, the first survey round in Romania was conducted in September and October, and the second in March and April. The financial literacy training was implemented from October 2024 to January 2025. Interestingly, the financial health trends in Romania follow an opposite pattern compared to Bulgaria: increases in savings, improved budgeting behavior, and an increased percentage of beneficiaries reporting debt for utilities. As in Bulgaria, the observed changes over time are likely influenced by seasonality patterns. Reported income47 is higher at round 2, which is likely driving the positive trends on savings: Households are typically able to save only once a certain income threshold has been met. A positive effect of financial literacy training is also a plausible hypothesis. The debts on utility bills are likely explained by overdue payments accumulating over the winter due to the high cost of energy. In fact, households reported higher expenditure on bills in the second round. Figure 10. Trend estimates: Financial literacy and financial health in Bulgaria and Romania Notes: All outcomes are standardized and at the household level. All regressions include individual fixed effects and robust standard errors. “Confident using bank account (no account)â€? was omitted for Bulgaria due to a small sample size (19 observations). *** significant at 1%, ** significant at 5%, and * significant at 10%. Confidence in using a bank account remained stable over the observed period in both Bulgaria and Romania. In Bulgaria, the reported confidence in using a bank account was already high at the time of the first data collection round for both households holding a bank account (80 percent) and those with no bank account (62 percent). In Romania, confidence in using bank accounts was high among 47 Average household income was 1,291RON in the first round and 1,653RON in the second round. 50 those with accounts (90 percent) but low among participants without accounts (14 percent). Without considering potential external factors, the trend analysis suggests that the pilot, and particularly the financial literacy training, did not necessarily raise unbanked beneficiaries’ self-confidence in their abilities to use bank accounts. In Bulgaria, participants’ employment declined, job search intensity remained relatively stable, and registrations with the public employment service increased. Figure 11 shows the trend estimates for labor market outcomes for Bulgaria and Romania. As argued for the financial health-related outcomes, some respondents at round 1 were engaged in seasonal jobs that were no longer available at the end of winter, leading to reduced employment both for beneficiaries and their household members. Job search intensity was slightly lower in round 2, possibly reflecting limited local job opportunities, although this change was not statistically significant. On the other hand, registrations with the public employment service increased due to the lower employment, the beneficiaries’ availability to work, and the social assistance cash-benefit program requirements.48 Figure 11. Trend estimates: Labor market participation in Bulgaria and Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5% and * significant at 10%. In Romania, employment increased at both the individual and household level, and job search intensified at the household level. It is worth noting that employment increased, though from a low base (4-5 percent). While there were no targeted interventions on employment in Romania, throughout pilot implementation, facilitators repeatedly communicated the importance of finding a job and reducing benefit dependency. The data includes information for only few respondents on occupation and employment sector, making it difficult to identify other possible explanations for the 48 Unemployed beneficiaries are required to register with the Employment Agency. 51 observed increase in employment. Among respondents participating in the pilot (treated population) who reported being employed at the time of the round 2 survey, all identified as self-employed. While most did not provide details about their self-employment, those who did mentioned working in recycling.49 Despite the observed increase in employment, the reported household income from labor remained unchanged between rounds. This suggests that employment could have been underreported at baseline and that the observed change could be a measurement error. The trends analysis suggests a worsening in some socio-emotional skills and subjective well-being in Bulgaria and some positive change in Romania. Figure 12 presents the results for these outcomes in both countries. Respondents in Bulgaria reported lower emotional resilience in the second survey round. The change in the other two socio-emotional skills indicators, goal-setting capacity and control over one's life, are also negative, but not statistically significant. There are two possible explanations for the findings in Bulgaria. First, the pilot could have raised participants’ awareness of their conditions without delivering substantial improvements, which may have negatively impacted their socio- emotional skills. Evidence suggests that setting aspirations too high can lead to disappointment and negatively impact socio-emotional skills indicators.50 Pilot participants also reported feeling more marginalized than at round 1. One hypothesis is that those participating in the pilot may have been “marginalizedâ€? by the rest of the community (non-participants), who might have considered the selection for additional benefits and support to be unfair (i.e., preferential treatment for those already receiving social assistance). Negative spillover effects of cash transfers like this have been documented.51 In contrast, no deterioration in indicators is observed in the Romanian setting. Instead, there is some improvement in participants’ sense of control over their lives. In this different context, identifying and actively working toward personal priorities may have contributed to an increased sense of agency among participants. 49 Other occupations mentioned during qualitative interviews and in the monitoring module include: salesperson in shops or transport/delivery companies, construction sector, cleaning services (for well-off Roma households), agricultural (cutting vineyards and fruit trees), street trading, traditional crafting (e.g., boilers, dustpans), opening small businesses/shops, and finding work abroad. 50 McKenzie, D., Mohpal, A., & Yang, D. (2022). Aspirations and financial decisions: Experimental evidence from the Philippines. Journal of Development Economics, 156, 102846. 51 Haushofer, J., Reisinger, J., & Shapiro, J. (2015). Your gain is my pain: Negative psychological externalities of cash transfers. Available at: https://files.givewell.org/files/DWDA%202009/Interventions/Cash%20Transfers/Haushofer_Reisinger_and_Sh apiro_2015.pdf. 52 Figure 12. Trend estimates: Socio-emotional skills, subjective well-being, and future income expectations in Bulgaria and Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. ***significant at 1%, **significant at 5% and *significant at 10%. The estimated trends in Bulgaria and Romania move in opposite directions and appear to be shaped by seasonal patterns and context. In Bulgaria, financial outcomes worsened, employment declined, and emotional resilience and marginalization, one of the subjective well-being indicators, deteriorated. The other two subjective well-being indicators, satisfaction with life and happiness, as well as expectations of future income, remained unchanged. In contrast, Romania showed improvements across some of the same dimensions (financial outcomes, employment). Although some estimates lose statistical significance after adjusting for multiple hypothesis testing, the overall pattern of results remains consistent. A summary table of the multiple hypothesis testing adjustments is provided in the Appendix (Table A26, columns (1) and (2)). Impact (Difference-in-Differences) Results: Romania The difference-in-differences analysis estimates the average treatment effect on the treated (ATT). The ATT measures the impact of the pilot on a series of outcomes for those who were assigned and received the treatment, as opposed to those who were eligible but did not participate.52 The ATT can be estimated only for Romania, where the assignment to treatment was randomized and data was collected on both control and treatment group in two rounds, before and after pilot implementation. Difference-in-differences estimates suggest there was no effect of the pilot on finance-related outcomes. While the trend analysis for Romania suggested positive changes for some financial-related 52 Technically, this is estimated using a two-stage least squares (2SLS) model, in which assignment to treatment serves as an instrument in a first-stage regression to predict actual treatment, and the second-stage regression estimates the effect of predicted treatment on the outcome only for those treated, thus an effect on the treated. 53 outcomes, estimates turn non-significant when compared to a control group. This suggests that the positive trends were driven by factors unrelated to the pilot. One possible explanation signaled by the data is that the trends were driven by seasonal income patterns. Treated respondents reported higher household income at round 2 compared to round 1, which in turn correlates positively with savings and budgeting at round 2. The diff-in-diff results suggest that these trends were similar for both the treatment and control groups. Figure 13. Difference-in-differences estimates: Financial literacy and financial health in Romania Notes: All outcomes are standardized and at household level. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5% and * significant at 10%. Heterogeneous effects by beneficiary gender were examined for two financial health outcomes: saving money and confidence in using a bank account among those without one. 53 A gender difference was detected only for the saving outcome, where the program had a significant positive effect on saving for male beneficiaries, while female beneficiaries experienced no measurable change (see Figure 14). No gender difference was found in the impact on banking confidence. Notice that the triple interaction coefficient (treatment*post*female) represents the difference in treatment effects between female and male beneficiaries. 53 See Table A27 in Appendix. 54 Figure 14: Impact estimates for outcome Saves Money by gender of beneficiary Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5% and * significant at 10%. The difference-in-differences estimates for labor market outcomes indicate a positive impact on employment at the household level, with more households reporting at least one employed member compared to the first round of the survey (Figure 15).54 One hypothesis is that the encouragement provided by facilitators to seek employment, as well as the identity reframing promoted by the implementers in Romania (“from dependency to earned incomeâ€?), prompted participants to take on available jobs.55 A more plausible hypothesis, however, is that pilot participants were more likely to report informal employment in the second round than in the first. The pilot messaging might have motivated participants to report any type of employment, including informal work, which was likely underreported at baseline. This explanation is supported by the fact that participants’ income from labor remained unchanged between survey rounds and is not statistically different from that of the control group. All those treated who reported a change from unemployment to employment between the first and second survey rounds were self-employed and male.56 While detailed information on occupations and sectors is largely missing, the few respondents who did provide details reported working in metal recycling and services. 54 The number of households with employed members increased from 4 to 24 over the observed period (out of 99 households interviewed at rounds 1 and 2). 55 The importance of having a job was included in the key enrollment messages and was one of the financial literacy training objectives (improving participants’ ability to mitigate risks associated with loans and debts , and developing strategies to increase income generation). 56 The partner of the main participant. Main participants were predominantly female. 55 Figure 15. Difference-in-differences estimates: Labor market participation in Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5% and * significant at 10%. A heterogeneous gender effect emerges only for registration with the public employment service (PES).57 There is no statistically significant difference by gender in overall employment status or active job seeking. Following the program, male participants are less likely to be registered with the PES, while female participants show no change (Figure 16). 57 See (1) (2) Saves money Confident using account (no account) treatment*post 1.13*** -1.35** (0.34) (0.56) treatment*post*female -1.27*** 1.50** (0.47) (0.66) Observations 661 509 R-squared 0.62 0.68 Standard errors in parentheses: * p<0.10, ** p<0.05, *** p<0.01 Table A28 in Annex 1. 56 Figure 16: Impact estimates for Registration with PES by gender of beneficiary Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5% and * significant at 10%. Pilot participants reported feeling less marginalized. Increasing the participants’ self-esteem and promoting cultural identity were the object of various pilot activities and training sessions. Additionally, pilot participants had opportunities to learn how different public services function through training sessions on access to public services and civil rights organized by the implementer. These experiences may have contributed to a greater sense of social inclusion. No impact was observed for the other subjective well-being outcomes indicators, socio-emotional skills, or income expectations. None of the difference-in-differences estimates for socio-emotional skills, subjective well-being, and future income were significant (Figure 17). 57 Figure 17. Difference-in-differences estimates: Socio-emotional skills, subjective well-being, and expectations in Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. ***significant at 1%, ** significant at 5%, and * significant at 10%. The intention-to-treat (ITT) specification shows a statistically significant effect only for the household-level employment variable. The ITT estimation treats all individuals assigned to the pilot as if they received the intervention, regardless of their actual participation. Consequently, the ITT coefficient reflecting the average impact of being offered the pilot is smaller than the ATT. Although the effect on household-level employment is positive, the result does not remain significant after adjusting for multiple hypothesis testing (see Appendix, Table A26). The corresponding graphs for financial literacy and health, labor market participation, and socio-emotional skills are in the Appendix (Figure A21, Figure A22, and Figure A23, respectively). In summary, in Romania, the results show an increase in reported employment at the household level and a reduction in feeling marginalized. The facilitators repeatedly encouraged participants to find work, with a positive impact on reported employment. Those who reported a change in employment status between the two rounds were self-employed males working in the recycling sector and services. A plausible hypothesis is that informal employment was underreported in the first round, with more accurate reporting in the second round due to identity reframing messages. Complementary data on incomes supports this hypothesis. The reduction in the degree to which the respondents feel marginalized is most likely driven by training sessions on self-esteem and empowerment or by interactions and relevant messages conveyed during the financial literacy training. The findings are not robust to multiple hypothesis testing. 58 4. Summary of Findings and Conclusions The pilot intervention aimed to enhance the socioeconomic inclusion of marginalized Roma households in Bulgaria, Romania, and Slovakia by promoting financial inclusion and personalized support through the Poverty Stoplight (PSL) methodology. The evaluation found that the pilot was successful in enrolling participants and delivering core components—such as financial literacy training, coaching, and referrals—in all three countries, albeit with significant variation in implementation quality and participant engagement. The outcomes of interest for the pilot covered five broad categories: financial literacy and financial health, labor market participation, socio-emotional skills, subjective well-being, and future income expectations. Improvements in financial literacy and financial health were anticipated due to the financial literacy training. Many participants identified employment as a priority; in response, implementers organized activities that aimed to enhance employment outcomes. The PSL processes, including coaching, were expected to strengthen participants’ socio-emotional skills. Helping participants define and pursue their goals through coaching and other implementer interventions was expected to improve their subjective well-being and optimism about the future, including their income expectations. The main findings are summarized below, beginning with the process evaluation and followed by the results of the trend analysis (Bulgaria and Romania) and the impact evaluation (Romania). Findings from the process evaluation cover overall implementation of the pilot as well as the business processes. Implementation of the pilot deviated in some respects from the intended intervention. There were significant variations across countries and even among localities in the same country. Inherent challenges arose during implementation, given the highly disadvantaged pilot localities and relatively short timeline. Other findings from the process evaluation include: • Many beneficiaries entered the PSL process with incomplete or erroneous information, a critical gap in outreach, given the focus of the PSL intervention model on fostering participants’ agency and motivation to improve their circumstances. Also, while most social assistance recipients opted to participate in Bulgaria and Slovakia, that was not the case in Romania. • The lack of consistency in procedures used for processes such as enrollment affected the trajectory of implementation in some communities. • The difficulties participants had in completing the PSL self-assessment and setting priorities placed additional demands on local implementers, specifically the facilitators and mentors coaching families. This points to the need to consider in future similar initiatives some combination of more intensive training, lower caseloads, and a higher level of supervision and support for coaching. • Mentors, facilitators, and trainers were able to establish relationships of trust with many participants. The extent of their institutional and community connections made a difference in the perceptions and assessments of pilot participants. • The use of incentives needs to be carefully considered, as it can have unintended negative consequences on community cohesion. • Even without the full support of local authorities in most cases, implementers delivered a range of services to a substantial number of participants, including training, workshops, 59 referrals, and one-on-one or group coaching. Most participants remained engaged in the pilot for its duration. In the categories of financial literacy and health, employment, and socio-emotional skills, the trends analysis shows positive results only for Romania. In Bulgaria, financial outcomes worsened, employment declined, and emotional resilience and a subjective well-being indicator (marginalization) deteriorated. The other two subjective well-being indicators and expectations of future income remained unchanged. Possible explanatory factors include seasonality (finance), social desirability bias (finance), and measurement error (employment). There are two possible explanations for the worsening of some socio-emotional skills and subjective well-being. First, the pilot could have raised participants’ awareness of their conditions without delivering substantial improvements. Second, those participating in the pilot may have been “marginalizedâ€? by other community members, who might have considered the participants’ selection for additional benefits and support to be unfair. On the other hand, Romania showed improvements for financial outcomes and employment. In addition to the positive effect of financial literacy training, seasonality might also have contributed to the finance outcomes. While there were no employment-related activities organized in Romania, facilitators communicated the importance of finding a job to reduce benefit dependency. Measurement error may also have been a factor. There is no deterioration in the indicators for socio- emotional skills, subjective well-being, and expectations of future income. Instead, there is some improvement in participants’ sense of control over their lives. Identifying personal priorities and actively working toward them may have contributed to an increased sense of agency among participants in Romania. The impact evaluation (Romania) shows positive effects only for reported employment and for reducing a sense of marginalization. In Romania, where a randomized control trial (RCT) was implemented, the pilot led to a statistically significant increase in reported household employment and a reduction in perceived marginalization. These outcomes align with the theory of change, which posits that coaching, goal-setting, and exposure to services can enhance agency and social inclusion. However, no significant impacts were observed on financial literacy or socio-emotional skills, suggesting that intermediate outcomes did not fully materialize within the shortened implementation period. The lack of effect on finance-related outcomes, contrary to the findings of the trend analysis, suggests that the positive trends were driven by factors unrelated to the pilot, such as seasonality. Factors driving the positive impact in employment at the household level are like those mentioned for the trends analysis: the encouragement of facilitators to seek employment, the identity reframing promoted by the implementers “from dependency to earned income,â€? and measurement error. The fact that participants felt less marginalized (subjective well-being) is likely to have been driven by workshops focusing on self-esteem and cultural identity, as well as sessions on accessing public services and civil rights. Both may have contributed to a greater sense of social inclusion. Other estimates for socio-emotional skills, subjective well-being, and future income are not significant. Shortened Implementation Timeline Constrained Outcome Realization. The pilot was implemented over approximately 12 months instead of the planned 18. This reduced the time available for participants to internalize financial literacy concepts, engage in sustained coaching, and achieve behavioral change. As a result, intermediate outcomes such as improved budgeting or goal-setting capacity may not have had sufficient time to emerge, weakening the link between activities and expected long-term impacts. 60 Participant Capacity and Program Design Misalignment. The PSL methodology assumes a level of self- reflection and goal-setting ability that many participants lacked. This misalignment required facilitators to provide more intensive support than anticipated, and in some cases, to complete assessments on behalf of participants. As a result, the intended mechanism of empowerment through self-assessment and planning was diluted, challenging a core assumption of the theory of change. Incentive Use Affected Motivation and Community Dynamics. While incentives such as food vouchers helped boost participation, especially in Bulgaria, they also distorted motivations and created perceptions of unfairness. This undermined the theory of change’s emphasis on intrinsic motivation and agency as drivers of change and introduced unintended social tensions that may have affected engagement and outcomes. Employment as a Common Priority but Weak Link to Services. Employment was frequently prioritized by participants in all countries, yet few employment-related services were delivered, especially in Romania and Slovakia. Bulgaria was the only country where some vocational training was provided. This gap between participant priorities and available services weakened the Theory of Change’s assumption that referrals and coaching would lead to improved labor market outcomes. Despite limitations, the pilot demonstrated that with appropriate support, even highly marginalized populations can engage in structured development processes. However, future interventions should: • Align program design more closely with participant capacities, simplifying tools and increasing facilitator training. • Extend implementation timelines to allow for sustained engagement and outcome realization. • Use incentives judiciously and transparently to avoid undermining intrinsic motivation. 61 Annex 1. Tables and Figures Table A16. Differences between interviewed and not interviewed households (RO) Coefficient interviewed Outcome (std. error) p-value Beneficiary female -0.02 (0.05) 0.646 Beneficiary age 1.19 (1.19) 0.314 Number of household members 0.49 (0.14) <0.001*** Number of children in household 0.28 (0.11) 0.008*** Minimum income benefit 0.01 (0.04) 0.007*** Family support allowance -0.05 (0.03) 0.104 Observations 456 Notes: Coefficients are obtained from a regression with outcome as dependent variable and indicator for interviewed and strata as regressors. Standard errors in parentheses. *** significant at 1%, ** significant at 5%, and * significant at 10%. Source: Administrative data. For FeteÈ™ti, data were collected for pilot purposes, while for Slobozia, data were obtained from the mayor's office. Table A17. Differences between assigned to treatment and control among those interviewed (RO) Coefficient assigned treatment Outcome (std. error) p-value Beneficiary female 0.06 (0.05) 0.270 Beneficiary age 0.64 (1.22) 0.599 Number of household members 0.01 (0.13) 0.925 Number of children in household -0.04 (0.10) 0.673 Minimum income benefit 0.01 (0.03) 0.868 Family support allowance -0.01 (0.03) 0.663 Observations 346 Notes: Coefficients are obtained from a regression with outcome as dependent variable and indicator for assigned to treatment and strata as regressors. Standard errors in parentheses. *** significant at 1%, ** significant at 5% and * significant at 10%. 62 Table A18. Differences between control, assigned to treatment - not treated, and treated (RO) Assigned to treatment, Control not treated Treated p-value p12 p23 p13 N=175 N=173 N=108 Beneficiary female 53% 68% 52% 0.692 0.750 0.605 0.392 45.9 Beneficiary age 44.0 (17.1) 43.1 (17.5) 0.287 0.231 0.134 0.586 (15.6) 2.9 2.8 Number of household members 3.0 (1.8) 0.406 0.735 0.200 0.277 (1.7) (1.8) 1.4 1.3 Number of children in household 1.3 (1.3) 0.812 0.606 0.547 0.862 (1.4) (1.4) <0.001 0.023 <0.001 0.004 Minimum income benefit 67% 49% 84% *** ** *** *** Family support allowance 50% 55% 48% 0.925 0.950 0.763 0.699 Notes: Coefficients are obtained from a regression with outcome as dependent variable and indicator for treatment status and strata as regressors. Standard errors in parentheses. *** significant at 1%, ** significant at 5%, and * significant at 10%. Source: Administrative data. For FeteÈ™ti, data were collected for pilot purposes, while for Slobozia, data were obtained from the mayor's office. Table A19. Difference between interviewed and not interviewed households (BG) Not Total Interviewed interviewed p-value N=152 N=136 N=16 Beneficiary female 89% 90% 88% 0.790 Beneficiary age 42.8 (14.2) 43.3 (14.2) 38.8 (14.5) 0.240 Number of household members 3.32 (1.9) 3.33 (1.9) 3.2 (1.6) 0.770 Highest educational level (in hh) 0.078* Primary education 48% 50% 29% Lower secondary education 36% 36% 43% Upper secondary education 16% 14% 29% Hh has income from salary 35% 37% 20% 0.190 Material needs benefit 80% 82% 63% 0.073* Child allowance 48% 47% 56% 0.490 Standard errors in parenthesis. *** significant at 1%, ** significant at 5% and * significant at 10%. Source: Initial PSL assessment. 63 Table A20. Difference between retained participants and dropouts (BG) Retained Total participant Dropout p-value N=136 N=99 N=37 Beneficiary female 90% 91% 87% 0.450 Beneficiary age 43.3 (14.2) 41.0 (11.9) 49.3 (17.7) 0.002*** Number of household members 3.3 (1.9) 3.6 (1.9) 2.7 (1.8) 0.025** Highest educational level (in hh) 0.040** Primary education 48% 78% 40% Lower secondary education 36% 21% 41% Upper secondary education 16% 6% 19% HH has income from salary 37% 23% 73% <0.001*** Monthly Social Assistance Benefit 82% 81% 84% 0.690 Means Tested Family Allowance 47% 52% 35% 0.089* Notes: Standard errors in parenthesis. *** significant at 1%, ** significant at 5% and * significant at 10%. Source: Initial PSL assessment. Table A21. Bulgaria: Qualitative research methods and participants Locality IDIs Conducted FGs Conducted Gradets 6 pilot participants: 3 FGs (18 pilot participants): • 4 continuing • 15 continuing • 2 dropped after self-assessment • 3 dropped after self-assessment Kotel 5 total: 1 FG (6 pilot participants): • 3 continuing • All continuing Novi 6 pilot participants: 2 FGs: Pazar • 3 continuing • FG1 (6 pilot participants): 3 • 2 dropped (employment & personal continuing, 2 dropped after self- reasons) assessment, 1 dropped for personal reasons • FG2 (5 pilot participants): all continuing Project 3 mentors (each locality) staff 2 facilitators (each municipality) 1 trainer (financial literacy) 1 coordinator of facilitators 1 project coordinator 64 Table A22. Romania: Qualitative research methods and participants Locality IDIs Conducted FGs Conducted Fetesti 10 total: 2 FGs (11 participants): • 4 pilot participants • 11 pilot participants • 4 non-participants • 2 facilitators Slobozia 13 total: 2 FG (10 participants): • 7 pilot participants • 10 pilot participants • 4 non-participants • 2 facilitators Trainers 2 trainers Coordinating 1 Project coordinator staff 1 Coordinator of facilitators Table A23. Slovakia: Qualitative research methods and participants Locality IDIs Conducted FGs Conducted Doľany 9 total: 1 FGs (9 participants) • 4 pilot participants • 5 non-participants SpiÅ¡ský Hrhov 6 total: 1 FG (5 participants) • 3 pilot participants • 3 non-participants Bardejov 6 total: 1 FG (5 participants) • 3 pilot participants • 3 non-participants Online 4 total: • 1 local coordinator • 2 facilitators • 1 trainer 65 Figure A18. Access to utilities and housing infrastructure Table A24. Financial literacy and financial health Bulgaria Romania # Responses Financial access Has bank account 74% 21% 197 Household confident using account (has account) 80% 90% 91 Household confident using account (no account) 62% 13% 102 Has bank card 97% 95% 94 Usage of bank account Deposited money last 12 months 84% 65% 96 Keeps money in bank account 29% 10% 89 Receives benefit in bank account 89% 0% 73 Withdrawal frequency 89 Several times a week 0.0% 5% Once a week 9% 5% Once every one or two weeks 31% 9% Once every three or four weeks 59% 76.2% Never 2% 4.8% Used card for purchase last 12 months 23% 67% 90 Savings and debt Saves money 27% 20% 194 Has a savings goal 8% 11% 172 Borrowed money in last 12 months 40% 15% 198 Observations 99 99 Source: Baseline surveys (survey not implemented in Slovakia). Notes: *** significant at 1%, ** significant at 5%, and * significant at 1%. 66 Table A25. Baseline outcome values Bulgaria Romania N=99 N=104 A. Financial literacy and financial health Saves money 27% 19% Has expenditure/budget plan 74% 29% Keeps money for bills separate 63% 36% No outstanding utility payments 86% 91% Confident using account (has account) 80% 91% Confident using account (no account) 62% 13% B. Labor market participation Employed - beneficiary 16% 4% Employed - household 21% 5% Looking for work - beneficiary 47% 1% Looking for work - household 49% 3% Registered with PES - beneficiary 54% 21% Registered with PES - household 58% 25% C. Soft skills and subjective well-being Goal-setting capacity 3.7 (0.7) 3.1 (0.8) Emotional resilience 3.6 (0.8) 2.8 (0.7) Satisfaction with life 5.5 (2.2) 4.1 (2.2) Not feeling marginalized 7.1 (2.4) 6.8 (2.2) Happiness 68% 40% Relative income in 4 years 43% 61% Source: Baseline surveys (survey not implemented in Slovakia) Notes: *** significant at 1%, ** significant at 5%, and * significant at 1%. 67 Figure A19. PSL priorities by gender Bulgaria, first priority Bulgaria, all priorities 0 10 20 30 40 50 0 10 20 30 40 50 Income and employment Income and employment Only Employment Only Employment Health and nutrition Health and nutrition Housing and infastructure Housing and infastructure Education and culture Education and culture Only Financial Literacy and Financial Only Financial Literacy and Financial Health Health Organization and participation Organization and participation Self-reflection and motivation Self-reflection and motivation women men women men Romania, first priority Romania, all priorities 0 10 20 30 40 50 0 10 20 30 40 Income and employment Income and employment Only Employment Only Employment Health and nutrition Health and nutrition Housing and infastructure Housing and infastructure Education and culture Education and culture Only Financial Literacy and Financial Only Financial Literacy and Financial Health Health Organization and participation Organization and participation Self-reflection and motivation Self-reflection and motivation women men women men Slovakia, first priority Slovakia, all priorities 0 20 40 60 80 0 20 40 60 80 100 Income and employment Income and employment Only Employment Only Employment Health and nutrition Health and nutrition Housing and infastructure Housing and infastructure Education and culture Education and culture Only Financial Literacy and Financial Only Financial Literacy and Financial Health Health Organization and participation Organization and participation Self-reflection and motivation Self-reflection and motivation women men women men 68 Figure A20. PSL first and second self-assessments Source: PSL database, 2025 69 Figure A21. Intention to treat estimates: Financial literacy and financial health in Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5%, and * significant at 1%. Figure A22. Intention to treat: Labor market participation in Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5%, and * significant at 1%. 70 Figure A23. Intention to treat: Socio-emotional skills, subjective well-being, and expectations in Romania Notes: All outcomes are standardized. All regressions include individual fixed effects and robust standard errors. *** significant at 1%, ** significant at 5%, and * significant at 1%. 71 Table A26. Multiple hypothesis tests (1) (2) (3) (4) BG trends RO trends RO ATT RO ITT p-values p-values p-values p-values Saves money .004 .033 1 1 Has expenditure/budget plan .005 .062 1 1 Keeps money for bills separate .012 .031 .941 1 No outstanding utility payments .013 .031 1 1 Confident using account (has account) .552 .561 - - Confident using account (no account) .473 .72 1 1 Employed - beneficiary .064 .03 1 1 Looking for work - beneficiary .149 .32 1 1 Registered with PES - beneficiary .005 .72 1 1 Employed - household .017 .001 .181 1 Looking for work - household .422 .03 1 1 Registered with PES - household .005 .72 1 1 Goal-setting capacity .203 .178 .757 1 Control over one's life .111 .067 1 1 Emotional resilience .012 .72 .757 1 Satisfaction .332 .561 1 1 Not feeling marginalized .004 .504 .276 1 Happiness .374 .561 1 1 Relative income in 4 years .374 .72 1 1 Notes: Sharpened False Discovery Rate q-values reported. They provide the expected proportion of rejections that are type I errors (false rejections). See Anderson (2008). Table A27. Heterogenous impact by gender of beneficiary for financial health outcome (1) (2) Saves money Confident using account (no account) treatment*post 1.13*** -1.35** (0.34) (0.56) treatment*post*female -1.27*** 1.50** (0.47) (0.66) Observations 661 509 R-squared 0.62 0.68 Standard errors in parentheses: * p<0.10, ** p<0.05, *** p<0.01 72 Table A28: Heterogenous impact by gender of beneficiary for employment outcomes (1) (2) (3) Employed Looking for work Registered with PES treatment*post 0.13 -0.96 -1.02** (0.51) (.) (0.50) treatment*post*female -0.06 1.13 1.14* (0.54) (.) (0.58) Observations 529 470 469 R-squared 0.80 0.64 0.76 Standard errors in parentheses: * p<0.10, ** p<0.05, *** p<0.01 73 Annex 2 Trend Analysis and RCT Specifications Trend Analysis Formally, the trend is estimated using the following regression framework: 𝑌𝑖 = 𝛽0 + 𝛽𝑡𝑟𝑒𝑛𝑑 𝑡𝑖 + 𝛾𝑖 + 𝑢𝑖 Where 𝑌𝑖 is the outcome of interest for household i, 𝑡𝑖 is as an indicator for the post period (i.e., it takes a value of 0 for observations at survey round 1 and a value of 1 for the period at survey round 2), and 𝛾𝑖 captures the household fixed effect. Fixed effects control for time non-varying factors such as educational level, innate ability, etc. All regressions are estimated with robust standard errors. The estimate of interest is 𝛽𝑡𝑟𝑒𝑛𝑑 . Survey attrition in both rounds needs to be examined—specifically, differences between pilot participants and those interviewed at baseline and endline—to assess the potential for differential attrition. Randomized Control Trial In the first stage, the actual participation is regressed on assignment to treatment: 𝐷𝑖 = 𝛼0 + 𝛼1 í µí±?𝑖 + 𝛾𝑖 + 𝑒𝑖 where 𝐷𝑖 is an indicator for household i of actually receiving the treatment and í µí±?𝑖 is the assignment to treatment of household i. In this regression, 𝛾𝑖 captures the household fixed effect. In the second stage, the predicted values of receiving treatment for household i, 𝐷𝑖∗, are used to estimate the causal impact: 𝑌𝑖 = 𝛽0 + 𝛽𝐴𝑇𝑇 𝐷𝑖∗ + 𝛾𝑖 + 𝑢𝑖 where 𝑌𝑖 is the outcome of interest for household i. The coefficient 𝛽𝐴𝑇𝑇 is the treatment on the treated and the main outcome of interest. All regressions are estimated with robust standard errors. Similar to the trend analysis, attrition at baseline and endline poses potential problems, and differential attrition at the different data-collection stages needs to be examined. Power Calculations The targeted sample size per country was 300, with an equal split planned between the treatment and control groups. With this sample size, it is possible to detect an effect size of 0.32, which is considered a small-to-medium effect. Power calculations are based on individual-level randomization and standard parameter values: a statistical power of 0.8 (the probability of correctly rejecting a false null hypothesis) and a significance level of 0.05 (the probability of a type I error). Table A29 illustrates how the minimum detectable effect varies with changes in sample size. When the sample size 74 decreases to 150, the minimum detectable effect increases to 0.46, which corresponds to a medium- to-large effect. These calculations indicate that the analysis may be underpowered.58 Table A29. Minimum detectable effect (MDE) by sample size Sample size MDE 350 0.30 300 0.32 250 0.36 200 0.40 150 0.46 58 See the Sample Selection section of the report. 75 Annex 3 Outcomes Measurement Financial literacy and financial health at household level Outcome Explanation The outcome is derived from the question “When you receive money from any source, do you leave aside some for savings?â€? (question FI.24) and is at the household level.59 The responses Saves money always, often, and sometimes are coded as yes, while responses rarely and never are coded as no. Responses from both respondent and partner were considered; if one of them said she had set money aside, the response was coded as a yes for that household. The outcome was derived from the question “Do you or does any other member of your household do the following? Make a plan to Has expenditure/budget manage your income and expenses for the month.â€? (question plan EXCO.2A) Households where the respondent said yes are coded as having an expenditure/budget plan. The outcome was derived from the question “Do you or does any other member of your household do the following? Make a plan to Keeps money for bills manage your income and expenses for the month.â€? (question separate EXCO.2C) Households where the respondent said yes are coded as having an expenditure/budget plan. The outcome was derived from the question “Does you household currently owe money for ... Select all that apply.â€? (FI.37) and No outstanding utility whether the respondent answered no to all of the following options: payments A Water supply, B Electricity supply, C Other housing related utilities, e.g. heating, phone bill etc. This outcome is derived from the question “How confident do you feel about using this bank account without help from others on a Confident using account (has scale from 1 to 5?â€? (FI.2). The variable is at the household level and account) is coded yes if one of the two respondents had an account and answered Somewhat confident, Fairly confident or Completely confident. This outcome is derived from the question “If you were to open an account at a bank or another type of formal financial institution, do Confident using account (no you think you could use it by yourself, without the help of another account) person?â€? (FI.21). The variable is at the household level and is coded yes if one of the two respondents said yes. 59 Two other questions were considered to capture the savings of the household. One is “In the PAST 12 MONTHS, have you, personally, saved or set aside any money for any reason? â€? The other is, “Do you or does any other member of your household do the following? Save money.â€? While the individual responses differed between the questions, the results of the econometric analysis were the same. 76 Labor market participation Outcome Explanation This outcome is derived form the question, “Which of these categories describes your current situation best? If you have two jobs, think about the one where you spend the most time.â€? (LA.1) Employed - beneficiary This question was asked of the respondent and their partner. The beneficiary is considered employed if he/she selected one of the following options for this question: 1. In paid work, 2. Self-employed or 3. Business owner with employee(s). This outcome is derived from the question, “Have you looked for Looking for work - work in the last 4 weeks?â€? (LA.17) This question was asked of the beneficiary respondent and their partner. The question was coded yes if the beneficiary answered yes. This outcome was derived from the question, “Are you currently Registered with PES - registered with the public employment service?â€? (LA.21) This beneficiary question was asked of the respondent and the partner. The question is coded yes if the beneficiary answered yes. This outcome is derived form the question, “Which of these categories describes your current situation best? If you have two jobs, think about the one where you spend the most time.â€? (LA.1) Employed - household This question was asked of the respondent and the partner. The beneficiary or the partner was considered employed if one of the two selected any of the following options for this question: 1. In paid work, 2. Self-employed, or 3. Business owner with employee(s). This outcome is derived from the question, “Have you looked for work in the last 4 weeks?â€? (LA.17) This question was asked of Looking for work - household respondent and their partner. The question was coded yes if the beneficiary or partner answered yes. This outcome is derived from the question “Are you currently Registered with PES - registered with the public employment service?â€? (LA.21) This household question was asked of the respondent and their partner. The question is coded yes if the beneficiary or partner answered yes. 77 Subjective well-being Source (if Outcome Explanation available) This outcome is derived from the following question: “All things World Value considered, how satisfied are you with your life as a whole these Survey days?â€? Use a scale of 1 to 10 where 1 means you are “completely Satisfaction dissatisfiedâ€? and 10 means you are “completely satisfied.â€? Where would you put your satisfaction with your life as a whole?â€? (WE.2) The variable value ranges from 1 to 10. This outcome is derived from the following question: “On a scale Not feeling from 1 to 10 where 1 means and 10 means how marginalized do you feel, in general.â€? (WE.4) The variable is reverse coded, and its value ranges from 1 to 10. This outcome is derived from the following question: World Value “Taking all things together, would you say you are (read out and Survey code one answer): 1 Very happy Happiness 2 Rather happy 3 Not very happy 4 Not at all happy (SES.1)â€? The answers very happy and rather happy are coded as happy, and the other two responses are coded as not happy. Expected improvement in relative income Source (if Outcome Explanation available) This outcome is derived from the following two questions: Life in (1) “Let's imagine this ladder. At the top of the ladder (indicate the Transition top step) are the people who have the largest amount of money, Survey the best jobs and the highest education. At the bottom of the ladder (indicate the bottom step) are the people who have the smallest amount of money, the lowest education or no education, Expected and the worst jobs or no jobs. In the middle of the ladder (indicate improvement the steps in the middle) are the people who have some money and in relative education and have decent jobs. The higher a family is on this income ladder, the better off it is. On which step of the ten is your household today?â€? (AS.10) (2) “And where on the ladder do you believe your household will be four years from now?â€? (AS.12) Respondents who said that they expected to be higher on the ladder in the future were coded as yes. Respondents who expected a worsening or no change on the ladder were coded as no. 78 Socio-emotional skills Source (if Outcome Explanation available) This outcome is derived from the question: Goal setting “Please indicate the degree to which you agree with each of the capacity following statements. 5 point: 1. Completely disagree 2. Mostly scale. disagree 3. Neither agree nor disagree 4. Mostly agree 5. Completely https://doi.or agree. (SES.1) g/10.7910/dv 1. I set short-term goals for myself. (Some examples of short-term n/3xwmdv goals include saving money for a small household purchase or learning a new skill) Goal 2. I set long-term goals for myself. (Some examples of long-term goals setting are finishing your education or saving money for a large household capacity purchase) 3. I set specific, clear goals for myself 4. I make plans to help me achieve my goals 5. I feel proud when I achieve my goals 6. I am able to prioritize multiple goals 7. Setting goals for myself is good for my success 8. Setting goals for myself is good for my household’s successâ€? The variable is coded as the mean of the responses to this question ranging from 1 to 5. The outcome is derived from the following question: “Some people World Value feel they have completely free choice and control over their lives, Survey while other people feel that what they do has no real effect on what Control happens to them. Please use this scale where 1 means “no choice at over one's allâ€? and 10 means “a great deal of choiceâ€? to indicate how much life freedom of choice and control you feel you have over the way your life turns out (code one number).â€? (SES.2) The variable value ranges from 1 to 10. This outcome is derived from the following question: BESSI 20 “Here is a list of activities or things you could do. For each one, please http://www.s select a response to indicate how well you can do that thing. For ebskills.com/ example, how well can you follow the instructions for an assignment? uploads/1/3/ Note that how well you can do something may be different from how 1/0/1310151 often you do it, or how much you like to do it. For each activity, you 87/bessi- Emotional should rate how well you can do that thing. Scale is 1 to 5. (SES.3) 20_form_self resilience 4. Calm down when I’m feeling anxious. .pdf 9. Keep a positive attitude. 14. Control my temper. 19. Find reasons to feel good about myself.â€? The variable is coded as the mean of the responses to this question ranging from 1 to 5. 79 • The emotional resilience outcome was derived from the Behavioral, Emotional, and Social Skills Inventory (http://www.sebskills.com/the-bessi.html). The set of questions is from BESSI 20 (http://www.sebskills.com/uploads/1/3/1/0/131015187/bessi-20_form_self.pdf). BESSI 20 measures five different skills domains: self-management skills, social engagement skills, cooperation skills, emotional resilience skills, and innovation skills. The questionnaire used for the survey included two out of five BESSI 20 domains: self-management skills and emotional resilience. We used the short-form set of items from the Behavioral, Emotional, and Social Skills Inventory (BESSI) developed by Soto et al. (2022). The BESSI has been validated for cross-cultural fit, as well as tested in multiple languages.60 • For goal-setting capacity, we used the revised and validated scale by the Measures for Advancing Gender Equality (MAGNET), tested with low income populations in Africa. • And for feeling of control over one’s own life, to proxy individual’s self-efficacy, we use the World Values survey item for this, also validated and translated for all focus countries, and also allowing for sample population to country average comparison. Selected literature using BESSI: • Soto, C. J., Napolitano, C. M., Sewell, M. N., Yoon, H. R., & Roberts, B. W. (2022). An integrative framework for conceptualizing and assessing social, emotional, and behavioral skills: The BESSI. Journal of Personality and Social Psychology, 123, 192-222. • Soto, C. J., Napolitano, C. M., Sewell, M. N., Yoon, H. J., Murano, D., Casillas, A., & Roberts, B. W. (2023). What I do and what I can do: Testing the convergence and incremental validity of social, emotional, and behavioral skills vs. traits for predicting academic success. Journal of Research in Personality, 104, 104382. • Sewell, M. N., Napolitano, C. M., Roberts, B. W., Soto, C. J., & Yoon, H. J. (2023). The social, emotional, and behavioral skill antecedents to college students’ volunteering during the COVID-19 pandemic. Journal of Research on Adolescence, 33, 618-631. • Soto, C. J., Napolitano, C. M., Sewell, M. N., Yoon, H. R., & Roberts, B. W. (2024). Going beyond traits: Social, emotional, and behavioral skills matter for adolescents' success. Social Psychological and Personality Science, 15, 33-45. • Napolitano, C. M., Soto, C. J., Sewell, M. N., Yoon, H. J., & Roberts, B. W. (in press). Changes in social, emotional, and behavioral skills are associated with changes in secondary school students’ important outcomes. European Journal of Personality. • Ringwald, W. R., Napolitano, C. M., Sewell, M. N., Soto, C. J., Yoon, H. J., & Wright, A. G. C. (in press). More skill than trait, or more trait than skill? Relations of (mis)matches between personality traits and social, emotional, and behavioral skills with adolescent outcomes. European Journal of Personality. Cronbach Alpha • Goal-setting capacity Bulgaria: 0.87 • Emotional resilience skills Bulgaria: 0.83 • Goal-setting capacity Romania: 0.93 • Emotional resilience skills Romania: 0.87 Limitations 60 For more details see http://www.sebskills.com/the-bessi.html 80 The questionnaire used in this study was not formally validated or calibrated prior to implementation, which may limit the reliability and comparability of responses across contexts. Without validation, it is uncertain whether the instrument consistently captures the intended constructs. The lack of calibration implies that the measurement properties of individual items were not statistically refined. In addition, fieldwork teams encountered some difficulties during data collection due to respondents’ low level of education. There were varying levels of respondent understanding and occasional inconsistencies in interpretation of certain items, which may affect data quality. 81