Report No: AUS0002179 WOMEN AND STEM IN EUROPE AND CENTRAL ASIA Blackboard in Minsk, Belarus. Photo by Roman Mager on Unsplash . © 2017 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “Munoz Boudet, Ana Maria, Rodriguez Chamussy, Lourdes, Chiarella, Cristina, and Oral Savonitto, Isil. 2021. Women in STEM in ECA. © World Bank.” All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. 2 Table of Contents Introduction .................................................................................................................................................. 4 Beyond the acronym: What do we mean by ‘STEM’? ............................................................................... 7 A framework to look at gender and STEM................................................................................................. 8 Education and STEM Relationship in ECA .................................................................................................... 11 Academic performance: Are girls worse than boys at math and science? .............................................. 12 Do aspirations, motivation, interest, and self-confidence of boys and girls reflect what we see in their performance patterns? ....................................................................................................................... 22 Explaining the differences in performance: girls need the alignment of more factors ....................... 33 STEM and Gender Dynamics in Tertiary Education ................................................................................. 34 Labor market outcomes for women in STEM fields and occupations ......................................................... 40 Behind the supply of STEM skills: Career aspirations and information on returns to education............. 41 School-work transitions for STEM and non-STEM graduates .................................................................. 44 Sectoral and occupational distribution of women and men in ECA ........................................................ 46 Returns to education, earnings and the gender pay gap in STEM ........................................................... 51 Policies to increase the participation of women in STEM ........................................................................... 56 What works? ........................................................................................................................................... 57 REFERENCES ................................................................................................................................................ 61 3 Women and STEM in Europe and Central Asia1 Introduction In the last decades, developed economies have witnessed significant declines in wages for low-skill workers, increases in employment in high-skill occupations, rapid diffusion of new technology, and expanding offshoring opportunities (Acemoglu & Autor, 2011). Labor markets in developed countries have reallocated labor from manual to cognitive jobs and from routine to non-routine work (Hardy, Keister & Lewandowski, 2015; Górka et al. 2017). Overall, workers are now required to do more complex tasks than before. In a changing labor market, education systems should impart the right skills, ideally both foundational ones and additional skills that will be amenable to adaptation and re-training to match with job opportunities. This is even more important in the ECA region given the context of the increasing share of older people who will depend on today’s generation of children when they join the working age population. Women’s access to and progress in what is generally referred to as ‘STEM fields’ of science, technology, engineering, and mathematics are topics that have been growing in significance in recent years (UNESCO, 2017; OECD, 2018). This is because (i) educational attainment and labor market outcomes have traditionally been unfavorable for women, but as women make strides in education, those gains are yet to be fully reflected in the labor market, representing an important missed opportunity (Hill, et al., 2011, Plantenga and Doneva, 2015); (ii) STEM occupations have been on the rise as people who are employed in fields related to technology and innovation play key roles in the growth, stability and global relevance of their economies in the world (Fayer, 2017; Hanson and Slaughter, 2016; OECD, 2018; World Economic Forum, 2018). In addition, people in STEM fields are well paid with better job security ad many STEM occupations have been predicted to grow faster than the average rate for other occupations (Lacey & Wright, 2009; National Science Board, 2010). Technological change that replaces routine work is estimated to have created more than 23 million jobs across Europe from 1999 to 2016 - almost half of the total increase in employment (World Bank, 2019). Evidence from Europe suggests technology is replacing workers in some jobs, but it also raises the demand for labor and leads to the creation of jobs in the technology sector (Gregory et al., 2016). Digital technologies also promote inclusion by boosting employment and earnings for women and men in the ICT sector or ICT occupations. ICT jobs pay well and can generate additional jobs through consumption and production spillovers. The median hourly earnings in the ICT sector and in ICT occupations are 1.5 times higher than in urban non-ICT sectors or non-ICT occupations in developing countries (World Bank, 2016). The higher pay reflects a workforce that is better educated than average and also reflects the relative 1 This report was produced by a World Bank team including Ana Maria Munoz Boudet, Lourdes Rodriguez Chamussy, Isil Oral Savonitto and Cristina Chiarella at the Poverty and Equity Global Practice. The work was funded by a grant of the World Bank’s Umbrella Facility for Gender Equality. The team benefited from comments by Nistha Sinha, Eliana Rubiano, Roberta Gatti and Oni Lusk-Stover. 4 scarcity of ICT workers in some countries, driving up the skill premium. These high-paying jobs create more demand and new jobs outside ICT as well. In the United States and Turkey, one job in the high-tech industry generates an average of 3 to 5 additional jobs elsewhere in the local economy. All this being said, the gender gap is still large, with men 2.7 times more likely than women to work in the sector and 7.6 times more likely to be in ICT occupations (World Bank, 2016). STEM professions have also been dubbed as the jobs of the future since innovation is mostly derived from advances in STEM-related disciplines. Employees of STEM occupations help drive innovation and competitiveness by generating new ideas and building new companies with these new ideas (Schlenker, 2015). Recent empirical research has documented that inventors are more likely to be educated in STEM fields compared to the average individual (Jung and Ejermo, 2014; Aghion et al., 2016; Bell et al., 2016; Akcigit, Grigsby and Nicholas, 2017). One specific example is in the United States where STEM workers are more likely to apply for, receive, and commercialize patents than non-STEM workers (Thomasian, 2011). STEM jobs are leading occupations for innovation and growth; therefore, they are in high demand but with low supply. According to a recent note from CEDEFOP, across the European Union, the top skill shortage occupations are ICT professionals, medical doctors, and STEM professionals (CEDEFOP, 2016). The report indicates that demand for ICT professionals is on the rise because almost every economic sector needs their skills, which are becoming more ubiquitous in the modern economy. Despite this, the supply of ICT and STEM graduates from upper-secondary and higher education is insufficient to meet this demand. In Europe (including Serbia and Turkey), the share of STEM tertiary students has increased from about 23% to 31% between 2013 and 2017, and the share of female students among those enrolled in STEM moved from 31% to 35% in the same period (source: Eurostat), an increase from the 22% observed in terms of STEM enrolment in 2006-2012. Demographic patterns will lead to STEM shortages if these trends persist: while high numbers of current STEM workers are approaching retirement, there will be relatively fewer young people to move through the education system towards labor markets with STEM skills (Dobson, 2013). The level of demand is such that some countries such as Bulgaria and Poland suffer from a STEM ‘brain drain’ as professionals emigrate for better jobs in countries like Germany and the UK (Shapiro et al., 2015). STEM-derived skills such as analytical thinking, problem-solving, ability to draw conclusions from research, and data-driven decision making can also be used to succeed in a variety of jobs, and are thus flexible in their use in a variety of settings, including management occupations (Deming and Noray, 2018). Increased technology use in the workplace means that more and more workers will end up needing the critical thinking and technical skills that come with STEM training. If the relevance and importance of STEM professions or skills continue to garner attention as an impetus for innovation and growth, an increasing number of jobs will require technical and specialized STEM knowledge. Many countries are looking to invest in creating a STEM proficient workforce to be competitive (Hill, et al., 2011; Rothwell, 2013). Here, it is crucial to make sure that relevant policies are in place to encourage interested women and men to pursue STEM degrees and make sure there are no barriers to entry for these degrees and professions later on. This could start from early years of schooling and continue with a transition from school to work as well as having opportunities in work life such as equal pay and promotions. 5 Gender segregation in occupations and education streams is inefficient, whether it is men and women are equal with respect to characteristics such as ability, in which case specialization gains derived from it will depend only on the extent of the market and the trade-off between short term gains and long-term losses (for example, if large gaps in income are derived from such specialization); and if we assume women and men have an equal distribution of talent, and what drives specialization are preferences, but such preferences are made in a social and normative environment that is not free of constraints (Plantenga and Doneva, 2015). Gender segregation thus implies a misallocation of talent across occupations with effects on productivity and welfare (Adda, et al., 2017). Based on a recent study, if women held digital jobs as frequently as men did, gains for European GDP each year would amount to 9 billion EUR (European Commission, 2013). In times of aging societies and shrinking labor forces, and in the context of economies that are increasingly knowledge-driven, the quantity and quality of workers in the key industries encompassed within the STEM category become especially critical for future growth and welfare (Schlenker, 2015). In the EU context, the capitalization of the potential and talent of women are needed to achieve the employment target set in the Europe 2020 strategy (Paniscu and Mladen, 2015). The lack of gender diversity within particular science disciplines is bound to hamper creativity, competitiveness, and the quality of output (Sikora and Pokropek, 2012). But before jumping on the STEM promotion bandwagon, we need to further examine where gender differences come from, whether they are indeed negative for women, and to identify what exactly we are referring to when talking about STEM and women. While, on the surface, there seems to be a consensus of the areas covered in STEM; in practice, the definition of the areas of study, occupations, or economic sectors included varies. Such variation is important when it comes to understanding gender differences and the value of studying or working in STEM. Irrespective of the definition - which we will discuss in the next paragraphs- as an approach in education, STEM adds value, regardless of further career choices. Teaching science and mathematics from kindergarten through higher education is one of the leading interdisciplinary education paradigms of today and plays a key role in transforming knowledge into innovative and valuable products (UNESCO, 2017). STEM education has the potential to contribute to economic development by helping raise creative and productive individuals equipped with the relevant new technology skills and promote innovative mindsets (National Research Council, 2011). When it comes to tertiary education, STEM streams of study are closely linked to future job opportunities. An important question related to what happens with STEM in primary and secondary education is, why female students are not choosing a STEM career. To date, most research on this subject has focused on the reasons behind women being “left out” of STEM fields (Hill, et al., 2011), but and less have looked at why women choose not to enter STEM careers (Stoet and Geary, 2018). Although overall, the number of women in STEM fields has gone up globally in recent years, systematic gender differences remain, which are then reflected in the labor market, where we see fewer women working in STEM occupations, and even in STEM sectors (Hill et al., 2011; OECD, 2018). A recent OECD study found that in the developed world, women account for just 25 percent of graduates in information and communications technology, and 24 percent in engineering (OECD, 2018). Most of the 3.9 billion people in the world who are offline are women; in 6 Africa, only 12 percent of women are online and in the developing world, women’s access to the internet is 25 percent below that of men (UNESCO, 2017). If women are less likely to show interest in STEM, to aspire, and to engage in STEM degrees; and even when engaging in the sector, they are less likely to be found working in STEM occupations or in a STEM-focused company, then it is not surprising that men continue to dominate STEM. And with that, it also comes that men are also more prominent at the upper levels of STEM professions and pay scales (Hill et al., 2011), as well as those more likely to get a job into these occupations and sectors. Beyond the acronym: What do we mean by ‘STEM’? In simple terms, STEM is an abbreviation for science, technology, engineering, and mathematics; but what exactly is included in such definition? From a review of many definitions (Brown, 2012; Koonce et al., 2012; Siekmann and Korbel, 2016), when referring to STEM, different people and institutions might be referring to different things. It is unclear whether it refers only to fields of study, and what the main differentiating element is among such fields of study (e.g. whether life sciences are part of STEM or not). When it relates to economic activity, what constitutes a STEM occupation or job is complex as a STEM job could be less about mirroring the STEM field of study, and more about the use of STEM knowledge in the main tasks of any job. In this scenario, STEM can refer to many different things. There has been considerable debate on what occupations to include in the STEM field as well as what types of jobs within STEM fields count (Deming and Noray, 2018; GAO, 2014; Grinis, 2017; Siekman and Korbel, 2016). Mostly, the discussion is centered around a few occupational groups such as educators (i.e. science and math teachers), healthcare professionals (nurses and doctors), and social scientists (psychologists and economists) as to whether or not they belong in the STEM category (Deming and Noray, 2018; GAO, 2014).2 An additional question that comes to mind would be the inclusion of administrative workers that work in STEM fields such as IT and scientific research (Smith and White, 2018). Such discussions are not gender neutral. If we look at the share of women and men in education or health, we will find that a large share of teachers, regardless of subject, are women. According to the OECD, around 97% of pre-primary schoolteachers, 82% of primary school teachers, and 63% of secondary school teachers are women (OECD, 2017). Similarly, women dominate the nursing profession. According to WHO Europe, in almost all countries in Europe, over 90% of nurses and midwives are female (WHO, 2017). These job trends also imply corresponding gender differences in education such as having more women in education and nursing programs of tertiary education institutions. However, it is not the aim of this report to solve what should or should not be included in the definition of STEM. For this report, when we talk about STEM we will refer to professional and technical occupations and study streams in the areas of life and physical 2 For example, in the US the National Science Foundation includes in STEM fields psychology, economics, sociology and political science 7 sciences,3 computer science, mathematics and engineering, a ‘narrow’ operational STEM definition that can be used across different countries,4 in line with some existing conventions.5 A framework to look at gender and STEM The interest in gender segregation in STEM, as in other sectors, arises mainly from the role segregation plays in gender inequality in the labor markets. Gender differences in productivity and earnings are systematic and persistent as women exhibit lower average productivity and earn lower wages than men across the board. These differences have been documented in both developed and developing countries, and although they have declined over time (primarily as a result of the reduction in the education gap), they remain significant (Blau and Kahn, 2016). Another form of segregation is self-selection into specific occupations. This is the notion that because of care and other responsibilities, women are more likely than men to choose an occupation that offers more flexibility and that do not require large or continual investments in skills. These tend to be occupations where the returns to skills and experience are lower and, other things equal, so are wages (Akerlof and Kranton, 2000: Görlich and de Grip 2009). Although the effects of such segregation are not always negative for women, in many instances, segregation constrains women’s employment choices and their access to higher-level jobs, while facilitating the undervaluation of female-dominated occupations (Burchell et al., 2014). Moreover, when examined closely, as will be done in the upcoming sections of this report, the disproportionately lower shares of women in STEM do not appear to be associated with issues of performance or ability but rather with preferences and aspirations (Schlenker, 2015). This is a similar trend as to why men are not encountered in care-related professions as frequently as women - which is a subject that has been associated with the societal worth of these ‘feminine’ professions.6 Preferences and aspirations barriers can also affect men, but they add to already existing tasks or preferences for women such as caring for children or needing flexibility in one’s job. In addition, preferences and aspirations of not only employees and students, but also of employers, educators, and society as a whole can exacerbate this gender segregation further. If women are not taking part in STEM-related education, they are underinvesting in their productivity. Any and all education can indeed make individuals more productive through increased skills (Duflo, Dupas & Kremer, 2017), and higher-skilled individuals are more employable, productive, and extend their labor market participation over their lifetime (Hanushek & Zhang, 2006). However, the foundational skills related 3 Medical professionals such as doctors and nurses were not included in the realm of STEM for this report. They were also excluded in the analyses involving PISA and LFS data. For PISA analysis, we included “life science professional and technicians” but left out “medical doctors” and “nurses and midwives.” For LFS analysis, we included “science and engineering professionals and associate professionals” and “ICT professionals and associate professionals” but left out “health professionals and associate health professionals.” 4 See Annex for ILO’s International Standard Classification of Occupations (ISCO). 5 See for example Eurostat Indicators on Human Resources in Science and Technology (https://ec.europa.eu/eurostat/web/science-technology-innovation/data/database) 6 Akerlof and Kranton (2000), Goldin (2013), Block, K., Croft, A. and Schmader, T., (2018). 8 to STEM appear to be of growing importance, particularly as flexibility, problem-solving, and other skills grow in importance over technical skills. International research (Acosta and Muller, 2018; Cunningham and Villaseñor, 2014; Zlatko and Ajwad, 2014; World Bank, 2019) has identified three dimensions of skills that matter for good employment outcomes and economic growth: cognitive skills, such as literacy, numeracy, creative and critical thinking or problem-solving (the last three strongly associated with STEM-related education); socio-emotional skills and behavioral traits, such as conscientiousness, grit or openness to experience (again features associated with STEM); and job-specific technical skills, such as the ability to work as an engineer. Cognitive and socio-emotional skills formation starts early in a person’s life, providing the necessary foundation for the subsequent acquisition of technical skills, largely in the education system, as well as in the workplace. Investment in such skills depend on many factors, from aspirations to having the correct and relevant information. In a graphic manner (diagram 1), we will create a storyline that there are STEM pathways, and these are formed early in life. A girl or boy’s interest and performance in math and science dur ing school years will be one part of their aspirations formation, which is also bound by exposure to role models and the views of parents and peers. Whether those aspirations transfer into entry and graduation of a STEM-related field of study will also be related to performance. Graduating from a STEM-related field of study will partly influence whether labor-market participation is in a STEM occupation, and/or a STEM-intensive sector. All these processes and decisions along the way will be influenced by information on STEM jobs and their characteristics, and information on their returns that will be weighed against other choices and preferences such as flexibility, time, or earnings. Diagram 1: STEM life-cycle of an individual 9 Taking this as a starting point, this report looks at: (i) how, and if, gender differences in STEM are present in Europe and Central Asia (ECA), and if so, what we can make of them; (ii) what might be the reasons of why women may or may not be engaged in STEM from education to the labor market; (iii) the mechanisms behind the pathways, as earlier barriers will make it more difficult to transition into STEM when initially choosing to be out – this transition is not always as challenging for non-STEM pathways. For women, barriers start earlier, are more salient, and choices and preferences shaped by peers seem to be stronger. Men’s barriers in terms of roles do not condition them out of STEM pathways but out of careers that require a lower investment in time and effort. Thus, this report is an exploratory one. We explore if there are differences between men and women in outcomes such as grades, enrolment, employment, and income. And then we explore the factors associated with those differences. Despite the attention that this topic is getting, this report also notes that there is a lack of clarity on definitions as well as patchy and incomplete data that affects our ability to make comparisons, follow trends, and contextualize a lot of the observed situations. We aim to cover as many countries in the region as possible, but this has not always been feasible. Hence, in some cases, a sub-set of countries, or specific country-cases are used. To address these questions, the report is organized into three chapters. The first one looks at education - including STEM content and fields of study that are within the education realm. The second one follows women into the labor market and looks at employment and wages in STEM sectors and for STEM occupations. The final section looks at policies, from the many initiatives and efforts in place to promote women’s participation in STEM, with a focus on those that have documented results. 10 Education and STEM Relationship in ECA The discussion on gender gaps in STEM education is not only about which programs men and women choose in tertiary education, but also about differences in performance in STEM-related education fields. There has been a major long-term international trend towards girls getting better results in school, and women being more likely to go to university than men, and ECA is no exception. Across ECA 70 percent of female and 66 percent of male students enroll in tertiary education,7 but women enroll in STEM-related fields in lower numbers, and, prior to tertiary education, they appear to, on average, underperform their male peers in STEM-related subjects in school, at least on the aggregate level. These differences are also relevant for those who don’t reach tertiary education and stay at the vocational or secondary education level, as the learning of STEM skills is as important as those who specialize in them in higher education. It is important to remember that education policy is multi-faceted in all countries. There are many layers and phases of education in an individual’s life span. While analyzing education policy in any context, one needs to think of access, quality, and outcomes. Various indicators can tell different parts of the story on education such as enrollment, graduation rates, and student performance. In this chapter, all of these indicators will be visited but the focus will be on student performance and outcomes as there is more comparable data to explore across ECA countries and it tells a more complete story in the STEM context. The gender gap in math and science performance has been attributed, as the gender gap in STEM has, to a number of different factors such as biological differences or pre-dispositions (Berenbaum, Bryk & Beltz, 2012; Blakemore, Berenbaum & Liben, 2009; Halpern, 2012), cultural and social norms (Andreescu et al., 2008; Ceci et al., 2009; Gevrek et al., 2018; Hartung et al., 2005; Low et al., 2005), parents’ and teachers’ behavior and expectations (Beilock et al., 2010; Gunderson et al., 2012), and self-confidence (Ellis, Fosdick & Rasmussen, 2016; OECD, 2016b), among others. Looking at these potential explanations matters to assess what we see as gender gaps in areas of study in tertiary education, understanding whether there is a path that starts earlier -in primary or secondary education- where girls and boys differ in their results and/or interest in math and science. We start this chapter by exploring deeper the differences between boys and girls in terms of performance in math and science, as these differences might be telling about what we observe later in career choices. Following that, we’ll look at the gender distribution in tertiary enrolment, partly as an outcome of the difference in performance and interest in math and science shown during the school years.8 7 WDI, 2017, gross enrollment ratio for ECA countries excluding high income. 88 To try and answer these questions we use two key international datasets on performance that are comparable between different countries and over time. These datasets are PISA (Programme for International Student Assessment) and TIMMS (Trends in International Mathematics and Science Study). PISA is a triennial international survey that aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students. TIMSS is a series of international assessments of the mathematics and science knowledge of 4th grade and 8th grade students around the world. [2] For TIMMS, we use the 2015 round. For PISA we focus on the last two waves as they had as a special focus science (2015) and math (2012), and in addition to the test scores, we also look at results from students, parents, teachers, and school principals surveys to complement to understand how different factors play a 11 Academic performance: Are girls worse than boys at math and science? The existence of gender gaps in test scores has been documented in the relevant literature for a wide range of countries following standardized assessments such as PISA and TIMMS. In particular, the lower performance of girls relative to boys in math is seen as the main reason why women are under-represented in STEM. Time trends of TIMSS from the last 20 years show that performance differences in favor of boys in mathematics and science have been reduced, especially in science and especially at the eighth grade as girls also started performing well in these subjects (Mullis, et al., 2016). In 1995, boys in most countries (15 of 26) performed better than girls in math and science – and this difference was much more pronounced in science. In 2015, boys performed better in only three of these 15 countries. For ECA countries, TIMSS 2015 results in mathematics achievement show that there are no salient differences between boys and girls, either in 4th grade or in 8th grade. Boys have an advantage in 4th grade in Croatia, which is statistically significant, and in 8th grade in Russian Federation, also significant. But girls also have a slight advantage in 8th grade in Kazakhstan and Turkey, although not statistically significant (Figure 1). Overall, short-term trends show increasing advantage for girls in science achievement, while 20-year trends show a great reduction in boys’ historical advantage in science (Mullis, et al., 2016b). This trend seems to resonate also in the ECA countries that took part in the latest TIMSS assessment. TIMSS 2015 science results show close to no gender differences in science achievement in 4th or 8th grade in ECA. Girls outperform boys in 4th grade in Bulgaria and Kazakhstan, which is statistically significant, and in 8th grade in Turkey, which is also significant (Figure 2). PISA, on the other hand, consistently reveals that on average girls underperform boys in math and outperform them in literacy. However, unpacking these trends reveals a more nuanced story for the case of ECA. For example, when we look at ECA countries, there are no significant differences by gender in performance in mathematics and science. Boys’ math scores are above girls’, but the difference is not statistically significant. Only literacy (or reading) shows a clear significant difference in favor of girls (Figure 3). At the country level, PISA 2015 math scores show a mixed picture of performance across genders. Only two countries show significant gender differences in math scores: boys perform better than girls in Poland and girls perform better than boys in Georgia. A similar story is true for science. Although there are no general significant differences in ECA, a few countries do have gender differences. Girls perform better than boys in science in the following countries: Latvia, Georgia, North Macedonia, and Albania. However, all countries in ECA have a significant gender gap in reading performance in favor of girls. (Figure 4 a,b,c). role in student achievement. The ECA countries in PISA are Albania, Bulgaria, Croatia, North Macedonia, Georgia, Kazakhstan, Kosovo, Moldova, Montenegro, Poland, Romania, Russian Federation, and Turkey. The latest two waves were conducted in 2012 and 2015. The ECA countries for TIMMS are Bulgaria, Croatia, Georgia, Kazakhstan, Poland, Serbia, Russian Federation, and Turkey. The survey has been administered every four years since 1995. The last international wave of TIMMS was in 2015. 12 When it comes to reading girls score on average 37 points higher than boys on the PISA reading scale- the equivalent of almost one full year of formal schooling. Figure 1: TIMSS: Mathematics Achievement 600 *** *** 500 400 300 200 100 0 Female Male Female Male 4th grade 8th grade Bulgaria Croatia Georgia Kazakhstan Poland Russian Federation Serbia Turkey International average Source: TIMSS, 2015. Figure 2: TIMSS: Science Achievement 600 *** *** *** 500 400 300 200 100 0 Female Male Female Male 4th grade 8th grade Bulgaria Croatia Georgia Kazakhstan Poland Russian Federation Serbia Turkey International average Source: TIMSS, 2015. 13 .Figure 3: Mathematics, Science and Reading Scores in ECA 490 480 470 460 450 440 430 420 mathematics science reading Girls Boys Source: OECD, PISA 2015 database. Y axis values represent scores on the PISA scale. 14 Figure 4a: Mathematics Scores in ECA Countries Figure 4b: Science Scores in ECA Countries 550 550 530 530 510 510 490 490 470 470 450 450 430 430 410 410 390 370 390 350 370 HUN MNE TUR RUS SVK GRC BGR MKD CZE SVN LTU GEO ALB EST LVA MDA POL HRV ROU RUS HUN TUR MNE GRC BGR LVA MKD SVN LTU GEO CZE SVK EST MDA ALB HRV POL ROU Girls Boys Girls Boys Figure 4c: Reading Scores in ECA Countries 550 500 450 400 350 300 HUN TUR RUS MNE LVA MKD BGR CZE SVK GRC LTU SVN GEO ALB EST MDA ROU HRV POL Girls Boys Source: OECD, PISA 2015 database. Y axis values represent scores on the PISA scale. 15 While already interesting in the lack of expected differences between girls and boys, aggregate scores do not necessarily show the many differences we can observe between girls and boys when it comes to their math performance. However, proficiency levels, among other metrics, allow seeing how much students know and if they can apply the information they gain in education into skills (in ‘real life’ situations). Here, we start seeing more noticeable gender differences. Figure 5 shows the percentage of low achievers and top performers in mathematics by gender. While again there is significant variation between countries and between boys and girls, a clear pattern emerges, inasmuch as proficiency levels are not very high in ECA, in countries that perform well (with more students in the top levels), there are fewer boys than girls at low levels of proficiency, and the reverse, more boys than girls, at higher proficiency levels. For example, for a top performance country like Poland, where the gender difference in performance favors boys, only 16 percent of boys and 19 percent of girls performed under Level 2 (the minimum level of proficiency), while 14 percent of boys and 10 percent of girls perform at the top two levels. In contrast, 75 percent of boys and 81 percent of girls performed below Level 2 in Kosovo. In a majority of ECA countries, more than half of the students who participated in PISA 2015 performed at the minimum proficiency level (level 2). In Georgia or Albania, where girls outperform boys, the difference is 6 and 4 percentage points respectively. Overall, a challenge for observing gender differences is that math proficiency levels are not very high in ECA, except for Poland, the Russian Federation and Hungary. Figure 5: There is significant variation between ECA countries and between genders with regards to proficiency levels in math 90 80 70 60 50 40 30 20 10 0 Boys Below Level 2 Boys Level 5 or above Girls Below Level 2 Girls Level 5 or above Source: OECD, PISA 2015 database. As for performance in science, female students seem to have done better on average based on scores only. In some countries like Croatia, Poland and Russian Federation male students had higher science scores. 16 However, in all remaining ECA countries, girls did better. Almost all ECA countries did worse than the OECD average scores in science with the exceptions of Poland and the Russian Federation. The level of proficiency in science is similar to math in terms of top-performing ECA countries. Both Poland and Russian Federation, have low levels of students below Level 2 (16 and 18 percent respectively for both boys and girls). On the other hand, in Kosovo, 70 percent of boys and 66 percent of girls performed below Level 2. In the majority of ECA countries, less than 1 percent of students performed at the top two levels in terms of science proficiency, and when observed, there is a higher proportion of boys performing at the top proficiency levels in science. In most countries that do relatively well in science such as Poland, Croatia and the Russian Federation, boys and girls perform similarly at lower levels, but boys do better at higher levels. In countries that have worse performance, boys do worse than girls at lower levels and have similar performance to that of girls in the higher levels. On the reverse side, when it comes to reading proficiency, there is a higher percentage of boys in the group of low achievers with a wide margin in every ECA country. In the highest performing country, Poland, 19 percent of boys are below the basic level of reading proficiency, while this rate is 10 percent for girls. As for high performers, girls outpace boys in almost every country, except for North Macedonia and Kosovo where there is 0 percent of students in Levels 5 and 6 on the reading proficiency scale. A further observation on performance distribution supports the previous findings on math, science, and reading performance by gender in ECA (Figure 6). More boys are performing at higher levels in the PISA math and science proficiency scales when performance distribution by gender is analyzed. However, there are more boys also on the lower end of the score distribution. The variance is greater for men than for women, who are less concentrated in the lower and upper ends of the distribution. Women’s performance distribution is more concentrated in the middle. There are significantly more girls that perform in the mid- range compared to boys, both in science and in math. For reading performance distribution, girls do better in all levels compared to boys: there are more boys at lower levels and fewer boys at higher levels. Figure 6: Performance distribution for math, science and reading in all ECA countries combined Math Science Reading .004 .004 .004 .003 .003 .003 kdensity mathmean kdensity readmean kdensity scimean .002 .002 .002 .001 .001 .001 0 0 0 0 200 400 600 800 0 200 400 600 800 0 200 400 600 800 x x x female male female male female male Source: OECD, PISA 2015 database. Note: The vertical lines represent proficiency levels 1 through 6 from left to right. 17 Following on initial findings by Herrera et al. (2018), performance can be further explored by looking at the specific contents included in the math and science sections of PISA -the subscales. Gender differences are mostly found in the “space and shape” subscale in favor of boys and to a lesser extent in the “quantity” subscale. These two scales use questions that are generally applied -i.e. estimate the square footage of a room; vis-à-vis the other more traditionally phrased questions -i.e. equation solving. In science, the only field that shows gender differences in favor of boys is “explain phenomena scientifically”. At the country level, there are important patterns that emerge. There are significant differences in performance in favor of boys for “explain phenomena scientifically” and “science content knowledge” (in Czech Republic, Poland, Russian Federation, Estonia, Croatia and Hungry). At the same time, there are significant differences in performance in favor of girls in: “evaluate and design scientific enquiry” (in Lithuania, Montenegro, Slovenia and Latvia), “interpret data and evidence scientifically” (in Slovenia, Latvia, Lithuania and Bulgaria), “procedure and epistemic knowledge” (in Montenegro, Slovenia, Lithuania, Latvia and Bulgaria), “living systems” (in Lithuania, Slovenia and Latvia). Figure 7: Scores for Mathematics Subscales 490 485 480 475 470 465 460 455 450 445 Female Male Female Male Female Male Female Male Change and Space and Shape Quantity Uncertainty and Data Relationships Figure 8: Scores for Science Subscales 480 475 470 465 460 455 450 445 Female Female Female Female Female Female Female Female Male Male Male Male Male Male Male Male Explain Evaluate and Interpret Science Procedure & Physical Living Earth & phenomena Design Data and Content Epistemic Systems Systems Science scientifically Scientific Evidence Knowledge Knowledge Systems Enquiry Scientifically Source: OECD, PISA 2015 database. 18 Taking advantage of the additional questions asked in PISA, we can also see that boys have higher self- efficacy in most mathematics questions and some science questions, particularly in the “hard -science” questions related to engineering. Girls have higher self-efficacy in specific mathematics questions such as solving a linear algebra equation, but less self-efficacy in the application of mathematical knowledge, which relates to the findings related to their performance in the subscales. In science, girls have higher self- efficacy in questions related to biological sciences. Following these findings, we investigate whether these self-efficacy questions have any effect on performance. We do this by estimating linear regressions on the effect of self-efficacy on performance, by gender with country fixed effects. We find that indeed math self-efficacy questions significantly predict higher math scores. Believing to be able to solve a linear algebra equation is the self-efficacy question with the greatest return on performance, for both boys and girls. Other questions that are important for both genders are believing to be able to apply mathematical knowledge, such as using calculating a discount rate, calculating area, etc. One question that appears significant for boys but not for girls is finding the distance between two places on a map (Figure 9). In science, the one self-efficacy question that has the strongest effect on performance for both boys and girls is if children believe they’re capable of explaining why earthquakes occur more frequently in some areas than in others, but the effect is stronger for boys, followed by being able to identify the most likely explanation for acid rain formation. Believing to be able to describe the role of antibiotics and predict how changes in the environment will affect the survival of ce rtain species have a more important role on girls’ science performance than in boys. Discussing how new evidence can lead to the understanding of life in Mars has a positive effect on boys performance and a negative effect on science performance for girls (Figure 10). Figure 9: Math Self-Efficacy Questions and Performance in Math 30 *** 25 *** 20 *** *** *** *** *** *** 15 *** *** *** 10 *** 5 ** 0 -5 ** -10 *** Using a train Calculating Calculating Understanding Solving an Finding the Solving an Calculating the schedule to how much how many graphs equation like actual distance equation like gas mileage of figure out how cheaper a TV square feet of presented in 3x+5= 17 between two 2(x+3) = (x + a car long it would would be after tile you need newspapers places on a 3)(x - 3) take to get a 30% to cover a map with a from one discount floor 1:10,000 scale place to another Female Male 19 Figure 10: Science Self-Efficacy Questions and Performance in Science 25 *** 20 *** 15 10 *** *** *** *** 5 ** ** ** ** ** 0 -5 ** ** *** -10 *** -15 *** Recognise the Explain why Describe the Identify the Predict how Interpret the Discuss how Identify the science earthquakes role of science changes to an scientific new evidence better of two question that occur more antibiotics in question environment information can lead you explanations underlies a frequently in the treatment associated will affect the provided on to change for the newspaper some areas of disease with the survival of the labelling of your formation of report on a than in others disposal of certain species food items understanding acid rain health issue garbage about the possibility of life on Mars Female Male Source: OECD, PISA 2015 database. To be able to understand what drives the gender gap in performance, we estimate a Blinder-Oaxaca decomposition on reading, mathematics and science performance, controlling for children’s characteristics, motivation, grade, whether the student is a repeater or attended 2 years or more of pre- school, educational material shortage, some school characteristics, learning climate, school segregation and school autonomy. We find a clear performance gap in favor of boys for mathematics, and in favor of girls for reading and science, as it was shown earlier. Student and school characteristics are important predictors of a gap in favor of girls for the three subjects, even though the final gap in mathematics is in favor of boys. But the unexplained part is in favor of boys for both mathematics and science. So, for mathematics, the positive effect for girls of the individual and school characteristics is overcompensated by the negative effect of the unexplained portion, such that the final gap ends up being negative for girls (in favor of boys). The opposite can be seen for science: the positive effect for girls of individual and school characteristics overcompensate the negative effect of unexplained variation. In other words, boys get a higher payoff than girls in terms of performance for their characteristics. In reading the unexplained variation is in favor of girls: girls get a higher performance payoff than boys for their characteristics. 20 Figure 11 Oaxaca Decomposition of the Reading, Mathematics and Science Performance Gender Gap Reading Mathematics Science 100% 50% 0% -50% Repeater Grade Individual Characteristics -100% Student Motivation 2y+ Preschool School Segregation School Characteristics Educational Material Shortage School Autonomy Source: OECD, PISA 2015 database. We also investigate if the performance gap can be explained by the differences in children’s motivation. To do that we compare the performance of the most motivated quantile with the performance of the least motivated children in the three subjects. We find that motivation is quite important in improving boys’ reading performance: most motivated students do perform better in reading than least motivated students. But motivation is not important for girls’ reading scores. In fact, least motivated girls perform slightly better than most motivated girls. Motivation doesn’t play a role in either boys’ or girls’ mathematics performance. But motivation does play a role in improving girls’ science performance, although not boys’ performance. Figure 12: Performance Gap between most and least motivated students 1.01 1.005 1 0.995 0.99 0.985 Reading Mathematics Science Female Male no performance gap Note: For each gender, quantiles of motivation were calculated. The performance gap is calculated as the performance of the most motivated quantile over the performance of the least motivated quantile, for each gender. All ECA countries in PISA are considered 21 The analysis of performance differences so far shows that, while in general terms gender differences are present, there is no clear evidence that boys do better than girls in math and science in ECA, and that the results are mixed. There is a slight advantage of boys at the highest level of the performance distribution in math and science, however, boys outnumber girls also at the lower end of the performance scale. Girls tend to do better than boys in the middle of the performance range both in math and science. There is great variation among the ECA countries for which we have data – in some countries the student performance is exceptional and close to or above OECD average. However, for most ECA countries, math and science performance is poor both for girls and boys. In countries that do relatively well in science such as Poland, Croatia and the Russian Federation, there is a similar proportion of girls and boys at lower levels of performance, but boys do better at higher levels. In Albania, Bulgaria, North Macedonia, Georgia, Hungary, Kosovo, Moldova, Montenegro, Romania and Turkey, boys do worse than girls at lower levels and have similar performance to that of girls in the higher levels. Do aspirations, motivation, interest, and self-confidence of boys and girls reflect what we see in their performance patterns? Boys’ and girls’ attitudes towards learning and aspirations for their future can be very different and that has a significant impact on their decision to pursue further education and on their choice of career. The relationship between gender stereotypes and the formation of career aspirations in children is important to explore to better understand what influences the likelihood of a young person aspiring to a STEM-related career. Children and young people still attach ‘female’ and ‘male’ labels to particular careers. The development of these labels begins at an early age and unfolds its most significant impacts after primary school (Senler and Sungur, 2009). This could lead young women to rule themselves out of careers that they might otherwise be successful in (Ertl et al., 2017; Schuster and Martiny, 2017; Flouri & Panourgia, 2012; Nosek et al., 2009) and they often apply these kinds of stereotypes to the assessment of their own abilities in STEM (Ertl et al., 2017). Stereotypes lower one's self-assessment and sense of competence (Marsh and Scalas, 2011). They also have a direct impact on career choices (Ertl et al., 2017; Schuster and Martiny, 2017). Another key factor in building aspirations is the way role models act around children. Parents' and teachers' expectations and attributions of abilities and achievements shape a child's self-concept (Ertl et al., 2017). If they do not rely on objective assessments of their child but on stereotypical evaluations which do not correspond to their child’s actual achievements, this could affect the child’s self -confidence. For example, parents tend to regard daughters as being less talented in mathematics and science and reinforce dysfunctional attribution patterns as a result, regardless of their child’s performance (Ertl et al., 2017). A recent study investigated to which degree individual stereotypes, support in school, and family support contribute to the self-concept in STEM. Gender stereotypes were found to be negatively related to students' STEM-specific self-concept in the selected sample. The study also reveals negative stereotypes and family-related influences can lower a woman's self-concept even though they presumably had good grades in STEM-related topics. One of the reasons for this might lie in stereotypes that attribute girls' achievements to diligence instead of talent (Ertl, et al., 2017). Another study investigated the gender gap 22 in math performance from the perspective of teacher implicit bias. Girls, especially those from disadvantaged backgrounds, lag behind boys significantly when assigned to teachers with high implicit stereotypes. Boys, on the other hand, are not affected by teacher bias. Teacher stereotypes affect high- school track choice, lower expectations about own ability, and lead to underperformance in male-typed domains such as STEM. Girls are more likely to consider themselves bad at math at the end of middle school if they are assigned to a teacher with a gender bias. Unconscious biases and implicit associations can form a significant barrier to equal opportunity. Female students compared to boys with the same performance are more likely to believe their achievement is the result of effort and less likely to believe it is the result of ability (Carlana, 2018). Self-efficacy is considered one of the most influential motivators and regulators of student behavior and performance. The term “self-efficacy” is used to describe students’ belief that, through their actions, they can produce desired effects, such as solving a difficult problem or achieving a personal goal. This, in turn, is a powerful incentive to act or to persevere in the face of difficulties (Bandura, 1977). Science self-efficacy refers to future-oriented judgments about one’s competency in accomplishing particular goals in a specific context, where meeting these goals requires scientific abilities, such as explaining phenomena scientifically, evaluating and designing scientific enquiry, or interpreting data and evidence scientifically (Mason et al., 2012). Better performance in science leads to higher levels of self- efficacy, through channels such as positive feedback received from teachers, peers and parents, and the positive emotions associated with it. At the same time, students who have low self-efficacy are at high risk of underperforming in science, despite their abilities (Bandura, 1997). If students do not believe in their ability to accomplish particular tasks, they may not exert the effort needed to complete the task, and a lack of self-efficacy becomes a self-fulfilling prophecy. Self-efficacy in science has been related to student performance, but also career orientation and choice of courses (Nugent et al., 2015). In PISA 2012 and 2015, students were asked to report on how easy they thought it would be for them to perform specific math and science-related tasks. Their answers differ by the type of question and by gender in ECA. Out of the 8 science questions posed, more girls said they could easily describe the role of antibiotics in the treatment of disease, identify the science question associated with the disposal of garbage, predict how changes to an environment will affect the survival of certain species, and interpret the scientific information provided on the labeling of food items. More boys, on the other hand, reported explaining why earthquakes occur more frequently in some areas than others, discussing how new evidence can lead them to change their understanding about the possibility of life on Mars, and identifying the better of two explanations for the formation of acid rain. And out of the 8 math problems presented to students, boys reported feeling confident in solving 6 of them significantly more than girls did (Figure 13). Boys and girls differ only in the sense that they are more comfortable with answering different categories of science and math questions. While girls feel more confident answering life science questions, boys feel more confident with physical and mathematical questions.9 9 Interestingly, this finding is not consistent with the average PISA 2015 finding that girls are more likely than boys to have low self-efficacy in science, making ECA countries different than the others included in the survey. 23 Girls, across the board, outpace boys in ambition and motivation in science, but lag behind in interest and application of specific science topics and interest and application of mathematics. Motivation and interest can be effective factors in what makes students succeed at school (Meece, et al., 2006; OECD, 2016c). More girls see themselves as ambitious and want top grades in their courses compared to boys. More girls say they want to be one of the best students in their class and that they want to be able to select from among the best opportunities when they graduate. Boys, on the other hand, agree more with wanting to be the best in whatever they do as a general statement (Figure 14). Self-efficacy levels, motivation and pressure, and stereotyping might manifest themselves in anxiety levels expressed by students, particularly concerning school performance. According to the PISA student survey data, there are significant differences in all anxiety questions by gender in ECA, for both science and mathematics-specific questions. Girls in ECA worry significantly more -they think that taking a test will be difficult for them; that they will get poor grades at school; and that they do not know how to solve a task at school. They also say that they feel anxious even when they are well prepared for a test and that they get very tense studying for a test. Boys agree with these statements to a much lesser degree. This also holds for math-specific questions. Girls agree more with negative statements, such as worrying about the difficulty of math classes, not being good at mathematics, and worrying that they will get poor grades in math. Boys agree more that they learn math quickly, they have always believed that math is one of their best subjects and that they understand even the most difficult work (Figure 15). 24 Figure 13: Girls have higher self-efficacy in life-sciences and boys in physical sciences and mathematics Science Math 0% 20% 40% 60% 80% 0% 20% 40% 60% 80% 100% Recognize the science question that underlies a Using a train timetable to newspaper report on a health work out how long it would issue* take to get from one place… Calculating how much Explain why earthquakes cheaper a TV would be after occur more frequently in a 30% discount*** some areas than in others*** Calculating how many Describe the role of square metres of tiles you antibiotics in the treatment of need to cover a floor*** disease*** Understanding graphs Identify the science question presented in newspapers associated with the disposal of garbage*** Solving an equation like 3x+5= 17*** Predict how changes to an environment will affect the Finding the actual distance survival of certain species*** between two places on a map with a 1:10,000… Interpret the scientific information provided on the Solving an equation like labelling of food items*** 2(x+3) = (x + 3) (x - 3)*** Discuss how new evidence Calculating the petrol can lead you to change your consumption rate of a understanding about the car*** possibility of life on Mars*** Identify the better of two Female Male explanations for the formation of acid rain*** Female Male Source: OECD, PISA 2012 and 2015 databases. Note: Bars show percentage of boys and girls who think they “could easily do” and “could do this with a bit of effort”. 25 Figure 14: Girls are more ambitious than boys in ECA 0% 20% 40% 60% 80% 100% I want top grades in most or all of my courses*** I want to be able to select from among the best opportunities available when I graduate*** I want to be the best, whatever I do*** I see myself as an ambitious person*** I want to be one of the best students in my class*** Female Male Figure 15: Girls in ECA report much higher anxiety levels compared to boys 0% 20% 40% 60% 80% 0% 35% 70% I often worry that it will be I often worry that it will be difficult for me in math… difficult for me taking a test*** I am just not good at math*** I get very tense when I have to do math homework*** I worry that I will get poor grades at school*** I get good grades in math I get very nervous doing math Even if I am well prepared problems for a test I feel very anxious*** I learn math quickly*** I have always believed that math is one of my best… I get very tense when I study for a test*** I feel helpless when doing a math problem In my math class, I understand I get nervous when I don't even the most difficult work*** know how to solve a task at school*** I worry that I will get poor grades in math*** Female Male Female Male Source: OECD, PISA 2015, 2012 database. Note: Bars show percentage of boys and girls that “strongly agree” and “agree” with each statement. 26 Aspirations, motivation, and anxiety of students might be affected not only by factors such as being a good or a bad student, but also by factors related to the family environment they live in: their parents’ level of education, whether they work or not, and in what occupation and sector, their household level of wealth, and whether or not the family lives in an urban or rural setting. Exposure to different opportunities of study and life experiences, the perceived ability to access these opportunities without being discriminated against due to any personal characteristic, and the attitudes of parents and teachers acting as role models and motivators, are among the factors that help build healthy aspirations for all students (Archer, et al., 2014). Family’s income level, access to resources and social status are crucial factors here as young people from privileged homes benefit from more home-based and external opportunities for education, access to resources for their plans, role models, knowledge about career possibilities, and informal networks. Their parents also tend to have high educational aspirations for them (Schoon, Martin, and Ross, 2007). As mentioned before, PISA is administered to students at the age of 15. This is a deciding time in a young person’s life as adolescence is a time when one begins to think about their future, when their aspirations become more closely aligned with their interests, their abilities, and the opportunities available to them, and when their vision of themselves can be influenced by the peers and adults around them (Beal and Crockett, 2010). Students’ expectations for their future influence what they choose to study and the activities they pursue, which, in turn, determine subsequent accomplishments (Nurmi, 2004). We use the PISA’s student survey to understand what 15-year-olds aspire to in terms of careers and how this can be connected to their performance and the gender gap in STEM in ECA.10 By looking to career aspirations, we can immediately see that boys aspire more to engineering professions and ICT jobs compared to girls. Mathematics, physics and life science professions have few students interested in them, so we don’t have enough data points for a gender-differentiated reliable estimate. However, what we find is that these aspirations are not related to the student’s performance. Both at low and high proficiency levels in math, science and reading; men aspire to STEM careers more than women. Girls aspire significantly more to what we could call ‘proxy STEM’ careers -that use some STEM-related content in an applied manner- such as architects and designers, medical doctors, veterinarians, dentists & pharmacists, teachers, business and administration, and health and associate professionals. (Figure 16). Overall, both boys and girls who aspire to STEM perform better in school than the ones that do not. Boys' performance distribution in math is slightly better than girls’ performance, but the distributions almost overlap for those that aspire to STEM. For performance in science, there are more boys at the lower and the upper ends of the performance distribution while girls dominated the mid-range scores for the overall student population. But when looking only at those that aspire to STEM, girls’ distribution is shifted towards the right of boys’ distribution. This means, that among those that aspire to STEM careers, the share of boys and girls performing at top levels is no longer significantly different from each other. 7.1 percent of girls and 7.5 percent of boys perform at top levels in math; and 4 percent of girls and 3.3 percent of boys perform at top levels in science. In contrast, there are three times as many girls as boys performing at top levels in 10 PISA includes a student survey that covers different behavioral areas such as motivation, interest, self-efficacy and anxiety. 27 reading among those that aspire to STEM. When looking at performance levels, again only among those that aspire to STEM 2.7 percent of girls perform in advanced levels while this rate is 1.6 percent for boys. Figure 16: More boys aspire to engineering professions and ICT jobs compared to girls in ECA STEM fields 0 1 2 3 4 5 6 7 8 9 Electrotechnology Engineers*** Ship and aircraft technicians*** ICT Professionals*** Physical and Engineering Technicians*** Engineers (excluding electrotechnology)*** Science and engineering*** ICT technicians*** Life science technicians** Physical and earth sciences** Life science professionals* Architects and Designers*** Mathematicians and statistics* non-STEM fields 0 5 10 15 20 25 Craft and related trade workers*** Plant and machine operators*** Skilled agricultural, forestry and fishery workers*** Armed forces*** Professionals Elementary occupations*** Legal, social, cultural assoc professionals*** Mining and construction** Managers*** Service and sales workers*** Business and administration associate professionals*** Buisiness and administration professionals*** Veterinarians*** Clerical support work*** Medical Doctors*** Legal, social and cultural professionals*** Teachers*** Dentists, pharmacists*** Health associate professionals*** Nurses and midwives*** female male Source: OECD, PISA 2015 database. Note: Graph is ordered in descending order relative to gender gap in favor of men, for both STEM and non-STEM fields 28 The role of parents as role models is influential in building aspirations. While p arents’ education does not make a difference in explaining children’s aspirations; whether the father works in a STEM field or not is significant in explaining both boys’ and girls’ aspirations. The father’s career increases the likelihood of aspiring to STEM by 3 percent for girls and by 11 percent for boys. However, the most important variable that explains a girl’s aspiration to STEM is whether her mother works in a STEM sector.11 If this is the case, it increases the probability of a girl aspiring to a STEM career by 11 percent, notably, the mother’s career does not make a significant difference for boys’ aspirations. Although whether parents expect their children to go into science fields or not does not eliminate the gender gap in their children’s aspirations, when parents expect their children to go into STEM fields, boys aspire more to STEM professions compared to girls. The perception of parents regarding their children’s aspirations might also affect the child’s aspirations. Under PISA, a set of questions were asked to parents about their child’s aspirations in science and mathematics.12 Overall, 30-40 percent of parents say their child shows interest in studying science beyond secondary school and that they expect them to do so, while for math these rates are between 5-10 percent. Girls’ parents have more of an expectation than boys’ parents that their child will go into a science-related career. The opposite is true for math. Parents of boys think more than parents of girls that their child shows an interest in working in a math-related career and expect their child will pursue it. (Figure 17).13 The same set of questions yield the same results for science when the parent category is broken down into parents who work in STEM fields and parents who do not work in STEM fields. Parents who work in STEM do not show any gender bias in their perception of their children’s interest, for example, mothers who work in STEM do not expect their sons to study math more than their daughters. Fathers, regardless of their occupation, expect their sons to pursue a math career more than their daughters, and mothers in non- STEM sectors still expect their sons to study a math career more than their daughters. 11 A recent study on the determinants of math achievement among children in early primary school found that among children of mothers with university education, there is no difference in the math achievement of girls and boys, which suggests that maternal education and social standing are important also from this perspective (Carneiro et al., 2017). 12 The questions about science are from the 2015 PISA parent questionnaire where the focal topic was science and the questions about math are from the 2012 PISA parent questionnaire where the focal topic was math. 13 All of these results are statistically significant for ECA. However, there are no significant differences for whether there is anyone in the family who worked in either science or math-related careers. 29 Figure 17: Parents opinion on child's aspirations in science and math 50% Science Female Male 40% 30% 20% 10% 0% Does anybody in your family Does your child show Do you expect your child will Has your child shown Do you expect your child will work in a science-related interest in working in go into a science-related interest in studying science study science after career? science-related career?** career?*** after secondary?*** secondary?*** 50% Math 40% Female Male 30% 20% 10% 0% Does anybody in your family Does your child show an Do you expect your child will Has your child shown Do you expect your child will work in a math-related interest in working in math- go into a math-related interest in studying math study math after career? related career?*** career?*** after secondary?** secondary?*** Source: OECD, PISA 2012 and 2015 databases. Note: Bars show percentage of parents of boys and girls that respond yes to each of the statements Student questionnaire results reveal that boys consider both science and mathematics to be more instrumental than girls do. More boys than girls believe that things they learn in science and mathematics classes will help them get a job; that what they learn in science and math classes is important because they need this for what they want to do later on; and that making an effort in science and math class is worth it because this will help them in the work they want to do later on Coupled with the ambition results above, these instrumental motivation findings suggest that girls tend to be more positive when it comes to stating how ambitious they are, but they find science and mathematics less instrumental for their future compared to boys. Despite girls’ higher ambitions, boys engage in science and mathematics-related activities at a significantly higher frequency than girls do in ECA.14 Boys report significantly more than girls that they watch TV 14 As a different comparison, we also looked at what kind of not science related activities boys and girls engaged in. The activities that girls do more than boys are studying for homework, reading a book or newspaper, talking to parents and working in the house. The activities that boys do more often than girls are watching TV, playing video games, working for pay and practicing sports. 30 programs about science, borrow or buy science books, visit websites about science topics, read science magazines, attend a science club, simulate a natural phenomenon in a computer program, simulate technical processes in a computer program, visit websites of ecology organizations, and follow science- related news via blogs. The same is true for mathematics activities. Boys report participating in a math club, programming computers, playing chess, working on math for more than 2 hours a day, taking part in math competitions, doing math as an extracurricular activity, helping their friends with mathematics and talking about mathematics problems significantly more than girls. When it comes to the specific topics, students are interested in learning in the area of science and mathematics, boys show significantly more interest in physical science topics, such as motion and forces, energy and its transformation, and the universe and its history. Girls show greater interest in biological science topics, such as biosphere and how can science help prevent a disease. When parents get asked about their child’s science involvement parents of boys reported more than parents of girls that their child watched TV programs read or listened to science-related content; boys also appear to be more likely seen as visiting websites about science topics and other activities related with science curiosity -such as engaging in construction play such as Legos; taking apart technical devices; fixing broken objects, toys or items; experimenting with a science kit, and playing computer games with a science content. This difference between boys and girls could have been an issue of availability and/or selection - girls may get dolls and not Legos or may not be encouraged to watch programs about science - as these are based on reports from parents and they could be biased or misremembered. Regardless of this fact that we cannot control for, the gender differences are statistically significant in most cases (Figure 18). Parents also report being more active in helping their child with a science or a math homework when the child was a boy. Parents of boys also engage more in discussing how science is used in everyday situations and obtaining science-related materials for their children. Girls’ parents, on the other hand, spend time talking to their child and discuss how well their child is doing at school more often than boy’s parents. For children with a declared interest in STEM for both boys and girls’ parents engage more in activities related to science and math, including discussing science-related career options with their child (Figure 19). Overall, girls’ parents seem to be more interested in following their child’s performance and talking to them about general topics, while boys’ parents seem to be doing more specific science and math -related activities with their children. This is similar to the pattern where boys reported to be more interested in specific science and math-related topics in and out of school compared to girls. We looked at the correlation between these two factors and found that the correlation is positive for all the combinations between children’s activities and what parents do with their children, for both genders. This means parents that discuss more science topics with their children have kids that spend more time on science-related activities. 31 Figure 18: More boys did all of the following science-related activities more than girls did when they were 10 years old according to their parents 50% 40% 30% 20% 10% 0% Watched TV Read books Watched, Visited web Attended a Construction Took apart Fixed broken Experimented Played programmes on scientific read or sites about science club play, technical objects or with a science computer about discoveries listened to science e.g.*** broken electronics science fiction*** electronic kit, or content*** toys*** chemistry set, used a Female Male micro*** Source: OECD, PISA 2015 database. Note: Bars show percentage of parents that report their sons and daughters engage in these activities “very often” and “regularly”. Figure 19: What parents do with their children Science Math- what parents do with their children (PISA) 0% 50% 100% 0% 50% 100% Discuss how well my Discuss how well my child is doing at… child is doing at school Eat main meal with my Eat main meal with my child around a table child around a table Spend time just talking to my child*** Spend time just talking to my child Help my child with science homework*** Help my child with math Ask how my child is homework*** performing in… Discuss how my child is Obtain science-related performing in math class materials*** Obtain math materials Discuss with my child for my child*** how science is used… Discuss with my child Discuss science related how math can be applied career options with… in everyday life*** Female Male Female Male Source: OECD, PISA 2012 and 2015 databases. Note: Bars for the first 3 categories (i) discussing how well child is doing at school, ii) eating main meal with child, and iii) spending time just talking to the child) show the percentage of parents who reported engaging in these activities “every day or almost every day”. Bars for the remaining categories (4 to 8) show the percentage of parents who report engaging in these activities during the previous academic year. 32 Explaining the differences in performance: girls need the alignment of more factors Pulling all the different elements discussed so far together, seeking to understand what explains performance scores in the three subjects, we estimate linear regressions controlling by gender, individual and school characteristics. What we find validates the results from the Oaxaca decomposition. Although we have previously identified there are no significant gender gaps in math and science, as we start adding individual and school characteristics as controls to these estimations, gender starts getting an increasing relevance: in favor of girls for reading and in favor of boys for mathematics and science. For example, take differences in aspirations and their relation to performance. Is it that boys start with more STEM focused aspirations and as a result perform better or is it that boys perform better in math and science and as a result aspire more to STEM fields? A regression analysis on aspirations reveals that boys who perform at the top levels (Levels 5 and 6) in math, and above average in science (above level 3) are more likely to aspire to STEM fields, while girls at top levels in math and science are not. A boy performing at top levels in math is 13 percent more likely to aspire to STEM fields than boys that do not perform at top levels. A boy performing above average in science is 4.8 percent more likely to aspire to STEM than boys performing below average. Girls performing at top levels in either math and science do not have significantly different probabilities of aspiring to STEM fields than girls performing at lower levels. The two more important factors, including one not discussed so far, is that gender differences in performance are largely driven by wealth and aspirations to STEM fields. Boys have better performance in math, but this is driven by their aspirations to STEM and by higher levels of wealth. A similar story is valid for science performance: more boys performing at top levels in science, but that is driven by their aspirations and relative wealth, which are important predictors of science scores. In fact, when these controls are added, girls have better performance than boys. An increase of one unit in the family wealth index constructed by OECD increases the likelihood that a girl aspires to STEM by 1.5 percent and by 1.3 percent for boys. When performance levels are disaggregated by wealth, gender score gaps in math appear to be stronger in favor of boys. This is consistent across wealth quintiles – it only disappears for the richest quintile. However, for the wealthiest families in ECA, boys’ and girls’ performance in math becomes similar. 33 Table 1: OLS Estimation on Reading, Mathematics and Science Performance Reading Mathematics Science Female 24.5*** -10.39*** -4.66** Child expects to have a STEM job when is 30 years old 13.51*** 19.33*** 18.48*** Father works in a STEM field 15.51*** 14.38*** 14.72*** Mother works in a STEM field 11.15** 10.18** 14.22*** Family wealth (WLE) 5.82*** 5.26*** 3.72*** Age -1.37 -4.32** -1.03 Grade Repetition -42.01*** -32.6*** -34.8*** International language at home -26.46*** -20.54*** -27.56*** Shortage of educational material (WLE) -3.98*** -2.7** -3.08** Attended ECEC for at least two years 4.31** 8.04*** 7.36*** Student Atttidudes, Preferences and Self-related beliefs: Achieving motivation ( 10.84*** 12.03*** 11.04*** Grade compared to modal grade in country 30.53*** 31.42*** 31.29*** School Inequality: ESCS SD at School level -19.12** -5.3 -10.58** School is located in rural area -23.52*** -16.76*** -19.46*** Class Size 0.59** 0.6** 0.57** School autonomy (Mean) -16.49** -9.51** -10.78** Disciplinary climate in science classes (WLE) 8.58*** 11.24*** 10.08*** Teacher participation (Sum) 2.85** 2.04** 2.71*** ISCED is vocational or pre-vocational -61.22*** -60.44*** -63.83*** Responsibility for resources 4.68*** 4.1** 4.47*** Responsibility for curriculum -1.93** -2.75** -2.15** Constant 501.89*** 544.6*** 505.2*** Country fixed effects x x x R-squared 0.28 0.28 0.26 Observations 41938 41938 41938 Note: * 10%, ** 5%, *** 1%. STEM and Gender Dynamics in Tertiary Education So far, PISA has allowed us to explore in-depth the main reasons why performance differences are observed between girls and boys during their secondary school years. We thus expect these differences, and differences in aspirations, to directly affect the choices these students will make when entering tertiary education and choosing a career stream. However, information on how students arrive at their career choices, and their entry and stay process in such career is less available at the regional level and/or in a standardized manner than for the case of secondary education. Country studies show that women and men systematically choose different fields of study in higher educational levels across all ECA countries. In Belarus, female university students tend to choose such majors as social protection, social sciences, pedagogy and health where the share of women reached 81 34 percent in 2009-2010, while young men account for 73 percent of students in construction, security, engineering and technology (World Bank 2014). In North Macedonia women are more likely to choose subjects related to social sciences, while men are more likely instead to choose engineering, production, or construction (World Bank 2013). In Croatia, women tend to select arts, humanities and social sciences, and are not encouraged to pursue education and careers in STEM (Vokić et al. 2016). In Montenegro, more working-age males study mathematics, engineering, and science-related fields than females (31 percent of males compared to 9 percent of females). Fifty percent of women complete schooling related to services, social sciences, business, and law, compared to 38 percent of men (World Bank 2013b). In Poland, although more than 50 percent of students in all fields of sciences are women, for Math, Physics, Engineering studies the percentage declines to less than 10 percent. Among Math, Science and Technology students, 30 percent were girls and 70 percent were boys (Porębska et al 2011). In Ukraine, female students make the overwhelming majority in the fields of healthcare, education and humanitarian sciences (more than 70 percent of all students with these majors in 2014/2015 academic year), while male students prevail among those who study transport and communications, engineering, architecture and construction. In Serbia, the Gender Equality Index indicator on segregation also reveals prominent gender discrepancies, with women concentrated significantly in the area of social sciences, humanities and arts (World Bank, 2016b). In addition to all these, men and women display different behavioral patterns even if they end up in STEM fields. According to a recent study, women are more likely to switch out of male-dominated STEM majors in response to poor performance, gender composition of classroom and stereotyping signals. The literature summarized above suggests that throughout ECA, women dominate tertiary fields of arts, education, health and welfare, humanities and social sciences. Women are present in STEM-related tertiary programs as well, albeit in lesser numbers compared to boys. Tertiary programs such as engineering, construction and IT appear to be male dominated across countries. The few comparable data points available -in this case, from UNESCO’s education statistics. UNESCO has tertiary level data on enrolment and graduation rates by gender and by program. 15 According to this data, in 2015, female enrolment in most STEM programs at tertiary level was much lower than other select programs (Figure 20). The ratio of women enrolled in Engineering, Manufacturing and Construction programs and ICT programs was quite low compared to the ratio of women in programs like arts and humanities, health and welfare and education. In Natural Sciences, Math and Statistics, there was a larger number of female students. All these trends are more or less similar in all ECA countries where data is available. Similarly, looking at the graduation rates from tertiary programs in ECA reveals that there are less women graduating from programs that are directly related to STEM – while there is some variation by country. Arts and humanities, health and welfare and education programs have a very dominant ratio of female graduates across the board (Figure 21). 15 It is important to note that the UNESCO education data does not include data for some ECA countries. One caveat is that UNESCO has a category called "engineering, construction and manufacturing", and it is not possible to separate engineering from construction and manufacturing as a purely STEM field of study. 35 Figure 20: Female enrolment in most STEM programs at tertiary level is much lower than other select programs 100 80 60 40 20 0 Engineering, ICT Natural Sciences, Arts and Humanities Health and Welfare Education Manufacturing, Math and Statistics Construction Albania Azerbaijan Belarus Bosnia and Herzegovina Bulgaria Croatia FYR Macedonia Hungary Kyrgyzstan Poland Romania Serbia Turkey Ukraine Source: UNESCO Data, 2015. Note: Poland data is from 2014 Figure 21: There are fewer female graduates in STEM fields in ECA than in other fields 100 80 60 40 20 0 Engineering ICT Natural Sciences, Math Arts and Humanities Health and Welfare Education Manufacturing and and Statistics Albania Constuction Azerbaijan Belarus Bosnia and Herzegovina Bulgaria Croatia FYR Macedonia Hungary Kyrgyzstan Poland Romania Serbia Turkey Ukraine Source: UNESCO Data, 2015. Note: Data for North Macedonia and Turkey is from 2014. 36 There is some literature on tertiary graduates in STEM fields in ECA countries that shed light on interesting trends. Flabbi (2012), in his analysis of a sample of OECD countries concludes that gender is a significant determinant of the field of study even after controlling for detailed individual characteristics. Being female increases the probability to choose Humanities and Health and decreases the probability to choose Sciences. According to Huyer (2015), in most countries in Eastern Europe women tend to be on par with men among tertiary graduates in science. However, differences in the sub-fields of study are persistent. Between 70 and 85 percent of graduates in health programs - that count as part of the science category for the purposes of this study - are women as opposed to around 20 to 30 percent in engineering. Based on Khitarishvili (2016), in a majority of Central Asia and CIS countries women represent more than 70 percent of graduates in education and more than 65 percent of students in health and welfare. In this region, women are significantly underrepresented in engineering, manufacturing, and construction, especially in Azerbaijan, where less than 20 percent of tertiary students in these subjects are female. Overall, in ECA the graduates of the Education program are dominantly female, and the graduates of the Engineering, Manufacturing and Construction are dominantly male. The gender gap in graduates is particularly high for these two programs. In Azerbaijan 40 percent of female graduates from tertiary education graduated from the Education program and this rate is 9 percent for Azeri males. In Belarus 48 percent of male tertiary graduates graduate from the Engineering, Manufacturing and Construction program while this rate was 11 percent for female graduates (Figures 22 a,b). Figure 22a: There is a wide gender gap between the number of graduates of tertiary in Education programs in ECA, 2015 (%) 40 Female Male 30 20 10 0 37 Figure 22b: Male students dominate the number of graduates in Engineering, Manufacturing and Construction programs in all ECA countries, 2015 (%) 50 40 30 20 10 0 Female Male Source: UNESCO Data, 2015. Note: Data for North Macedonia and Turkey is from 2014. Box 1: STEM and gender in Bahcesehir University in Istanbul, Turkey To get a more thorough insight into how gender differences are juxtaposed in tertiary education, administrative data about the student body and various STEM-related programs from Bahcesehir University in Istanbul have been analyzed. This box aims to shed light into how STEM programs in a tertiary education institution differ with regards to their gender composition and what students do after graduation. According to data shared by Bahcesehir University, there is considerable gender variance between registration in STEM-related programs. There are programs that are clearly male dominated such as computer engineering, electric- electronics engineering, mechatronics engineering, software engineering, construction engineering and energy systems engineering. These programs have a very low number of female students. This information reveals that programs that are considered ‘more technical’ are mostly male dominated. Female students seem to heavily gravitate towards health-related fields or architecture instead. Despite this, in the last three academic years, more female students in STEM programs had a GPA of 3 or above compared to their male counterparts. It is also interesting to observe what happens after graduation from STEM programs: 47 percent of their female alumni are not working. 64 percent of those not working are looking for employment and 22 percent is pursuing further education. Among the 53 percent who are working, the majority is working in the construction sector. This is followed 38 by technology, electrics-electronics, information, finance and education. The fact that almost half the women who graduated from one of the top universities in Turkey is not working although a majority of them are looking for jobs might be due to a variety of reasons such as lack of access, high reservation rates, lack of vacancies and/or socio- cultural factors. However, with the current information at hand, it is not possible to speculate on which of these reasons might be more effective in keeping women out of jobs in their fields of study. Figure B1: Registered Students in STEM Programs, by gender (2016-17 academic year) 100 Women Men 80 60 40 20 0 Source: Administrative data from Bahcesehir University, 2017. 39 Labor market outcomes for women in STEM fields and occupations This section provides a panorama of women’s participation and outcomes in labor market STEM fields and occupations in the ECA region. It explores different sources of available data to analyze the relationship between constraints -before and in the labor market- to women’s participation in STEM jobs and the overall outcomes in the labor market for women and men. Based on identified cross-country comparable sources of data, this section first presents a regional characterization of the demand for STEM skills in the labor market. Second, it portrays the gender-specific supply of skills for STEM jobs and describes how it correlates with the trends in education outcomes presented in the previous section. Finally, the gender gaps in outcomes in the labor market for STEM sectors and occupations are described. The ECA region overall, shows trends that are similar to the rest of the world in terms of employment increasing towards the service sectors and higher skills (Figures 23, 24). The typical pattern of change in occupational structure has been marked with a substantial fall in demand for agricultural skills and a rise in demand for occupations in the service sector. The distribution of employment by skill level over time also shows that the rate of the low skilled stayed more or less the same in the last two decades, higher skilled individuals have been on the rise in the labor market and this shift seems to be happening because of the drop in the rate of people with medium level skills. Over the last 25 years, in Europe and Central Asia as a region, the demand for skills related to complex non-routine, cognitive tasks has increased, however the dynamics of the demand for manual/routine skills are heterogeneous across countries (World Bank 2017, Bussolo et al. 2018). Figure 23: Evolution of employment by sector in ECA, Figure 24: Evolution of employment by skill level in 1997-2017 (% of total employment) ECA, 2000-2017 70 60 60 50 50 40 40 30 30 20 10 20 0 10 2000 2007 2017 0 Employment in Employment in Employment in Skill levels 3 and 4 (high) agriculture services industry Skill level 2 (medium) 1997 2007 2017 Skill level 1 (low) Source: World Bank Databank, 2018 Source: ILO, 2018. 40 The regional average hides however a very important heterogeneity among countries. For example, high- skill employment is lower in the Eastern part of the region; an emerging process of polarization is happening in more Western European countries, while in post-soviet countries the share of low-skill workers has actually increased. STEM skills seemed to have played a role in this shift of employment structure. In the European Union, STEM occupations were an important driver of overall employment growth in the past decade. While total employment over the period 2000-2011 grew by 8%, STEM employment increased by 34% – bringing the share of STEM employment in total employment to 7% by 2011 (Goos, Hathaway, Konings and Vandeweyer, 2013). Furthermore, demand for STEM professionals is expected to grow by 8% in the decade leading to 2025, compared to the forecast of 3% for all occupations. Employment in STEM-related sectors is also expected to rise by around 6.5% by 2025 (Cedefop, 2015). Finally, the returns for STEM jobs have grown more relative to other jobs, and STEM workers are among the highest paid in today’s economy as a result of this rise in demand. An apparent shortage of STEM skills in the United States and Western Europe, does not seem to characterize the labor markets of Eastern and post-soviet countries (U.S. Department of Commerce, 2011, Carnevale et al. 2011). If at all, the shortage in these economies will be driven by workers of a more educated caliber migrating to fill the mismatch between the STEM workforce supply and the demand in the US and countries like the UK and Germany. Behind the supply of STEM skills: Career aspirations and information on returns to education Existing literature has identified patterns of systemic variation in career intentions among students and the composition of the workforce (Christensen and Knezek, 2017). Occupational outcomes seem to be associated with occupational aspirations and student-specific demographics, in particular gender. Available data for countries in the ECA region points to aspirations playing a central role in the discussion on women’s place and prominence in STEM fields. Career aspirations are particularly important at secondary school as young people are about to make important decisions about their future in terms of choosing which studies to pursue. Such decisions are ultimately critical for the labor market prospects of youth. Focusing only on STEM-related professions, going back to the student survey in PISA shows that 9 percent of boys said they wanted to become engineers, 5 percent ICT professionals and around 5 percent technicians when asked about what they want to do when they are 30 years old. In turn, 2 percent of girls expressed interest in engineering while ICT and technician jobs were not among the preferred choices for girls at all. A high proportion of girls were interested in legal, social and cultural professions and teaching (around 35 percent total). 13 percent of girls were interested in teaching while this rate was 5 percent for boys. More girls said they wanted to be doctors compared to boys. Interestingly, there is no significative difference between the percentage of girls and boys who said they wanted to become managers. As noted in the previous chapter, in all ECA countries, boys aspire to STEM--related careers more than girls do. They 41 also graduate from STEM-related tertiary programs at a higher rate than girls. In fact, there are many more boys and girls graduating from STEM fields even though the aspirations at a younger age seem to be less pronounced (Figure 25). Figure 25: Higher aspirations to STEM degrees and jobs among boys/young men than among girls/young women in ECA countries 70 female male 60 50 40 30 20 10 0 graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM graduated from STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM aspires to STEM Moldova Montenegro Russia Macedonia Bulgaria Croatia Poland Romania Turkey Georgia Source: PISA 2015; UNESCO, 2015, STWT, 2015 Note: Those that those that aspire to STEM and those that graduated from STEM are two separate cohorts coming from different databases. Aspirations to STEM are from PISA 2015 and graduation from STEM are mostly from UNESCO 2015, except for: North Macedonia and Turkey (2014) and Georgia (2012). Data for Montenegro and Russian Federation is from STWT 2015. The figure consolidates them to show information about two different stages of the individual’s life cycle. Career aspirations for a certain field or a specific occupation are usually a product of different determinants coming together for that individual. Below, we will discuss some of those determinants that could potentially be driving gender differences in aspirations to STEM careers. These include access to information of different career trajectories and job vacancies, and the returns to STEM careers in comparison to other occupational choices. Access to information on job vacancies, salary levels, and the expectations of employers is an important piece of the puzzle while discussing what shapes aspirations and career choices. If girls possess the information on what it would mean to have a STEM degree for future job satisfaction and earnings prospects, this could potentially make a difference in them aspiring to pursue these degrees and careers in the first place. Information failures can be a substantial barrier that discourages women from looking for a job or finding one that fits their skills and needs. A recent report (World Bank 2016a) found that young women in ECA have difficulty understanding where and how to look for jobs and that their lack of 42 experience and networks make it harder for them to find jobs. Lack of information may lead students to underestimate the returns of a specific degree and discount opportunities in that field. A study in Moldova (World Bank 2016b) found that information gaps about different educational choices make it hard for young people to make optimal decisions about their future. The study reveals that many students do not form clear occupational aspirations even after making their educational decisions, partly because of a scarcity of labor market information. About half of general school students and two‐thirds of older students felt they did not have enough information about the labor market. An important point speaking to the main premise of our study is the following: the Moldova study found that across all educational levels, women were more likely to report having an information deficit - among 12th graders, 55 percent of the women felt they did not have sufficient information compared to 45 percent of the men. In addition, the Moldova study tackles an interesting question about perceived returns to further education. Figure 26 shows the wage respondents think they would earn at age 30 with different educational levels (represented by different shaped dots) relative to what they would earn if their highest education was a 9th grade education. According to this, all students correctly said continuing their studies after 9th grade will pay off and that university education has the highest returns. However, different groups had differing perceptions about the returns to intermediate levels of education, while professional school and university students clearly identify that someone with a 3‐year professional school diploma earns significantly more than someone with a 12th grade diploma, they are mistaken. According to the 2014 Labor Force Survey data, the two degrees are similarly remunerated for workers aged 25–34. Also, professional school students perceived that there are no significant returns to a collegium diploma, but the Labor Force Survey results demonstrate that the difference in earnings between collegium graduates and holders of a 3‐year professional school diploma is significant. These are good examples of misperceptions that can potentially have a significant impact on how students make decisions about educational investments. Student surveys in Moldova also revealed that information services should reach not only students but also their social networks, especially parents as they are often co‐decision makers for their children but they feel unprepared to advise them. Many students rely on the Internet for information but it does not reach all of them. Students with less‐educated parents demonstrate the highest information deficits, potentially exacerbating an already existing inequality of opportunity. 43 Figure 26: Reported expected wages for different education achievements and actual returns, Moldova Source: World Bank (2016b) using MJSDS 2014/2015 data and LFS 2014 Notes: Significant differences in the wage index compared to the base category for the 3-year professional school program. * 10%, ** 5%, *** 1%. There is evidence of effective interventions to improve access to information on education and training in the labor markets. These range from forming labor market observatories that provide relevant stakeholders with information that can inform their educational and labor market choices; offering information sessions in the school system to provide students with relevant information that can influence their choices; providing incentives to employers to hire new entrants to help bridge some of the information and network gaps that make it challenging for these groups to access their first job; and putting in effect counseling and matching services to help improve job search including job fairs, job shadowing and mentoring. Defraine et al. (2014) provide evidence from the US that Millennials possess a strong desire for work/life- interaction, which runs counter to the traditional lab-training model endorsed by many lab directors. Changing the information and perceptions about the work environment in scientific and technical workplace is shown to have a sizeable impact on female interest in STEM participation. School-work transitions for STEM and non-STEM graduates One of the key findings of this study is that STEM graduates do not necessarily end up working in STEM sectors and occupations, or even working at all. Robust evidence of pathways of transition for STEM and non-STEM graduates into economic activity status and labor trajectories would require longitudinal data. Based on available data, in this section we attempt to describe what are patterns of these transitions. The results of a linear probability model on entering the labor market for Turkey, Russian Federation and Moldova given STEM and non-STEM graduation we find that studying STEM translates into 2.6 percent overall increase in the probability of being employed (in any sector or occupation) in Turkey. The effect of 44 studying STEM is even greater for women, as studying STEM gives them an extra 2.7 percent increase in the probability of being employed. However, women have a 30 percent lower chance of being employed in the first place so adding up the effects, women who study STEM are 25 percent less likely to be employed than a man who did not study STEM. In summary, although studying STEM slightly helps women entering the job market, it is not enough to overcome the significant gender barrier. In the Russian Federation, studying a STEM field does not have a significant effect on entering the labor market overall. Women get a significant extra 4 percent increase in the probability of being employed after studying a STEM field. But this effect cancels out with the decrease of 12 percent in the likelihood of being employed that all women have. So, overall, women that studied STEM are 8 percent less likely to find a job than men that did not study STEM. In Moldova, both men and women get an increase of 12 percent in the probability of getting a job if they studied STEM. All women have 3 percent more chances of being employed. So overall, men that studied STEM are 11 percent more likely to find a job, and women 14 percent more likely in Moldova. Data also show that inactivity rates in the labor market for both men and women are higher for those that graduated from STEM fields in ECA, but -as is known also in other regions- the inactivity rates are much lower for men regardless of whether or not they studied STEM (Figure 27). The case of Turkey is interesting in that is the country with available data that shows the least difference in inactivity rates between STEM and non-STEM graduates, both for men and women. Figure 27: Share of inactive population by education and gender 60 50 40 30 20 10 0 men women men women men women men women men women men women men women men women Turkey Bulgaria Czech Republic Estonia Croatia Hungary Poland Romania Graduated from STEM Graduated from non-STEM Source: Team calculations based on latest LFS of the respective countries (circa 2015-2017) With respect to the job search process, a recent study found that men and women have a similar speed of transition to their first job only in the first few months after leaving education. After this time, the differences between men and women diverge, with men having a higher likelihood to find a first job than 45 women across all time periods. Overall, in comparison to men, women have a significantly slower transition to their first job. When women study male-typical fields such as STEM, they have significantly slower entry into the labor market (Mills and Präg, 2014). Available data shows that in Turkey, men that study STEM spend less time searching for a job compared to women and compared to men who did not study STEM. The opposite is true in the case of women: women who study STEM spend more time looking for a job compared to men regardless of what they studied and compared to other women who did not study STEM. So, the entry into the job market for women who studied STEM seems harder to penetrate compared to everyone else. In the Russian Federation, men that graduate from a STEM field take less than men that did not graduate from a STEM field, but women that graduate from a STEM field take as much as men that did not study STEM. Women that did not study STEM are the first ones to find a job once unemployed.16 Table 2: How many months did it take you to find a job? TURKEY RUSSIAN FEDERATION Non STEM graduate STEM graduate Non STEM graduate STEM graduate MALE 7.31 6.86 6.00 5.30 FEMALE 8.67 8.96 4.71 5.90 Source: Turkey LFS 2017, Russian Federation RLMS 2017 Sectoral and occupational distribution of women and men in ECA There are usually differences in how women and men are distributed into occupations in any country. Some jobs are clearly dominated by men while others by women. Historically, the male dominated jobs have been the technical jobs with higher returns and higher possibilities for career advancement. This occupational gender segregation is economically inefficient because it aggravates skill shortages and inhibits the potential of women and men by restricting their movement into professions that would best fit their preferences and abilities (Hill et al. 2010).. It also contributes to a big problem of supply and demand of skills as the supply of STEM graduates in a lot of countries lags behind the demand for STEM skills from employers (Cervantes 1999, Jordan and Yeomans 2003, Roberts 2002). Figure 28 summarizes the employment distribution in ECA countries for which data was available by select occupations and gender. On average women and men occupy different spaces in the labor market in all 16 Access to administrative data from Bahcesehir University, a private university in Istanbul, Turkey, shows that after graduation from STEM programs, 47 percent of female graduates are not employed after 6 months of graduation. 64 percent of those not working are looking for employment and 22 percent are pursuing further education. Among the 53 percent who are working, the majority is working in the construction sector. This is followed by technology, electrics-electronics, information, finance and education. 46 ECA countries. The occupations being focused on here are clerical support workers, service and sales workers, craft and related trades workers and plant and machine operators and assemblers. The reason for this focus is that the first two are occupations dominated by women and the last two are occupations dominated by men in ECA across the board. Figure 28: Employment distribution by select occupations and by gender, 2016 30 25 20 15 10 5 0 Women Women Women Women Women Women Women Women Women Women Women Men Men Men Men Men Men Men Men Men Men Men ARM AZE BGR HRV CZE HUN MKD POL ROM SRB TUR Clerical support workers Service and sales workers Craft and related trades workers Plant and machine operators, and assemblers Source: ILO A recent study in Bulgaria (World Bank 2017) shows that while men - aged 25-64 with at least upper secondary education -are more equally distributed between STEM (48%) and non-STEM occupations (52%), women with the same level of education in the same age group are highly concentrated in non-STEM occupations and only 10 percent of them work in STEM occupations. Numbers confirm this high concentration of women in the traditionally female occupations in Bulgaria (Table 3). Table 3: Distribution of professionals and technicians by category of occupation in Bulgaria MEN WOMEN TOTAL STEM 47.64 10.48 23.91 TEACHING 7.71 19.56 15.28 HEALTH 9.23 26 19.94 OTHER PROFESSIONALS AND TECHNICIANS 35.42 43.96 40.87 Source: Bulgaria LFS 2013 47 Even when the numbers of men and women in a certain STEM sector are not that different from each other, the types of jobs and opportunities may be very different for women and men. An example of this would be the ICT sector in Armenia. A study (Inoue et al 2017) found that higher than average salaries and opportunities for skills enhancement make ICT a highly desirable field for all students. In fact, at an ICT company interviewed for this report, women comprised 40 percent of the staff, but they were concentrated in lower-skilled jobs and tasks. Male employees of this company predominantly win the patents and awards while women are not rewarded in a similar fashion for their creativity, either because they are not encouraged to act on their creative ideas as strongly as men or because there are fewer opportunities for them in the first place. How about for those that graduated from STEM programs? Do people who graduate from STEM degrees end up in sectors that involve STEM? The short answer is it depends on the country and it changes a lot between women and men (Figure 29). The first pattern that stands out is that not a lot of STEM graduates end up with ‘pure’ STEM jobs such as science and engineering and ICT related fields, regardless of gender. If you add all STEM jobs up, only one ECA country, Croatia, have 25 percent of STEM educated women in STEM jobs and this is not even the case for men. Among those that do work in STEM fields, female STEM graduates seem to be concentrated more on the science and engineering side compared to ICT, although the differences can be negligible in some cases. To look at a more specific example, we turned once again to the survey data in Turkey. We found that men who studied STEM have a 13 percent greater chance of working in a STEM occupation while this rate is 11 percent for women who studied STEM. Keeping in mind that this all depends on job availability, overall labor market performance and labor regulations which can all change dramatically depending on the country, it is important to note that women do not seem to be lagging behind men in finding a job in science and engineering. Among countries with available data, the only exception to this seems to be Hungary and the reasons behind that need to be looked at in more detail to reach any conclusions. The takeaway here can be that STEM educated women and men have similar access to science and engineering jobs, men end up with ICT jobs more often, and that not a lot of STEM graduates end up in STEM professions regardless of gender. 48 Figure 29: What portion of STEM graduates end up working in a STEM sector? 30 25 20 15 % 10 5 0 men women men men men men men men men women women women women women women women Turkey Bulgaria Czech Estonia Croatia Hungary Poland Romania Republic Science and engineering professionals and associate professionals Information and communication Source: Team calculations based on latest LFS of the respective countries (circa 2015-2017) Note: these shares represent the proportion of women and men that graduated from STEM that end up working in science and engineer professions and information and communication It is also important to look at what other professions STEM graduates end up going into. This differs by country and by gender and is influenced by what kinds of pathways are available to shift professional focus and how possible it is to further one’s education to go into another profession if one wants to or has to. Figure 30 shows the top three non-STEM professions that STEM graduates end up with by gender in select ECA countries. First takeaway from this is that men and women end up in very different job families. A lot of women end up in sales jobs. In Bulgaria, almost one quarter of women who graduated from a STEM program work as salespeople. Also popular is teaching. This is especially the case in Turkey where one fifth of female STEM graduates end up becoming teachers. The third most popular area of work for female STEM graduates is as business and administration associate professionals. This job group includes sales and purchases agents and brokers, business service agents, administrative assistants and secretaries. Men with STEM degrees who end up elsewhere, on the other hand, usually work as machinery related trade workers, plant operators or building workers. A seemingly gendered professional pattern can be observed here between men and women. Women end up doing more desk jobs and teaching whereas men do more technical but manual labor-intensive jobs. 49 Figure 30: Where does most STEM graduates end up working? Women 25 20 14.44 15 % 10 7.83 8.37 6.62 5.48 4.79 4.05 5 1.82 0 Turkey Bulgaria Czech Republic Estonia Croatia Hungary Poland Romania Teaching professionals Models, salesperson Business and Administration associate professionals Men 20 15 10 % 5 0 Turkey Bulgaria Czech Republic Estonia Croatia Hungary Poland Romania Metal, machinery and related trade workers Drivers and mobile plant operators Building and related trade workers Source: Team calculations based on latest LFS of the respective countries (circa 2015-2017) It can be challenging to fully understand the extent of STEM involvement for men and women enough to make a meaningful inference. What we find are ‘gradients’ of STEM, from a full-STEM individual -someone who studied a STEM degree and works in a STEM occupation in a STEM sector, to a non-STEM individual on the opposite end of the spectrum. Figure 27 tries to capture these degrees of STEM involvement for men and women in ECA countries where all three pieces of information are available – namely what share of the population has STEM degrees, what portion has a STEM occupation and how many of those work in a STEM sector. What we find is that the choice of STEM career is not deterministic of where an individual may end up in the labor market, so not everyone who studies STEM ends up in the STEM sector and/or with a STEM job while some people who were not educated in a STEM field do have jobs in the STEM sector. This makes the boundaries of what we can consider STEM in the labor market more difficult to define than in education. In fact, the graph shows that people who did not study STEM but ended up in a STEM work field are the majority in every country for both genders. The other clear pattern reveals that is that for all other categories, men dominate the numbers. 50 Figure 31: Degrees of STEM involvement 12 10 8 6 4 2 0 men women men women men women men women men women men women men women men women Turkey Bulgaria Czech Republic Estonia Croatia Hungary Poland Romania stem degree, stem occ, stem sector stem degree, stem occ, non-stem sector stem degree, non-stem occ, stem sector non-stem degree, stem occ OR stem sector Source: Team calculations based on latest LFS of the respective countries (circa 2015-2017) Returns to education, earnings and the gender pay gap in STEM In the majority of ECA countries, returns to higher education have increased in the last few decades. The changes in the wage structure points to an increase in wage premia for white collar skills, particularly professional and managerial skills. As a rule, countries where returns to education are high are the same countries where employment had already shifted towards more skilled occupations. The increase in returns to education has mostly been driven by an increase in returns to tertiary education. Wages for people with tertiary education have increased dramatically, while those to secondary education have remained stable. In Hungary, the wage returns to university education more than doubled in the wake of the transition. In the Russian Federation, a worker with tertiary education earns over twice as much as a worker with primary education and nearly 60% more than a worker with general secondary education. 17 Another study that confirms that adults with a tertiary degree are 10% age points more likely to be employed, and will earn 56% more than adults who completed only upper secondary education. They are the first ones to recover from economic downturns.18 In most ECA countries, unemployment rates are lowest for those with advanced levels of education. Figure 32 summarizes the returns to tertiary education in ECA countries for which data was available. According to the analysis, returns to finishing tertiary education are positive and significant for all countries analyzed in ECA. They're also greater than finishing upper secondary education. Finishing upper secondary education pays more than having less than upper secondary education, but finishing tertiary education pays even more. For women, things are a bit different in that although it pays more to study tertiary 17 Murthi and Sondergaard, 2012. 18 OECD, 2017. 51 education overall, this payoff is less for women compared to men. This does not mean that the returns to tertiary education are lower for women, but that women overall (including those with tertiary education) get paid less than men. So, it is the effect of gender rather than the returns to education. Figure 32: Average returns to tertiary education among full-time workers (ages 25-64) in ECA (Mincer equation19) 0% 10% 20% 30% 40% 50% 60% 70% 80% Bulgaria *** Czech Republic *** Croatia *** Hungary *** Poland *** Romania *** Serbia ** % increase in avg hr earnings Men Women Difference men-women: *** p<0.01, ** p<0.05, * p<0.1 Source: EU SILC, 2015. Note: Baseline group of comparison is men with less than upper secondary education. Focusing specifically on returns to studying STEM and returns to specific fields of study, we turn to the Turkey and Moldova Labor Force Survey (LFS) data and the Russian Federation Longitudinal Monitoring Survey (RLMS). According to the regression analysis on returns to STEM fields of education, we find that having studied STEM translates into a 5% overall increase in hourly income for everyone combined in Turkey and 7% for everyone in the Russian Federation. The return that women get for studying STEM is not significantly different than men. However, women, regardless of their field of study, earn 8% less hourly income than men in Turkey and 25% less in the Russian Federation. As a consequence, the increase for having studied STEM does not compensate the lower returns they start off with. So, even after studying STEM, they still get 2.7% less hourly income than men that did not study STEM in Turkey and 25% less in 19 Earnings can be explained as a function of schooling and labor market experience using the Mincer equation. The equation provides estimates of the average monetary returns of one additional year of education. This provides policymakers with important information about how to invest in education. 52 the Russian Federation. In Moldova, studying STEM also pays more and, again, it pays more for men. But this effect is driven mainly by studying tertiary education20. As for the analysis of returns to different fields of study in Turkey, the lower hourly returns that women get is found to be driven by the returns to specific fields of study. Health and law are the fields of study that translate into the highest overall hourly returns: 21 and 9% increase respectively compared to having studied education. However, the hourly returns of these same two fields for women are much lower. Women get only a 10% increase in hourly income for having studied health and a 3% decrease in hourly income for having studied law compared to men who studied education. Despite this, health is the best paid field of study for women, followed by mathematics. Women get a 4% increase for having studied math compared to men that studied education. Besides math, health and education, all other fields have negative returns for women. This is, in all other fields women earn less than men who studied education. The lowest returns for women are in the fields of social services, transport and environmental protection, manufacturing, and arts and agriculture where women get more than 30% less hourly income than men who studied education. In addition, women who studied computing also get 30% less than men who studied education. In the Russian Federation, after controlling for gender, experience, education level and job sector; fields of study are not significant predictors of earning different hourly income, with the exception of STEM fields of study. Studying science and engineering gives both men and women an increase of 20% in hourly income compared to men and women that studied a field related to sales and services. And studying information and communication technologies translates into a 27% increase in hourly income for both men and women. These effects are not different between men and women (although women make 30% less hourly income, regardless of their field of study). In Moldova, STEM fields of study are amongst the best paid fields of study. Studying science and engineering yields an increase of 12% in hourly income (although not significant) and studying information and communication technologies yields a significant increase of 22% in hourly income. Women have an extra premium of 20% for studying these fields of study. Although since they make 27% less hourly income regardless of their field of study, they end up getting a lower premium for studying STEM compared to men. Information of earnings of men and women in ECA is useful in understanding if and how STEM jobs differ from other jobs when it comes to returns. Although cross-country comparable data in scant, analysis of ILO data from 18 countries in the region show that it pays to work in a STEM job for both men and women in an overwhelming number of ECA countries, but men are at a clear advantage in terms of earnings even within STEM jobs. Figure 29 shows male and female earnings for professional, scientific and technical activities in the ECA labor market in comparison to average monthly earnings of all professions combined for both sexes as a benchmark. In all ECA countries, men in STEM professions have higher earnings than 20 Significant effects for studying STEM fields disappear once we control for highest level of education achieved. 53 women in the same professions.21 Also, in almost all ECA countries, STEM jobs garner higher earnings than the average of all jobs. Figure 33: Does it pay to work in STEM? Average monthly earnings in USD PPP 4000 3500 3000 2500 2000 1500 1000 500 0 Male earnings: professional, scientific and technical activities Female earnings: professional, scientific and technical activities Average monthly earnings (all professions, both sexes) Source: ILO Statistics Note: The data for SRB is from 2017; data for ALB, ARM, AZE, CZE, KYR, MOL, ROM, SVK, UKR is from 2016; data for BGR, HUN, KAZ is from 2015; data for EST, MKD, POL, SVN, TUR is from 2014; data for HRV is from 2011. To enhance the findings on earnings, a Oaxaca decomposition analysis for Turkey using the Labor Force Survey is presented in this report. The results can be seen in the table below. According to this, the gender gap in hourly income is 8%, in favor of men, after controlling for selection in the labor market. This gap is explained by the difference in the characteristics of men and women such as fields of study, experience, job sector, and the probability of being employed, rather than being explained by the "payoff" of such characteristics. The coefficients (or the unexplained part) actually negatively explain the gap. This means that women receive a better "payoff" for their characteristics than men when they have those, so this helps close the gender pay gap. The analysis shows that the single most significant factor affecting monthly wages is the Mills ratio, which means selection into the labor market is the greatest factor in explaining the income gap in favor of men. Work experience is the other important factor that explains this gap. Studying a STEM field, on the other hand, is negatively related to the gap: It is associated with a decrease in the gap by 25%. Working in a stem sector is also associated with a decrease of 6.7% in the gender pay gap. 21 A study on wage expectations in STEM careers found that when choosing STEM majors, women already expect to make less money than men do in the same field (Osikominu and Pfeifer, 2018). 54 Table 4: Oaxaca decomposition of monthly wages. Turkey 2013 Threefold and twofold decomposition Detailed twofold decomposition of wage of wage gaps, original scale Threefold decomposition Gap 1 Sig. Gender gap, % 8.40*** Explained 18.7 *** Endowments 15.54*** STEM field -2.6 *** Coefficients -7.26** STEM sector -0.68 *** Interaction 1.17 Experience -5.03 *** Region -0.48 ** Mills ratio 17.3 *** Twofold decomposition Explained 16.21*** Unexplained 18.6 *** Unexplained -6.71*** STEM field -5.6 *** STEM sector 0.50 ** Experience -64.2 *** Region 1.93 ** Mills ratio -12.9 *** N.of observations 35,051 Constant 9.17 ** Source: Turkey LFS 2013 A similar Oaxaca decomposition for the Russian Federation and Moldova show different results. Contrary to the case of Turkey, for these two countries, we find that the unexplained part predicts the gender gap in favor of men. That is, women receive a lower payoff for their characteristics compared to men. In fact, their characteristics help close the gender gap. In particular, education, region and experience are the most important predictors that help close the gender gap for Russian Federation; while again, selection into the labor market helps increase the gap in favor of men. In Moldova, the most important variable that helps closing the gender gap is education, followed by studying a STEM field. In summary, although more cross-country comparable research is need, aggregate data for 18 countries in the region show that it pays to work in a STEM job, especially for men who have a clear advantage in terms of earnings even within STEM jobs. 55 Policies to increase the participation of women in STEM Increasing the participation of women in Science, Technology, Engineering, and Mathematics (STEM) educational fields and careers has been a topic of interest and debate for researchers and policy practitioners in the recent years. As this report emphasized, the number of women in STEM fields has been on the rise, but room for improvement still exists, since women remain behind men in STEM in terms of access and opportunities. This chapter looks at what works to increase the participation of women in STEM fields by trying to answer some key questions that have been highlighted throughout our study. These questions are: (i) how to ensure that women studying in STEM fields in tertiary education have access to STEM jobs after graduation; (ii) how to guarantee women who already have STEM jobs to keep those jobs, make the same amount of money as men and move up in their careers; and, maybe most importantly, (iii) how to work in the education realm to create aspirations for STEM careers and improve performance in math and science topics for girls in the first place. Sustaining aspirations and expectations in the face of different challenges at different stages of life is also an important issue to explore that would make a positive difference for women. These questions are the ones that emerged from the previous chapters of this study as key questions to address to promote female students’ interest and motivation in STEM topics as well as women’s opportunities in STEM fields of work. A key thing to note is that there are numerous factors that impact the answer to these questions for all women such as interests, aspirations, access to relevant information, family background and salaries. Hence, it is important to address the challenges that women face in entering STEM fields and staying in these fields with regards to all these factors so as to sustain continued policy success. In order to address these questions, two efforts were undertaken. A systematic review of experimental and quasi experimental intervention related to women in STEM (further discussed below), and a literature review of all other documented interventions targeting women and Science, Technology, Engineering and Mathematics (STEM) in ECA countries, irrespectively whether they reported rigorous impact evaluation results or not. In this second effort, we were only able to document 35 different interventions -noted that we were doing a literature review, and there is a wide range of smaller interventions that are not found in the literature but are being implemented by NGOs and governments. Of the ones found, most of them were aimed at students and/or women in the academic sector (faculty and similar). Along those lines, the kinds of activities identified largely focus in the education area, and are generally small-scale, based on certain universities or schools -although a small set of national- level initiatives was identified. Despite the various categories of interventions reviewed, an element that seems to be quite common among them is the inclusion of hands-on or practical experiences for participants. About one third of the interventions reviewed focused on activities that relate to training; and another third included program features that could be categorized as mentoring schemes, and the majority of interventions included an information component for either girls interested in STEM, to promote interest, or to generally inform those that might be gatekeepers (e.g. teachers) of girls getting into STEM education. The type of intervention was rarely evaluated. In fact, as noted in the previous sector, and highlighted by other authors, there is a dearth of properly designed evaluations of interventions aimed at raising the participation of 56 women in STEM (Valla and Williams 2012), and even when an evaluation was included, most of the studies lacked a proper control group identification for the results. So we focus this chapter on the findings of a systematic review (Henninger et al 2019) of the existing studies and their effect sizes in order to better understand what works to increase the presence of women in STEM.22 The review covered literature and studies from 2000-2017, irrespectively of country, as long as rigorous impact evaluation results were presented. Many types of interventions and programs have been developed and implemented across countries to address women in STEM presence issues over the last few decades. Studies that included an assessment of the impact of these interventions aimed at raising women’s profiles in STEM fields are rare but increasing in number. While the policy review or the systematic review allow us to directly answer some of the above questions, some conclusions from this chapter shines a light on how to move forward to get more women into STEM as we will go over policies and programs that attempted to encourage change and/or had any impact. What works? There is a wide spectrum of STEM-related interventions across different fronts. However, there are only a few government level policy actions in this area. They add to NGO-led programs and pilots implemented in academic settings, which account for the bulk of interventions in the STEM area. A crucial caveat is that, generally speaking, most of these programs and policies have not been evaluated for impact so it is not possible to know whether or not they are effective in what they set out to do. This section focuses solely on those studies that had some measurement of effects or impact evaluation component attached to them to be able to deduce how effective they have been. We conducted a systematic review is to explore the nature and results of interventions seeking to address female underrepresentation in STEM fields of study and employment. We systematically reviewed experimental and quasi-experimental studies targeting outcomes of participation of women in STEM. The initial search returned 2,100 articles, which after an initial filter were reduced to 256, of which only 21 were included in the final review. The main criteria for inclusion required that the documented intervention targeted STEM-related barriers for women or girls, that the analytical approach included a quantitative research design for the measurement of a counterfactual, and they pass an assessment of risk bias and quality of reporting.23 The set of interventions studied in the 21 papers included in the systematic review are mostly in the education domain and encompass more representation of female teachers in math; exposing girls to non- traditional or gender-neutral roles in the learning environment; female peer mentoring; group learning; 22 Systematic review is a type of literature review that uses methods to collect secondary data, critically appraise research studies, and synthesize those two. They are designed to provide a thorough summary of current evidence relevant to a research question. Systematic reviews of randomized controlled trials are key to the practice of evidence-based research. 23 These papers formed the basis of an initial literature review, including studies from thirty-three countries; fourteen of these countries were within the Europe and Central Asia (ECA) region. Sample sizes are found to be generally very small - between 12 and 1,664 observations. Studies are mainly from developed countries, with the USA dominating. 57 single-sex learning settings; facilitation of advanced course taking; psychological intervention on affirmation training; and, internet delivered “growth mindset” training. Within the domain of STEM work fields, the interventions tested in the literature included in this review include: Communication of flexible and work-life balanced STEM workplace; increased female representativity in scientific committees; encouragement about STEM activities; and recruitment of STEM academics. Drawing on the main barriers identified in the analysis affecting women and girls participation in STEM, the studies were grouped by their outcome of interest in four categories: (a) aspirations and motivation to pursue education or career in STEM; (b) achievement or performance linked to educational progress in STEM; (c) participation or representation, for example course taking, field of study chosen, persistence in the field of study and labor participation in STEM sector and occupation; and, (d) beliefs and attitudes. Figure 34 summarizes the standardized effect sizes and shows that, overall the categories of interventions reviewed show small effects in participation, retention, and training of women in STEM.24 The review finds that interventions altering teaching/learning environments, aiming to remove stereotype threat or psychological bias for girls leading to and resulting from self-selection seem to have an effect on a range of outcomes from performance and achievement to girls’ interest and attitudes towards STEM fields. In particular, the most promising interventions that are shown to have an impact are those involving role models in girls’ lives helping them with self-confidence and aspiration to STEM fields; and group learning activities on STEM subjects in a mixed gender environment, where early interventions might support with changes in the future trajectory of engagement in STEM. Further along, providing information about more work-life balanced workplace environment seem to have an effect encouraging female participation in STEM careers for students in tertiary education. The limited set of studies looking at interventions focusing on women already in the labor market was too narrow (e.g. focusing on tenure-track academics in the USA) and seem to relate largely with de-biasing selection committees. 24 Two studies did not provide enough information allowing calculation of the effect sizes (Jackson 2012 and Unkovic et al 2015). 58 % -4 -2 0 2 4 6 10 Coyle & Liben (2016): Video game highlighting 8 women doing traditionally male jobs (preschool… Weisgram & Bigler (2007): Gender-Discrimination Info Session Betz, D. and D. Sekaquaptewa (2012) Feminine Source: Own elaboration Math and Science Role Models Interest / aspirations Miyake et al (2010): Values Affirmation Exercise *Eisenkopf et al (2015): Female Math Teacher in single-sex classrooms *Antecol et al (2013): Female Math Teacher *Eisenkopf et al (2015): Single-Sex Schooling Walton et al (2015): Mitigation of "chilly climate" in STEM fields Hossain & Tarmizi (2012): Group Learning on boy's Carrel et al (2010) Female Math Teacher Achievement / performance Hossain & Tarmizi (2012):Group Learning overall Hossain & Tarmizi (2012): Group Learning girls Notes: Blue markers denote effects that are not statistically different from zero. Effect size *Bagues et al (2015): Gender Composition of Scientific Committees Joensen & Nielsen (2016): Elected Advanced Mathematics in High School Fabert (2014): Growth Mindset Training (*) Denotes lack of information to replicate confidence intervals for the calculated effect size. Smith et al (2015): Gender Bias Training for Hiring Committee Williams & Ceci (2015): Determining the role of gender bias in tenure-track faculty hiring Carnes et al (2015): Info sesion to break the Gender Bias Habit among faculty in STEM Figure 3434: Effect Sizes of Studies Included in the Systematic Review Carrel et al (2010) Female Math Teacher (All Participation / representation students) Carrel et al (2010) Female Math Teacher (High- performing students) Weisgram & Bigler (2007): Gender-Discrimination Info Session Walton et al (2015):Mitigation of "chilly climate" in STEM fields Cheryan et al (2011): Non-Stereotypical Role Models DeFraine et al (2014): Work/Life Balance Attitudes / beliefs Presentation vs. Standard Lab Presentation DeFraine et al (2014): Higher flexibilitiy and Work/Life Balance 59 In Sum This study departed from an exploratory question. Is it correct to assert that women in ECA are not present in STEM, and that low representative is negative for women? To what extent women’s limited participation in STEM relates to women’s preferences, and/or to observable constraints? Our findings point to less definitive answers and a more cautious approach to the topic. During the course of the study we found evidence that the notion of women in STEM is a more complex issue than identified at the start. While there are somewhat clearer definitions when it comes to school-level education on what can be understood as STEM (at least when it relates to school subjects), such clear demarcations disappear when entering discussions on career streams and labor market participation. We also find that the observed differences between women and men are strongly path-dependent and multifaceted, and at all stages, multiple and overlapping constraints affect women’s choice for a STEM focus, ranging from aspirations, anxiety, and parental support in early education, to social norms, information, social networks, and labor market prospects in tertiary education and in the transition from education to employment. Finally, in the labor market, the demarcation of what constitutes a STEM occupation or job, and the limited information available to assess returns, wage gaps, and other job-related characteristics, make it hard to draw conclusive insights. However, the complexity of the issues identified indicates that there are not clear and immediate solutions and a lot more work, interventions, and policy testing is required. This study could not find a systematic and consistent barrier to be common to ECA countries but rather an overlap of different types of gaps. More gender equality s crucial to promote more women in STEM professions as gender equality policies and practices get you better results in STEM. While there is a lot of different schemes and programs to get more women into STEM than the ones we were able to document, our findings suggest that if policy makers only focus on STEM specific policies as a way to empower women in STEM fields, they might be missing the big picture. This conclusion stems from the observation that female-friendly STEM policies may be under the risk of being reduced to increasing information on STEM careers, or, related to ITC, to coding teaching and camps for girls. This potentially reduces the discussion to a narrow view of what STEM is and what is STEM's potential for women’s economic empowerment. Policies must focus on opening pathways for women from all backgrounds by offering a range of skills that can be applied to many different jobs and sectors that do not only need to be STEM-related ones. To have more women navigating STEM fields requires having more women in the labor market in the first place. There are many other policies proven to work in promoting gender equality that might work for STEM as well. Making male-dominated sectors more female friendly, as well as making female dominated sectors more male friendly is a first step. In addition to this many different interventions and programs that introduce family-friendly workplace practices such as child care availability in the workplace and flexible work arrangements, those that introduce gender and age quotas in the candidates interviewed for a certain job and not just a quota for hiring women can have a positive effect in STEM fields by making a difference for women overall. 60 REFERENCES Alon, S. and DiPrete, T.A. (2015). Gender Differences in the Formation of a Field of Study Choice Set. Sociological Science 2: 50-81. Andreescu, T., Gallian, J. A., Kane, J. M., & Mertz, J. E. (2008). Cross-cultural analysis of students with exceptional talent in mathematical problem solving. Notices of theAmerican Mathematical Society, 55(10), 1248 –60. Archer, L., DeWitt, J. and Wong, B. (2014). Spheres of influence: what shapes young people’s aspirations at age 12/13 and what are the implications for education policy? Journal of Education Policy Vol. 29, Issue 1, 2014. Arcidiacono, P., Aucejo, E., & Hotz, V. (2016). University Differences in the Graduation of Minorities in STEM Fields: Evidence from California. The American Economic Review,106(3), 525-562. Bandura, A. (1977), Social Learning Theory, Prentice-Hall, Englewood Cliffs, NJ Bandura, A. (1997), Self-Efficacy: The Exercise of Control, Freeman, New York, NY. Barker, L. J. and Aspray, W. 2006. The state of research on girls and IT. J. M. Cohoon and W. Aspray (eds), Women and Information Technology: Research on Underrepresentation. London, The MIT Press, pp. 3-54. Beal S.J., Crockett L.J. (2010). Adolescents’ occupational and educational aspirations and expectations: Links to high school activities and adult educational attainment. Developmental Psychology. 2010; 46(1):258–265. Beasley, M. A. and Fischer, M. J. 2012. Why they leave: The impact of stereotype threat on the attrition of women and minorities from science, math and engineering majors. Social Psychology of Education, Vol. 15, No. 4, pp. 427-448. DOI: 10.1007/s11218-012-9185-3. Beilock S. L., Gunderson E. A., Ramirez G., Levine S. C. (2010). Female teachers' math anxiety affects girls' math achievement. Proc. Natl. Acad. Sci. U.S.A. 107, 1860–1863. 10.1073/pnas.0910967107 Berenbaum, S.A., Bryk, K.L. & Beltz, A.M. (2012). Early androgen effects on spatial and mechanical abilities: evidence from congenital adrenal hyperplasia. Behavioral Neuroscience 126 (2012): 86-96. Bian, L., Leslie, S. J. and Cimpian, A. 2017. Gender stereotypes about intellectual ability emerge early and influence children’s interests. Science, Vol. 355, No. 6323, pp. 389-391. Blakemore JEO, Berenbaum SA, Liben LS. Gender development. New York: Psychology Press; 2009. Block, K., Croft, A. and Schmader, T., (2018) Worth Less? Why Men (and Women) Devalue Care-Oriented Careers. Frontiers in Psychology, 9, p.1353. Boll, C., Rossen, A. and Wolf, A. (2016). The EU gender earnings gap: Job segregation and working time as driving factors. HWWI Research Paper 176. Burchell et al. (2014). A new method to understand gender segregation in the European labor market. European Commission. Bussolo, M., M.E. Dávalos, V. Peragine, and R. Sundaram. (2018) Toward a New Social Contract: Taking On Distributional Tensions in Europe and Central Asia . Europe and Central Asia Studies. Washington, DC: World Bank Carlana, M. (2018). Implicit Stereotypes: Evidence from Teachers’ Gender Bias. IZA Discussion Paper Series No. 11659. Bonn, Germany: 2018. Carneiro, P., Cruz-Aguayo, Y. and Schady, N. (2017). Where the Girls Are Not: Households, Teachers, and the Gender Gap in Early Math Achievement. IDB WORKING PAPER SERIES No. IDB-WP-807. Washington, D.C. Ceci, S. J., Williams, W. M., & Barnett, S. M. (2009). Women’s underrepresentation in science : Sociocultural and biological considerations. Psychological Bulletin, 135(2), 218–61. 61 CEDEFOP, (2016), “Skill Shortage and Surplus Occupations in Europe,“ European Centre for the Development of Vocational Training, November 2016 Christensen, R. & Knezek, G. (2017). Relationship of middle school student STEM interest to career intent. Journal of Education in Science, Environment and Health (JESEH), 3(1), 1- 13. Cvencek, D., Meltzoff, A. N. and Greenwald, A. G. 2011. Math-gender stereotypes in elementary school children. Child Development, Vol. 82, No. 3, pp. 766-779. Ellis J, Fosdick BK, Rasmussen C (2016) Women 1.5 Times More Likely to Leave STEM Pipeline after Calculus Compared to Men: Lack of Mathematical Confidence a Potential Culprit. PLoS ONE 11(7). Ertl B., Luttenberger S. and Paechter M. (2017). The Impact of Gender Stereotypes on the Self-Concept of Female Students in STEM Subjects with an Under-Representation of Females. Frontiers in Psychology 8:703. European Commission (EC) (2013). Women active in the ICT sector. European Commission. Flabbi, L. (2012). Gender differences in education, career choices, and labor market outcomes in a sample of OECD countries. World Bank WDR 2012 Background paper. Flouri, E. & Panourgia C. (2012). Do primary school children's career aspirations matter? The relationship between family poverty, career aspirations and emotional and behavioural problems. Centre for Longitudinal Studies Working Paper 2012/5. London: Institute of Education Galdi, C., Cadinu, M. and Tomasetto, C. 2014. The roots of stereotype threat: When automatic associations disrupt girls’ math performance. Child Development, Vol. 85, No. 1, pp. 250-263. DOI: 10.1111/cdev.12128. Gevrek, Z. E., Neumeier, C., Gevrek, D. (2018). Explaining the Gender Test Score Gap in Mathematics: The Role of Gender Inequality. IZA Discussion Paper Series No. 11260. Bonn, Germany: 2018. Gunderson E. A., Ramirez G., Levine S. C., Beilock S. L. (2012). The role of parents and teachers in the development of gender-related math attitudes. Sex Roles 66, 153–166. 10.1007/s11199-011-9996-2 Halpern D.F. (2012). Sex differences in cognitive abilities. 4th ed. New York: Psychology Press; 2012. Hartung, P. J., Porfeli, E. J., & Vondracek, F. W. (2005). Child vocational development: A review and reconsideration. Journal of Vocational Behavior, 66(3), 385–419. Hill, C., C. Corbett and A. St. Rose (2011). “Why so few? Women in Science, Technology, Engineering and Mathematic”, AAUW Washington D.C. Huyer, S. (2015). Is the gender gap narrowing in science and engineering? UNESCO science report: towards 2030. Khitarishvili, T. (2016). Gender Dimensions of Inequality in the Countries of Central Asia, South Caucasus, and Western CIS. Working Paper No. 858. Kugler, A D, C H Tinsley and O Ukhaneva (2017), “Choice of Majors: Are Women Really Different from Men?”, NBER Working Paper No. 23735. Lacey, T. A., & Wright, B. (2009). Occupational employment projections to 2018. Monthly Labor Review, 132(11), 82- 123. Leslie, S. J., Cimplan, A., Meyer, M. and Freeland, E. 2015. Expectations of brilliance underlie gender distributions across academic disciplines. Science, Vol. 347, No. 6219, pp. 262-265. DOI: 10.1126/science.1261375 Lippa, R. A. 2005. Gender, nature and nurture, 2nd edn. Mahwah, NJ, Lawrence Erlbaum Associates Low, K. S. D., Yoon, M., Roberts, B. W., & Rounds, J. (2005). The stability of vocational interests from early adolescence to middle adulthood: A quantitative review of longitudinal studies. Psychological Bulletin, 131(5), 713 –37. Mann, A. and DiPrete, T.A. (2016). The Consequences of the National Math and Science Performance Environment for Gender Differences in STEM Aspirations. Sociological Science 3: 568-603. 62 Marsh, H. W., and Scalas, L. F. (2011). “Self-concept in learning: reciprocal effects model between academic self- concept and academic achievement,” in Social and Emotional Aspects of Learning, ed S. Järvela (Amsterdam: Elsevier), 191–197. Mason, L. et al. (2012), “Besides knowledge: A cross -sectional study on the relations between epistemic beliefs, achievement goals, self-beliefs, and achievement in science”, Instructional Science, Vol. 41/1, pp. 49 -79, Master, A., Cheryan, S. and Meltzoff, A. N. 2014. Reducing adolescent girls’ concerns about STEM stereotypes: When do female teachers matter? Revue Internationale de Psychologie Sociale, Vol. 27, No. 3, pp. 79-102 Meece, J. L., Anderman, E. M., & Anderman, L. H. (2006). Classroom goal structure, student motivation, and academic achievement. Annual Review of Psychology, 57, 487-503 Michelmore, K. and S. Sassler. (2016). Explaining the Gender Wage Gap in STEM: Does Field Sex Composition Matter? RSF: The Russell Sage Foundation Journal of the Social Sciences, 2(4), 194-215. doi:10.7758/rsf.2016.2.4.07 Milam, J. 2012. Girls and STEM Education: A Literature Review. Atlanta, Georgia Institute of Technology. Mills M. and Präg, P. (2014). Gender Inequalities in the School-to-Work Transition in Europe. Rand Europe Short Statistical Report No. 4. Mullis, I.V.S., Martin, M.O., & Loveless, T. (2016b). 20 Years of TIMSS: International Trends in Mathematics and Science Achievement, Curriculum, and Instruction. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Mullis, I.V.S., Martin, M.O., Foy, P. & Hooper, M (2016). TIMSS 2015 International Results in Science. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. National Research Council (2011). Successful K-12 STEM education: Identifying effective approaches in science, technology, engineering, and mathematics. Washington, DC: National Academy Press. National Science Board (2010). Science and engineering indicators 2010 (NSB 10-01). Arlington, VA: National Science Foundation. Nosek, B. A., Smyth, F. L., Sriram, N., Lindner, N. M., Devos, T., Ayala, A., et al. (2009). National differences in gender- science stereotypes predict national sex differences in science and math achievement. Proc. Natl. Acad. Sci. U.S.A. 106, 10593–10597. Nugent, G. et al. (2015), “A model of factors contributing to STEM learning and career orientation”, International Journal of Science Education, Vol. 37/7, pp. 1067-1088, http://dx.doi.org/10.1080/09500693.2015.1017863. Nurmi, J. -E., (2004). Socialization and self-development: Channeling, selection, adjustment, and reflection. In R. Lerner & L. Steinber (Eds.). Handbook of adolescent psychology, 2, 85-124. OECD (2016), PISA 2015 Results (Volume I): Excellence and Equity in Education, PISA, OECD Publishing, Paris. OECD (2016b): The ABC of Gender Equality in Education. OECD Publishing, Paris. OECD (2016c), PISA 2015 Results (Volume III): Students’ Well-Being, PISA, OECD Publishing, Paris. OECD (2017), The Pursuit of Gender Equality: An Uphill Battle, OECD Publishing, Paris. OECD 2017. Education at a Glance 2017, OECD Publishing, Paris. Osikominu, A. and Pfeifer, G. (2018). Perceived Wages and the Gender Gap in STEM Fields. IZA Discussion Paper Series No. 11321. Bonn, Germany: 2018. Pasnicu, D. and Mladen, L. (2015). The Analysis of the Perceptions about Gender Equality on the Labour Market in Romania. Procedia of Economics and Business Administration. Porębska, A., Mitkowski, S.A., Dąbrowski, A.M. and Zegarmistrz, P. (2011). Female students at technical universities - gender as the factor determining the choice of engineering studies. World Transactions on Engineering and Technology Education Vol.9, No.4, 2011. 63 Rothwell, J. (2013). The Hidden STEM Economy. Brookings Institution: Washington. Schlenker, E. (2015). The labour supply of women in STEM. IZA Journal of European Labor Studies 20154:12. Schoon, I., P. Martin and A. Ross (2007), 'Career transitions in times of social change. His and her story', Journal of Vocational Behavior, 70(1), 78–96. Schuster, C., and Martiny, S. E. (2017). Not feeling good in STEM: effects of stereotype activation and anticipated affect on women's career aspirations. Sex Roles 76, 40–55 Senler, B., and Sungur, S. (2009). Parental influences on students' self-concept, task value beliefs, and achievement in science. Span. J. Psychol. 12, 106–117. Sikora, J. and Pokropek, A. (2012). Gender segregation of adolescent science career plans in fifty countries. Science Education. Sondergaard, L. and M. Murthi , with D. Abu‐Ghaida, C. Bodewig, and J. Rutkowski (2012). Skills, Not Diplomas— Managing for Results in Education Systems in Eastern Europe and Central Asia. Directions in Development Series. Washington, DC: World Bank. Spencer, S. J., Steele, C. M. and Quinn, M. 1999. Stereotype threat and women’s math performance. Journal of Experimental Psychology, Vol. 35, No. 1, pp. 4-28. DOI: 10.1006/jesp.1998.1373 Starobin, S. S. and Laanan, F. S. 2008. Broadening female participation in science, technology, engineering, and mathematics: Experiences at community colleges. New Directions for Community Colleges, Vol. 2008, No. 142, pp. 37-46. DOI: 10.1002/cc.323 Storage, D., Horne, Z., Cimpian, A. and Leslie, S. J. 2016. The frequency of ‘brilliant’ and ‘genius’ in teaching evaluations predicts the representations of women and African Americans across fields. PLOS ONE, Vol. 11, No 3, e0150194. DOI: 10.1371/journal.pone.0150194. Teitelbaum, M. (2014). Falling Behind?: Boom, Bust, and the Global Race for Scientific Talent. Princeton; Oxford: Princeton University Press. Thomasian, J. (2011). “Building a Science, Technology, Engineering, and Math Education Agenda: An Update of State Actions.” National Governors Association Center for Best Practices, 2011. Available at https://www.nga.org/files/live/sites/NGA/files/pdf/1112STEMGUIDE.PDF. U.S. Department of Commerce, (2011), STEM: Good Jobs Now and for the Future; Carnevale, A.P., Nicole Smith, and Michelle Melton, 2011, STEM: Science, Engineering, Technology, and Mathematics” (Washington: Georgetown University Center for Education and the Workforce) UNESCO (2017). Cracking the code: girls' and women's education in science, technology, engineering and mathematics Vokić, N.P., Ćorić, D.S. and Obadić, A. (2016). To be or not to be a woman? Highly educated women´s perceptions of gender equality. Wigfield, A. and Eccles, J. S. 2000. Expectancy-value theory of motivation. Contemporary Educational Psychology, Vol. 25, pp. 68-81. DOI: 10.1006/ ceps.1999.1015. World Bank (2011). World Development Report 2012: Gender Equality and Development. Washington, DC. World Bank (2013). North Macedonia Gender Diagnostic: Gaps in Endowments, Access to Economic Opportunities and Agency. The World Bank. World Bank (2013b). Montenegro Gender Diagnostic: Gaps in Endowments, Access to Economic Opportunities, and Agency. The World Bank. World Bank (2014). Belarus Country Gender Profile. The World Bank. World Bank (2016). World Development Report 2016: Digital Dividends. Washington, DC. 64 World Bank (2016a). Voices of Europe and Central Asia: New Insights on Shared Prosperity and Jobs World Bank (2016b). From Aspirations to Occupations: The Role of Information in Educational and Labor Market Decisions in Moldova. World Bank (2016c). Women's Access to Economic Opportunities in Serbia. The World Bank. 65