Policy Research Working Paper 10540 Rule-Based Civil Service Evidence from a Nationwide Teacher Reform in Mexico Juan Bedoya Rafael de Hoyos Ricardo Estrada Human Development Global Practice August 2023 Policy Research Working Paper 10540 Abstract This paper studies the effect of a civil service reform on the hires drove this change. Two channels explain these effects. skills profile of new teachers in Mexico. The reform man- First, the reform decreased the prevalence of discretionary dated the use of rule-based recruitment over discretionary hires, who tended to be drawn from the bottom of the skills hiring. The results show that the reform led to hiring teach- distribution. Second, the reform improved the screening ers with higher cognitive skills. The paper also shows that an efficiency of rule-based hiring, making cognitive skills more improvement in the bottom of the skills distribution of new important determinants of hiring outcomes. This paper is a product of the Human Development Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at rdehoyos@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Rule-Based Civil Service: Evidence from a Nationwide Teacher Reform in Mexico∗ Juan Bedoya† Rafael de Hoyos‡ Ricardo Estrada§ Keywords: civil service, hiring methods, teacher hiring, teacher quality JEL codes: I21, J24, J45, M5 ∗ This paper was written under an agreement with the Federal Education Ministry of Mexico and the National System of the Professional Teaching Service, allowing us to access data without compromising con- fidential information. We thank Felipe Barrera-Osorio, Gianmarco Leon-Cilliota, Maria Lombardi, Marcos Fern´andez-Guti´errez, Mauricio Romero and participants in a seminar at the Universidad del Pais Vasco, the Impact Evaluation Network of LACEA Annual Meeting and AEDE Meetings. We are grateful to Florencia Buccari, Janina Cuevas, and Veronica Michel for their excellent research assistance. † Universidad de Cantabria. E-mail: juan.bedoya@alumnos.unican.es. ‡ The World Bank. E-mail: rdehoyos@worldbank.org. § CAF–development bank of Latin America. E-mail: restrada@caf.com. 1 Introduction An efficient bureaucracy is a cornerstone of state capacity and economic development. How- ever, history shows that putting that cornerstone in place is often challenging (Grindle, 2012). For example, there is a consensus (going back to the Weberian definition of a professional bureaucracy) that the meritocratic recruitment of public officials is vital to a well-functioning bureaucracy. Yet, international indexes of bureaucratic quality indicate that many govern- ments in low– and middle–income countries do not recruit public officials meritocratically (Besley et al., 2021). This pattern calls for a better understanding of the mechanisms used to recruit civil servants and the factors shaping their effectiveness. Teachers are among the most important public sector employees. They are crucial factors in the quality of educational services and, therefore, critical to productivity and long-term growth. At the individual level, the positive impact of being assigned to a highly effective teacher persists into adult life (Rivkin et al., 2005; Chetty et al., 2014). Furthermore, teachers account for a significant part of public sector employment. In Mexico, around 24 percent of public sector employees are teachers, according to our calculations using data from the national labor force survey. In this paper, we study the effect of a civil service reform on the skills profile of new teachers. We do so in the context of a nationwide education reform adopted in Mexico in 2014, which revamped the teachers’ civil service. (We refer to this as the “SPD reform” in the rest of the paper.) Among other changes, the SPD reform mandated centrally managed, competitive examinations to determine hire and promotion decisions, proscribing a discre- tionary system in which local officials and teachers’ union representatives took such decisions at the state level. The SPD reform scaled up a previous reform that introduced competi- tive examinations for teacher hiring in response to criticisms about opacity, corruption, and absence of merit in the discretionary system (Section 2). We focus the analysis on the effects of the SPD reform on newly hired teachers’ cognitive skills. Teacher quality is multidimensional, but several papers have shown that teachers with higher cognitive skills tend to be more effective—they add more value to student learning (Rockoff et al., 2011; Gronqvist and Vlachos, 2016; Hanushek et al., 2019; Neilson et al., 2019). Indeed, it would be surprising if teaching effectively did not require having at least basic levels of cognitive skills. Yet, using data from an international assessment of the adult population, Estrada and Lombardi (2020) show that many teachers in Latin America, including Mexico, have low levels of literacy and numeracy skills. For the empirical analysis, we use personnel data to construct a dataset of primary 1 and lower-secondary public school teachers hired in Mexico between 2012 and 2017. We link this data to results from the competitive examinations used to select new teachers during the same period. This way, we can identify teachers hired through the discretionary process before and after the SPD reform, plus successful and unsuccessful candidates taking the competitive examinations. Our measure of teachers’ (and rule-based applicants’) cognitive skills comes from their results in “ENLACE Media Superior” (hereafter, ENLACE), a census- based national standardized test applied at the end of secondary school (grade 12) between 2008 and 2013. We focus the analysis on recent university graduates who became (or applied to become) teachers in the public education system between 2012 and 2017 (Section 3). Results show that teachers hired after the SPD reform have higher cognitive skills than teachers hired before the reform. The ENLACE score of new teachers increased between 2.7 to 3.8 percentile points after the SPD reform. An improvement in the bottom of the skills distribution of newly hired teachers drives this change. Because the SPD reform was implemented at the national level, the above effects are estimated using time variation. Yet, we provide evidence supporting their causal interpreta- tion. Notably, our analysis rules out the main threat to identification: the possibility that the documented results are explained by secular trends or shocks that affect the pool of individuals from which teachers are selected. We study several potential mechanisms to understand the forces behind these changes. First, the SPD reform increased the share of teachers hired via competitive examinations from 63.6 percent in 2012 to 76.8 percent in 2014 and 86.4 percent in 2016. However, it did not wholly eliminate discretionary hiring, highlighting the implementation challenges in contexts with weak state capacity. Second, the SPD reform increased the skills gap favoring rule-based over discretionary hires. A sharp increase in the skills of rule-based hires the year Mexico implemented the SPD reform and a progressive decline in the skills of the discretionary hires explain this enlargement of the skills gap. Our results show that the improvement in the skills of rule-based hires is driven by an improvement in the screening of applicants, with only a modest contribution from the self-selection channel. Given the promise of making merit the key criterion of personnel decisions throughout the teaching career, the SPD reform could have attracted higher-skilled individuals into teaching. However, we find only small gains in rule-based applicants’ skills. Second, the reform improved the screening efficiency of rule-based hiring, making cognitive skills stronger determinants of hiring outcomes. That is, the SPD reform made it more likely that those with higher skills would be hired. Before the SPD reform, a one-percentile- 2 point increase in an applicant’s ENLACE score was associated with an increase of 0.13 percentage points in the probability of being hired as a teacher. During the SPD regime, the corresponding increase was 0.43 percentage points. This paper contributes to several literatures at the intersection of personnel, education, and public economics. We contribute, first, to the literature on personnel economics of the state and, more specifically, to a recent strand of papers that study the effect of rule-based and discretionary hiring on the profile of civil servants (Estrada, 2019; Neilson, Gallegos and Calle, 2019; Brassiolo, Estrada and Fajardo, 2020; Colonnelli, Prem and Teso, 2020; Dahis, Schiavon and Scot, 2020; Brassiolo, Estrada, Fajardo and Martinez-Correa, 2021; Munoz and Prem, 2021).1 The paper also contributes to the literature on teacher hiring in developing countries (Estrada, 2019; Neilson, Gallegos and Calle, 2019; Araujo, Heineck and Cruz- Aguayo, 2020; Brutti and Torres, 2021). Our findings complement those by Estrada (2019), who—studying the period previous to the SPD reform—finds that teachers hired under the rule-based mechanism were significantly more effective at increasing student learning than discretionary hires. Relative to this paper, our contribution is to document the selection patterns (self-selection and screening) shaping the skills profile of rule-based hires. Our paper also connects to the literature on civil service reforms, which mainly focuses on the U.S. context (Rauch, 1995; Ujhelyi, 2014; Ornaghi, 2019; Moreira and P´ erez, 2021). We contribute to this literature by studying a large civil service reform in a different setting. Finally, we contribute to the literature on decentralization, particularly to the studies that highlight that the weak capacity of some local governments can compromise the success of decentralization reforms. (See reviews in Bardhan (2002); Mookherjee (2015).) Our findings point out one factor that could contribute to such weakness: a less meritocratic selection of bureaucrats. 2 Institutional Context In Mexico, state governments are responsible for the operation of preschool, primary, and lower secondary schools, whereas the federal government defines the national curriculum, monitors the system’s performance, among other regulations, and finances a large share of the public education budget. Prior to 2008, state education ministries and the teachers’ union were responsible for the selection and promotion of teachers.2 In practice, the union had close to full control in the hiring and promotion decisions in many states through, for example, the 1 In related work, Xu (2018); Voth and Xu (2020) study the effect of using discretion in promotion decisions in the context of the British Empire. 2 SNTE for its acronym in Spanish. See more about SNTE in Estrada (2019). 3 appointment of state officials who were also union leaders and the inclusion of participation in union activities as a requirement for promotions (Santibanez, 2008). Personnel decisions were characterized by a high use of discretion and opacity. The system was widely criticized for lack of merit in decisions, including the practice of entitlement (which allowed teachers who were retiring to pass on their jobs to relatives) and the selling of teaching positions. A 2008 reform named the “Alianza por la Calidad de la Educaci´ on” or ACE introduced the use of national competitive examinations to hire teachers for public schools. (See details about the ACE reform in Estrada (2019).) The ACE used test scores from national stan- dardized examinations organized by the Federal authorities as the main criteria for hiring teachers. However, the use of competitive examinations in ACE was not mandatory. The decision of what share of vacancies would be filled by rule-based hiring was jointly taken by state authorities and the teachers’ union, and the rest were assigned at the discretion of the teachers’ union. Thus, the union was entitled to select a share of new hires in every state, and representatives from both state authorities and the union made promotion decisions. The ACE reform also did not introduce changes in other dimensions of the teaching career—it was limited to the selection of new teachers. In 2013, the Mexican Congress enacted a major education reform that included revamp- ing the teachers’ civil service system. It expanded in scale and scope the policies implemented by ACE. However, there were some important differences between ACE and SPD. (Table 1 summarizes the main characteristics of rule-based hiring under both reforms.) The newly created teachers’ civil service (named “Servicio Profesional Docente” or SPD) mandated the use of competitive examinations to determine the hiring and promotion of teachers in preschool , primary, and secondary education.3 The SPD used standardized examinations to screen applicants. These assessments were designed specifically for this purpose and were the main criterion to make hiring offers. Like ACE before it, SPD faced fierce opposition from the teachers’ union. Despite this and other implementation challenges, the federal Ministry of Education and the National Institute for the Evaluation of Education implemented in July of 2014 the first SPD examination for teacher selection, a process that took place every year until 2018. In 2019, Mexico canceled the SPD reform because of a change in the political party in office, bringing this era of education reform to a close. See Figure A.1 for a summary timeline. A detailed description of the reform and the political context is available in Islas, Calef and Aparicio (2021). 3 The SPD reform did not cover either private schools or upper secondary schools operated by public universities. 4 3 Data 3.1 Data Sources We obtain information on primary and lower secondary school teachers hired from 2012 to 2017 from the following three databases: • Registro Nacional de Maestros, RENAME: quarterly administrative dataset on school personnel (3rd quarter of 2011–4th quarter of 2012). Identifies teachers hired in 2012. • Censo de Escuelas, Maestros y Alumnos de Educaci´ on B´asica, CEMABE: census of schools, teachers and students carried out by the National Statistics Office, INEGI (September–December 2013). Identifies teachers hired in 2013. • Fondo de Aportaciones para la N´ omina Educativa, FONE: quarterly administrative dataset on school personnel (1st quarter of 2015 – 2nd quarter of 2018). Identifies teachers hired from 2014 to 2017. The Federal Ministry of Education assembles the RENAME and FONE databases. They cover all public school teachers paid with federal funds, except those based in Mexico City schools. No official documents report the total number of teachers on the payrolls of state governments. However, according to our estimates, they accounted for around 13 percent to 15 percent of the teachers hired in 2013. Most teachers held positions funded by the federal government and are part of our analysis. Information on the individuals who applied under the rule-based hiring process is based on micro-data from ACE for 2012–13 and SPD for 2014–17. There are no comparable records of “applicants” to the discretionary process. To the best of our knowledge, there is no documentation of the process or criteria used to select and promote teachers in the discretionary process. The ENLACE exam is our measure of teachers’ cognitive skills. ENLACE was a census- based standardized test applied to all students in their final year of upper secondary school (grade 12) between 2008 and 2013. The test assessed achievement in mathematics, literacy, and a rotating subject. The main purpose of the assessment was to provide feedback to stakeholders in the education system, and its results did not have consequences for students’ graduation or admission into the next schooling level. Participation in ENLACE was op- tional, but more than 90 percent of all students sat for the test each year. de Hoyos, Estrada and Vargas (2021) find that ENLACE test scores are strong predictors of schooling and labor market trajectories and conclude that ENLACE effectively captured the cognitive skills it 5 was designed to measure. 3.2 Merging and Variables Included in the Final Dataset Using the RENAME, CEMABE, and FONE databases, we construct a data set of new teachers inclusive of name, gender, birth date, type of teaching position, assigned school(s), and year of hire.4 We identify as new teachers those individuals who do not appear in any record from previous school years—the datasets do not include information on hiring dates.5 We merge the results from the ACE and SPD entry examinations with the teachers’ data, building a dataset that contains applicants’ population ID, their results in the entry examination, and the type of teaching position they applied to.6 The data does not include hiring offers or acceptances. Hence, we define rule-based hires as those who apply to the rule- based process, either the ACE or SPD examinations, and are part of the teacher payroll the following school year. Newly hired teachers who are on the payroll but did not participate in a rule-based process are considered discretionary hires. We transform the individual ENLACE test scores into year-specific percentiles of the overall distribution of the mean score in math and literacy. Among the 4.9 million students who sat for the test between 2008 and 2012, 95 percent have a complete population ID. We use the population ID to merge the ENLACE data with data from the merged teachers’ and applicants’ database described above. Due to data limitations, we are not able to merge the information on the unsuccessful rule-based applicants in 2012 to their ENLACE scores. The final dataset includes information on all new hires, rule-based and discretionary, for primary and lower-secondary public schools between 2012 and 2017, while for rule-based applicants it covers the 2013-2017 period.7 3.3 Samples Used in the Analysis We identify a total of 181,590 individuals hired as primary and lower-secondary school teach- ers during the 2012–17 period. Because of their age, many of these individuals finished sec- 4 Across the different databases, we track individuals using their population ID (CURP for its acronym in Spanish) and taxpayer ID (RFC). 5 Because CEMABE was designed to cover all public school teachers, it includes teachers paid with federal or state funds. To avoid counting state-funded teachers as federally funded teachers, we exclude all teachers in CEMABE who are based in schools that do not appear in either RENAME or FONE. 6 Because test scores under SPD are only available in brackets corresponding to five performance cate- gories, we construct a corresponding variable for the ACE test scores—reported as a continuous variable— that mimics the distribution of SPD applicants among these categories. 7 The final dataset can be downloaded from xaber.org.mx along with a document explaining in detail how it was constructed. 6 ondary school before ENLACE was administered for the first time in 2008. For the 2012 cohort of new teachers, the first in our analysis, we can only identify the ENLACE score of those hired four years after concluding secondary school.8 Hence, our analysis is based on the sample of 24,914 individuals hired four years after finishing upper secondary. We refer to these individuals as “recent graduates.”9 Panel A of Table 2 shows descriptive statistics. Both all new teachers and recent university graduates are majority female, the new teachers are, on average, seven years older than the recent graduates, and the recent graduates are more likely to have been hired through a rule-based process (whether before or after the SPD reform). Their ENLACE scores are, on average, at the 67th percentile of the national distribution. Panel B of Table 2 shows some descriptive statistics for the 468,846 individuals who applied to the rule-based process during the period 2013–17. The majority are female, on average 29 years old, and 14 percent succeeded in being hired. The 55,660 recent university graduates have an even higher proportion of females and are about seven years younger on average. Recent graduates tended to perform well on the standardized test used in the competitive examinations. Thirty-eight percent (66 percent) scored in the top quartile (half) of their entry examination test, and 30 percent of them were hired. They have an average ENLACE score at the 61.5th percentile of the national distribution. 4 Empirical Strategy We are interested in identifying the effect of the SPD reform on the skills profile of new teachers and in shedding light on its underlying mechanisms. Because the SPD reform had a national reach and was implemented simultaneously across the country, we rely on time variation for our first set of results: 2017 Yit = α + βπ · 1[π = t] + ΓXst + θs + it (1) π =2013 Where Yit is the ENLACE test score (in percentiles) of individual i hired at year t. 1[π = t] is a vector of dummy variables that indicate the relative time (in years) with respect to 2012, the year prior to the enactment reform, and βπ are the coefficients of interest. Xst is a vector of labor market variables as controls and Γ is the associated vector of parameters. θs 8 A teaching degree in Mexico typically requires four years of studies. 9 We use the sample of teachers hired four or five years after finishing upper secondary for robustness checks (using 2013 as a baseline). 7 is a vector of state fixed effects and it is an error term. We present heteroskedasticity-robust standard errors. There was almost a one-year gap between the announcement and the implementation of the SPD reform, which could have led to anticipation effects. We deal with such a possibility by using 2012 (one year before the reform was announced and two years before it was implemented) as the baseline year. To explore heterogeneous effects along the distribution of Yit , we complement the main specification (equation 1) with a quantile regression model: 2017 Qτ (Yit ) = βπ (τ ) · 1[π = t] + Γ(τ )Xst + θs + uit (2) π =2013 Where Qτ (Yit ) is the ENLACE test score of individual i hired at year t at the τ th quantile and βπ (τ ) and Γ(τ ) are vectors of coefficients at the various quantiles. In equations 1 and 2, the effect of the SPD reform is identified using time variation. Hence, it faces two important threats to identification. First, the effect of the reform might be confounded with a secular trend unrelated to the SPD reform on the ENLACE scores of new teachers. To mitigate this concern, we use a dynamic specification of treatment effects— instead of a simple before-and-after specification—which allows exploring in a desegregated way the evolution of the new teachers’ profiles. Furthermore, by using the year-specific percentiles of ENLACE test scores as the outcome variable, we concentrate on changes in the relative position of newly hired teachers in the national distribution of cognitive skills, holding constant changes in overall test scores across years. We do not have information about long-term trends in the skills profile of the teaching force in Mexico, but a string of papers from other contexts has documented a secular decline—mostly in developed countries ¨ (Nickell and Quintini, 2002; Corcoran et al., 2004; Fredriksson and Ockert, 2008), but also in Chile in Latin America (Neilson, Gallegos and Calle, 2019)—which in our specification would lead to an underestimation of the effects of the SPD reform on the skills profile of new teachers. A second threat to identification is related to potential shocks that affect the selection process into teaching in a contemporaneous way to the SPD reform. We do not have any information about a contemporaneous policy or economic shock that could produce such a pattern. Nonetheless, we include in the estimation of equations (1) and (2) a set of covariates on economic conditions at the local (state) level that could affect occupational decisions in 8 the labor market: the state’s GDP per capita, unemployment rate, and the mean wages earned by tertiary-educated individuals working in non-teaching occupations Furthermore, we can directly observe the skills profile of individuals who self-select (apply) into the rule-based process. As we rely on time variation, we cannot separate con- temporaneous shocks to applicant quality from the potential effect of the SPD reform on the skills profile of rule-based applicants. This is not problematic though, as in Section 6, we find that the implementation of the SPD reform was followed by only a modest change in the skills profile of rule-based applicants, which—no matter the source—cannot explain the improvement in the skills profile of new teachers that we present in the next section. 5 Main Results We find a clear improvement in the skills profile of teachers hired after the SPD reform. Figure 1a shows point estimates for the changes with respect to 2012 in the average ENLACE scores of the teachers hired every year—coming from the estimation of equation 1.10 The point estimates for the changes in the post-reform years vary from 2.7 to 3.8 percentiles of the ENLACE test score distribution and are all statistically significant at the 95 percent level. For context, teachers hired in 2012—our baseline year—had average ENLACE scores in the 65th percentile. We do not find clear evidence of an anticipation effect. The coefficient for 2013 has a small magnitude (-1.0 percentiles) and is not statistically significant. An improvement in the bottom of the skills distribution explains most of the improve- ment that the SPD reform produced in the skills profile of new teachers. The SPD reform was accompanied by a sharp improvement in the skills of teachers at the 0.1 quantile. We estimate equation 2 to study whether the change in the profile of teachers hired after the reform varied across the skills distribution. Figure 1b reports the corresponding point es- timates for changes with respect to 2012 to test scores for teachers in the 0.1 and the 0.9 quantiles. The point estimates for the changes in the post-reform years vary from 4.6 to 6.2 percentiles of the ENLACE score and are all statistically significant at the 95 percent level. To evaluate the magnitude of these point estimates, consider that in 2012 new teachers in the 0.1 quantile had ENLACE test scores at the 24th percentile of the national test distribution. The corresponding point estimates for changes to teacher test scores in the 0.9 quantile are small in magnitude (around 1 percentile) and not statistically significant. This lack of an 10 Figure A.2a in the Appendix presents the raw yearly means, and Table A.1 in the Appendix reports the point estimates plotted in Figure 1a. It also presents results from estimations without the controls for the state-level labor market conditions and using yearly changes in these variables instead of levels. Results are consistent across specifications. 9 effect may be because the test scores of incoming teachers at the 0.9 quantile were already high in 2012 (equivalent to the 94th percentile of the overall ENLACE distribution), so there was little room to improve.11 Due to data limitations, we focus our analysis on recent graduates—teachers hired four years after finishing secondary school. Figure A.4 in the Appendix shows that our results are robust to the inclusion of individuals hired five years after finishing secondary school, using 2013 as the baseline year. Unfortunately, we cannot go beyond this extension. 6 Mechanisms Our results suggest that the rule-based personnel policies implemented with the reform had a positive effect on the skills profiles of new teachers. We turn now to evidence that could support a causal relationship and shed light on the mechanisms linking the SPD reform with the improvement in teachers’ skills. 6.1 The Prevalence of Rule-Based Hiring We study first the extent to which the SPD reform effectively increased the prevalence of rule-based hiring in the public education system. Figure 2 reports the share of new teachers hired under the rule-based method. We make three observations. First, rule-based hiring was the most common way to select teachers even before the SPD reform, with 63.6 percent of the teachers hired through a competitive examination in 2012 as a result of the ACE reform. Second, the SPD reform significantly increased the prevalence of rule-based hiring—by 14 percentage points in its first year of implementation and 23 percentage points two years later. (See point estimates in Figure A.5 in the Appendix.) Third, the use of discretionary hiring—albeit to a lesser extent—persisted after the SPD reform.12 6.2 The Skills Gap between Rule-Based and Discretionary Hires The skills gap between rule-based and discretionary hires widened after the SPD reform. Figure 3a plots yearly means of ENLACE scores by hiring method. Three things stand out. First, rule-based hires have, on average, higher ENLACE scores than discretionary hires 11 A similar—though less stark—pattern is found from comparing the 0.25 and 0.75 quantiles—see figures A.2b and A.3 in the Appendix. 12 We might have overestimated the proportion of rule-based hires among total hires. Because we do not observe job offers, we define rule-based hires as individuals who apply to the rule-based process and are hired as new teachers. Our definition might encompass individuals who, despite having participated in a competitive examination, were hired through a discretionary process. 10 starting from the baseline year. Rule-based hires in 2012 have, on average, a 3.4 percentile higher ENLACE score than discretionary hires. The gap increased to 6.7 in 2013 and to 14.5 in 2014—the first year in which the reform was implemented. Second, there seems to be a discontinuity in the skills profile of rule-based hires when the reform is implemented. Third, the skills profile of discretionary hires follows a downward trend. The ENLACE scores of this group decreased on average by 3.2 percentiles per year from 2012 to 2016, and regression estimates for the skills gap between rule-based and discretionary hires by year of hiring (Figure 3b) confirm the widening skills gap between these two groups of teachers. The widening skills gap between rule-based hires and discretionary hires is mainly explained by a downward trend in the skills of discretionary hires and a marked improvement in the skills of rule-based hires between 2013 and 2014. (The point estimate for the difference between these two years is 4.4 percentiles, with p-value of 0.01.) The marked improvement in the skills of rule-based hires between 2012 and 2014 is startling. Figures 3c and 3d present raw yearly percentiles and quantile regression estimates for the skills gap between rule-based and discretionary hires. The gap at the 0.1 quantile increased from 5 percentiles in 2012 to 20 percentiles in 2014. In contrast, the corresponding gap at the 0.9 quantile was 2 percentiles in 2012 and 3 percentiles in 2014.13 Combined with the increased prevalence of rule-based hiring, the widening skills gap between rule-based and discretionary hires and—particularly—its large increase in the year the SPD reform was implemented support the hypothesis that this reform is responsible for the documented improvement in the skills profile of new teachers. The sharp improvement in the skills profile of rule-based hires—but not discretionary hires—in 2014 is evidence against the concern that the improvement in the skills profile of new teachers was the result of a secular trend in the profile of individuals who self-select into the teaching profession (which would affect both rule-based and discretionary hires), a matter which we investigate further below. 6.3 Self-Selection and Screening in Rule-Based Hiring Two channels could explain the documented change in the skills gap between rule-based and discretionary hires: changes in self-selection (in the pool of individuals from which new teachers are drawn) and screening. We study the importance of both mechanisms in the rule-based process; we cannot do the same for the discretionary process because we only 13 See OLS and quantile regression estimates for the yearly change in the skills gap with respect to 2012 in Figures A.6a and A.6b in the Appendix; and raw yearly percentiles and quantile regression estimates for the 0.25 and 0.75 quantiles in Figures A.7a and A.7b in the Appendix. 11 observe individuals who were hired, not the full pool from which they are selected. Self-Selection The SPD reform had at most only a modest effect on the skills profile of the pool of individuals who decided to apply to rule-based hiring. Figure 4a shows point estimates for the change with respect to 2013 in the ENLACE score of rule-based applicants. Between 2013 and 2014, there is a small drop in the average ENLACE score of rule-based applicants (point estimate is -1.4 percentiles, with p-value of 0.00). From 2015 on, there are no statistically significant changes with respect to 2013. At the bottom of the skills distribution, there was a modest improvement in the profile of individuals who self-selected into the rule-based process. Figure 4b plots the corresponding quantile regression estimates. For the 0.1 quantile, there is no observed change between 2013 and 2014, and then there is a significant increase of around 1.5 percentiles from 2015 on. For the 0.9 quantile, there are significant decreases of around 1 percentile in 2014 and 2017 with respect to 2013, and no observed change in 2015 and 2016 with respect to 2013.14 To interpret the magnitude of these point estimates, consider that with respect to 2013, the average ENLACE score of rule-based hires in the post-SPD years improved by around 3.4–4 percentiles and by 5–7 percentiles at the 0.1 quantile of the rule-based hires’ distribution (Figure A.9 in the Appendix). This further weakens the concern that the main results could be explained by a secular trend or shock—independent of the reform—to the skills profile of individuals who self-select to teaching. Screening The SPD reform improved the efficiency of rule-based hiring, making cognitive skills more important determinants of hiring outcomes. Figure 5 reports local means of the probability of being hired among rule-based applicants by percentile of the ENLACE score for the peri- ods before and after the SPD reform. As expected, the relationship between the probability of being hired and the ENLACE score changed significantly with the implementation of the SPD reform, making skills a stronger predictor of hiring. (Figure A.10 in the Appendix reports similar results by year to show that this change is not the result of a secular trend.) While ACE was in effect, a one-percentile-point increase in the ENLACE score was associ- ated with an increase of 0.13 percentage points in the probability of being hired (controlling 14 We obtain a similar picture if, instead of focusing only on individuals who finished secondary school four years before applying to a teaching position, we include in the sample those who finished four or five years prior, as shown in Figure A.8 in the Appendix. 12 for state fixed effects, see Table A.3 in the Appendix), whereas during the SPD regime the corresponding increase was of 0.43 percentage points. Although an applicant with an EN- LACE score at the 10th percentile had the same (unconditional) probability of being hired in both the ACE and SPD examinations (of 10.9 percent), an applicant with an ENLACE score at the 90th percentile had a 20.9 percent chance of being hired during the ACE examination and a 43.5 percent chance of being hired during the SPD one. The SPD regime was more successful than its rule-based predecessor, ACE, in making hiring outcomes more sensitive to applicants’ cognitive skills. Both regimes used standardized tests to screen applicants. However, the institutions responsible and the length and contents of the tests differed. Figure 6 plots local means of the probability of scoring in the top quartile (half) of the SPD (ACE) entry examination by the applicant’s ENLACE score, and Table A.4 in the Appendix presents the corresponding regression estimates—controlling for state and type of teaching position fixed effects. In line with the previous results, the SPD examination strengthened the relationship between screening and cognitive skills. For example, an ACE applicant with an ENLACE score at the 50th percentile had an (unconditional) probability of scoring at the top quartile (half) of the entry examination distribution of 9.8 percent (23.6 percent), whereas an SPD applicant with the same ENLACE score had a probability of 21.3 percent (58.0 percent) of scoring at the top quartile (half) of the entry examination. 7 Conclusions This paper shows that the implementation of the SPD reform improved the skills profile of new teachers. This progress was mainly driven by changes in the bottom of the skills distribution of newly hired teachers. Two main channels explain this result. First, the reform decreased the prevalence of discretionary hires, which were drawn disproportionately from the bottom of the skills distribution. Second, the reform improved the screening efficiency of rule-based hiring, making cognitive skills a more important determinant of hiring outcomes. The SPD reform decreased, but did not eradicate, the use of discretionary hiring. This finding had not been documented before in this context, but it echoes evidence from a similar teacher reform in Colombia (Brutti and Torres, 2021). The incomplete reach of these reforms speaks to the challenges of implementing civil service reforms in contexts where institutions might be weak and incentives for non-compliance strong. The skills profile of discretionary hires was already worse than those of rule-based hires and worsened after the SPD reform. This evidence is consistent with the idea that the abuse of discretion in hiring can lead to the selection of lower-skilled individuals because 13 they are the ones who have fewer labor market options and therefore benefit the most from non-meritocratic access to a job (Estrada, 2019). The SPD reform was based on the promise of making merit the key criterion of per- sonnel decisions through the teaching career, in contrast to the accusations often made to the discretionary regime. One could expect that such a change in personnel policy made teaching a more attractive career for higher-skilled individuals, a possibility reinforced by the fact that the reform eliminated application restrictions among university graduates from fields different from the ones exclusively related to teaching (“escuelas normales” as they are known in Mexico). We only find a modest improvement in the skills profile of rule-based applicants. However, we cannot discount the hypothesis that the self-selection effect would have grown in importance in the medium- to long-term (because our post-reform period of analysis only comprises four years). Several countries in Latin America have implemented teacher reforms to make rule- based hiring mandatory. The list includes Brazil, Chile, Colombia, Ecuador, Mexico, and Peru. The results presented in this paper highlight two important dimensions to evaluate the effect of such reforms on teacher quality. One is its overall efficiency vs. the alternative regime. The other is the technical efficiency of the assessment used to screen teachers. The results presented in this paper show how a rule-based civil service system can improve the skills profile of incoming public officials. However, the ambitious reform under analysis faced a political backlash that led to its cancellation. This demonstrates that the political implementation and the generation of a broad coalition of support among public servants and other stakeholders are as important as the technical content of personnel policies for the long-run success of civil service reforms. 14 References Araujo, Maria Daniela, Guido Heineck, and Yyann´ u Cruz-Aguayo, “Does Test- Based Teacher Recruitment Work in the Developing World? Experimental Evidence from Ecuador,” IZA Discussion Papers 13830, Institute of Labor Economics (IZA) October 2020. Bardhan, Pranab, “Decentralization of Governance and Development,” Journal of Eco- nomic Perspectives, December 2002, 16 (4), 185–205. Besley, Timothy J, Robin Burgess, Adnan Khan, and Guo Xu, “Bureaucracy and Development,” Working Paper 29163, National Bureau of Economic Research August 2021. Brassiolo, Pablo, Ricardo Estrada, and Gustavo Fajardo, “My (running) mate, the mayor: Political ties and access to public sector jobs in Ecuador,” Journal of Public Economics, 2020, 191 (C). , , , and Julian Martinez-Correa, “Family Rules: Nepotism in the Mexican Judiciary,” CAF Working Paper #2021/09 October 2021. Brutti, Zelda and Fabio S´anchez Torres, “Turning around teacher quality in Latin America: Renewed confidence and lessons from Colombia,” Economic Analysis and Policy, 2021. Chetty, Raj, John N. Friedman, and Jonah E. Rockoff, “Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood,” American Economic Review, September 2014, 104 (9), 2633–79. Colonnelli, Emanuele, Mounu Prem, and Edoardo Teso, “Patronage and Selection in Public Sector Organizations,” American Economic Review, October 2020, 110 (10), 3071–99. Corcoran, Sean P, William N Evans, and Robert M Schwab, “Changing labor- market opportunities for women and the quality of teachers, 1957-2000,” American Eco- nomic Review, 2004, 94 (2), 230–235. Dahis, Ricardo, Laura Schiavon, and Thiago Scot, “Selecting Top Bureaucrats: Ad- mission Exams and Performance in Brazil,” Working Paper, Available at SSRN October 2020. 15 de Hoyos, Rafael, Ricardo Estrada, and Mar´ ıa Jos´e Vargas, “What do test scores really capture? Evidence from a large-scale student assessment in Mexico,” World Devel- opment, 2021, 146, 105524. Estrada, Ricardo, “Rules versus discretion in public service: Teacher hiring in Mexico,” Journal of Labor Economics, 2019, 37 (2), 545–579. and Maria Lombardi, “Skills and selection into teaching: Evidence from Latin Amer- ica,” CAF Working Paper #2020/06 August 2020. Fredriksson, Peter and Bj¨ ¨ orn Ockert , “The supply of skills to the teacher profession,” Scandinavian Journal of Economics, 2008, 110 (2), 277–296. Grindle, Merilee S., Jobs for the Boys Patronage and the State in Comparative Perspective, Cambridge, MA: Harvard University Press, 2012. Gronqvist, Erik and Jonas Vlachos, “One size fits all? The effects of teachers’ cognitive and social abilities on student achievement,” Labour Economics, 2016, 42, 138 – 150. Hanushek, Eric A, Marc Piopiunik, and Simon Wiederhold, “The value of smarter teachers: International evidence on teacher cognitive skills and student performance,” Journal of Human Resources, 2019, 54 (4), 857–899. Islas, Paul Moch, Anne K. Calef, and Cristina Aparicio, 2013 Mexico’s Education Reform: A Multi-dimensional Analysis, Cham: Springer International Publishing, Mookherjee, Dilip, “Political Decentralization,” Annual Review of Economics, 2015, 7 (1), 231–249. Moreira, Diana and Santiago P´ erez, “Civil Service Reform and Organizational Prac- tices: Evidence from the Pendleton Act,” Working Paper 28665, National Bureau of Eco- nomic Research April 2021. Munoz, Pablo and Mounu Prem, “Managers’ Productivity and Recruitment in the Public Sector: The Case of School Principals,” Technical Report March 2021. Neilson, Christopher, Sebastian Gallegos, and Franco Calle, “Screening and recruit- ing talent at teacher colleges using pre-college academic achievement,” Working Paper 2019. Nickell, Stephen and Glenda Quintini, “The consequences of the decline in public sector pay in Britain: a little bit of evidence,” The Economic Journal, 2002, 112 (477), F107–F118. 16 Ornaghi, Arianna, “Civil service reforms: Evidence from US police departments,” 2019. Rauch, James, “Bureaucracy, Infrastructure, and Economic Growth: Evidence from U.S. Cities During the Progressive Era,” The American Economic Review, 1995, 85 (4), 968— -79. Rivkin, Steven G, Eric A Hanushek, and John F Kain, “Teachers, schools, and academic achievement,” Econometrica, 2005, 73 (2), 417–458. Rockoff, Jonah E, Brian A Jacob, Thomas J Kane, and Douglas O Staiger, “Can you recognize an effective teacher when you recruit one?,” Education Finance and Policy, 2011, 6 (1), 43–74. Santibanez, Lucrecia, “Reforma Educativa: El papel del SNTE,” Revista Mexicana de Investigacion Educativa, April-June 2008, 13 (037), 419–443. Ujhelyi, Gergely, “Civil Service Rules and Policy Choices: Evidence from US State Gov- ernments,” American Economic Journal: Economic Policy, May 2014, 6 (2), 338–80. Voth, Joachim and Guo Xu, “Discretion and Destruction: Promotions, Performance, and Patronage in the Royal Navy,” Mimeo November 2020. Xu, Guo, “The Costs of Patronage: Evidence from the British Empire,” American Eco- nomic Review, November 2018, 108 (11), 3170–98. 17 Figures Figure 1: New teachers: Change in ENLACE scores with respect to 2012 (a) OLS estimates (2012: mean = (b) Quantile regression estimates 64.9) (2012: p10 = 24.3, p90 = 94.1) 5 9 Change in ENLACE score (in percentiles) Change in ENLACE score (in percentiles) 7 3 5 1 1 3 -1 -3 -1 -3 -5 2012 2013 2014 2015 2016 2017 -5 2012 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 Notes : Panel (a) shows the difference in percentiles of the ENLACE score of newly hired teachers with respect to 2012, corresponding to the βπ estimates of equation (1). Panel (b) shows the difference in percentiles of the ENLACE score of newly hired teachers for quantile 0.1 in blue and quantile 0.9 in red with respect to 2012, corresponding to the βπ (0.1) and βπ (0.9) estimates of equation (2). Regressions include state fixed effects, state job market controls, and robust standard errors. Confidence intervals at the 95 percent level are shown in bars. 18 Figure 2: New teachers: Share of Rule-Based hires .85 .8 Share of ruled-based hires .7 .75 .65 .6 2012 2013 2014 2015 2016 2017 Notes : The figure shows the share of rule-based newly hired teachers for each of the sample years. As explained in Section 3.1, we define rule-based hires as individuals who apply to the rule-based process (ACE or SPD) and are hired as teachers the following school year. 19 Figure 3: New teachers: ENLACE scores of Rule-Based vs. Discretionary Hires (a) Means (b) OLS estimates: yearly gaps 75 20 ENLACE Score (in percentiles) Gap in ENLACE Score (in percentiles) 70 15 60 65 10 55 5 50 2012 2013 2014 2015 2016 2017 Discretionary 0 Ruled-based 2012 2013 2014 2015 2016 2017 (d) Quantile regression estimates: (c) Percentiles yearly gaps 100 30 ENLACE Score (in percentiles) 80 Gap in ENLACE Score (in percentiles) 60 20 Notes : 40 10 20 2012 2013 2014 2015 2016 2017 Discretionary 0.9 Ruled-Based 0.9 0 Discretionary 0.1 2012 2013 2014 2015 2016 2017 Ruled-Based 0.1 quantile 0.1 quantile 0.9 Panel (a) shows the ENLACE score in percentile means for new rule-based and discretionary hires for each year in the sample. Panel (b) shows the estimates of the annual gaps in ENLACE scores in percentiles between new rule-based and discretionary hires; regressions include state fixed effects and robust standard errors. Panel (c) shows the ENLACE score in percentiles at quantiles 0.1 and 0.9 for new rule-based and discretionary hires for each year in the sample. Panel (d) shows estimates of the annual gaps in ENLACE score in percentiles at quantiles 0.1 and 0.9 between new rule-based and discretionary hires; regressions include state fixed effects and robust standard errors. For Panels (b) and (d), confidence intervals at the 95 percent level are shown in bars. 20 Figure 4: Rule-Based applicants: Change in ENLACE scores with respect to 2013 (a) OLS estimates (2013: mean = (b) Quantile regression estimates 59.0 ) (2013: p10 = 19.6, p90 = 92.5) 3 Change in ENLACE Score (in percentiles) 5 Change in ENLACE Score (in percentiles) 1 1 3 -1 -1 -3 2013 2014 2015 2016 2017 -3 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 Notes : Panel (a) shows the difference in percentiles of the ENLACE score of rule-based applicants with respect to 2013, corresponding to the βπ estimates of equation (1). Panel (b) shows the difference in percentiles of the ENLACE score of rule-based applicants for quantile 0.1 in blue and quantile 0.9 in red with respect to 2013 corresponding to the βπ (0.1) and βπ (0.9) estimates of equation (2). Regressions include state fixed effects and robust standard errors. Bars show confidence intervals at the 95 percent level. 21 Figure 5: Rule-Based applicants: Probability of being hired by ENLACE score .6 .4 Share of hires .2 0 0 20 40 60 80 100 ENLACE score (in percentiles) ACE ACE (linear fit) SPD SPD (linear fit) Notes : The figure shows probability of being hired by each percentile in ENLACE score for ACE and SPD rule-based applicants. A linear fit is included for each system. 22 Figure 6: Rule-based applicants: Probability of obtaining a high entry examination score by ENLACE score 1 .8 Probability .4 .2 0 .6 0 20 40 60 80 100 ENLACE score (in percentiles) Top quartile-SPD Top quartile-SPD (quadratic fit) Top half-SPD Top half-SPD (linear fit) Top quartile-ACE Top quartile-ACE (quadratic fit) Top half-ACE Top half-ACE (linear fit) Notes : The figure shows the probability of scoring in the top 25 percent and top 53 percent of the ACE and SPD entrance exams for each of percentiles in ENLACE score. For the probability of scoring in the top 53 percent in any of the systems a linear fit was included, in the other hand, for the probability of scoring in the top 25 percent in any of the systems a quadratic fit was included. 23 Tables Table 1: Teacher hiring reforms: ACE and SPD characteristics Characteristics ACE SPD Institutions Regulations National Committee with represen- National Institute for the Evaluation in charge of and oversight tatives from the Federal Ministry of Education and Federal Ministry of competitive of Education and teachers’ union Education. examinations (Comisi´ on Nacional Rectora). Implementation state committees with representa- Federal and state ministries of edu- tives from state Ministry of Educa- cation. tion and teachers’ union (Comit´ es Estatales de Seguimiento). Exam’s content(*) 1. Course content; 1. Course content; 2. Teaching skills; 2. Teaching skills; 3. Intellectual skills; 3. Intellectual skills; 4. Regulations, ethics and school 4. Regulations, ethics and school management. management. Exam’s length (number of questions) 80 175-240(**) Hiring criteria Weighted average of test score in Test score in standardised entry standardised entry exam and (op- exam. tionally, if chosen by the State Min- istry of Education) bachelor’s GPA. Assignment of teachers to schools Responsibility of state committees Applicants can choose among avail- with representatives from state Min- able vacancies in order of their rank- istry of Education and teachers’ ing established for hiring decisions. union. Eligibility criteria Bachelor degree from authorised Bachelor degree from any higher ed- public and private teacher training ucation institution. Specialization institutions. Specialization varies varies according to the position. according to the position. Notes : (*) In both cases, exams were specific for each type of position (primary teacher, secondary math teacher, etc). ACE comprised only one exam, while SPD was divided into two exams (only in the case of teachers applying to fill in a vacancy for indigenous languages took a third exam). (**) According to the 2014–15 Exam Study Guide, the total number of questions amounted to 175. Subsequent guides indicated a total of 240 questions. 24 Table 2: New teachers and rule-based applicants: Descriptive statistics Panel A - New Teachers 2012-2017 All Recent graduates (4 years) Recent graduates (4 & 5 years) (1) (2) (3) Mean 0.649 0.691 0.686 Female SD 0.477 0.462 0.464 Mean 29.529 22.283 22.664 Age SD 8.864 0.749 0.918 Mean 67.342 64.508 ENLACE Score (in percentiles) SD 25.012 25.634 Mean 0.457 0.784 0.732 Rule-based hire SD 0.498 0.411 0.443 Observations 181,590 24,914 39,615 Panel B - Rule-based applicants 2013-2017 All Recent graduates (4 years) Recent graduates (4 & 5 years) (1) (2) (3) Mean 0.692 0.764 0.752 Female SD 0.461 0.424 0.432 Mean 29.147 22.294 22.611 Age SD 7.138 0.766 0.927 Mean 61.547 59.594 ENLACE score (in percentiles) SD 26.570 26.834 Mean 0.241 0.377 0.346 Top-quartile score in entry examination SD 0.415 0.484 0.476 Mean 0.505 0.659 0.625 Top-half score in entry examination SD 0.498 0.474 0.484 Mean 0.136 0.300 0.282 Rule-based hire SD 0.343 0.458 0.445 Observations 468,846 55,660 79,530 Notes : Panel A shows summary statistics for new teachers in the period 2012–17. Column (1) shows statistics for all new teachers, including those we cannot merge with ENLACE. Column (2) shows statistics for recent graduates who entered the system 4 years after graduating from secondary school. Column (3) shows statistics for recent graduates who entered the system 4 or 5 years after graduating from secondary school. Panel B shows summary statistics for rule-based applicants for the period 2013–17. Column (1) shows statistics for all rule-based applicants, including those we cannot merge with ENLACE. Column (2) shows statistics for recent graduates who approached the rule-based system 4 years after graduating from secondary school. Column (3) shows statistics for recent graduates who approached the rule-based system 4 or 5 years after graduating from secondary school. 25 Appendix. Additional Figures and Tables Figures Figure A.1: Timeline and data sources ACE SPD reform SPD reform is canceled First ACE competitive Last ENLACE test grade First SPD competitive examination 12 examination 2008 2009 2011 2012 2014 2015 2017 2018 2007 2010 2013 2016 2019 First ENLACE test grade 12 BEFORE AFTER Dataset used to identify new teachers: RENAME CEM FONE Notes : This timeline illustrates the significant changes in teacher personnel policies that the Mexican education system experienced over 2007–19. The figure also shows the data sources used to identify the newly hired teachers (RENAME, CEM, and FONE), the competitive examinations (ACE and SPD), and the ENLACE 12th grade scores. 1 Figure A.2: New teachers: ENLACE score (a) Mean (b) Percentiles 100 70 ENLACE Score (in percentiles) 80 68 ENLACE score (in percentiles) 60 66 Notes : 40 64 20 2012 2013 2014 2015 2016 2017 62 Quantile 0.25 Quantile 0.75 Qunatile 0.1 60 2012 2013 2014 2015 2016 2017 Qunatile 0.90 Panel (a) shows the mean percentile ENLACE scores for newly hired teachers for each year in the sample. Panel (b) shows the ENLACE score in percentiles at quantiles 0.1, 0.25, 0.75 and 0.9 for newly hired teachers for each year in the sample. 2 Figure A.3: Quantile regression estimates 9 Change in ENLACE score (in percentiles) -3 -1 1 3-5 5 7 2012 2013 2014 2015 2016 2017 quantile 0.25 quantile 0.75 Notes : The figure shows the difference in percentiles of the ENLACE score of newly hired teachers for quantile 0.25 in blue and quantile 0.75 in red with respect to 2012 corresponding to the βπ (0.25) and βπ (0.9)estimates of equation (2). Regressions include state fixed effects and state job market controls and robust standard errors. Confidence intervals at the 95 percent level are shown in bars. 3 Figure A.4: New teachers (4 & 5 years after graduating from secondary school): Change in ENLACE scores with respect to 2013 (a) OLS estimates (b) Quantile regression estimates 9 5 Change in ENLACE score (in percentiles) Change in ENLACE score (in percentiles) 7 3 3 5 1 1 -1 -3 -1 -3 -5 2013 2013 2015 2016 2017 -5 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 Notes : The figure (a) shows the difference in percentiles of the ENLACE score of newly hired teachers with respect to 2013, corresponding to the βπ estimates of equation (1). The figure (b) shows the difference in percentiles of the ENLACE score of newly hired teachers for quantile 0.1 in blue and quantile 0.9 in red with respect to 2013 corresponding to the βπ (0.9) and βπ (0.1) estimates of equation (2). Regressions include state fixed effects and state job market controls and robust standard errors. Confidence intervals at the 95 percent level are shown in bars. 4 Figure A.5: OLS estimates: change in share of rule-based hires (2012: mean = 67.7 ) .25 .2 Share of ruled-base hires .1 .05 0 .15 2012 2013 2014 2015 2016 2017 Notes : The figure shows the difference in the share of rule-based hires within the newly hired teachers with respect to 2012, corresponding to the βπ estimates of equation (1). Regressions include state fixed effects, state job market controls, and robust standard errors. Confidence intervals at the 95 percent level are shown in bars. 5 Figure A.6: Rule-based - Discretionary hires skills gap: Change with respect to 2012 (a) OLS estimates (b) Quantile regression estimates 30 15 Change in ENLACE Score (in percentiles) Change in ENLACE Score (in percentiles) 20 10 10 5 0 0 -10 -5 2012 2013 2014 2015 2016 2017 12 13 14 15 16 17 quantile 0.1 quantile 0.9 20 20 20 20 20 20 Notes : Panel (a) shows the estimates of the skills gap in percentiles of the ENLACE score of rule-based over discretionary hires with respect to 2012. Panel (b) shows the estimates of the skills gap in percentiles of the ENLACE score of rule-based over discretionary hires with respect to 2012 for quantiles 0.1 and 0.9. Regressions include state fixed effects and state job market controls and robust standard errors. Bars show confidence intervals at the 95 percent level. Figure A.7: Rule-based vs. discretionary hires: ENLACE score (b) Quantile regression estimates: (a) 25th and 75th percentiles yearly gaps 100 40 ENLACE Score (in percentiles) Gap in ENLACE Score (in percentiles) 80 30 60 Notes : 20 40 20 10 2012 2013 2014 2015 2016 2017 Discretionary 0.25 Ruled-Based 0.25 0 Discretionary 0.75 2012 2013 2014 2015 2016 2017 Ruled-Based 0.75 quantile 0.25 quantile 0.75 Panel (a) shows the ENLACE score in percentiles at quantiles 0.25 and 0.75 for rule-based and discretionary hires for each year in the sample. Panel (b) shows the estimates of the annual gaps in ENLACE score in percentiles at quantiles 0.25 and 0.75 between rule-based and discretionary hires; regressions include state fixed effects and robust standard errors. For Panel (b), bars show confidence intervals at the 95 percent level. 6 Figure A.8: Rule-based applicants (4 & 5 years): Change in ENLACE scores with respect to 2013 (a) OLS estimates (b) Quantile regression estimates 3 Change in ENLACE Score (in percentiles) 5 Change in ENLACE Score (in percentiles) 1 1 3 -1 -1 -3 2013 2014 2015 2016 2017 -3 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 Notes : The figure (a) shows the difference in percentiles of the ENLACE score of rule-based applicants (4 & 5 years) with respect to 2013, corresponding to the βπ estimates of equation (1). The figure (b) shows the difference in percentiles of the ENLACE score of rule-based applicants (4 & 5 years) for quantile 0.1 in blue and quantile 0.9 in red with respect to 2013 corresponding to the βπ (0.9) and βπ (0.1) estimates of equation (2). Regressions include state fixed effects and robust standard errors. Confidence intervals at the 95 percent level are shown in bars. 7 Figure A.9: Rule-based hires: Change in ENLACE score with respect to 2012 and 2013 (a) OLS estimates with respect to (b) Quartile regression estimates 2012 respect to 2012 -3 -1 1 3 5 7 9 11 13 15 Change in ENLACE score (in percentiles) 7 Change in ENLACE score (in percentiles) -3 -1 1 3 5 -5 2012 2013 2014 2015 2016 2017 -5 2012 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 (d) Quartile regression estimates with (c) OLS estimates respect to 2013 respect to 2013 9 -5 -3 -1 1 3 5 7 9 11 13 15 17 Change in ENLACE score (in percentiles) Change in ENLACE score (in percentiles) -3 -1 1 3 -5 5 7 2013 2014 2015 2016 2017 2013 2014 2015 2016 2017 quantile 0.1 quantile 0.9 Notes : Panel (a) shows the difference in percentiles of the ENLACE score of rule-based newly hired teachers with respect to 2012, corresponding to the βπ estimates of equation (1). Panel (b) shows the difference in percentiles of the ENLACE score of rule-based newly hired teachers for quantile 0.1 in blue and quantile 0.9 in red with respect to 2012, corresponding to the βπ (0.1) and βπ (0.9) estimates of equation (2). Panel (c) shows the difference in percentiles of the ENLACE score of rule-based newly hired teachers with respect to 2013, corresponding to the βπ estimates of equation (1). Panel (d) shows the difference in percentiles of the ENLACE score of rule-based newly hired teachers for quantile 0.1 in blue and quantile 0.9 in red with respect to 2013, corresponding to the βπ (0.1) and βπ (0.9) estimates of equation (2) Regressions include state fixed effects, state job market controls and robust standard errors. Bars show confidence intervals at the 95 percent level. 8 Figure A.10: Rule-based applicants: Probability of being hired by ENLACE percentile and year .6 Share of hires .2 0 .4 0 20 40 60 80 100 ENLACE score (in percentiles) ACE ACE (linear fit) SPD - 2014 SPD - 2014 (linear fit) SPD - 2015 SPD - 2015 (linear fit) SPD - 2016 SPD - 2016 (linear fit) SPD - 2017 SPD - 2017 (linear fit) Notes : The figure shows probability of being hired by each percentile in ENLACE score for ACE and SPD (yearly). A linear fit is included. 9 Tables Table A.1: OLS estimates: New teachers: Change in ENLACE score with respect to 2012 (1) (2) (3) 2013 -0.800 -0.781 -1.004 (0.758) (0.761) (0.776) 2014 2.835*** 2.969*** 2.697*** (0.619) (0.631) (0.629) 2015 3.131*** 3.516*** 3.170*** (0.585) (0.644) (0.586) 2016 2.146*** 2.645*** 2.179*** (0.580) (0.696) (0.588) 2017 3.609*** 4.492*** 3.835*** (0.595) (0.867) (0.618) No. of obs. 24,904 24,904 24,904 State FE Yes Yes Yes Job market controls No Yes No Job market lagged variation controls No No Yes Notes : The table shows estimates of the βπ coefficients for equation (1), each column with a different set of controls. The dependent variable is the ENLACE score in percentiles. New teachers incoming in the cycle 2012–13 are the excluded category. Job market controls include the state unemployment rate and the average monthly income of workers with at least 13 years of education in sectors other than education services. Job market variation controls are the annual percentage changes of the job market controls. Robust standard errors * p < 0.10, ** p < 0.05, *** p < 0.01. 10 Table A.2: Quantile regression estimates: New teachers: Change in ENLACE score with respect to 2012 (q 0.1) (q 0.9) 2013 -0.436 1** (1.878) (0.502) 2014 4.685*** 1.000** (1.370) (0.437) 2015 6.964*** 1.000*** (1.264) (0.388) 2016 5.882*** 1.000** (1.173) (0.389) 2017 6.717*** 1.000** (1.319) (0.409) No. of obs. 24,904 24,904 State FE Yes Yes Job market lagged variation controls Yes Yes Notes : The table shows estimates of the βπ coefficients for equation (2). The first column shows the estimates for quantile 0.1, and the second column shows the estimates for quantile 0.9. The dependent variable is the ENLACE score in percentiles. New teachers incoming in the 2012–13 cycle are the excluded category. The job market variation controls are the annual percentage changes of the state unemployment rate and the average monthly income of workers with at least 13 years of education in sectors other than education services. Robust standard errors * p < 0.10, ** p < 0.05, *** p < 0.01 11 Table A.3: OLS estimates: Rule-based applicants: Probability of being hired by ENLACE score (1) (2) (3) (4) (5) (6) (7) ACE SPD All 2014(SPD) 2015(SPD) 2016(SPD) 2017(SPD) ENLACE Score 0.00127*** 0.00431*** 0.00127*** 0.00412*** 0.00432*** 0.00404*** 0.00476*** (0.000155) (0.0000718) (0.000155) (0.000141) (0.000146) (0.000148) (0.000138) SPD -0.0290*** (0.0107) SPD*ENLACE 0.00308*** (0.000171) No. of obs. 7,635 48,023 55,658 10,828 11,987 12,673 12,535 Notes : This table shows the regression coefficients of the ENLACE score in percentiles as the independent variable and a dummy variable taking the value of 1 if the rule-based applicant was hired and 0 otherwise as the dependent variable. Column (1) shows the results for the ACE rule-based applicants and column (2) for the SPD rule-based applicants, column (3) for all the sample including an interaction of a dummy variable taking the value of 1 for 2014 and after and 0 otherwise and ENLACE score and columns (4) to (7) for each year of the SPD rule-based applicants. Year fixed effects included for columns (1) and (2). Robust standard errors * p < 0.10, ** p < 0.05, *** p < 0.01 12 Table A.4: OLS estimates: Rule-based applicants: Probability of obtaining a high entry examination score by ENLACE score (1) (2) (3) (4) Top quartile SPD Top quartile ACE Top half SPD Top half ACE ENLACE Score 0.00801*** 0.00553*** 0.00822*** 0.00851*** (0.0000635) (0.000165) (0.0000665) (0.000168) No. of obs. 51,642 7,609 51,642 7,609 Notes : This table shows the regression coefficients of the ENLACE score in percentiles as independent variable and a dummy variable taking the value of 1 if the rule-based applicant scored in the Top 25 percent of SPD (column 1), Top 25 percent ACE (column 2), Top 53 percent SPD (column 3), Top 53 percent ACE (column 4) and 0 otherwise as the dependent variable. State fixed effects and a dummy variable taking the value of 1 for those applying for high school positions and 0 otherwise are included as controls. Robust standard errors * p < 0.10, ** p < 0.05, *** 13