Unraveling a Secret: Vietnam's Outstanding Performance on the PISA Test

This paper seeks to find an empirical explanation of Vietnam's outstanding performance on the Programme for International Student Assessment (PISA) in 2012. Only a few developing countries participate in the assessment. Those who do, with the unique exception of Vietnam, are typically clustered at the lower end of the range of the Programme for International student Assessment scores. The paper compares Vietnam's performance with that of a set of seven developing countries from the 2012 assessment's data set, using a cut-off per capita GDP (in 2010 purchasing power parity dollars) of $10,000. The seven developing countries' average performance lags Vietnam's by more than 100 points. The "Vietnam effect" is difficult to unscramble, but the paper is able to explain about half of the gap between Vietnam and the seven countries. The analysis reveals that Vietnamese students may be approaching their studies with higher diligence and discipline, their parents may have higher expectations, and the parents may be following up with teachers regarding those expectations. The teachers themselves may be working in a more disciplined environment, with tabs being kept on their own performance as teachers. Vietnam may also be benefiting from investments in pre-school education and in school infrastructure that are disproportionately higher when compared with Vietnam's per capita income level.


Policy Research Working Paper 7630
This paper is a product of the Education Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at esedmik@ worldbank.org. This paper seeks to find an empirical explanation of Vietnam's outstanding performance on the Programme for International Student Assessment (PISA) in 2012. Only a few developing countries participate in the assessment. Those who do, with the unique exception of Vietnam, are typically clustered at the lower end of the range of the Programme for International student Assessment scores. The paper compares Vietnam's performance with that of a set of seven developing countries from the 2012 assessment's data set, using a cut-off per capita GDP (in 2010 purchasing power parity dollars) of $10,000. The seven developing countries' average performance lags Vietnam's by more than 100 points. The "Vietnam effect" is difficult to unscramble, but the paper is able to explain about half of the gap between Vietnam and the seven countries. The analysis reveals that Vietnamese students may be approaching their studies with higher diligence and discipline, their parents may have higher expectations, and the parents may be following up with teachers regarding those expectations. The teachers themselves may be working in a more disciplined environment, with tabs being kept on their own performance as teachers. Vietnam may also be benefiting from investments in pre-school education and in school infrastructure that are disproportionately higher when compared with Vietnam's per capita income level.

Introduction
Vietnam participated in the Programme for International Student Assessment (PISA) for the first time in 2012 and its performance has been much higher than other developing countries that take part in this OECD led initiative. PISA scores of 15 year-olds in Mathematics, Reading and Science are calibrated to an OECD mean of 500 and standard deviation of 100 points. Only a few developing countries take part in PISA, perhaps because most of them have results much lower than the OECD countries. In the OECD-PISA 2012 database, there are seven countries other than Vietnam with a per capita GDP (in 2010 PPP dollars) below US$ 10,000 -Albania, Colombia, Indonesia, Jordan, Peru, Thailand and Tunisia. At US$ 4,098, Vietnam's GDP per capita is the lowest of this group. Figure 1 indicates a positive, albeit non-linear correlation between GDP per capita and PISA test scores. Vietnam, represented by a red star, lies much above the other developing countries clustered in the lower left hand corner of Figure 1. With a mathematics mean score of 511, Vietnam is more aligned to Finland (519) and Switzerland (531), rather than Peru (368) and Colombia (376). The weighted average mathematics score of the seven developing countries is 383. It is helpful to understand the significance of the 128 point difference of the seven countries as compared with Vietnam. According to a recent OECD publication [OECD, 2013a], "an entire proficiency level in mathematics spans about 70 score points -a large difference in the skills and knowledge students at that level possess. Such a gap represents the equivalent of about two years of schooling in the typical OECD country." Applying this heuristic would imply a nearly 3 year difference in educational attainment between Vietnam and the group of seven developing countries in the PISA database. It should be noted at the outset that crosssection data from one application of PISA does not permit causal inference, but correlations can still provide useful insights. The difference is not only for mathematics and not just in the mean score, but spanning the entire test distribution, as can be seen in Figure 2.

(c) Reading
A range of alternative classifications are possible to organize the explanatory factors available in the OECD-PISA database. Figure 3 presents four sets of factors, starting clockwise from the right. This is admittedly an arbitrary classification, utilized merely for expository purposes as we consider each of the constituent variables in turn. The approach of this paper is as follows. We begin in Section 2 by examining closely the mean differences between Vietnam and the collective group of seven developing countries, termed as "Dev7" for this paper (not to be confused with the G-7 of wealthy countries). Comparing means in this context is a first pass at understanding the performance anomaly of Vietnam on empirical grounds. Do Vietnamese 15 year olds somehow enjoy better cultural, social or civic endowments to balance their economic disadvantages? An examination of mean differences will provide us with a first set of tentative hypotheses.
The insights provided by mean differences need to be explored further by a regression of the test scores on the explanatory variables. Large differences in means may not amount to much if the associated variables are not correlated with test scores. In Section 3 we adopt the regression methodology used by Fryer and Levitt to understand differences in test score results of black children in the first two years of schooling in the United States [Fryer and Levitt, 2004]. Fryer and Levitt are able to explain away all of a 0.62 standard deviation negative achievement gap for black kindergarten children. In our case, we are able to explain about half of a larger 1.28 standard deviaton positive achievement gap for Vietnam compared to Dev7 countries. The lower ability of the Fryer-Levitt method to explain the "Vietnam gap" is probably accounted for by the fact that per capita GDP lower than US $ 10,000 is the only common support across diverse economic, political and educational systems.
The Fryer-Levitt method deepens the understanding from mean comparisons, but what it does not reveal may be as interesting as what it does. Our Fryer-Levitt adaption is based on a pooled regression of eight developing countries, where we follow the fate of the magnitude of the coefficient of the dummy variable representing the Vietnamese students in the sample. However, we also need to investigate structural differences in the effects of endowments between Vietnam and Dev7 countries. In Section 4, we adopt an approach first used to explain variation in PISA performance between Germany and Finland by Andreas Ammermueller [Ammermueller, 2007]. This is an adaptation of the popular Oaxaca-Blinder decomposition of the wage earnings equation to uncover evidence of discrimination on the basis of gender [Blinder, 1973] and [Oaxaca, 1973]. In this section, we examine closely the structural differences between Vietnam and the Dev7 countries, including the contribution of differences in endowments and the coefficients to the gap in test scores.
Even a multi-variate regression approach only proves correlation with nothing more than a hint regarding causation, and so far we have only one year (2012) of PISA data for Vietnam. Even though we cannot uncover causality, there are useful policy related conclusions that we can derive from the analysis presented in this paper. There is a veritable industry of papers regarding Finland's PISA performance, directed mostly toward other OECD countries with lower scores, for instance the United States. Vietnam's superlative performance points to a similar future stream of research, with the added advantage of relevance for developing countries. Section 5 provides concluding ideas that might be among the first of many more such ideas for future investigations of Vietnam's performance.

Endowment Differences
Utilizing the categorization of explanatory factors presented in Figure 3, this section analyzes mean differences in explanatory factors on students, parents, teachers and schools. All variable means presented in the tables are statistically different at the 95% significance level, unless otherwise noted in the footnotes and figures in parentheses represent standard deviations. PISA documentation, especially the technical report - [OECD, 2014a] provides rich definitions and explanations of the variables used. Appendix tables A2, A3 and A4 of this paper accordingly provide references mapping the variables used in this paper and the original PISA variable names. Table 1 begins an exploration of differences in mean values between Vietnamese and Dev7 student characteristics. The absence of differences is sometimes as important as the presence of differences. Table 1 indicates no differences by age or gender of students. The PRESCHOOL variable shows the first instance of a large statistically significant difference. While 78.88% of Dev7 students reported attending pre-school, the number of students attending pre-school from the Vietnam sample was 91.20% -a sizable difference that is both statistically and economically significant. The relationship between pre-school and later educational outcomes has been studied very closely over the years. Longitudinal impact evaluation studies regarding the Perry Pre-school project and Head Start in the US are among the most cited studies in the economics literature 1 . We can also see from the numbers of REPEAT in Table 1 that PISA takers in Vietnam were three times less likely to have repeated a grade in the past (6.79% compared to 19.15%). The findings regarding PRESCHOOL and REPEAT indicate the possible importance of the trajectory of the student prior to high school. Repetition rates are difficult as comparative indicators of system quality because of the variations across countries in curriculum and standards, but REPEAT is another interesting variable to keep in mind as a possible clue to the mystery of Vietnam's PISA performance. As in some other East Asian cultures, Vietnamese parents expect their children to study hard. Though Mark Twain, translated into Vietnamese, is quite a best seller for young readers in Vietnam, truancy from school is not perceived benevolently by parents. 2 Table 1 indicates a consistently lower truancy rate for the three variables used. The question refers to the past two complete weeks of school and we can see that Vietnamese students are less likely to have been late for school, have fewer days of unexcused absence and skip fewer classes. 3

Student Characteristics
The final set of variables in Table 1 concerns parental background and wealth at the students' home, including cultural resources and books at home which may work to stimulate cognitive development. The PISA database includes a number of indices to measure aspects such as wealth. These indices are based on underlying data regarding occupations and possessions. The scaling of raw data to indices is described in detail in the PISA technical report [OECD, 2014a]. For HISEI, which describes parental occupation status, the OECD mean is 50 and the OECD standard deviation is 15. Table 1 shows that HISEI for Dev7 parents stands at 40.42 and is thus much higher than 26.60 for Vietnamese parents. MISCED refers to the International Standard Classification of Education (ISCED) developed by UNESCO. Table 1 shows that the average level of mother's education (MISCED) for Dev7 was just over 3, meaning Upper Secondary education, while for Vietnam the mean was just over 2, meaning Lower Secondary education. The WEALTH index is set for an OECD mean of zero and standard deviation of 1. Dev 7 countries wealth level was -1.5 and Vietnam's was -2.1, which is consistent with the data regarding occupational classification and mother's education. These findings indicate the close correlation of these variables with GDP per capita. Another interesting finding concerns the indices CULTPOS, cultural possessions and HEDRES, educational resources at home which have an OECD mean 0 and a standard deviation 1, as well as BOOK N, the number of books in family home. CULTPOS includes classical literature, books of poetry and works of art. HEDRES includes reference books and books to help with school work as well as a study desk and "a quiet place to study". These three variables are also in line with per capita income -with the Dev7 mean being lower than the OECD mean, and Vietnam being lower than the Dev7 mean. One explanation regarding Vietnam's PISA performance can probably be ruled out -it does not seem likely that Vietnamese households spend a disproportionately higher amount of their income on acquiring possessions such as books and other objects that would give their children an edge in life.
indicates that Vietnam is a positive outlier on discipline and authority orientation [Dalton and Ong, 2005]. 3 In the student's questionnaire, there is a telling question -student's have to agree or disagree on a four point Likert scale to the statement "If I had different teachers, I would try harder at school.". Converted into an index, the mean for Vietnam at 0.363 is lower than that for Dev7 at 0.525. This suggests a tendency in Vietnamese students for greater self-responsibility.

Student Effort
The phenomenon of primary and high school children taking extra classes to supplement in-school instruction in Vietnam is well known, see [Ha and Harpham, 2005] and [Dang, 2007]. Table 2 indicates that while Dev7 students spent roughly 4.7 hours in such classes (total of OUTMATH, OUTLANG and OUTSCIE), the Vietnamese student spends nearly 2 hours more for a total of 6.6 hours per week in such classes, with the difference being highest for OUTMATH. Vietnamese students also spent about 1 additional hour per week doing homework (total of ST57Q01 and ST57Q02) compared to Dev7 students. The highest difference in this set of variables concerns the variable ST57Q04, which relates to extra classes taught by a commercial company. While most of the schools in Vietnam are public or government schools, it is interesting to note that students report nearly 5 hours of commercially provided extra lessons, while the total for Dev7 countries is only about 2 hours per week. Collectively, these variables indicate that Vietnamese students spent about 16 hours per week studying outside of school, compared to 13 hours per week for Dev7 students.

Student Attitudes
PISA applications in each test round have a focus on one of the subjects and in PISA 2012 the focus subject was mathematics. Mathematics happens to be the subject where the mean score difference is highest between Vietnam and Dev7 countries. The PISA questionnaire for students includes a very interesting series of questions regarding students' perceptions of their abilities, their effort and their reported practices. The details of these questions can be found in the PISA technical report [OECD, 2014a]. Typically, each question includes a set of Likert scaled items to which the student provides a discrete response on a four point agree-disagree scale. These responses are then combined under specified algorithms to provide an index value. For instance, MATWKETH, is meant to measure a student's "mathematics work ethic". Students either agree or disagree with a set of 9 items on a 4 point likert scale -strongly disagree, disagree, agree and strongly disagree. The items include items such as "I work hard on my mathematics homework", and "I listen in mathematics class", "I keep my mathematics work well organized". In the case of MATWKETH, when a student agrees/strongly agrees with a positive statement, or disagrees/strongly disagrees with a negative statement, he or she would tend to be deemed to have a stronger work ethic towards mathematics. The raw data from the Likert scale is converted into an index using IRT scaling procedures, so that the mean for OECD countries is 0 and the standard deviation is 1. Table 3 indicates a most interesting finding regarding a range of such indices from the PISA database. The upper panel in Table 3 indicates a set of indices for which the scores of Vietnamese students are lower than the scores of Dev7 students. For example, the score for MATWKETH is 0.45 for Dev7 and 0 for Vietnam. The variable SUBNORM is supposed to measure subjective norms regarding mathematics. This construct relates to a student's perceptions regarding how other people in the student's life value mathematics. It includes items such as "my friends enjoy taking mathematics tests" and "my parents believe it's important for me to study mathematics." Presumably, when this measure is high, the student has a high subjective norm for mathematics. Table 3 shows that the resulting mean for Dev7 countries is 0.72 and the corresponding value for Vietnam is -0.09. The index SCMAT includes items such as "I learn mathematics quickly" and "I have always believed that mathematics is one of my best subjects". Vietnamese students, who scored more than 1 standard deviation above the Dev7 students on the PISA math test, scored half a standard deviation lower on SCMAT. What is going on here?
This mini-mystery within the overall mystery of Vietnam's PISA performance can possibly be resolved by looking at some further indices. The lower panel of Table 3 reports on indices where the balance tips to the other side -these are indices where Vietnamese students have a higher mean value than Dev7 students. These three indices bear close examination. PERSEV consists of items that purport to capture perseverance with a task or a problem to resolve; ANXMAT is a negative index (less is better) that deals with mathematics anxiety (for example, an item included in this index states that "I get very nervous doing mathematics problems"); MATINTFC relates to future mathematics intention, including items such as "I am planning on majoring in a subject in college that requires lots of mathematics".
One possible explanation, as indicated in the heading of the Table 3 panels, is that Vietnamese students are brought up in a culture that stresses the importance of modesty and humility as a pathway to learning. They may find it difficult to say great things about themselves, because of cultural norms against bragging or boasting. The lower panel in Table  3, on the other hand includes items that are less prone for an immodest interpretation. To say that you are not afraid of mathematics may not be perceived as bragging. In this context, the Vietnamese students are less anxious and more confident about the future role of mathematics in their life. 4

Mathematics Curriculum
In addition to beliefs and perceptions of students regarding mathematics in general, PISA also seeks to closely investigate the issues related to the content of mathematics instructions. PISA incorporates a very interesting approach to avoid or minimize the bragging or overclaiming problem referred to in the previous sub-section. The index FAMCON is constructed out of a response to a question about mathematical concepts for which students are asked "How familiar are you with the following items?" The list of items includes items such as 'Linear Equation', 'Quadratic Function' and 'Cosine.' The list of items also includes three nonsensical items or pseudo-concepts that sound fancy: 'Proper Number','Subjunctive Scaling' and 'Declarative Fraction'. These items are termed as "FOIL", and are used as trick items to calibrate the response for over-claiming on part of the students. The index without correction is presented as FAMCON, and the index with correction is presented as FAMCONC. It is quite fascinating that with FAMCON, the "uncorrected" version, Dev7 students come out apparently better than Vietnam students, with a mean value of 0.26 as compared to 0.12. Unfortunately, this also included familiarity with non-existent items like 'subjunctive scaling' -or bragging. With the corrected version, FAMCONC, the Vietnamese students turn out to do much better, with a mean value of 0.43 as compared -0.54 for Dev7, as can be seen in Table 4.

Parental Support at School
The publication of the bestselling book [Chua, 2011] "Battle Hymn of the Tiger Mother" in 2011 ignited a firestorm of controversy. The book gave prominence in popular culture to a vast academic literature regarding parenting styles and the perceived higher performance of children from Asian immigrant families in the US and other Western countries. One of the ways that parents influence their children's educational outcome is through the interaction that parents have with their child's teachers and others at school. The PISA data includes a question that tries to examine parental expectations towards schools. The question SC24 includes a statement "There is constant pressure from many parents, who expect our school to set very high academic standards and to have our students achieve them." 6 Table 5 indicates a higher level of PARPRESSURE (an index derived from SC24) for Vietnam, compared to Dev7. Another question (SC25) asks school principals about the proportion of parents that take part in a set of 12 activities. While the question does not specify which parent (or both) may be involved, the variables, that may contain more than one of these activities, have been named after the mother for ease of exposition. TIGERMOM refers to the reported proportion of parents who discussed their child's behavior or the child's progress "on their own initiative", to differentiate from cases where parents might have done so following the initiative of the teacher, termed as DUTYMOM. Table 4 shows a slightly higher number on DUTYMOM for Vietnamese parents compared to Dev7, but a greater difference, more than ten percentage points for TIGERMOM. VOLU-MOM refers to parents volunteering in various non-academic activities, such as field trips or carpentry and yard work. Vietnamese parents appear to have a slight advantage with regard to VOLUMOM, yet a much higher one when considering TEACHMOM, which refers to parents volunteering as assistants to the teacher -38.28% compared to 12.18% for Dev7. Vietnamese parents also appear to be much more active in fund raising, looking at FUND-MOM, though they may have less formal influence through school committees.

Teacher Characteristics
Conventional measures regarding student-teacher ratios and teacher certification show some advantage for Vietnam over Dev7 as shown in Table 6. The overall student-teacher ratio is not much different for Vietnam and Dev7 and stands at roughly 20 students per teacher. However, there are more specialized mathematics teachers per student in Vietnam, as shown by the values for SMRATIO (121 in Vietnam compared to 188 for Dev7). There is a higher percentage of certified teachers in Vietnam and higher reported professional development in mathematics (SC35Q02). A very interesting variable from a policy point of view regards the incentives for teachers. School principals were asked to what extent performance appraisal or other forms of feedback are related to incentives for teachers in seven different forms, from salary and bonus to public recognition and greater job responsibilities. The answers were to be given on a 4 point scale: 'No change', 'A small change', 'A moderate change' and 'A large change'. We converted the rating into a Rasch index, scaled to an OECD mean of 0 and standard deviation of 1. The mean for Dev7 for this index, TCH INCENTV was -0.03 for Dev7 and 0.27 for Vietnam, indicating greater presence of teacher incentives in Vietnam. The final set of variables in Table 6 deal with the way that quality assurance regarding teacher performance is carried out, with help of a mentor, peer, supervisor or external inspector. These variables indicate a higher prevalence of oversight for teachers in Vietnam, with the difference being greatest for external inspections (86.64% in Vietnam compared to 58.82% in Dev7 countries).

Pedagogical Practices
Pedagogical practices are an outcome of a complex interaction between curriculum and related educational policies, economic possibilities and the cultural and historical context. It is difficult to trace differences in these practices in a quantitative survey. 7 Table 7 presents a few variables that seek to capture variation in pedagogical practices. They indicate the higher prevalence of national policies in Vietnam regarding the use of computers in the classroom and the use of a standardized curriculum that specifies what has to be taught each month. There is no difference with regard to the use of a single textbook. There is some difference in the use of formative student assessment, with slightly higher percentage of use of assessments to monitor teachers and schools in Vietnam. COGACT represents an OECD-PISA index variable based on response to student reports regarding classroom practices such as teachers requiring students to reflect on a problem or develop new procedures rather than rely on common practices. This variable shows a much lower level of cognitive activation in Vietnam (-0.33) compared to 0.30 for Dev7. In the final set of classroom management variables, an interesting variation can be seen in DISCLIMA, an index variable that measures disciplinary climate in class, and is higher for Vietnam (0.38) than Dev7 (-0.02).  Table 8 indicates interesting basic differences between Vietnam and Dev7 school characteristics. Vietnamese schools are about half as likely to be private schools (8% compared to 17%) and less dependent on funding from student fees; in Vietnam, student fees account for 17% of the school's financing, compared to 26% on average for Dev7. One very useful comparison comes from a question regarding the geographic location of the high school. The percentage of schools reported in a VILLAGE (defined in PISA by population below 3,000 inhabitants), was 46% of high schools in Vietnam compared to 14% of High schools in Dev7 countries. With CITY, defined by a population above 100,000 inhabitants, we find only 23% Vietnamese schools in cities, compared to 41% of high schools located in cities for Dev7 countries. The average class size in Vietnam is higher, with 43 students compared to 35 students in Dev7 countries, and the schools in Vietnam are bigger, with average enrollment of 1,303 students compared to 1,057 in Dev7. There is also a slightly higher percentage of girls in Vietnamese schools.

School Resources
The comparison of Vietnam and Dev7 regarding school resources may be showing that Vietnam makes a deeper effective investment in education (Table 9). Schools in Vietnam have a lower number of computers per student (0.22) compared to a Dev7 (0.39). However, the ratio of computers connected to the Internet is slightly higher in Vietnam (78% compared to 76%). Indices on quality of school educational resources (SCMATEDU) show Vietnam with -0.4941 value and Dev7 with -0.8145 value, and similar higher Vietnam level exists for quality of physical infrastructure at the school (SCMATBUI). There is also a higher proportion of schools that offer additional math classes. These differences indicate that Vietnam has made it a priority to invest in Basic Education that compensates to some extent for its income disadvantage compared to the Dev7. With regard to extra-curricular activities; there is a mixed picture. Not all extra-curricular activities are shown in Table 9, but some indicate lower prevalence in Vietnam compared to Dev7 -for instance school band and math club (not shown, with similar pattern are chess club, IT club, art club). Some activities have higher prevalence in Vietnam -school play/musical, mathematics competition, and sports (not shown here). It would appear that even for extra-curricular activities, the prevalence of activities that require greater effort or competition are more prevalent in Vietnam compared to Dev7. With regard to school leadership and autonomy, there appears to be less autonomy and more accountability in Vietnam. The index variable SCHAUTON indicates a Dev7 mean value of -0.2542, higher than the Vietnam mean value of -1.0419 (recall that indices are set to OECD mean of zero). Teachers in Vietnam have lower chances to participate in school management -TCHPARTI indicates a Dev7 mean value of -0.2169 compared to 1.6445 for Vietnam. Principals in Dev7 are more likely to say that they communicate and act on school goals (LEADCOM), but there is much higher prevalence of public posting of school achievement data (SCORE PUBLIC) in Vietnam. Interestingly, even Dev7 countries have high levels of achievement tracking data by authorities (80% of schools report this -SCORE AUTHRITS). Finally with regard to the school climate, indices described further in the PISA documentation, STUDCLIM (student climate) is roughly even between Vietnam and Dev7, but TEACCLIM (teacher climate), that includes variables such as teacher absenteeism and teacher expectations of students, is higher for Vietnam.

Preliminary conclusions from comparison of endowments
In summary, the mean comparisons between Vietnam and Dev7 students finds a number of potentially insightful results. Consider the four-fold classification of factors presented in the conceptual diagram of Figure 3 -students, parents, teachers and the school, the findings are summarized below.
Students: Students in Vietnam are more likely to have attended pre-school and less likely to have repeated grades in the past. They are likely to behave more disciplined at school, skip fewer classes, and assume greater responsibility for their own learning. Vietnamese students are less likely to brag about their abilities and experience and yet work harder, especially out of school, in extra classes. They tend to have lower anxiety about mathematics and higher confidence about the usefulness of mathematics in their future.
Parents: Parents in Vietnam are likely to be more involved in the school life of their children than parents of students in Dev7 countries. Though time spent on homework help is similar in both groups, Vietnamese parents are more likely to volunteer and take part in fund-raising for the school and help the teachers as classroom assistants. Vietnamese parents are also more likely to seek to meet the teacher to discuss their child's progress or the child's behavior on their own initiative. Principals in Vietnam report higher levels of parental pressure.
Teachers: Teachers have similar levels of formal education in both groups, but Vietnamese teachers may have had more recent professional development activities. There are more specialist mathematics teachers at high schools in Vietnam, and teachers overall are also more likely to be certified. The performance of teachers is more likely to be monitored in Vietnam, with higher emphasis on student achievement and on making information about that achievement public. Teachers also tend to have lower autonomy, more likely to be sub-ject to centralized policies and work in an environment with higher prevalence of incentives for performance. Principals report fewer problems with regard to teacher absenteeism, which squares with an explanation about a Confucian heritage culture.
Schools: Vietnam has a much lower level of economic development compared to the Dev7 countries, which is reflected in lower levels of educational attainment of parents and lower level of home possessions, including so called cultural possessions such as artwork and books. Also, comparatively more Vietnamese students go to school in villages and small towns, reflecting the national population distribution. Yet, two things are striking about schools -although schools have fewer computers compared to Dev7 countries, these computers are as likely as Dev7 countries to be connected to the internet. Also, indices regarding quality of school infrastructure and school educational resources are less deficient in Vietnam compared to Dev7, which is indicative of substantive investments in schools in the past few decades.
Overall, across these four domains of information, it seems likely that the PISA data set is able to detect significant cultural differences between Vietnam and Dev7 countries. There appears to be some influence of policy, looking at student achievement assessment and teacher incentives, and higher levels of centralized controls, but the effectiveness of such policies is also likely tied to cultural factors. Unlike the 'World Values Survey' the set of PISA instruments is not suited to clearly identify cultural differences, for instance through responses regarding beliefs, attitudes and practices defined specifically to discriminate between cultures. While mean differences provide interesting hints, they are essentially bi-variate correlations. In order to tell us more about the correlations, which ones are more important than others, and whether indeed some unobservable 'Vietnamese culture' variable may be a plausible explanation, we need to unravel the mystery further through a study of multi-variate correlations. We do this first by using the Fryer-Levitt approach.

Regression Approach I: Fryer-Levitt
We are now ready to investigate the secret a bit further by deepening our analytical approach beyond a mere comparison of means. We adopt a simple methodology that is easy to understand and interpret. Our approach closely follows [Fryer and Levitt, 2004] who sought to explain the black-white achievement gap in the first two years of schooling for children in the United States. For the results presented in this section, we pool the student level data from Vietnam and Dev7 countries. Recall that Dev7 stands for the seven developing countries in the 2012 PISA dataset with a per capita income below the cut-off of US$10,000. The reason for focusing on developing countries is that we want to have a common support with regard to a country's wealth. If a rich country shows outstanding results, perhaps it may be of interest to other rich countries which do not do as well, but it is hardly of great interest to a poor country. But if a poor country does very well, and stands out from the pack of poor countries that mostly do poorly in PISA, readers from poor countries want to know what can explain such a phenomenon, since it clearly cannot be attributed to the wealth of the country (as captured albeit imperfectly by per capita GDP). We start by looking at the Mathematics scores, with the identical approach being used for the other two PISA disciplines; Reading and Science.

Mathematics
We estimate a weighted least squares regression of student level test scores as follows: 8 A key estimate of interest is γ, the coefficient on V IET N AM , a 0 or 1 dummy variable.
Regressions are run in a sequence, starting from one without any covariates in X, and then adding variables in groups to expand X in consecutive columns in Table 10. Column (1) in Table 10 shows that the Vietnam dummy has a coefficient of 128.05, when no other covariates are added. By construction, this is the absolute difference in means between Vietnam and the Dev7 countries. Next, we want to see the extent to which observable variables included in the PISA dataset can help to explain this large gap of 128.05. 9 The first set of variables included in the regression reported in column (2) concern the students themselves. The student characteristics were -if students went to pre-school, repeated a grade in the past, and how often they are late for school (ST08Q01) or skipped classes (ST115Q01). With these variables included, the coefficient on the dummy or "the Vietnamese advantage" or "gap", comes down by nearly 20 points, or roughly 0.2 standard deviation units, to 108.91.
In other words, one key reason that the Vietnam gap is so high is because of these student related variables -this result was hinted at in the endowment comparison presented earlier in Section 2. Note that of the four student variables used in column (2), only two are statistically significant. Figures in parenthesis represent t-values.
As we add variables to expand the group of covariates for Mathematics (Table 10), Reading (Table 11) and Science (Table 12) we follow a trial and error method, depending on whether or not, within each group of variables (students, parents, teachers and schools), the inclusion of the specific variable leads to a reduction in the Vietnam dummy coefficient. We retain the variable if it leads to a reduction in the Vietnam dummy coefficient, even if the variable itself may or may not be statistically significant. In this approach, we are not interested in accurately capturing the size of the coefficients other than the one for the Vietnam dummy. Neither are we seeking to maximize the explanatory power of the regression. 10 After considering student related variables, the second set of variables relates to the home background and parents of students. Mean comparisons showed that PISA indices on family wealth or parental education are much lower for Vietnam compared to the other seven countries. Clearly, Vietnam's higher PISA scores cannot be explained by higher parental wealth or prental education. Inclusion of these variables would increase the coefficient on the Vietnam dummy and would take us away from our objective. If Vietnam had enjoyed the Dev7 levels of those variables, the Vietnam gap would have turned to be larger than 128 points. Hence, the parent variables that are retained and presented in column (3) are only those variables that reduce the Vietnam dummy. The reduction amounts to only 11 points compared to the nearly 20 point reduction for student variables. Only the variable PARPRESSURE is statistically significant in this group and indeed has a sizeable impact on the score. This variable reports on principals claiming that "there is constant pressure from many parents, who expect our school to set very high academic standards and to have our students achieve them."  The third set of variables, related to teachers is presented in column (4) and results in a reduction of the Vietnam dummy by only 3 points to a level of 95.13. This does not mean that teachers are unimportant as a reason for Vietnam's superior performance in mathematics, only that the observed teacher related variables in the PISA dataset do not collectively help as an explanation. In the regression itself, one can see that PROPCERT, proportion of certified teachers affects mathematics scores positively. The same goes for a few of student assessment related variables, including TCM STUASS and STU FEEDB, that relate to the use of student assessment and student written feedback to assess teachers.
The final set of variables, related to schools is presented in column (5), that results in a further reduction of the dummy coefficient by 12 points, to a level of 77.26. There are interesting insights from some of these school variables. COMP USE does not have a positive effect on the mathematics score, but COMPWEB does -recall that Vietnam does relatively better on internet connectivity compared to mere availability of computers. The presence of SCMATEDU and SCMATBUI, quality of school educational resources and quality of physical infrastructure, helps explain the gap, recall from Table 9 that Vietnam has superior endowments compared to the Dev7 countries. Table 10 shows that extra classes organized by the school (SCL EXTR CL) and the "systematic recording of data including teacher and student attendance and graduation rates, test results and professional development of teachers" (QUAL RECORD) are also part of the story of explaining the Vietnam test score gap. Public dissemination by the school of test results (SCORE PUBLIC) has a positive coefficient, which is also one of the variables where Vietnam appears to have twice as many schools following this practice.
As we successively add variables in the regression equation, the number of observations in the regressions drops significantly due to missing values in the data set. Comparing available observable values to other variables for the dropped observations does not seem to indicate a systematic bias in this attrition. The 2012 PISA round included a so called 'rotated' module for the student's questionnaire. The idea of the rotated module was to ask three additional sets of questions in a systematic division that increases the overall data available without burdening the respondents with excessively long questionnaires. The main caveat of the 'rotated' design is that these additional three sets of questions/variables only cover two-thirds of students each, and any combination of two sets would only cover one-third of students. Regressing on variables of the rotated modules, thus significantly reduces the sample size. The regression tables with rotated variables included can be downloaded from the github site for this paper. The lowest possible value for the Vietnam dummy in the Mathematics regression was 49.59, adding gap-decreasing variables from the second rotated module (appendix table A5 online). This relates to a 61% reduction of the Vietnam 'gap'. The rest of the gap is explained due to effects not measured by PISA.   For scores on Reading (table 11), the gap is lower to begin with: 105.16 compared to 128.05 for mathematics. The pattern revealed by the regession is quite similar: the student specific variables bring down the Vietnam dummy coeffcient to 85; the parent related variables accounts for a further 9 points, with only PARPRESSURE being statistically significant.   Teacher related variables for Reading only account for a further decline of 2 points, and school variables reduced the dummy by 14 points to a level of 60.31 reported in column (5) of Table 11. Appendix Table A6, available on the online github repository, reports the regressions with rotated variables included which brings the dummy coefficient down to a lowest level of 52.04. Finally we look at the Science results.

Science
The gap between Vietnam and Dev7 is largest for science scores, with column (1) in Table  12 indicating the dummy coefficient at 134.56. The pattern of reduction in the dummy is again similar to the previous two subjects -with the student specific variables accounting for the biggest decline (19 points) , followed by school (13 points), parents (11.5 points) and teacher related variables (2 points). The final specification reduces the dummy to 88.54. Together with rotated variables, presented in Appendix Table A7, available on the online github repository, the lowest value for the Vietnam dummy is 80.09, a total reduction of 55 points explained, but still leaving a large portion unattributable through the Fryer-Levitt method.

Summarized insights from Fryer-Levitt
We can see that across the three test subjects, the Vietnam dummy comes down by nearly 50 points, or half a standard deviation. We find that student, parent and school related variables appear to explain part of Vietnam's superior performance in PISA.

Students:
The student related variables reflect two policy elements that could be useful for other countries that seek to learn from Vietnam. The investment made by Vietnam in pre-schools appears to have long lasting effects, and indeed in Vietnam the government continues to invest deeply not only for universal pre-school, but also for early childhood care services even prior to pre-school. A policy lesson can also be derived from the effect of class repetition -cause and effect is difficult to extract in the case of repetition and test score performance, but one can see that repetition is much lower in Vietnam (Table 1) and the regression coefficient on a student being a repeater has a large negative value, even in the final specification with all other variables included. The other student related variables regarding being late for school and skipping classes perhaps do not have clear policy implications for other countries but help us understand a cultural effect regarding Vietnam.
Parents: As noted in the text, the household wealth/possessions, parents' education levels and socio-economic indices reflect Vietnam's per capita GDP and act against explanations of the test score gap. Applying this trend, Vietnam would have benefited from a much higher gap if it had been a wealthier country, ceteris paribus. There is an "advantage" that Vietnamese children have in having more demanding parents, though perhaps Vietnamese teenagers may not always see it that way. Parents are demanding not only of their children, but apparently also of schools and generally parents appear to back up their demands by contributing on their own as volunteers. Interestingly, even though the individual coefficients of parent related variables are not statistically significant except for one variable, the variables appear to collectively influence the dummy coefficient up to one-tenth of a standard deviation of test scores. As Amy Chua [Chua, 2011] attested, parental attitudes and behav-iors are deeply influenced by cultural norms. There is a policy lesson here concerning the freedom of access provided to parents to take part in the school life. Sometimes schools tend to be insular places without much scope for parents to contribute, but measures to harness parents' contributions in their time as well as in cash and kind may yield positive results.
Teachers: Teachers are widely recognized to be the most important factor in many studies of student achievement. Yet, in this case, the inclusion of a number of teacher related variables does not appear to be useful in explaining Vietnam's achievement gap. It is interesting to note that in the regressions, the variables individually tend to have statistically significant coefficients, but do not affect the dummy coefficient. One teacher variable, the proportion of certified teachers, is clearly economically and statistically significant, and it is one where Vietnam has an advantage (80% vs. 68% certified). Variables that relate to the use of student assessment in providing feedback to teachers on their performance are seen to be important. The presence of other assessment and feedback related variables are also in line with intuition. It is possible that the advantages which Vietnam enjoys with regard to teachers are 'swamped' by the effects of variables for which Vietnam does not have an advantage, so the net result is that the gap is not explained by PISA related teachers variables. It is also possible that the effect of teachers is particularly context specific, revealing a weakness of the pooled regression approach of Fryer and Levitt. This last explanation is further investigated in the next section of the paper.
School School resources matter with regard to PISA results in the international perspective, as the scatter plot in Figure 1 motivating this paper clearly shows. Developing countries with the notable exception of Vietnam are clustered at the bottom left hand side. And there is a positive slope with high scoring countries tending to be on the higher income side. In this section, we see that the effort made by the Vietnamese government to invest in education plays an important part in explaining the achievement gap. Even though Vietnam may be poor with regard to per capita income, it is not as poor with regard to the quality of educational resources and the quality of physical infrastructure. This can be seen in Figure  4 comparing the average PISA 2012 mathematics scores with a school infrastructure quality index. Compared to Figure 1, where Vietnam is the country with the lowest GDP per capita in the PISA 2012 data set and thus placed in the left hand side of the figure, Vietnam moves more towards the middle in Figure 4. Using SCMATBUI (quality of school infrastructure), Vietnam jumps ahead 11 places, with a similar story (not shown) regarding SCMATEDU (quality of educational materials). A key reason is the investments by the Vietnamese government in schools in smaller towns and rural areas as classified by PISA, given that the dispersion of school infrastructure is lower in Vietnam compared to other countries. A key weakness of the Fryer-Levitt approach is that the pooled regression does not allow for regression coefficients to be different across countries. In this section, we set aside the Fryer-Levitt method to look at the data from a different analytical perspective. We use the Oaxaca-Blinder decomposition (OB) method, that has recently become quite popular for PISA analysis after the initial work by [Ammermueller, 2007] comparing the PISA results of Finland and Germany.

Overview of the OB Decomposition
The objective of OB is to decompose the mean differences. In this case it is the mean difference in Vietnam's PISA performance with each of the DEV7 countries individually. Extensions of OB allow for decompositions to be made throughout the distribution rather than only at the mean values, but we leave such an extension for further research regarding Vietnam's PISA performance. OB is based on a simple algebraic rearrangement of terms of the OLS regression of test scores. The mean outcome difference to be explained (∆Ȳ ) is simply the difference of the mean outcomes for Vietnam and the comparison country. Let us denote the scores asȲ V andȲ O , respectively: Now, as the OLS error terms are of mean zero by construction, (2) can be represented by In the twofold version of OB that we use in this paper, (3) can be represented either as depending on which country is used as a reference country. We focus on this paper on the approach of (4), using Vietnam as the reference, and leave (5) and additional OB variations to subsequent research.
We base the choice of regression specification on the findings so far. In line with per capita GDP of Vietnam compared with the Dev7 variables, there are a series of income level or wealth related variables for which Vietnam does poorly in comparison with Dev7. We term these variables WEALTH related variables. We include in it all variables for which Vietnam has poorer endowments, and the term WEALTH denotes that they have higher mean values in Dev7 countries, which are wealthier than Vietnam. These variables have typically not been included in the Fryer-Levitt regressions as they would have exacerbated rather than reduced the Vietnam gap. For example, the set includes mother's highest level of education, or MISCED, which is one ISCED level higher for Dev 7 as compared to Vietnam. A second set of variables, which we term as ED WEALTH are typically variables for which Vietnam does better and are good for education results. For instance, Vietnam has a higher level for PRESCHOOL and for time spent by students in mathematics lessons after school (OUTMATH). These two sets of variables together constitute the specification of the X V and X O vectors. Now it is an established result that the matrix decomposition reported in equation 8 also holds at the variable level, as the overall decomposition is nothing else but the sum of variable level decompositions [Hlavac, 2015]. We extend this notion further to look at aggregations by the two sets of variables -ED WEALTH and WEALTH. In the equations below, we have m variables in the EDWEALTH set and n variables in the WEALTH set:

Findings from OB Decomposition
In Tables 13 through 15 we present the mathematics score findings from the OB decomposition for Vietnam compared with each of the Dev7 countries arranged by geographic area. We present the mean differences again between Vietnam and Dev7, then countries from Latin America (Colombia and Peru), Eastern Europe & Central Asia (Albania), Middle East & North Africa (Jordan and Tunisia), East Asia & Pacific's (Indonesia and Thailand) as well as Shanghai. The number "Sum Total ED WEALTH + WEALTH" in the bottom row indicates the mean difference between Vietnam and each of the Dev7 countries. The top panel in each of Tables 13 through 15 presents the ED WEALTH variables -these are variables related to educational performance where Vietnam does better on mean values compared to the Dev7 countries. The lower panel in each of the tables presents the WEALTH variables -typically related to a country's income level, they are variables where the Dev7 countries do better. The paired column of numbers for each country indicates a hypothetical counter-factual.
The 'Endowments' column shows how much of the mean difference arises from a difference in the mean values, keeping the coefficients fixed at the level of Vietnam. The 'Coefficients' column indicates how much of the difference is due to the difference in coefficients between Vietnam an the compared country, keeping the characteristics fixed at the mean level of the compared country. We begin the analysis by looking at Latin American countries Colombia and Peru in Table 13. The mean difference for Colombia is positive 106 points. Interestingly, differences on ED WEALTH endowments make a negligible contribution and the coefficients on ED WEALTH only account for positive 8% of the difference. The endowments on WEALTH indicate a 14% reduction for Colombia by adopting Vietnamese endowments, and a large 103% reduction from coefficients. The Colombia EDWEALTH coefficient column indicates some interesting results. While PRESCHOOL (students having attended PRESCHOOL) would have had a positive 21 point impact, note the negative 14 point impact on the variable called TIGER-MOM (parents proactively following up with teacher regarding student's performance) and negative 10 point impact on STU FEEDB (teachers obtain written feedback from students). This might indicate that some of the features of countries are related to cultural factors that come together as a package -being a 'Tiger Mom' may help the child in Vietnam, but perhaps not as much in Colombia! A similar interpretation is possible regarding the variables TCM INSPE and TCM OBSERVER (teachers benefit from class room observation by external inspectors and principal/senior school staff) and SCL EXTR CL (extra classes at school). These have a negative value in the endowment column as well as the coefficients column. This means that if Colombian students had Vietnamese characteristics on these variables, the mean result for Colombia would have been lower than it already is. The interpretation would be that what is good for Vietnamese students, in a Vietnamese context as measured by PISA, may not be good for Colombian students. However, this finding should be taken with some caution as there are also some variables, like OUTMATH (time spent in extra classes for math outside of school) which have positive values on the endowments and the coefficients.
Finally from Table 13 we can see that for Peru, ED WEALTH endowments make a positive 10% contribution and the coefficients make a negative 11% contribution. ED WEALTH variables that make a positive contribution in the coefficients column include PRESCHOOL, SHRS (hours of science instruction) and all the parent related variables. On the WEALTH endowments, there is a positive 4% contribution and a negative 100% contribution from coefficients.   Table 14 shows that the mean difference between Albania and Vietnam for mathematics was positive 143 points, the highest Vietnam advantage amongst all Dev7 countries. The OB decomposition indicates that if Albanian 15 year olds had Vietnamese endowments on ED WEALTH variables, their score would have been higher by 18% and if they had Vietnam's coefficients on those variables, their score would have been higher by 64%. Looking at the bottom panel, with Vietnam's WEALTH endowments, Albania's score would have been reduced by 6% and with Vietnam's WEALTH coefficients while retaining its own characteristics would have lowered the score by 46%. Interpretation needs to be made with care, for instance, a big boost would have come from the coefficient values on repetition, but this is probably driven by the rarity of repetition in Vietnam.
With Jordan, the gap to be explained is positive 118 points. On the ED WEALTH variables, endowment differences with Vietnam make for a positive 15% contribution and the coefficients make for a negative 15% contribution. Vietnamese endowments on ED WEALTH would make a reasonable contribution for Jordanian students, but the coefficients would pull in the opposite direction. Six variables contribute mainly to this negative direction -LATESCHOOL (number of times arriving late in the schoolday), TCM OBSER (teacher classroom observation by principal or senior school staff), STU FEEDB (written feedback from students for teacher), SCL EXTR CL, EXC6 MATHCOMP (mathematics competition as extra-curricular activity) and SCMATBUI (index of quality of school infrastructure). On the WEALTH side, there is a positive 12% contribution of endowments and a negative 109% contribution of coefficients. The variables which contribute to this anomalous result include PRIVATESCL, LHRS (hours of language instruction) and RATCMP15 (available computers for 15 year olds).
The mean difference for Tunisia is positive 113 points. Tunisia indicates the highest positive value of ED WEALTH endowments amongst all countries (35%) and also the highest negative value on the coefficients (-52%). The results for WEALTH for Tunisia indicate a positive 7% contribution on endowments and a negative 86% contribution on coefficients. The constituent variables have made their appearance in the previous commentary -for example, the contributions to the negative value on ED WEALTH coefficients for Tunisia come from LATESCHOOL (-13.6), NOREPEAT (-24.3), TIGERMOM (-4.6), SCMATBUI (-13.05), and SCL EXTR CL (-17.72).
Next, we turn to Table 15 which includes the remaining set of Dev7 countries from the East Asia & Pacific's region -Thailand and Indonesia as well as the additional case of Shanghai, which has much better results even than Vietnam, and is included as an East Asian counterpoint to the Dev7 countries.  Table 15 indicates that in the case of Indonesia, the gap to be explained is positive 140 points. Endowments and coefficients on ED WEALTH account for positive 19% and positive 23% of the gap. The ED WEALTH variables explain more as was the case for Colombia. When we look at the WEALTH related variables, we see that the endowments of Indonesia would not have made such a big impact, pointing to the fact that Indonesia is closest to Vietnam amongst the Dev7 countries on per capita income. Of a similar magnitude like Colombia, we can see that the WEALTH coefficients of Vietnam would set back Indonesian students by negative 75%.
Thailand is the country with the lowest difference in mathematics score, only positive 73 points behind Vietnam. The OB decomposition for Thailand indicates a negative 6% contribution of endowments on EDWEALTH and a positive 25% contribution on coefficients. On the WEALTH set of variables, the contribution for Thailand was negative 11% on endowments and negative 177 % on coefficients. The results from Shanghai, added to the mix as a counterpoint to Dev7 countries, do not seem to provide much additional insight. The biggest exlanation of difference between Vietnam and Shanghai would the coefficients on ED WEALTH and WEALTH, with 111% and 91% respectively. 11 Overall, the OB decompositions support the previous findings. In five of the seven Dev7 countries, the ED WEALTH variables show a positive contribution on endowments -meaning that if the other countries have had Vietnam's endowments on ED WEALTH variables, their performance would have been better. On the coefficients side of ED WEALTH, we see a different picture -the contributions range from +64% for Albania to -52% for Tunisia, with other countries ranged in between. The two Asian countries (Indonesia and Thailand) have similar contributions of 23% and 25%. The predictable WEALTH set of decompositions is of less interest to us as in most cases there are small effects on endowments and large negative effects on coefficients.
With cross-section data in a non-experimental context, it is very difficult to make definitive conclusions, and only tentative ones can be made that hint at some answers. The findings on Dev7 countries indicate that it is possible that a number of advantages that Vietnam enjoys, depicted in ED WEALTH, can only function effectively in combination. One way to consider this is through a cultural lens -meaning that there is something specific to Vietnamese culture, that enables Vietnam to benefit from hard working students and teachers, with the guidance of committed and involved parents, even in cities and small towns.

Conclusion
This paper has sought to focus attention and find insights regarding a most remarkable PISA 2012 result -the superlative performance of Vietnam, a country with the lowest per capita income amongst all PISA participants. Vietnam, with a mean PISA math score of 511 is not one of the very top performers. However, when compared with other lower middle income countries that took part in PISA 2012, Vietnam is a clear outlier. The following three concluding points can be made as a result of the analysis presented in this paper.
1. Half the gap can be explained: Even though the PISA dataset is rich and covers many aspects related to the achievement of student scores with international standardization of measures, with all the available variables, we could explain at best about 50% of the performance gap of Vietnam. The PISA 2015 application will be especially interesting to study as it will provide another important data point and enable a trend analysis to be conducted.

Cultural factors are likely very important:
A combination of three sets of factors appear to be the most potent explanation for Vietnam's performance: First, Vietnamese students work harder -we see they have less instances of skipped classes and being late for school, spend about the same time or more learning in school and substantial extra time studying after school. While at school, Vietnamese students are more disciplined and focused on their studies. Second, Vietnamese teachers appear to benefit from a closer supervision of their work by the school principal and others, and there may be a stronger harmony between the hard working students and their teachers. Third, parents may have an important role to play, by taking an active part in combining high expectations of their children, following up with their children's teachers and contributing at school.
3. Resources do appear to matter: When we compare PISA performances across the range from lower income non-OECD countries to the high income OECD countries, we find a clear positive trend. Vietnam has so far been the only PISA outlier, with a performance on par with much wealthier countries, and in fact one of the top performing countries in Science. The analysis indicates that Vietnam may be reaping the benefits of policies regarding investments in education -the most important factor probably being the higher level of access to pre-school. A second factor is the investment in school infrastructure, especially in cities and small towns.
The unique combination of focused educational investments beyond its income level and a cultural heritage that has positive behavioral implications for students appear to be part of the story behind Vietnam's educational success.