WPS8378 Policy Research Working Paper 8378 Asymmetric Information on Noncognitive Skills in the Indian Labor Market An Experiment in Online Job Portal Futoshi Yamauchi Shinsaku Nomura Saori Imaizumi Ana Areias Afra Chowdhury Education Global Practice March 2018 Policy Research Working Paper 8378 Abstract This paper examines the impact of noncognitive (socio-emo- test results were shown generally enjoyed a higher proba- tional) skills on job market outcomes, using a randomized bility of unlock. That is, employers are more interested in control trial implemented in an online job portal in India. those for whom they can see personality test results. Such Job seekers who registered in the portal were asked to take a a relationship was not seen in the pre-test period, which Big-Five type personality test and, for a random subsample confirms that the results are unlikely to be spurious. The of the test takers, the results were displayed to potential study also finds a significant impact among organized, calm, employers. The outcomes are measured by whether a imaginative, and/or quiet applicants (no effect is detected potential employer shortlisted a job seeker by opening among easy-going, sensitive, realistic, and/or outgoing (unlocking) his/her application and background informa- applicants), which seems to display employers’ preference. tion. The results show that the treatment group for whom This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at snomura@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Asymmetric Information on Noncognitive Skills in the Indian Labor Market An Experiment in Online Job Portal1 Futoshi Yamauchi2, Shinsaku Nomura, Saori Imaizumi, Ana Areias, Afra Chowdhury JEL Classifications: J23, J24, C81, C93 Key words: Noncognitive skills, Job portal, Randomized control trial, India 1 We would like to thank John Gibbons, Sean Blagsbets, and Vir Kashyap (formerly Babajob, which was merged into Quikr Jobs in June 2017) for implementing the proposed experiment and participants at the KDI School of Public Policy and Management 20th anniversary commemorative workshop for useful comments and suggestions. Financial supports are received from the World Bank’s Jobs Trust Fund and the Strategic Research Program. We are grateful to Pankhuri Shrivastava (Quikr Jobs) for continued support to this study. The opinions and conclusions expressed in this paper are ours and do not necessarily reflect positions of the World Bank and its member governments. Any remaining errors are ours. 2 Corresponding author. International Food Policy Research Institute, 1201 Eye Street, NW, Washington DC 20005; Email: f.yamauchi@cgiar.org; Phone: +1-202-862-6469. 1. Introduction Asymmetric information on both workers and employers is one important source of frictions in the labor market. Employers often have to rely on observables to infer unobservable characteristics of workers such as innate ability. In a classic example, educational attainment is taken as an observable to signal ability under some circumstances, which leads to an inefficient equilibrium in the labor market (Spencer,1973; Rothchild and Stiglitz, 1976; Wilson, 1977). In this setting, workers (students) are encouraged to invest in education simply to prove their ability rather than to augment their labor productivity, in contrast to the conventional human capital theory (Becker, 1962). Though it has been increasingly recognized that roles played by non-cognitive (socio-emotional) skills are quite important in the workplace (Heckman and Kautz, 2002; Barrick and Mount, 1991), such skills of job applicants are typically unobservable to employers. In this paper, we examine the impact of non-cognitive skills on job market outcomes by introducing a unique randomized control trial into an online job portal in India. The recent literature highlights the importance of non-cognitive skills, including soft skills, personality traits, abilities, character skills, and socio-emotional skills, as a determinant of life outcomes (Almlud, et al., 2011; Kautz, et al., 2014). Studies show that non-cognitive skills are equally as significant as cognitive ability to explain labor market outcomes (e.g., reviewed in Kautz, et al., 2014). Non-cognitive skills can be also formed and malleable until later ages. Our study uses a relatively well-accepted taxonomy of non-cognitive skills called the Big Five (OCEAN: Openness to Experience, Conscientiousness, Extraversion, Agreeableness, and Neuroticism).3 For example, Conscientiousness of the Big Five - the tendency to be organized, responsible, and hardworking - is associated with job performance and wages (Barrick and Mount, 1991; Hogan and Holland, 2003; Nyhus and Pons, 2005; Salgado, 1997). The experiment we propose below aims to break a typical setting in the labor market that, to a large extent, non-cognitive skills of job seekers are unknown to potential employers. In this attempt, the online job portal proves very useful. Currently, global online job portals such as LinkedIn, Indeed, Monster and Career Builder connect employers and job seekers in many countries. A growing number of local online job portals have also emerged, including formerly Babajob (merged into Quikr Jobs in June 2017; all Babajob users were carried 3 Other measurements are also proposed to capture a certain aspect of non-cognitive skills, such as Grit (Duchworth, et al., 2007). The Big-Five measurements depend on self-reported answers, but researchers also proposed to use actual behavioral decisions to measure non-cognitive skills, such as risky and reckless behaviors measured in the adolescent years (Heckman, et al., 2014) and tenth grade participation in sports, academic clubs, and fine arts activities (Lleras, 2008). As discussed in Section 3, the answers by job seekers might be biased as those who decided to take the test might have an intention to look good. However, the sample of test takers is randomly split into treatment and control, so our empirical results are unbiased under the above-mentioned conditionality. 2 to Quikr Jobs). 4 The job portals provide information on job vacancies and help expand access to job information for those who have internet connectivity. These online platforms, global or local, provide real- time streams of labor market data that have remained largely unused by researchers as well as policy makers.5 A rapid expansion of job portals is a manifestation of their usefulness for both job seekers and employers. A great advantage of online job portals is low marginal cost to acquire information. In online job portals, job seekers and employers can easily access each other. Kroft and Pope (2014) and Mang (2012) discuss that there has been a large-scale shift from posting job listings in newspapers and other print media toward websites and that this process has lowered the cost of acquiring employment information. Moreover, several studies from developed countries have found that the use of employment websites has reduced unemployment rates (Beard et al., 2012; Kuhn and Mansour, 2014), though an impact on wages remains unclear (Kuhn and Mansour, 2014; Shahiri and Osman, 2014). However, as Shahiri and Osman (2014) point out, it is important to keep in mind that a large number of employers and job seekers still cannot fully utilize employment websites due to limited internet infrastructure, high user fees, and/or a lack of knowledge about information technology. Some studies also found that a greater use of cell phones improves employment outcomes in developing countries (Klonner and Nolen, 2010 for South Africa; Aker, 2011 for Niger; Burga and Barreto, 2014 for Peru), implying that information constraints are a significant source of inefficiency in the labor market. In this paper, we use an online job portal in India (Babajob, described in the next section) to experiment on information asymmetry of job seekers’ non-cognitive skills. Job seekers who registered in the portal were asked to take a Big-Five type personality test and, for a random sub-sample of the test takers, the results were displayed to potential employers. In this experiment, outcomes are measured by whether a potential employer assesses a seeker by opening (unlocking) his/her application and background information. Though whether or not to take the test is a voluntary decision (thus creating a selectivity issue), whether or not the results are displayed to potential employers in Babajob is random. Therefore, we can compare outcomes between those for whom the results were shown and not shown to analyze the impact of non- 4 Currently the platform developed in formerly Babajob was part of the Quikr Jobs platform. In this paper, we refer to Babajob since the randomized control trial was introduced into Babajob prior to their transition to Quikr Jobs (June 2017). 5 For example, LinkedIn, a social networking service for businesses and professionals launched in 2003, had data on 433 million individuals as of Q1 2016. The site is available in over 200 countries worldwide and in 20 different languages. The data obtained through LinkedIn have been primarily used for business purposes, rather than for policy formulation, though various industry- and skills-focused analyses have been published on LinkedIn’s official blog. Tambe (2014) used skills data from LinkedIn to measure employers’ investment in big-data-related human resources management. 3 cognitive skills. The execution of our experiment was fast once it was programmed and tested in the system, as the number of participants in Babajob is large (i.e., big data environment). The paper is organized as follows. Section 2 describes the Indian labor market and Babajob. Section 3 describes in detail the experiment on non-cognitive skills. Section 4 displays empirical results. The results show that the treatment group for whom test results were shown generally enjoyed a higher probability of unlock (one step before being shortlisted). That is, employers are more interested in those for whom they can see personality test results. Such a relationship was not seen in the pre-test period, which confirms that the above results are unlikely to be spurious. We also found a significant impact among organized, calm, imaginative and/or quiet applicants (no effect was detected among easy-going, sensitive, realistic and/or out-going applicants), which seems to display employers’ preference. Implications are discussed in the concluding section. 2. Empirical Setting India has made remarkable progress in economic growth as well as poverty reduction over the past few decades. India’s gross domestic product (GDP) grew at an average rate of 7.3 percent per year between 2007 and 2012. This contributed to a substantial decline in the incidence of poverty, and an estimated 138 million people rose above the poverty line during the period. Despite its robust growth, India still faces major challenges to improve labor productivity and match the supply and demand of workforce skills.6 In India, online job portals emerged in the late 1990s, but only began to flourish in the past decade as mobile phone and internet use became more widespread and social networks expanded. The share of mobile phone subscribers more than tripled from 20 percent of the population in 2007 to 70 percent in 2014, and the number of fixed broadband subscriptions increased eightfold during the same period. There are now about 20 job search portals, many of them focusing solely on the Indian labor market.7 Babajob, once established 6 The Indian labor force is large, mostly informal and relatively young. India’s population reached 1.295 billion in 2014, including 497 million workers. The size of the labor force has been expanding at an annual net growth rate of 4.2 million for the past 10 years. The labor force participation rate is 54 percent, with a relatively high participation rate among men (80 percent) and low rate among women (26 percent). Only 16 percent of the labor force is engaged in wage employment (18 percent of male workers and 12 percent of female workers), and a large majority work in the informal sector. Moreover, 54 percent of the country’s population is under 25 years of age. While the country’s relatively young population has the potential to yield significant demographic dividends, ensuring sufficient employment opportunities for a young workforce also presents a critical challenge for the government. India faces a lack of highly trained workers and a large share of unskilled youth. As a result, job creation and skills development are critical priorities. To address these challenges, the government launched the National Policy for Skill Development and Entrepreneurship in 2015, and its 12th Five-Year Plan set a goal of training 400 million workers by 2022. 7 Though not exhaustive, 22 firms were identified by compiling a list from government agencies and search engines. 77 percent of those firms provide job-search services, while a few provide job-matching services. Some platforms 4 in 2007, became one of the leading job-matching websites in the country. Between July 3, 2007 and May 24, 2017 there were 1,286,812 ads posted by 524,672 employers and 529,023 employers and 8,218,720 job seekers were registered with Babajob. Babajob matched workers to potential employers in both formal and informal sectors. In order to reach disadvantaged populations, Babajob provides a variety of access options including standard websites, mobile sites, interactive voice response (IVR), text messaging and web applications.8 A large number of the jobs listed in Babajob are at the entry level, with the largest share of listings in 2015 classified as clerical support. 70 percent of the jobs advertised were in the 10 most populous cities: Bangalore, Delhi, Mumbai, Chennai, Hyderabad, Pune, Kolkata, Thane, Patna, and Lucknow. In 2015, the average offered salary was 13,182 rupees (Rs.) per month.9 The average offered salary for professional- level jobs is Rs. 14,900, 17 percent higher than that for non-professional jobs, Rs. 12,739. By city, the average salary for professional jobs ranged from Rs. 16,970 in Mumbai to Rs. 12,757 in Patna, while the average for non-professional jobs ranged from Rs. 14,184 in Delhi to Rs. 10,742 in Patna. 3. Experiment In mid-February 2017, a shorter version of the Big-Five personality test (Big Five Inventory-10 in Rammstedt and John, 2007)10 was introduced to job seekers who were relatively active in Babajob in the past several months. Test takers were then randomly split into two groups: those for whom the results were displayed to potential employers and those for whom the results were kept confidential. Those who decided to take the test were informed that the results may be displayed in their profiles as additional information when they apply for a position. Big-Five Personality Test focus on entry-level jobs, while others mainly advertise technology or senior management-level jobs. Job-matching platforms use different techniques to match workers with job opportunities, including leveraging social networks, providing curated job information based on a individual profiles and connecting local recruiters with candidates. 68 percent of the platforms focus on jobs located in India, with some portals concentrating on specific cities. 8 The Babajob platform works to connect job seekers and employers. Job seekers can create a profile in Babajob, search and apply for jobs online or offline, all for free. Employers can create profiles in Babajob and post their hiring requirements for free in a service that resembles online job classifieds, or they can opt for the paid, premium service (RapidHire) that offers a facilitated hiring experience. All job posts are alive for 90 days. RapidHire jobs are promoted more heavily on the site for a certain period depending on the plan opted for (e.g. the basic plan promotes jobs for 15 days). RapidHire also offers other services, such as additional screening, executive recruitment support, unlocking more information about candidates, and SMS promotion of the posted job to relevant job seekers within a certain radius. 9 Using an exchange rate of US$1=Rs. 67 (the average rate during the first half of 2016) this amount is equivalent to about US$197. 10 We used BFI 10 plus 1 additional item on Agreeableness suggested by Rammstedt and John (2007). 5 The Big-Five personality test aims to measure non-cognitive (socio-emotional) skills through the following five components: (i) openness (to experience), (ii) conscientiousness, (iii) extraversion, (iv) agreeableness and (v) neuroticism. Details on each component are below (Table 4.2 of Jonh, Naumann and Soto, 2008): Openness to experience describes the breath, depth, originality and complexity of an individual’s mental and experiential life. Conscientiousness describes socially prescribed impulse control that facilitates task- and goal-directed behavior, such as thinking before acting, delaying gratification, following norms and rules, and planning, organizing, and prioritizing tasks. Extraversion implies an energetic approach toward the social and material world and includes traits such as sociability, activity, assertiveness and positive emotionality. Agreeableness contrasts a prosocial and communal orientation toward others with antagonism and includes traits such as altruism, tender-mindedness, trust and modesty. Neuroticism contrasts emotional stability and even-temperedness with negative emotionality, such as feeling anxious nervous, sad and tense. Pilots The experiment was first developed through pilots. Two different pilots were introduced to test both the content of the assessment and the survey tools planned to be used. The first pilot was composed of a telephone survey of 100 job seekers in our target population, using the Big Five Personality Test (24 question version adopted in the World Bank’s Skills Toward Employment and Productivity (STEP) Measurement Survey (Pierre et al. 2014)). For this survey, Babajob’s Consumer Insights Team was instructed to call job seekers and give the questions exactly as they were written without any explanation, even if the respondent requested it. The survey was administered from June 22 to June 24, 2016. Half of the job seekers were told that the results of this survey would be shared with potential employers, while the other half were told that the survey was purely for research purposes and the results would not be shared. The goals of this pilot were to ensure that job seekers understood the questions of the assessment and how to answer them, and to investigate whether job seekers were likely to change their answers if they believed the results would be shown to employers. Overall, the job seekers understood the questions being asked (only 2.7% of all responses given were the “Respondent couldn’t answer” option), and there was not significant deviation between the two groups. 6 For the second pilot, the team tested our planned data collection methods. As described later, the team decided to pursue recruitment to the RCT primarily through SMS messages to job seekers, sending a link to the survey instrument in the text. This pilot aimed to test the effectiveness of the recruitment, as well as gave the opportunity to experiment with the combinations of messaging and SMS timing to determine how to increase the number of respondents. From November 29 to December 1, 2016, a series of SMS blasts were sent to job seekers over a week with a link to a survey tool that allowed the users to fill out the Big Five Personality Test (11 question version). Overall, we sent approximately 124,000 SMS to 44,000 job seekers. They clicked the link in the SMS 3,865 times, with 1,010 completing the survey. The team experimented with different variations in the content and frequency of the messages, starting with sending three SMS to each seeker spread over three hour intervals. In our final SMS blasts, sending one SMS and waiting at least 24 hours before subsequent blasts was found as the most efficient conversions. Technology Design A variety of iterations were tested when determining how to run the main intervention/data collection (i.e. getting job seekers to take the personality test). In the original design, the team was planning to build the functionality into the Android app, as users on this platform have data-rich profiles and high application per applicant numbers. Due to sample size requirements and the smaller numbers seen in app downloads and usage, the team decided to focus on job seekers using Babajob’s mobile web platform. Building the assessment into the registration process was also considered, but this would not allow for proper sampling and threatened Babajob’s registration/application numbers, so the team switched to a direct recruitment method, prompting users through SMS to take the test. The team planned to use a survey software outside of the Babajob platform to collect the required data (one stage of the RCT pilot), but this approach provided several technical difficulties and meant that updates to users’ profiles with the personality test results would only happen periodically. Instead, the team decided to have job seekers take the test through Babajob’s chatbot. Through this, job seekers would be able to click a link in the recruitment SMS, enter their registered mobile number and begin the personality assessment. Below are screenshots of the test introduction, the process of answering the questions, and the job seeker’s view of the results: 7 After a job seeker finishes the assessment, they get to see their scored results immediately. They can then go on to search local jobs that match their profile and apply for them over the chatbot itself, helping Babajob re-engage the participants of the study to increase the effect. The results of the test are also automatically uploaded to their Babajob profiles and will be shown on their applications to various jobs depending on if the job seeker is in the treatment group. The job seeker is allotted to a group based on their Babajob User ID, a number created sequentially on their registration. Even-numbered User IDs are in the treatment group, while those with odd-numbered User IDs are in the control group (hidden results). Test Results The scores were constructed by giving each of the four-option answer scale 10 = almost never, 30 = sometimes, 70 = often and 90 = almost always. In each component, the scores were averaged. For example, if two answers are almost never (10) and often (70) in agreeableness, the score is (10+70)/2= 40. Personality traits are labeled by a simple word in each dimension using the cut-off point of 50. For example, if the score is above (not including) 50 in extraversion, the person is characterized as “outgoing”; if the score is below or equal to 50, the person is characterized as “quiet”. To choose labels, we avoided to use negative words. Simple and intuitive words were identified to best describe the two opposite poles of Big Five from Table 4.4 of Jonh, Naumann and Soto (2008). Below is a table of the personality trait labels we decided to use in the experiment: Scores above 50 Scores below or equal to 50 8 Extraversion Outgoing – Likes to be Quiet – Likes to spend time around people and enjoys alone and doing solitary being the center of attention. activities such as reading and Is talkative and energetic. writing. Prefers to concentrate Prefers to work in groups. on a single activity at a time. Agreeableness Cooperative – Sensitive and Competitive – Ambitious. Not warm-hearted, gets along well afraid to criticize others or with others. Tends to avoid make difficult decisions. Is conflict and has a difficult rational and may be perceived time saying ‘no’. as insensitive. Emotional stability Sensitive – More emotional, Calm – Tends to be calm and (Neuroticism) can get worried and nervous unemotional. Handles stressful easily. May be vulnerable to situations well. Does not get stress. easily upset. Openness to experience Imaginative – Fanciful and Realistic – Practical, likes curious, likes to try new working with things rather than things. Creative and in touch with ideas. Prefers familiar with their feelings. Tends to routines to new experiences. be liberal. Tends to be conservative. Conscientiousness Organized – Hard-working Easy-going – Tends to be more and reliable, efficient. Likes laid back, less goal-oriented, and to think carefully before less driven by success. More acting. Prefers working in a flexible and spontaneous. structured setting. Display For displaying the results of the assessments to potential employers, the team has modified our application tracking system to include the test results in an easy to read format where employers are likely to see them. Below are screenshots of this new feature: Non-expanded version (what an employer first sees): Expanded version (what an employer sees on clicking “Show More” 9 As can be seen in the above screenshots, we include personality trait labels and scores. This reporting is done both on Babajob’s Mobile and Desktop web platforms. For job seekers whose results are to be hidden, their profile would only include the normal information associated with an applicant without the personality test scores. Data Collection The data collection was started on February 18, 2017 through SMS blasting the link to a randomized list of job seekers who were active in the last three months (to increase the likelihood that they are still looking for jobs). Each day, we have messaged a new group of job seekers, waiting a few days before again messaging those who did not complete the personality test yet. Through this, we are recruiting participants to the RCT in a trackable fashion. Results Tracking 10 The team was able to track the results of the RCT without additional development based on Babajob’s existing data tracking methods. Results tracking was performed through May 24, 2017.11 Babajob tracked every application made by a user, storing a record of the same, as well as anytime an application is “Unlocked” (the primary result we are tracking, which signifies an employer is placing value on the applicant). In addition to this, Babajob has tracked which job seekers have completed the personality test and their answers. Descriptive Analysis In this sub-section we show descriptive results to characterize the experiment. Personality traits were distributed as follows (including both treatment and control). There is potential bias in the distributions due to voluntary decisions made by seekers to take the test. Therefore, the distribution is not representative of personality traits in the general population. For the same reason, there can be differences between this result and the first pilot done by phone as the pilot was conducted under strict confidentiality. Except in extraversion, the traits are one-sided: cooperative in agreeableness, organized in conscientiousness, imaginative in openness, and calm in emotional stability. Those who decided to take the test might have an intention to make them look attractive to potential employers by choosing particular answers. Next, the treatment and control groups are compared in personality test scores as well as traits. In our experiment, 51.5% of the sample is in treatment (Table 2). Since the treatment is a random sub-sample of test takers, it is designed that the treatment and control groups are probabilistically ex-ante identical. Table 3 shows that personality traits (defined in this study) are balanced between the treatment and control groups. Two-sided t tests could not differentiate the two groups (the equality of means is not rejected at the conventional level). Table 4 compares gender and education distributions by treatment status. Though we do not perform a test to compare the two groups, it is evident that the two distributions extraordinarily resemble. Similarly, age distributions are compared between the treatment and control groups (Figure 1), which confirms that the two groups are balanced in age. The above descriptive results appear to provide a good justification to identify the impact of non-cognitive skills (additional information on profiles) on job market outcomes. 11 There are variations in the tracking period. The last test taker was recorded on May 16. 11 4. Results We have confirmed that the treatment and control groups are similar in individual characteristics and personality test scores. The only difference between the two groups is whether the test scores were displayed to potential employers (treatment). Data were collected in the portal even before the test, so we have information on their job search behavior and outcomes prior to the experiment. Therefore, this setting offers us four different situations: (a) treatment after experiment, (b) control after experiment, (c) treatment before experiment and (d) control before experiment. In (c) and (d), the distinction between treatment and control is trivial because they have not taken the test at that stage, thus whether or not the test results were displayed in the system has no significance. The situations in (a) and (b) are in our primary interest and we conjecture that the difference between treatment and control after experiment should not hold before experiment if that is attributed to the intervention defined above. The system records an incident that an employer opens (unlocks) an application. As described earlier, we use the unlock incidence as the outcome measure to detect the impact of displaying non-cognitive skill information on potential employers’ reactions. In Babajob, there are two types of job advertisements: paid and unpaid. In paid advertisements, employers who post those advertisements pay fees to have additional services from Babajob such as a preliminary screening to improve matching. In unpaid advertisements, such a service is not attached, so employers have to check applications by themselves. To unlock is the first action by employers in order to see details on the job applicant for short-listing. In the analysis, we use unpaid job advertisements because the number of unlocks in paid job advertisements was very small during the period under our experiment. Apparently, the number of unlocks is (positively) correlated with that of applications, so we include the number of applications to control on the right hand side. The intervention dummy is interacted with the number of applications to capture two possible effects: (a) direct effect and (b) indirect effect through the rate of unlock per application. Estimation is also executed in a sub-sample to know differentiated effects by job seekers’ characteristics. We look at their personality traits (revealed in the test), gender and job category. For example, results in the conscientiousness part of the test provide an indication as to whether the person is likely to be organized or easy-going and it is possible that the impact of personality traits information display could differ between organized applicants and easy-going applicants. Similarly, we can compare cooperative and competitive applicants; imaginative and realistic; calm and sensitive; quiet and out-going. Possible gender difference is also an interesting issue. If female and male applicants to a job vacancy are not treated equally for some cultural reasons (e.g., the job is traditionally handled by males), the employer may want to know more details of the female applicant to assess her suitability to the job. Alternatively, if female characteristics, 12 both observable and unobservable, are in small variations, employers may want to gather more information from male applicants. In some jobs, workers are required to face customers face to face and soft skills may be more important in those jobs. Table 5 presents the benchmark results. The estimation used the sample of job seekers if they had been in the Babajob platform more than 30 days prior to the test and the number of applications is less than 30. As hypothesized earlier, we confirm that the disclosure of personality test results (treatment) has a positive and significant impact on the number of unlocks after controlling the number of applications and its interaction with the treatment indicator only if the post-intervention period is used. In the pre-intervention period, the treatment indicator is insignificant, which indicates that the above result is unlikely to be spurious. Even in the pre-test period, the number of applications is positive and significant, which confirms that the positive relationship between the numbers of applications and unlocks remains stable before and after the intervention. The qualitatively same results were confirmed even when gender, education and age are controlled (though the sample size is reduced to missing data for some observations). In Table 6, we relax one of the assumptions to define the estimation sample. The number of days staying in the system prior to the test is 60 days. The results remain qualitatively the same. Only after the intervention, the treatment indicator has a positive and significant effect on the number of unlocks. Employers may be looking for job seekers who possess a certain set of personality traits. Our experimental design enables us to look deeply into this issue. Since we saw earlier that personality trait distributions are identical between the treatment and control groups, we can examine the treatment effect separately for different personality trait groups, e.g., the organized versus the easy-going. Table 7 shows the results by component-wise personality trait. Interestingly, we found a significant impact among organized (high conscientiousness), calm (low neuroticism), imaginative (high openness) and/or quiet (low extraversion) applicants (no effect was detected among easy-going, sensitive, realistic and/or out-going applicants), which seems to display employers’ preference. Cooperative and competitive are not easily differentiated in the above results (t values are 1.81 and 1.76, respectively). The results imply that employers generally prefer job seekers who are organized, clam, imaginative and/or quiet. Next, we split the sample by gender to check gender-specific treatment effects. Table 8 presents the results for each gender group. Interestingly, we observe a significant and positive (direct) effect of the intervention among males, but such an effect was not detected among females. Instead, the estimation shows that the marginal effect of the number of applications significantly increases with the intervention, i.e., the unlock rate becomes higher among females. It is possible that insignificance of the direct effect among females 13 maybe due to a relatively small sample size (power). To cross check potential difference of personality trait patterns between males and females, Table 9 shows the percentage of test takers (by gender) who are labeled as cooperative, organized, imaginative, calm and/or quiet. Though there are some differences by gender, we do not see any large gaps between the two groups. A similar sub-sample analysis is performed on job categories. Here we pick up categories that have relatively large numbers of observations in the sample. They are Office Clerk, BPO, Sales, Receptionist and Driver. Table 10 reports the results by job category group. The treatment (direct) effect is significant and positive in Office Clerk, Sales and Receptionist, in which workers normally need to physically face customers. In contrast, the effect is not significant in BPO and Driver. In these categories, they face customers by phone (BPO) or while in transportation if they drive a taxi or bus (truck drivers do not face customers). The results seem to imply that non-cognitive skills are considered to be valued in jobs for which physical customer interface is important. However, it is also important to be reminded that some job categories have relatively small sample sizes, likely not enough to render sufficient confidence in the estimates. 5. Conclusions The innovative randomized control trial introduced in an online job portal has shown that (i) non-cognitive (socio-emotional) skills play potentially important roles in the hiring process and (ii) employers seem to have certain (stereo-typical) preference when looking for employees. In other words, information on non- cognitive skills is usually quite private, thus employers generally do not have such knowledge on job applicants and may have to imperfectly infer it from observable (easily verifiable) data. In this sense, asymmetric information on non-cognitive skills can be a significant source of inefficiency prevailed in the labor market. Our study stands very uniquely in the current big data research. While the majority of analyses on big data apply different methodologies to such rich data, we successfully introduced an experiment into a job portal platform (big data). We are aware that the recruitment of test takers was on a voluntary basis, but differentiation of the takers into treatment and control was random, which offers an ideal setting to test roles played by non-cognitive skill asymmetric information in the job market. References 14 Aker, J.C., 2011. Dial “A” for agriculture: a review of information and communication technologies for agricultural extension in developing countries, Agricultural Economics, 42(6): 631-647. Almlund, M., A. Duckworth, J. J. Heckman, and T. Kautz (2011). Personality psychology and economics. In E. A. Hanushek, S. Machin, and L. W¨oßmann (Eds.), Handbook of the Economics of Education, Volume 4, pp. 1–181. Amsterdam: Elsevier. Barrick, M. R. and M. K. Mount, 1991, The Big Five personality dimensions and job performance: A meta-analysis, Personnel Psychology 44 (1), 1–26. Beard, T. R., Ford, G. S., Saba, R. P., & Seals, R. A., 2012, Internet Use and Job Search. Telecommunications Policy, 36(4), 260-273. Becker, G., 1962, Investment in human capital: A theoretical analysis, Journal of Political Economy 70 (9). Burga, P. I. R. and M. E. G. Barreto, 2014, The effect of Internet and cell phones on employment and agricultural production in rural villages in Peru, Manuscript, Harris School of Public Policy, University of Chicago. Duckworth, A.L., C. Peterson, M.D. Matthews, D.R. Kelly, 2007, Grit: Perseverance and passion for long-term goals, Journal of Personality and Social Psychology, 92 (6), pp. 1087-1101. Heckman, J.J. and T. Kautz, 2012, Hard evidence on soft skills, Labour Economics, 19 (4), August 2012 Heckman, J. J. and S. Mosso, 2014, The economics of human development and social mobility. Annual Review of Economics 6 (1), 689–733. Hogan, J. and B. Holland, 2003, Using theory to evaluate personality and job-performance relations: A socioanalytic perspective. Journal of Applied Psychology 88 (1), 100–112. John, Naumann and Soto, 2008, Paradigm shift to the integrative Big Five trait taxonomy: History, measurement and conceptual issues, Chapter4, In O. P. John, R. W. Robins, & L. A. Pervin eds., Handbook of personality: Theory and research, New York, NY: Guilford Press. Kautz, T., J.J. Heckman, R. Diris, B. Weel, and L. Borghans, (2014), Fostering and measuring skills: Improving cognitive and non-cognitive skills to promote lifetime success, NBER Working Paper 20749 Klonner, S., & Nolen, P. J., 2010, Cell Phones and Rural Labor Markets: Evidence from South Africa. Proceedings of the German Development Economics Conference, Hannover 2010 56, Verein für Socialpolitik, Research Committee Development Economics. Kroft, K., & Pope, D. G., 2014, Does Online Search Crowd out Traditional Search and Improve Matching Efficiency? Evidence from Craigslist, Journal of Labor Economics, 32(2), 259-303. 15 Kuhn, P., & Mansour, H., 2014, Is Internet Job Search Still Ineffective? Economic Journal, 124(581), 1213-1233. Lleras, C., 2008, Do skills and behaviors in high school matter? The contribution of noncognitive factors in explaining differences in educational attainment and earnings. Social Science Research 37 (3), 888– 902. Mang, C., 2012, Online job search and matching quality, Ifo Working Paper No.147, Ifo Institute – Leibniz Institute for Economic Research at the University of Munich. Nyhus, E. K. and E. Pons, 2005, The effects of personality on earnings, Journal of Economic Psychology 26 (3), 363–384. Pierre, Puerta, Valerio and Rajadel, 2014, STEP skills measurement surveys: Innovative tools for assessing skills, Social Protection & Labor Discussion Paper No.1421, World Bank. Rammstedt and John, 2007, Measuring personality in one minute or less: A 10-item short version of the Big Five Inventory in English and German, Journal of Research in Personality 41: 203-212. Rothschild, M. and Stiglitz, J., 1976, Equilibrium in competitive insurance markets: An essay on the economics of imperfect information, Quarterly Journal of Economics 80: 629-649. Salgado, J. F., 1997, The five factor model of personality and job performance in the European Community, Journal of Applied Psychology 82 (1), 30–43. Shahiri, H., and Z., Osman, 2014, Internet Job Search and Labor Market Outcome, International Economic Journal, 1-13. Spencer, M., 1973, Job market signaling, Quarterly Journal of Economics 87: 355–374. Tambe, P, 2014. Big data investment, skills, and firm value, Management Science 60 (6): 1452-1469. Wilson, C.A., 1977, A model of insurance markets with incomplete information, Journal of Economic Theory 16: 167-207. 16 Table 1 Personality traits Agreeableness Cooperative: 2017 Competitive: 853 Conscientiousness Organized: 2001 Easy-going: 869 Openness Realistic: 1022 Imaginative: 1848 Emotional stability Calm: 2058 Sensitive: 812 Extraversion Quiet: 1461 Outgoing: 1409 Table 2 Sample composition by treatment status # obs % Control 1392 48.50 Treatment 1478 51.50 Total 2870 100.00 17 Table 3 Personality traits by treatment status and t-test results Treatment Control # obs Mean # obs Mean p value Agreeableness 1478 61.857 1392 60.944 0.1322 Conscientiousness 1478 67.896 1392 67.787 0.8736 Openness 1478 62.465 1392 62.098 0.6462 Emotional stability 1478 69.133 1392 68.671 0.5298 Extraversion 1478 57.410 1392 56.983 0.5959 Mean equality is tested under two side and unequal standard deviation assumptions. Table 4 Characteristics by treatment status (number of observations) Gender Treatment Control Uninformed 2 4 Male 1099 1024 Female 329 329 Education Treatment Control Uninformed 558 522 Primary 1 3 Secondary 334 302 Post-Secondary/Tertiary 585 565 Gender reported had a large number of uninformed cases: 179 in treatment and 147 in control. The results shown above are gender imputed from their given names in the case of being uninformed. 18 Table 5 Benchmark results After intervention Before intervention Treatment 0.0353** 0.0058 (2.44) (0.51) # applications 0.0317*** 0.0178*** (4.16) (5.01) Treatment * # applications -0.0132 -0.0068 (1.49) (1.42) R squared 0.1296 0.0751 N obs 1795 1791 *** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period. Table 6 Robustness After intervention Before intervention Treatment 0.0408** 0.0010 (2.29) (0.08) # applications 0.0269*** 0.0170*** (3.57) (4.14) Treatment * # applications -0.0128 -0.0058 (1.43) (1.02) R squared 0.1079 0.0670 N obs 1281 1278 *** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 60 days at the time of taking the test and applied for less than 30 positions during the tracking period. 19 Table 7 Personality [agree/con/open/emo/extra, after/before] Cooperative After intervention Before intervention Treatment 0.0343 (1.81)* 0.0144 (1.18) # applications 0.0352 (3.34)*** 0.0187 (4.37)*** Treatment * # applications -0.0150 (1.25) -0.0114 (2.22)** Competitive After intervention Before intervention Treatment 0.0403 (1.76)* -0.0085 (0.38) # applications 0.0246 (2.69)** 0.0155 (2.41)** Treatment * # applications -0.0102 (0.89) 0.0012 (0.13) Organized After intervention Before intervention Treatment 0.0374 (1.97)* 0.0051 (0.39) # applications 0.0294 (3.06)*** 0.0174 (3.82)*** Treatment * # applications -0.0095 (0.86) -0.0092 (1.72)* Easy-going After intervention Before intervention Treatment 0.0313 (1.57) 0.0040 (0.17) # applications 0.0360 (2.94)** 0.0186 (3.27)*** Treatment * # applications -0.0221 (1.60) 0.0010 (0.10) Imaginative After intervention Before intervention Treatment 0.0405 (2.08)** 0.0038 (0.25) # applications 0.0330 (3.32)*** 0.0192 (4.46)*** Treatment * # applications -0.0167 (1.46) -0.0031 (0.49) Realistic After intervention Before intervention Treatment 0.0262 (1.26) 0.0105 (0.68) # applications 0.0296 (2.51)** 0.0151 (2.49)** Treatment * # applications -0.0065 (0.47) -0.0133 (2.15)** 20 Calm After intervention Before intervention Treatment 0.0343 (2.19)** 0.0057 (0.41) # applications 0.0308 (3.46)*** 0.0184 (4.39)*** Treatment * # applications -0.0154 (1.53) -0.0060 (1.03) Sensitive After intervention Before intervention Treatment 0.0299 (0.94) 0.0078 (0.42) # applications 0.0339 (2.34)** 0.0163 (2.46)** Treatment * # applications -0.0047 (0.27) -0.0099 (1.24) Quiet After intervention Before intervention Treatment 0.0419 (2.14)** 0.0009 (0.05) # applications 0.0375 (3.24)*** 0.0190 (3.86)*** Treatment * # applications -0.0238 (1.87)* -0.0047 (0.67) Outgoing After intervention Before intervention Treatment 0.0266 (1.24) 0.0120 (0.84) # applications 0.0258 (2.66)** 0.0162 (3.17)*** Treatment * # applications 0.0005 (0.04) -0.0094 (1.52) *** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period 21 Table 8 Gender Males Females Treatment 0.0384** -0.0031 (2.39) (0.14) # applications 0.0357*** 0.0060 (4.16) (1.41) Treatment * # applications -0.0197* 0.0228** (1.99) (2.22) R squared 0.1513 0.1135 N obs 1381 395 *** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period. Table 9 Personality traits by gender Male Female Agreeableness (% Cooperative) 69.38 73.14 Conscientiousness (% Organized) 69.24 70.93 Openness to experience (% Imaginative) 65.66 60.51 Emotional stability (% Calm) 72.35 69.35 Extraversion (% Quiet) 51.95 49.29 22 Table 10 Job category Office clerk Sales Receptionist BPO Driver Treatment 0.0550* 0.0756** 0.1290* 0.0483 0.0006 (1.92) (2.07) (1.91) (0.91) (0.02) # applications 0.0487** 0.0439** 0.0828** 0.0442 0.0066 (2.62) (2.62) (2.05) (1.45) (1.21) Treatment * # applications -0.0207 -0.0356* -0.0590 -0.0268 0.0010 (0.90) (1.89) (1.34) (0.81) (0.11) R squared 0.1722 0.1471 0.3007 0.2038 0.0326 N obs 367 175 143 225 133 *** 1%, ** 5%, * 10% significance. Numbers in parentheses are absolute t values using robust standard errors. The sample consists of job seekers who were in the system more than 30 days at the time of taking the test and applied for less than 30 positions during the tracking period. 23 Figure 1 Age distribution by treatment status (kernel density) .1 .08 .06 .04 .02 0 0 20 40 60 age Treatment Control 24 Appendix Personality test questions used in the first pilot test 1 Are you talkative? 2 When doing a task, are you very careful? 3 Do you come up with ideas other people haven't thought of before? Do you like to share your thoughts and opinions with other people, even if you don't know them 4 very well? 5 Do you get very upset in stressful situations? 6 Do you finish whatever you begin? 7 Do people take advantage of you? 8 Do you work very hard? For example, do you keep working when others stop to take a break? 9 Do you forgive other people easily? 10 Do you tend to worry? 11 Are you very interested in learning new things? 12 Do you prefer relaxation more than hard work? Do you enjoy working on things that take a very long time (at least several months) to 13 complete? 14 Do you enjoy beautiful things, like nature, art and music? 15 Do you think about how the things you do will affect you in the future? 16 Are you very polite to other people? 17 Do you work very well and quickly? 18 Do you get nervous easily? 19 Are you generous to other people with your time or money? 20 Are you outgoing and sociable, for example, do you make friends very easily? 21 Do you think carefully before you make an important decision? 22 Are people mean/not nice to you? 23 Do you ask for help when you don’t understand something? 24 Do you think about how the things you will do will affect others? Personality test questions (BFI-11) used in the RCT 1 Is reserved 2 Is generally trusting 3 Tends to be lazy 4 Is relaxed, handles stress well 5 Has few artistic interests 6 Is outgoing, sociable 7 Tends to find fault with others 8 Does a thorough job 9 Gets nervous easily 10 Has an active imagination 11 Is considerate and kind to almost everyone 25