80614 Making Sense of Financial Capability Surveys around the World A Review of Existing Financial Capability and Literacy Measurement Instruments FEBRUARY 2013 Making Sense of Financial Capability Surveys around the World A Review of Existing Financial Capability and Literacy Measurement Instruments © 2013 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judg- ment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemina- tion of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522- 2422; e-mail: pubrights@worldbank.org Acronyms AFI Alliance for Financial Inclusion ANZ Australia and New Zealand Banking Group Limited COMEC Comitê de Regulação e Fiscalização dos Mercados Financeiro, de Capitais, de Seguros, de Previdência e Capitalização ENEF Estratégia Nacional de Educação Financeira FC Financial Capability FCCP Financial Capability and Consumer Protection FCP Financial Consumer Protection FI Financial Inclusion FSA Financial Services Authority FSS Financial Service Survey FTC Federal Trade Commission HRS Health and Retirement Study LSMS Living Standards Measurement Study OECD/INFE Organization for Economic Cooperation and Development International Network for Financial Education RTF Russia Financial Literacy and Education Trust Fund WB The World Bank Acknowledgments The lead authors of this review are Valeria Perotti, Siegfried Zottel, Giuseppe Iarossi, and Adedayo Bolaji-Adio. Aurora Ferrari and Douglas Pearce (World Bank) provided valuable guidance and comments throughout the preparation of the review. Insightful comments were also received from Richard Hinz (World Bank). The document was peer reviewed by Anna Maria Lusardi (George Washington University), Lewis Mandell (SUNY Buffalo), Nataliya Mylenko (World Bank), and Mehnaz Safavian (World Bank). Substantive contributions were received by Adele Atkinson (OECD), Daryl Collins (Bankable Fron- tiers), Kim Dancey, Kammy Naidoo, Sabine Strassburg (FinMark Trust), Jeanne Hogarth (Federal Reserve Board), Elaine Kempson (University of Bristol), Leora Klapper (World Bank), Maria Lúcia Leitão, Susana Narciso (Bank of Portugal), Annamaria Lusardi (George Washington University), Kathryn Maloney (Commission for Financial Literacy and Retirement Income), and Jonathan Sib- ley. Research assistance by Hoda Osman is also gratefully acknowledged. In addition to World Bank funding, financial support for this review work has been generously pro- vided by the Russia Trust Fund for Financial Literacy and Education. Table of Contents 1. PURPOSE OF THE REVIEW AND HOW TO READ IT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2. FINANCIAL CAPABILITY: WHY IT IS IMPORTANT AND HOW SURVEYS CAN HELP . . . . . . . . . . . 7 2.1 FINANCIAL LITERACY AND CAPABILITY: WHY THEY ARE IMPORTANT AND HOW THEY RELATE TO FINANCIAL INCLUSION AND FINANCIAL CONSUMER PROTECTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 HOW A SURVEY CAN SUPPORT POLICY AND RESEARCH IN THIS AREA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 2.2.1 MEASUREMENT OBJECTIVES FOR FINANCIAL CAPABILITY SURVEYS. . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.2.2 MEASUREMENT OBJECTIVES FOR FINANCIAL INCLUSION SURVEYS . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.2.3 MEASUREMENT OBJECTIVES FOR FINANCIAL CONSUMER PROTECTION SURVEYS . . . . . . . . . . . . . 12 3. WHAT DATA ARE NEEDED TO INFORM POLICY OBJECTIVES? . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1 POLICY OBJECTIVE 1: IMPROVE FINANCIAL CAPABILITY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1.1 WHAT CONCEPTS CAN BE MEASURED FOR FINANCIAL CAPABILITY?. . . . . . . . . . . . . . . . . . . . . . . . . . 15 3.1.2 WHAT QUESTIONS CAN BE USED TO MEASURE FINANCIAL CAPABILITY?. . . . . . . . . . . . . . . . . . . . . . . 16 3.2 POLICY OBJECTIVE 2: PROMOTE FINANCIAL INCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.2.1 WHAT CONCEPTS CAN BE MEASURED FOR FINANCIAL INCLUSION?. . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.2.2 WHAT QUESTIONS CAN BE USED TO STUDY FINANCIAL INCLUSION?. . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3 POLICY OBJECTIVE 3: STRENGTHEN FINANCIAL CONSUMER PROTECTION . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.3.1 WHAT CONCEPTS CAN BE MEASURED FOR FINANCIAL CONSUMER PROTECTION?. . . . . . . . . . . . . 27 3.3.2 WHAT QUESTIONS CAN BE USED TO STUDY FINANCIAL CONSUMER PROTECTION?. . . . . . . . . . . . . 27 4. SURVEY DESIGN AND DATA QUALITY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.1 WHO SHOULD BE INTERVIEWED?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.2 WHAT SHOULD BE CONSIDERED TO ENSURE COMPARABILITY ACROSS COUNTRIES?. . . . . . . . . . . . . . . . 31 4.3 WHAT SHOULD BE CONSIDERED TO ENSURE COMPARABILITY OVER TIME? . . . . . . . . . . . . . . . . . . . . . . . . 31 4.4 HOW CAN DATA QUALITY BE ENSURED? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.4.1 HOW TO DEVELOP A VALID AND RELIABLE QUESTIONNAIRE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.4.2 WHAT MUST BE CONSIDERED FOR A SOUND SAMPLING DESIGN? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.4.3 WHICH MODES OF INTERVIEWING CAN BE CONSIDERED?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 5. TRANSLATING DATA INTO POLICY: ANALYZING SURVEY RESULTS. . . . . . . . . . . . . . . . . . . . . . 39 5.1 HOW TO DEVELOP INDICATORS FOR SETTING BENCHMARKS AND TARGETS AND FOR MONITORING PROGRESS OVER TIME . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 5.2 HOW TO IDENTIFY POTENTIAL DEMOGRAPHIC TARGET GROUPS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.3 HOW TO IDENTIFY POTENTIALLY USEFUL TYPES OF INTERVENTION. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 APPENDIX A: POLICY OBJECTIVES AND CONCEPTS MEASURED . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 APPENDIX B: SURVEY REVIEWS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 APPENDIX C: SURVEY BIBLIOGRAPHY. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187 1 1 Purpose of the Review and How to Read It The purpose of this review is to identify, compare, and contrast existing measurement ap- proaches in the area of financial literacy and capability, and in the related areas of financial inclusion and financial consumer protection. The review is intended to be a reference tool for policy makers, practitioners, and researchers interested in conducting a survey on these topics who may not be familiar with the different approaches that have been developed to date, and who need to identify the appropriate combination of methods for their policy or research objectives. This report will be particularly useful for policy makers and regulators who prioritize financial inclusion and/or financial literacy, or who are introducing financial education strategies according to the high-level principles developed by the Organization for Economic Cooperation and Development International Network for Financial Education and recently endorsed by the G20 leaders (OECD/INFE 2012). The objective is not to describe all the instruments that have been developed, but to focus on the key approaches. The instruments described in this review were selected because they are (1) innovative, namely, they have addressed new concepts or introduced new methods for measuring them; (2) well-established, because they have been either suc- cessfully repeated over time, or replicated/adjusted to other settings, or frequently cited in the literature; (3) focused on the demand side of these phenomena, by studying house- holds and personal finance as opposed to enterprises and business finance; and (4) well documented (questionnaires and/or detailed documentation are available). The scope of the review is also limited to baseline assessments and does not include impact evalua- tions, which are an important source of measurement tools but typically focus on measur- ing very specific outcomes. Based on these criteria, 20 key approaches were identified. A full list with policy objectives and countries of implementation is provided in table 2, which also lists other studies that have used similar approaches. The paper presents the purpose, content, and methodology of the 20 key approaches reviewed. Chapter 2 discusses the role that surveys can have in informing policy and research in this area and provides examples of specific survey objectives. Chapter 3 describes and compares the content of the various instruments on financial capability, fi- nancial inclusion, and financial consumer protection, and provides examples of common- ly used questions. Chapter 4 describes the methods used for survey implementation, and highlights technical issues that can affect data quality. Finally, chapter 5 describes the analytical methods that have been used to present the results and to construct in- 2 Review of Existing Financial Capability and Literacy Measurement Instruments dicators. Detailed information about each study is provided in standalone summaries in appendix B. Table 1 outlines the key questions addressed by the review and provides a quick reference for readers interested in finding answers to specific questions. Table 1: Review Structure Policy perspective Why are financial literacy and capability important and how are Chapter 2, Section 1 they related with financial inclusion and consumer protection? How can a survey support policy and research in this area? Chapter 2, Section 2 Who are the key counterparts for implementing a survey? Chapter 2, Section 2 Guidance on survey implementation What concepts can be measured? §§ Financial Capability Chapter 3, Section 3.1.1 §§ Financial Inclusion Chapter 3, Section 3.2.1 §§ Financial Consumer Protection Chapter 3, Section 3.3.1 What questions can be used to measure them? §§ Financial Capability Chapter 3, Section 3.1.2 §§ Financial Inclusion Chapter 3, Section 3.2.2 §§ Financial Consumer Protection Chapter 3, Section 3.3.2 What is the most common question for each topic across the reviewed approaches? §§ Financial Capability Chapter 3, Section 3.1.2 §§ Financial Inclusion Chapter 3, Section 3.2.2 §§ Financial Consumer Protection Chapter 3, Section 3.3.2 Who should be interviewed? Chapter 4, Section 4.1.1 What should be considered to ensure comparability across Chapter 4, Section 4.1.2 countries? What should be considered to ensure comparability over time? Chapter 4, Section 4.1.3 How can data quality be ensured? Chapter 4, Section 4.2.2 Guidance on using survey results for policy How to develop indicators for setting targets/monitoring Chapter 5, Section 5.1 progress How to identify potential demographic target groups Chapter 5, Section 5.2 How to identify potentially useful types of intervention Chapter 5, Section 5.3 FC: Financial Capability Table 2: List of Reviewed Surveys with Policy Objectives and Countries of Implementation FI: Financial Inclusion Big bullets denote the major concept(s) covered by each survey FCP: Financial Consumer Protection Survey Name FC FI FCP Countries Implemented Replications and Similar Studies ANZ-Retirement Commission New Zealand ANZ Survey of Adult Financial Literacy in Australia, 2008 Financial Knowledge Survey American Association of Retired Persons (AARP), Consumer Behavior, Experiences and Attitudes: A Comparison by Age Group, 1999 AARP, Consumer Experience Survey: Insights on Consumer Credit Consumer Fraud in the United States: Behavior, Fraud and Financial Planning, 2003 United States The Second FTC Survey Telemarketing Fraud Victimization of Older Americans: An AARP Survey, 1996 AARP, Off the Hook: Reducing Participation in Telemarketing Fraud, 2003 Consumer Protection Diagnostic Report India, 2010 Consumer Protection Diagnostic Cambodia Consumer Protection Diagnostic Report, 2009 Kenya Study Kenya (CP Kenya) Malaysia Consumer Protection Diagnostic Report, 2009 Latvia Diagnostic Review of Consumer Protection Financial Access in Kenya Kenya Enhancing Financial Innovation & Access in Nigeria (FinAccess) 2009/2011 Financial Capability, Financial Competence, and Wellbeing in Rural Fiji Fijian Households (Fiji Fin Cap) Financial Diaries Bangladesh, India, and South Africa Financial Services Authority (UK United Kingdom FSA) Consumer Awareness Survey Financial Capability in Ireland, Financial Regulator, 2007 Financial Capability: New Evidence for Ireland, 2009 Canadian Financial Capability Survey, 2009 Financial Capability in the United States, FINRA, 2009 Financial Services Authority (UK United Kingdom Financial Education in Italy: The First Scientific Measurement of FSA) Measuring Financial Capability Financial Culture in Italy, 2008 Financial Capability of Austrian Households, 2007 Summary of financial insight among the Dutch, CentiQ, 2008 Baseline survey of financial literacy, 2008 FC: Financial Capability Table 2: List of Reviewed Surveys with Policy Objectives and Countries of Implementation FI: Financial Inclusion Big bullets denote the major concept(s) covered by each survey FCP: Financial Consumer Protection Survey Name FC FI FCP Countries Implemented Replications and Similar Studies Botswana, Ghana, Kenya, Lesotho, Malawi, Mozambique, Namibia, FinScope Nigeria, Pakistan, Rwanda, South Africa, Swaziland, Tanzania, Uganda, and Zambia Global Findex 148 countries Junior Achievement Young Enterprise & Citi Bank Survey Financial Literacy Project Survey, 2004 College Students’ Financial Literacy: Dead or Alive? Jump$tart Survey United States Personal Financial Literacy Among College Students Survey, 1998 Junior Achievement “Teens and Money Survey� Capitol One Survey of High School Students Albania, Armenia, British Virgin Islands, Czech Republic, Estonia, OECD/INFE Measuring Financial Germany, Hungary, Ireland, Literacy Malaysia, Norway, Peru, Poland, South Africa and United Kingdom Singapore National Financial Literacy Singapore Survey Austria, Belgium, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Special Eurobarometer 230 Luxemburg, Malta, Netherlands, Poland, Portugal, Slovak Republic, Slovenia, Spain, Sweden, Republic of Cyprus, and United Kingdom Survey on the Financial Literacy of Portugal the Portuguese Population FC: Financial Capability Table 2: List of Reviewed Surveys with Policy Objectives and Countries of Implementation FI: Financial Inclusion Big bullets denote the major concept(s) covered by each survey FCP: Financial Consumer Protection Survey Name FC FI FCP Countries Implemented Replications and Similar Studies Financial Literacy, Retirement Preparation and Pension Expectations in the Netherlands, 2011 United States Financial Literacy, Retirement Planning, and Household Wealth, the (Replicated in Germany, India, Netherlands, 2011 U.S. Health and Retirement Study Financial Literacy and Retirement Planning in New Zealand, 2011 (HRS) 2004 Indonesia, Italy, Japan, Sweden, Netherlands, New Zealand, and Financial Literacy and Retirement Planning in Germany, 2011 Russia) Financial Literacy and Retirement Planning in Sweden, 2011 Financial Literacy, Retirement Planning, and Household Wealth, Italy, 2011 U.S. Survey of Consumers Nov/Dec United States 2001 World Bank Financial Capability and Azerbaijan, Bulgaria, Bosnia & Consumer Protection surveys (WB Herzegovina, Romania, Russia, FCCP) and West Bank & Gaza World Bank Livings Standards Measurement Survey /FinScope Ghana Financial Services Experiments (Ghana FSS) World Bank/Russia Financial Literacy Armenia, Colombia, Lebanon, and Education Trust Fund Financial Mexico, Nigeria, Papua New Capability Survey Guinea, Turkey, and Uruguay 7 2 Financial Capability: Why It Is Important and How Surveys Can Help Financial Literacy and Capability: Why They Are Important and How 2.1  They Relate to Financial Inclusion and Financial Consumer Protection Across the world, policy makers are concerned with identifying strategies for improving households’ financial well-being, deepening the financial sector, and increasing its stabil- ity. Over the past few decades, these objectives have turned attention toward the financial capability of individuals. On one hand, citizens who make good financial decisions and interact effectively with providers of financial services are also more likely to achieve their financial goals and therefore improve their household’s welfare. On the other hand, finan- cially included citizens contribute to financial sector development, which in turn is strongly related to economic growth. Financial capability is the internal capacity to act in one’s best financial interest, given socioeconomic environmental conditions. It therefore encompasses the knowledge, atti- tudes, skills, and behaviors of consumers with regard to managing their resources and understanding, selecting, and making use of financial services that fit their needs. Financial capability has therefore emerged as a strategic policy objective that comple- ments the financial inclusion and financial consumer protection agendas. These objec- tives are highly interconnected: for example, in order to foster participation in financial markets, consumers need to be better informed about the features of financial products, have the confidence to interact with providers of financial services, know how to choose and use these products and where to seek advice, and be aware of their rights and avail- able redress mechanisms; adequate financial infrastructure must be developed to promote access and use of these services; and financial consumer protection mechanisms must be established in order to protect consumers (the least capable in particular) from risks associated with interactions with providers of financial services. Because financial capability is a relatively new area, alternative definitions and approaches to its measurement coexist. For example, in this review the term “financial literacy� refers to one aspect of financial capability—the knowledge and awareness of financial concepts and products—while other practitioners use it as a synonym for “capability.� The definition used in this review is consistent with the framework developed by the Financial Inclusion and Consumer Protection Service Line at the World Bank and informed by the work of the Russia Financial Literacy and Education Trust Fund (RTF). Given the existence of alterna- 8 Review of Existing Financial Capability and Literacy Measurement Instruments tive definitions, the terms used in this review do not necessarily reflect how they have been used in the surveys we present. Surveys of financial capability can measure some directly observable aspects of this in- ternal capacity, as well as the interaction between the internal capacity and the enabling environment. For example, knowledge or cognitive skills are directly measureable: if a respondent has the relevant knowledge, she will tend to answer the question about knowl- edge correctly (assuming the question is well designed). On the other hand, skills related to managing money cannot be observed independently from external factors such as the level of resources and accessibility of financial products. For example, even if a person is capable of planning for her old age, she may not do so because her income is too low even to cover present essential needs. For this reason, measuring attitudes (such as reasons for doing/not doing something, or being concerned about certain issues) helps to provide a more nuanced picture of a person’s capability. External factors that can affect how the internal capacity manifests itself into actual behav- iors include financial resources, social norms and obligations, financial infrastructure, and existing financial consumer protection mechanisms. In what follows, we describe under “fi- nancial inclusion� all the issues related to access and use of financial products, and under “financial consumer protection� the issues related to existing financial consumer protection mechanisms and institutions. The three policy objectives of improving financial capability, promoting financial inclusion, and strengthening financial consumer protection of course overlap. Therefore, some survey questions that are described under one policy objective may also be relevant for another, and the tables in appendix A help to identify these cases. 2.2 How a Survey Can Support Policy and Research in This Area The purposes a survey serves depend on the content of the questionnaire, the sampling strategy adopted, and the statistical methods used to analyze the data. Surveys can cover a variety of subject areas and have different types of end users, including policy makers and researchers. Regardless of the specific area of interest (capability, inclusion, or con- sumer protection), there are a few broad measurement objectives for which surveys are typically used. Policy makers frequently use surveys as a diagnostic tool to inform the design of national strategies. According to a recent report by the OECD (Grifoni and Messy 2012), at least 36 countries have established or are in the process of designing a national strategy for financial education. The main advantage of developing a national strategy is the enhanced level of coordination among different stakeholders who are typically interested in the area of financial capability—central banks, banking associations, financial supervision entities and financial consumer protection institutions, ministries of finance, and education, and statistical offices. According to the OECD study, 80 percent of the countries have used a survey as a diagnostic method to identify the key priorities for their national strategies. Conducting a survey is a very common method used by policy makers to understand the nature and extent of the issues they want to address through intervention. For example, when little is known about how to best use resources to address a particular issue, a sur- Review of Existing Financial Capability and Literacy Measurement Instruments 9 vey can identify the key weaknesses or the most vulnerable population groups that can be targeted through intervention. A survey can also identify possible areas of intervention. This kind of assessment can be repeated over time to measure progress toward the policy objectives; in some instances comparisons are made not only over time, but also across space by benchmarking results to similar assessments conducted in other countries. A more specific objective that surveys can help achieve is the impact evaluation of a par- ticular intervention. This type of assessment typically requires a survey to be conducted at least twice (before and after the intervention), and questions are specifically designed to measure outcomes that were targeted through the intervention. As a consequence, survey questionnaires that have been designed for a broader purpose may not be appropriate for use in an impact evaluation, and vice versa. Finally, some surveys allow for more in-depth research on the causal relationship among different phenomena of interest. For example, surveys that cover a broad range of topics can be used to analyze the relationship among these (for example, links between financial capability and financial inclusion, or between financial capability and financial welfare). Some survey features, in particular whether the survey has been repeated over time with the same respondents, will affect the range of analytical methods that can be used and the scope of the analysis (whether only simple correlations can be reported or if a causal interpretation can be provided). The following sections present examples of how surveys can support policy making or re- search in the areas of financial capability, financial inclusion, and financial consumer protec- tion. As chapter 3 shows in detail, most of the reviewed surveys cover multiple topics and they could be used to achieve several purposes, whether or not this purpose is explicitly stated in the survey report. In order to have a complete overview of all the surveys that cover specific subjects, the discussion in chapter 3 and the tables in appendix A should be consulted. 2.2.1 Measurement Objectives for Financial Capability Surveys The most common purpose of financial literacy and capability surveys is to provide a baseline assessment of weak areas and to identify potential target population groups. For surveys focusing on financial knowledge and awareness, this means identifying specific knowledge gaps and potential target population groups for financial education programs. For example, the special module designed for the 2004 U.S. Health and Retirement Study and the related international project discussed by Lusardi and Mitchell (2011a); the New Zealand Financial Knowledge Survey; the Kenya Financial Access survey (FinAccess); and the OECD/INFE Financial Literacy survey—all had the purpose of pinpointing specific concepts and topics on which knowledge is limited, and to identify population groups for whom financial education programs could be targeted. In Kenya and in countries partici- pating in the OECD/INFE survey, these assessments were specifically meant to provide a baseline and benchmarks for national strategies on financial education. In surveys that measure financial behavior, the objective is to identify weak areas of behavior and the least capable subgroups of the population. For example, this was done in the U.K. survey conducted for the Financial Services Authority (U.K. FSA) and in the Financial Capability Survey developed by the World Bank for the RTF. 10 Review of Existing Financial Capability and Literacy Measurement Instruments Box 1: Jump$tart Surveys and Improving Financial Literacy in the United States Since 1997, the Jump$tart Coalition for Personal Finance has conducted a biennial Personal Financial Survey to measure the levels of financial literacy of U.S. high school seniors. The Jump$tart surveys were the first studies to indicate low levels of financial literacy among young American adults. When subsequent surveys showed that literacy scores were falling even lower, many states decided to mandate courses in financial education to try to reverse this situation. Interest in financial literacy also grew among federal agencies. The U.S. De- partment of Treasury has created an Office of Financial Education to address the problem of low levels of financial literacy among Americans. On January 22, 2008, President George W. Bush signed an Executive Order creating, for the first time, a President’s Advisory Coun- cil on Financial Literacy. Based on the evidence provided by some of Jump$tart’s studies, the Council was charged with improving financial literacy for all Americans. A different purpose of surveys that focus on knowledge is the identification of potential channels for delivering information or financial education. For example, the Financial Lit- eracy Survey of the Portuguese population has been designed to understand channels through which information about financial products could be provided to potential consum- ers, while the National Financial Literacy Survey conducted in Singapore was intended to measure awareness of the existing financial education programs and determine the most effective delivery channels for these programs. Surveys that measure financial behavior in addition to knowledge also have the purpose of studying the relationship between these two aspects. This was a specific purpose for the special module attached to the U.S. Survey of Consumers in 2001, for the 2004 HRS module, and for the survey conducted in Fiji on Financial Capability, Competence and Wellbeing (Fiji Fin Cap). Most financial capability surveys aim to understand people’s financial behavior and at- titudes with respect to personal or household finances, though different surveys measure different behaviors. Some surveys focus on understanding behaviors related to the use of financial products (Jump$tart, Portugal Financial Literacy, New Zealand Financial Knowl- edge, Survey of Consumers, FinScope), while others are concerned with more general aspects of financial behavior, such as making a budget, planning ahead, or dealing with unexpected expenses. (U.K. FSA Financial Capability, WB/RTF, OECD/INFE, Fiji Fin Cap, Kenya FinAccess). Understanding the details of households’ financial practices is usually beyond the scope of a typical survey of financial capability. First, because it is sometimes difficult to evaluate whether adopting a specific practice is more or less capable than adopting another, and second because understanding these details requires more questions than is manageable in a standard survey. However, there are experimental studies, such as the financial diaries described by Collins and others (2009), where a relatively small number of households are Review of Existing Financial Capability and Literacy Measurement Instruments 11 visited every two weeks, to document their financial transactions over periods that range from six months to a year. The majority of the surveys on financial capability aim to represent the entire adult popula- tion. Two exceptions among studies of financial knowledge and awareness, which instead focus on a subgroup of the population, are Jump$tart, representing high school and college students, and the HRS, a survey of individuals of ages 50 and older. Providing benchmarks across countries is a purpose that most financial capability sur- veys try to achieve. Examples of international projects with a survey instrument explicitly designed for use across different countries include the survey developed by the Russia Financial Literacy and Education Trust Fund managed by the World Bank and the Finan- cial Literacy survey carried out by the OECD/INFE. The U.K. FSA Financial Capability study had a national focus; however, its approach was later followed and adapted to a local context by several other countries. Similarly, the U.S. HRS 2004 module on financial knowledge has been adapted in an international project covering eight different countries. Box 2: The National Strategy for Financial Education (ENEF), Brazil. In 2010, Brazil formally established a National Strategy for Financial Education (ENEF), which was developed by a working group comprised of four financial regulators- the Finan- cial System Supervision and Regulation Committee (COREMEC). In developing the new strategy, COREMEC conducted a survey in 2008, the National Measurement of Financial Literacy, to assess existing levels of financial capability and financial inclusion in Brazil. The results of the study revealed significantly low levels of financial knowledge and awareness, and inadequate levels of household savings. Based on the results of the survey, and fol- lowing consultations with public and private sector stakeholders, ENEF was developed to accomplish the following goals: (1) promote financial education for consumers at all income levels; (2) develop financial infrastructure to expand financial inclusion and; (3) improve the efficiency of financial markets. A school-based financial education pilot project was intro- duced in 2010 to empower students and their parents to make sound financial decisions. To assess the effectiveness of this financial education program, and before scaling-up through- out Brazil, the government engaged with the World Bank and the RTF to evaluate and fund the initiative. The rigorous evaluation of the intervention identified positive impacts on stu- dent’s and their parent’s financial knowledge, attitudes, and behavior. 2.2.2 Measurement Objectives for Financial Inclusion Surveys The common purpose for all surveys that measure financial inclusion is to determine what kind of financial products people have access to and use. A few studies, such as the Fin- Scope surveys, the new Global Findex project, the OECD/INFE survey, the WB/RTF sur- vey, and the World Bank’s Financial Capability and Consumer Protection surveys provide internationally comparable data on participation in financial markets, although FinScope 12 Review of Existing Financial Capability and Literacy Measurement Instruments and the Global Findex provide much greater detail because financial inclusion is their main purpose. FinScope is a large, standalone survey carried out in several developing countries and it includes a large number of questions about banking service usage, credit products, savings and insurance products (including informal products), and products for remittances. The Global Findex project was implemented in 148 countries as part of the Gallup World Poll, and it is a source of systematic indicators of account penetration, use of formal or community-based saving mechanisms, and formal or informal borrowing across the world. Because the survey is implemented in association with a larger survey, the Fin- dex questionnaire is relatively short and includes about 20 questions. Another key purpose of financial inclusion surveys is to understand reasons for limited use of financial products. Potential issues include the limited availability or accessibility of infra- structures (number of bank branches, technology for mobile banking, etc.), the affordability of financial products and services, and consumer satisfaction with their range and quality. Examples of studies designed to assess these problems are the Kenya FinAccess survey, FinScope, the Ghana Living Standards Measurement Study (LSMS)/FinScope, and the studies conducted in Portugal and in New Zealand. Only a few surveys have also sought to understand how people use financial information and their attitudes toward financial issues. The survey conducted in Portugal aimed to identify potential channels to provide information on financial products, identifying inves- tors’ habits and level of sophistication. The Special Eurobarometer 230 survey on Public Opinion on Financial Services had instead the objective of measuring attitudes and views on financial issues and priorities, preferences for payment methods, and experience with cross-border financial services. While collecting information on the use of financial services, one survey also intended to answer some methodological questions. The LSMS/FinScope survey in Ghana tested whether the information collected is affected by the measurement approach (asking one person about household usage versus asking each member about their own use) and by which member of the household was interviewed. The results showed that rates of house- hold usage were comparable when the household head was interviewed or a full enu- meration of individual use was done, whereas interviewing a randomly selected household member led to a less complete picture of household use of financial services. Finally, analyzing the relationship between access to banking and financial competence is another purpose for which a reviewed survey has been designed. Though the main focus of the survey carried out in Fiji is on measuring financial capability, it also captured informa- tion on financial inclusion to assess the relationship between these objectives. 2.2.3 Measurement Objectives for Financial Consumer Protection Surveys Measuring financial consumer protection issues is the specific purpose of only a few sur- veys. These are Jump$tart, the Financial Literacy Survey in Portugal, the Kenya Consumer Protection Diagnostic Study, the New Zealand survey, the US Federal Trade Commission (FTC) fraud survey, the U.K. FSA Consumer Awareness Survey, and the Financial Capabil- ity and Consumer Protection (WB FCCP) surveys implemented by the World Bank. Review of Existing Financial Capability and Literacy Measurement Instruments 13 For improving the effectiveness of existing mechanisms and institutions, some surveys aim to assess the level of awareness of financial consumer protection laws and sources of financial information. The Jump$tart survey assesses the level of awareness of financial consumer protection laws among high school and college students; the Portugal survey focuses on respondents’ knowledge of sources of financial information and redress mecha- nisms. Awareness of these mechanisms was also one of the topics included in the Kenya survey, which aimed to assess whether users of financial services understand their rights and responsibilities. Similarly, the Consumer Awareness Survey carried out by the FSA in the U.K. was designed to measure consumer awareness of financial regulation in order to evaluate the FSA’s effectiveness in identifying and mitigating risks. A specific purpose of the FTC Fraud survey is to identify which population groups are most likely to become victims of financial fraud. This survey does this by collecting information about past experience with financial fraud and identifying the characteristics of the victims. Results are used to support the Federal Trade Commission in targeting education cam- paigns to particular groups of consumers. Assessing consumer attitudes toward financial service providers and regulators is another important, though less common, purpose. This is a main goal of the WB FCCP surveys developed by the World Bank’s Financial Inclusion and Consumer Protection Service Line. For instance, the WB FCCP surveys ask questions about the level of trust in and satisfac- tion with services provided by financial institutions or trust in financial regulators as well as confidence in the existing redress mechanisms. The New Zealand survey, which had among its objectives to support the reform of financial consumer protection laws, also asked respondents about their attitudes when interacting with financial advisers (whether they ask for their qualifications and experience). Finally, the U.K. Consumer Awareness Survey also measured the level of consumer confidence in the regulatory regime. 15 3 What Data Are Needed to Inform Policy Objectives? Though the studies presented in this review focus on different aspects of financial ca- pability, financial inclusion, and financial consumer protection, all studies measure these subjects across the following four dimensions: knowledge, skills, attitudes, and behavior. Studies seek to measure aspects that are directly observable: (1) measuring knowledge in order to better understand the information gaps that affect people’s ability to effectively manage their finances; (2) looking at numeracy and literacy skills that may explain differ- ences in knowledge or behavior; (3) looking at observable attitudes to understand how psychological variables shape financial behavior; and (4) collecting data on behavior to gain insight into how people actually manage their personal or household finances, and the financial tools they use in doing so. This chapter takes a look at how different studies have used questions about knowledge, attitudes, skills, and behavior to assess financial capability, financial inclusion, and finan- cial consumer protection. The section is organized around these three policy objectives in order to present which survey data can be used to inform a specific objective. In addition, the chapter reports examples of commonly used survey questions—these are meant to serve purely as illustrations and are not necessarily the best or most rigorously tested questions.1 For each policy objective, Tables 1-3 in Appendix A present the key concepts measured by each survey. 3.1 Policy Objective 1: Improve Financial Capability 3.1.1 What Concepts Can Be Measured for Financial Capability? As mentioned in chapter 2, studies of financial capability seek to measure, given specific so- cioeconomic conditions, aspects of a person’s internal capacity to make self-beneficial finan- cial decisions. This internal capacity to manage financial resources is measured by assessing financial knowledge (literacy), attitudes, and skills. In addition, surveys measure behaviors and attitudes that result from the interaction of this internal capacity and external conditions. Given the multidimensional nature of financial capability, one area of internal capacity that can be measured to inform policy is financial knowledge or literacy. Studies of financial knowledge seek to measure understanding of the concepts deemed necessary for navigat- 1 Due to different levels of rigorousness used in testing questions, this review cannot express a view on the qual- ity of the example questions used. 16 Review of Existing Financial Capability and Literacy Measurement Instruments ing the formal economy. In general, these studies measure financial knowledge across the following three dimensions: 1. Knowledge of financial concepts: knowledge of general finance, including under- standing of basic concepts, such as saving, investing, lending, the effects of infla- tion, budgeting, and risk diversification, among others. 2. Knowledge of financial products and services: awareness of the economic ser- vices and products provided by the finance industry, and the risks associated with using these financial services and products. 3. Know-how or practical knowledge: understanding of how to manage own personal finances, including how to use financial services and products; how to save, bor- row, and invest; and how to protect oneself against financial fraud. Another aspect of internal capacity that can be measured in relation to financial capability is composed of cognitive skills. By measuring skills, policy makers can better understand the potential for people’s financial behavior under certain conditions. Accordingly, a num- ber of studies gather data on respondents’ numeracy and literacy skills in addition to as- sessing levels of financial knowledge. In order to understand how and why people make financial decisions, surveys of financial capability also collect data on observable psychological factors, like attitudes. The studies reviewed here gather information on attitudes including preferences for receiving financial education, reasons for and proclivity toward managing budgets and expenditures, and perceptions of financial service providers. Finally, assessing financial behavior offers an important avenue for understanding the in- teraction between the internal capacities described above (knowledge, attitudes, skills) and external socioeconomic environmental conditions. The types of financial behavior that are measured by the reviewed surveys can be classified into four main areas: money man- agement (managing day-to-day finances), long-term planning (preparing for emergencies and retirement), financial decision-making (ability to choose appropriate financial prod- ucts), and seeking advice. 3.1.2 What Questions Can Be Used to Measure Financial Capability? Measuring Financial Knowledge (literacy) Knowledge of Financial Concepts One approach to measuring financial knowledge is to design questions that gather infor- mation about people’s understanding of a few key economic concepts. The types of finan- cial concepts deemed important may vary by country and context, but these usually include knowledge of inflation, interest rates, risk diversification, and taxation. Review of Existing Financial Capability and Literacy Measurement Instruments 17 The 2004 U.S. Health and Retirement Study, for example, included a three-question mod- ule on financial knowledge based on economic models of saving and portfolio choice. Questions were designed to measure concepts that are considered indispensable for mak- ing everyday financial decisions, and to allow for differentiation among levels of financial knowledge. The survey module was administered in the form of a test, and it gathered data on respondent knowledge of (1) simple interest rate compounding; (2) the effects of inter- est rate and inflation on savings; and (3) portfolio diversification and differences in rates of return. Below is an example from the HRS study of a commonly asked financial literacy question (Lusardi and Mitchell 2011a): 1. Knowledge of Financial Concepts Imagine that the interest rate on your savings account was 1% per year and inflation was 2% per year. After 1 year, how much would you be able to buy with the money in this account? 1. More than today 2. Exactly the same 3. Less than today 4. Do not know 5. Refuse to Answer The World Bank’s Financial Capability and Consumer Protection surveys measure finan- cial knowledge similarly to the HRS study, but include further measures, for instance, to test if people can think in real rather than nominal terms (money illusion). Other surveys measure similar concepts found in the WB FCCP and HRS studies, including the Singa- pore National Financial Literacy Survey, the Fiji Fin Cap study, and the Portuguese Finan- cial Literacy Survey. As illustrated in Table 1 in Appendix A, these studies test respondent knowledge of inflation and compound interest. Other studies go beyond measuring savings-related concepts, to gathering information on people’s understanding of budgeting, investing, lending, and borrowing. For instance, the 2009 New Zealand Financial Knowledge Survey and the U.S. Survey of Consumers test respondents’ knowledge of lending, budgeting, investing, and equity, as well as knowledge of inflation and compound interest. Survey questions were based on a Financial Knowl- edge Framework that detailed the knowledge and skills required for a person to be finan- cially literate at the basic and advanced levels. Some questions ask respondents to define financial terms like a budget, term deposit, asset, liability, capital gain, and savings, while other questions tests understanding of financial concepts. Knowledge of Financial Products and Services Policy makers, however, may want to look beyond people’s understanding of basic finan- 18 Review of Existing Financial Capability and Literacy Measurement Instruments cial concepts and design questions that measure knowledge and awareness of financial products and services. Studies that focus on knowledge of financial products and services generally collect information on two areas: 1. Awareness of financial services and products, and understanding of the functions of financial service providers; and 2. Understanding of the risks and costs associated with using financial products and services. There exist a variety of useful examples of studies that measure awareness of financial tools and services that can be used for managing personal or household finances. The FinAccess 2009 survey, for example, collects data on respondents’ awareness of finan- cial products, such as savings accounts, credit cards, ATM cards, pension, credit bureau, checks, shares, and insurance. Delving further, the survey also asks respondents about their awareness of specific Kenyan financial services providers. Similarly, the FinScope study includes questions that measure Tanzanians’ awareness of savings accounts, cur- rent accounts, insurance, debit cards, and ATMs. The OECD/INFE study collects informa- tion on people’s awareness of a range of financial products. Below is an example of a question from that study of a commonly asked question to gauge respondent awareness of financial products (OECD INFE 2011). 2. Awareness of financial products Please can you tell me whether you have heard of any of these types of financial products. 1. A pension fund 2. An investment account, such as a unit trust 3. A mortgage 4. A bank loan secured on property 5. An unsecured bank loan 6. A credit card 7. A account 8. A savings account 9. A microfinance loan 10. Insurance 11. Stocks and Shares 12. Bonds 13. Mobile Phone payment 14. Prepaid Payment 15. Don’t know response given to question as a whole 16. Refused to respond to the question ad a whole 17. Record if the respondent has (heard of/holds/choose) none of the above. Review of Existing Financial Capability and Literacy Measurement Instruments 19 A number of studies assess whether people understand the basic features and advantag- es of financial products. One well-known financial literacy survey, the Jump$tart biennial Personal Financial Survey, includes a number of questions to test respondent knowledge of the features of different types of financial products. Similarly, the 2009 New Zealand Financial Knowledge Survey asks respondents about the advantages of using Internet banking and the services provided by the national pension plan. The Portuguese Financial Literacy Survey asks respondents what is required to access the minimum bank services, and tests their knowledge of Internet banking. The Survey of Consumers 2001 module in- cludes a number of true or false questions that test knowledge of the functions of financial service providers. For instance, the module asks questions regarding whether the bank is responsible for informing a consumer of an impending overdraft and whether credit repair agencies can remove negative credit information. A related question measures people’s understanding of the costs associated with using fi- nancial services and products. As part of the Consumer Protection Diagnostic Study of Ke- nya, the Financial Sector Deepening Program respondents were asked if they understood the fees charged by their banks. The Portuguese Financial Literacy Survey asks similar questions, including whether respondents knew of the commission and fees charged by banks on current accounts and interest rates and fees for overdrafts. The Survey of Con- sumers also includes a question to test respondent understanding of finance fees charged for using credit cards (Hogarth and Hilgert 2003). Know-How and Practical Knowledge Studies of financial knowledge sometimes collect data on whether people know how to apply economic concepts and use financial products to manage their finances. The Fiji Fin Cap consisted of about 300 questions including some on people’s financial know-how (Sib- ley 2009). For instance, the survey asks respondents, “How would you set up a household budget?� and lists a number of possible answers. The instrument includes a number of other questions that test respondent knowledge of how to manage a budget, keep financial records, compare financial information, review a financial agreement, seek financial ad- vice, make payments, and keep money safe. In addition, the survey assesses respondent knowledge of how to seek recourse in the event of being unable to repay loans. A variety of surveys, for example, include questions on whether people know how to make payments, save, prepare budgets, manage risk, and plan for the long term. The FSA finan- cial capability study includes an interesting question on how well people knew the amount in their bank accounts. Other surveys focus on whether respondents know how to effec- tively access and use financial services and products. The Kenya FinAccess surveys, the Portuguese Financial Literacy Survey, the Survey of Consumers, and the Jump$tart study also include questions that measure knowledge of how to use financial services and prod- ucts. This includes knowing how to open and use bank and saving accounts, make pay- ments, obtain loans, and transfer payments or remittances. For example, the Fiji Financial Capability survey asks respondents: 20 Review of Existing Financial Capability and Literacy Measurement Instruments 3. Know-How: Accessing Financial Services 1 What is one type of identification that you need to take to the bank to open an account? 1. Birth certificate 2. Drivers license 3. Baptism certificate 4. Letter from employer 5. Passport 6. Other (specify) 7. Don’t know 8. Don’t understand the question 9. Refused Similarly, the Portuguese Financial Literacy Survey asks respondents: 4. Know-How: Accessing Financial Services 2 What is required in order to access the Minimum Bank Service? 1. Have a low income 2. Be unemployed 3. Not have a bank account 4. Does not know 5. Does not answer Another important concept that surveys measure is whether respondents know their rights and know how to seek recourse for unfair financial practices or fraud. Table 1 in Appendix A presents a number of different surveys that cover this concept. For instance, the Por- tuguese Financial Literacy Survey asks respondents what entities they would consult in the event of a misunderstanding or disagreement with a bank (Banco de Portugal 2011). Other surveys that measure people’s knowledge of how to resolve problems with financial service providers include the Fiji survey, Kenya Consumer Diagnostic study, Jump$tart survey, and FinAccess. Testing Skills: Literacy and Numeracy In gauging financial capability, a number of studies consider it important to collect data on cognitive skills such as literacy and numeracy. Literacy questions are usually presented at the beginning of the survey in order to gauge whether the respondent (1) can read and (2) understands concepts presented in elementary terms. The FinScope survey, for instance, begins by asking respondents to read a simple question not related to financial literacy. Review of Existing Financial Capability and Literacy Measurement Instruments 21 The survey also includes a test designed to measure numeracy. Similarly, Fiji Fin Cap sur- vey respondents are shown a card with written information and are then asked questions about it to evaluate their literacy skills. The Fiji Fin Cap instrument also includes questions designed to test numeracy skills, for example asking respondents to do simple mathemati- cal questions, including the following (Sibley 2009): 5. Numeracy skills 1 Please read the card. [A prize of $18 is shared equally between six people]. How much will each person receive? Generally, questions designed to measure knowledge of economic concepts are presented as a math quiz that tests numeracy skills. These mathematical questions can be found in the New Zealand Financial Knowledge, the OECD/INFE, and the U.S. HRS 2004 stud- ies. The U.K. FSA’s Measuring Financial Capability: An Exploratory Study (FSA FinCap) includes a seven-question money quiz that tests knowledge of financial concepts as well as numeracy skills. For example, the survey asks respondents the following question (Kempson and others 2005): 6. Numeracy skills 2 Suppose you saw the same television on sale at a discount in two different shops. The original price of the television is £250. One shop is offering a discount of £30 off the original price, the other is offering a discount of 10% off the original price. Which is the better deal- £30 off or 10% off? Assessing Attitudes Another aspect of financial capability that is measured by the reviewed studies is related to attitudes and motivations. As described above, personal views, beliefs, or psychologi- cal traits can affect people’s decision-making and thus form a part of their internal finan- cial capability. Therefore, a number of surveys design questions to measure psychological characteristics that can be directly observed. Some studies measure motivations using scales developed in psychological studies, or simply inquire about reasons for adopting certain kinds of behavior. The World Bank’s Russia Financial Literacy Trust Fund study (WB/RTF survey), for example, contains a set of questions that measure impulsiveness, attitudes toward the future, and achievement ori- entation. Other studies, like the Singapore National Financial Literacy survey, FinAccess, 22 Review of Existing Financial Capability and Literacy Measurement Instruments and the Special Eurobarometer 230, include questions about people’s preferences toward or reasons for long-term planning, lending, saving, and day-to-day money management. Similarly, a number of surveys ask questions about proclivity toward certain types of be- havior. As illustrated in Table 1 in Appendix A, these surveys include the Kenya 2009 Fi- nAccess, the Portuguese Financial Literacy Survey, FinScope, and the OECD/INFE. The New Zealand study gathers data on respondents’ propensity toward saving, managing their expenses, planning for the future, investing, and borrowing. One major study in the field of financial capability, FSA FinCap, includes a number of questions on people’s pro- clivities toward money management (e.g., impulse purchases, delay in paying bills, saving vs. spending), planning ahead (e.g., living for today vs. tomorrow), and seeking financial advice and obtaining financial products. The following question from the U.K. FSA study presents one example of a common attitude question on financial capability (Kempson and others 2005): 7. Preferences I am now going to read you some things that other people have said about managing money. Please tell me how strongly you agree or disagree with them. “I am impulsive and tend to buy things even when I can’t afford them.� “I am more of a saver than a spender.� “I prefer to buy things on credit rather than wait and save up.� “I would rather cut bank than put everyday spending on a credit card I couldn’t repay in full each month.� “I am very organised when it comes to managing my money day to day.� “I am never late at paying my bills.�1. Birth certificate 1. Agree 2. Tend to agree 3. Tend to disagree 4. Disagree strongly 5. Don’t know 6. Refused Measuring Financial Behavior Financial capability is often expressed and measured in terms of behaviors that result from the interaction of internal capabilities and external factors. Surveys measure financial behavior in a number of different areas that can be classified according to the following four dimensions identified in the FSA FinCap study: money management, planning ahead, making choices, and staying informed. Money management questions focused on how people managed their credit and bills, used credit, and managed their bank accounts. Planning ahead questions measured how Review of Existing Financial Capability and Literacy Measurement Instruments 23 respondents would deal with an unexpected drop in income, “lumpy� expenses, and wheth- er they had made pension provisions. The instrument gathered data on what products respondents purchased, how they made financial decisions, and how they stayed informed on financial matters (Atkinson, Kempson, and Collard 2005). The WB/RTF study builds upon the U.K. FSA study to assess levels of financial capability in a group of low- and middle-income countries. Like the FSA Financial Capability study, the WB/RTF survey study includes many questions on behavior. In particular, it looks at how people manage their expenditures, plan for the future, prepare for emergencies, choose between financial products, and make financial decisions. Below is a sample question from the WB/RTF survey study that offers an example of a usual question on money manage- ment in the case of insufficient funds: 8. Money Management Behavior What do you do when you run short of money for food or other necessary items? 1. Borrow from family, friend or work colleague 2. Cash gifts from family or friends 3. Borrow from employer/salary advance 4. Borrow from bank/use credit card/go into overdraft 5. Borrow from a local moneylender 6. Borrow from another type of lender 7. Use savings 8. Find extra work/work extra hours 9. Sell something 10. Spend less on essentials/necessary items (e.g. food) Spend less on non-essentials (e.g. spending on self/partying/going out/alcohol, 11.  cigarettes, hair) 12. Other The Financial Diaries are a unique example of a financial capability study that measures people’s money management, long-term planning, and financial decision making behavior on a daily basis for an extended period of time. Unlike most other surveys, the Financial Diaries aim at understanding the details of household financial practices instead of provid- ing a broad assessment of financial capability. The study gathers information on how fami- lies living on less than US$2 dollars a day in South Africa, Bangladesh, and India manage their cash flow, by recording their household balance sheets every two weeks for about six months. The instrument was tailored to household circumstances, but largely collected information on: • How well people planned ahead or would plan ahead; • How people dealt with an unexpected drop in income and unexpected expenses; • How they anticipated expenses and planned for retirement; 24 Review of Existing Financial Capability and Literacy Measurement Instruments • The money respondents borrowed and repaid; • Respondent savings and withdrawals; • The costs associated with financial behavior; and • What strategies proved successful for managing respondent finances 3.2 Policy Objective 2: Promote Financial Inclusion 3.2.1 What Concepts Can Be Measured for Financial Inclusion? The level of financial inclusion of a population is the result of the interaction between in- dividual financial capabilities and external factors such as financial infrastructure and re- sources. Generally, studies of financial inclusion are designed to measure attitudes and behaviors by looking at the two concepts of access and usage. 1. Access: studies of financial access measure a respondent’s ability to use avail- able financial products and services. Studies of access usually assess barriers to financial services and products by looking at issues such as legal documentation, physical proximity, and affordability. 2. Usage: studies of financial usage are concerned with the actual usage of financial services and products. These surveys generally gather details on the regularity, frequency, and duration of time that people use financial services. Surveys may also look at the concept of quality when studying financial inclusion. Measures of quality focus on whether financial products and services fit people’s lifestyles, people’s attitudes toward products and services, and their experiences using financial products and services (AFI 2010). Table 2 in Appendix A presents the range of concepts measured by surveys of financial inclusion. 3.2.2 What Questions Can Be Used to Study Financial Inclusion? Assessing Behavior and Financial Inclusion Instruments on financial inclusion usually include a variety of questions on people’s behav- ior regarding the ownership and use of financial services. This includes: • The types of formal and/or informal financial products/mechanisms people use and have access to (informal or formal savings and credit systems, mobile banking, home banking, remittance systems, insurance systems, etc.); • The frequency of usage of informal and/or formal financial products/mechanisms (savings, checking, credit, insurance, and/or remittance products/mechanisms); • How people use different types of financial products/mechanisms (save, lend, bor- row, pay bills, insurance, and/or remittance transfers, etc.); • The number of financial products and services people use/possess; • The source of information that influenced choice of financial product/service; and • How people access financial services and products. Review of Existing Financial Capability and Literacy Measurement Instruments 25 The World Bank’s Global Financial Inclusion study (Global Findex), for example, measures financial inclusion by collecting data on behavior across four indicators: account penetra- tion, saving, borrowing, and managing risks. Questions on account penetration capture information on account ownership, how respondents use accounts, the frequency of ac- count usage, the use of mobile money, and how people access their accounts. Saving and borrowing questions look at what informal or formal financial methods people use to save, as well as the frequency with which they use these tools. Finally, questions on managing risks focus on the types of insurance products respondents use to plan for emergencies. The following from the Tanzania FinScope study is a typical question on what products people own: 9. Ownership of Financial Products We are going to talk about your personal experience with banking products and services. Please tell me about your experience with each of the following, using these options: CUR- RENTLY HAVE, USED TO HAVE BUT NO LONGER USE, NEVER HAD. Transaction products 1. Jk funds/Mamilion ya Kikwete 2. Post bank Account 3. Savings Account 4. Fixed Deposit Account 5. Current Account 6. ATM 7. Debit Card Credit & Loan from Banks 8. Personal Loan from a bank 9. Bank overdraft 10. Credit card 11. Loan to buy a house from a bank 12. Loan to buy land from a bank 13. Loan to build a house from a bank Two other financial inclusion studies, FinScope and FinAccess, include detailed inquiries about how people use and access specific types of informal and formal financial prod- ucts and services. FinAccess includes questions on respondent past and current usage of financial services and products, and FinScope asks questions on how respondents use financial products to manage their money. Unlike the Global Findex study, FinAccess and FinScope include specific questions about how respondents access financial services and products. Both surveys ask respondents how they travel to the nearest branch of their fi- 26 Review of Existing Financial Capability and Literacy Measurement Instruments nancial service provider, and how much it costs to travel there by public transportation. For example, the 2009 FinAccess survey asks (Financial Sector Deepening 2010): 10. Access to Financial Products and Services 1 What is the average time you take to travel to the nearest branch of…[Read name of institu- tion] (FSD, 2010): 1. Under 10 minutes / Chini ya dakika 2. About 10 to 30 minutes / Karibu dakika 3. About 30 minutes to 1 hour / Karibu nusu saa hadi lisaa moja 4. About 2 hours / Karibu masaa mawili 5. About 3 hours / karibu masaa matatu 6. About 4 hours / Karibu masaa manne 7. About 5 hours / Karibu masaa matano 8. About 6 hours / karibu masaa sita 9. 7 hours or more/ Zaidi ya masaa saba Another study, the Ghana Living Standards Measurement Survey (LSMS), presents an ex- perimental approach to measuring financial inclusion. The LSMS survey tests whether the identity of the respondent or the inclusion of product-specific cues in questions affect the reported rates of household usage of financial services. Unlike FinAccess and FinScope, the survey measures household financial services usage and behavior, and asks broad questions about the use of different types of financial service providers rather than specific products. Measuring Attitudes and Financial Inclusion To better explain financial access and usage, a number of studies gather data on how and why people choose or use different types of financial products and services. One common question in financial inclusion surveys is to ask respondents about their reasons for or against choosing different kinds of financial services and products. As illustrated in Table 2, Appendix A, the Global Findex, FinAccess, FinScope, FSA study, and Portuguese Financial Literacy Survey, among others, include questions on this topic. The Special Eu- robarometer 230, for instance, asks respondents why they prefer to use a certain method of payment, and what features are important when choosing a financial product. Some surveys also ask respondents about their reasons for not using services or their perceived barriers to accessing and using financial services and products. The 2005 Spe- cial Eurobarometer 230, for instance, includes a number of specific questions asking what obstacles respondents face in using certain financial services in the European Union. The Review of Existing Financial Capability and Literacy Measurement Instruments 27 Global Findex, FinAccess, and FinScope studies ask a similar question. For example, the Global Findex asks: 11. Access to Financial Products and Services 2 Please tell me whether each of the following is a reason why you, personally, DO NOT have an account at a bank, credit union or other financial institution. A. They are too far away B. They are too expensive C. You don’t have the necessary documentation (ID, wage slip) D. You don’t trust them E. You don’t have enough money to use them F. Because of religious reasons G. Because someone else in the family already has an account 1. Yes 2. No 3. (DK) 4. (Refused) 3.3 Policy Objective 3: Strengthen Financial Consumer Protection  hat Concepts Can Be Measured for Financial Consumer Protection? 3.3.1 W Financial consumer protection studies assess how people interact with financial service providers within a specific regulatory environment. This includes measuring attitudes, such as the level of trust in and perceptions of regulators and financial service providers; experi- ences with financial services and products; and behaviors, such as actions performed or mechanisms used to report and resolve problems or to mitigate risks.  hat Questions Can Be Used to Study Financial Consumer Protection? 3.3.2 W Behavior and Financial Consumer Protection In order to address financial consumer protection issues, a number of studies look at how people behave when it comes to resolving problems with financial service providers. The World Bank’s Financial Capability and Consumer Protection (WB FCCP) study, for in- stance, includes a number of questions on how respondents resolved issues with financial service providers. In addition, it asks what actions they took when they were unsatisfied with a financial service or provider. The U.K. FSA study also includes a series of questions that capture how respondents resolved disputes with financial service providers and where they took up their complaints. Similarly, the Portuguese Financial Literacy Survey asks respondents what entities they have consulted in the event of a disagreement or misunder- standing with a bank (Banco de Portugal 2011). 28 Review of Existing Financial Capability and Literacy Measurement Instruments Measuring Attitude and Experience To a large extent, instruments that measure financial consumer protection issues col- lect data on people’s attitudes (trust and perceptions) and experiences. Table 3 in Ap- pendix A details the various types of attitude and experience questions that surveys of financial consumer protection measure. The Kenya Consumer Protection Diagnostic study, for instance, gathers information on people’s experiences with fraud as well as with financial service providers more generally. In the survey, respondents were asked if they had ever had experience with pyramid schemes and if they had lost money. In addition, respondents were asked for their perceptions and satisfaction with informal and formal financial services providers. The WB FCCP survey, for instance, captures specific information on the level of people’s satisfaction with services offered by finan- cial institutions and government institutions. The WB FCCP survey collects data on the level of trust people have in regulatory and supervisory institutions and in various kinds of financial service providers. The 2007 U.S. Federal Trade Commission (FTC)’s Consumer Fraud Study offers an inter- esting example of an instrument designed to collect information on fraud. Implemented in 2003 and 2005, the FTC survey was constructed to measure people’s experiences with specific types of financial fraud. Questions were based on those experiences that had the greatest number complaints in the FTC’s Consumer Sentinel Database of Fraud Com- plaints and had led to enforcement actions. For example, respondents were asked if they had paid an advance fee to obtain a loan or credit card, been billed for product or service they didn’t receive, purchased membership in a pyramid scheme, or had been billed for a product they had not agreed to. The FTC survey is unique in that it focuses on people’s experiences with fraud and not on their experiences with any particular financial service provider (Anderson 2007). Finally, a number of studies of financial consumer protection gather information on peo- ple’s confidence in resolving disputes with financial service providers. The FSA Consum- er Survey, for example, asks respondents about their confidence in their ability to resolve a problem with a financial service provider. It includes the following common financial consumer protection study question (FSA 2010): 12. Confidence in Resolving Problems If you needed to make a complaint to a financial firm, how confident would you be that the firm would resolve your complaint fairly? 1. Very confident 2. Fairly confident 3. Neither confident nor unconfident 4. Fairly unconfident 5. Very unconfident Review of Existing Financial Capability and Literacy Measurement Instruments 29 As illustrated in Table 3, Appendix A, the Special Eurobarometer 230 likewise includes a number of questions that ask respondents to rate the likelihood (from very easy to very difficult) of winning a dispute with a bank and an insurance company. The WB FCCP sur- vey also asks people about their confidence that occurring conflicts with financial service providers will be resolved in a timely and just manner by existing recourse mechanism. Likewise, the Kenya Diagnostic Survey asked respondents the reasons for or against seeking recourse in the event of a dispute. 31 4 Survey Design and Data Quality 4.1 Who Should Be Interviewed? The specific policy objective of policy makers when they decide to implement a survey has certain implications with regard to the unit of analysis that should be considered. Potential units of analysis could be firms, households, or individuals. Since surveys of firms are beyond the scope of this study, all reviewed surveys collect either individual- or household- level data. The most common approach among the surveys reviewed is to use individuals as the unit of analysis; they gather specific information about one randomly selected per- son within the household. Exceptions are the Financial Diaries and Ghana’s FSS survey, which gather data on all household members or the household head, who is then asked to provide information about the whole household. If policy makers are interested in knowing, for example, which parts of the population are financially excluded, household-level data on access and usage will most likely underestimate the problem because they ignore in- equalities within the household by assuming that financial products owned by a household can be accessed and used by all household members to the same extent. 4.2 What Should Be Considered to Ensure Comparability Across Countries? Being able to compare and benchmark across countries can yield huge benefits. If a proj- ect aims at comparing results internationally, the topics covered and questions asked will have to be adjusted, sometimes at the expense of addressing country-specific issues. Cross-country studies are of course best implemented through a joint, coordinated, and simultaneous effort by all the countries involved (as was done for the OECD/INFE survey, the WB/RTF or Global Findex survey). Another option may be just replicating what other countries have already done. This option requires particular caution though, because the simple translation of survey instruments developed in other settings is not sufficient. Ad- equate testing and fine-tuning of the instrument will always be required in order to ensure that the questions have the same meaning across countries. Equally important is the use of consistent sampling methodology. 4.3 What Should Be Considered to Ensure Comparability over Time? Conducting a survey more than once and carrying out a follow-up assessment is necessary to evaluate trends in the phenomenon of interest. This can be done in different ways, for example, by simply conducting the survey again with a new random sample (repeated cross- 32 Review of Existing Financial Capability and Literacy Measurement Instruments sectional survey), or by following the original sample and interviewing the same respondents again (longitudinal or panel survey). Panel surveys are useful to assess specific interven- tions and their impact on individuals or households, because they measure changes for the same units over time. They are less useful for overall monitoring or in cases of broad types of interventions when the interest is focused on changes among the population as a whole. Re-interviewing the same respondents over time requires contact details to be collected, and to allocate more time and resources to trace the same people from one survey round to the next and to incentivize them to participate again. Among the surveys we reviewed, only Jump$tart, the New Zealand survey, and the WB FCCP survey implemented in Russia have been repeated over time. Only the last re-interviewed the same sample. Independent from the type of survey conducted, the same questions should be used to ensure comparability over time. However, before re-using the same questions it is advised to check that they are still meaningful. For instance, if policy makers are interested in monitoring changes in the proportion of people who are aware of the minimum payment required to open a bank account and this amount changed over time, the question has to be modified accordingly. A repeated cross-sectional survey should also use a similar sam- pling methodology. 4.4 How Can Data Quality Be Ensured? The quality of the data collected in a survey is critical in informing the design of the pol- icy intervention. The conclusions that policy makers can draw from surveys are only as good as the quality of the survey data. Survey design, therefore, is important for effective evidence-based policy development. An overview of the survey design of the reviewed surveys is presented in table 6. When designing a survey, the following properties are particularly important for ensuring high data quality: (1) development of a valid and reliable questionnaire and (2) a sound sampling design that ensures representativeness and preci- sion of results. Finally, the mode of interview (3) will also be discussed as a criterion for evaluating a particular survey design. 4.4.1 How to Develop a Valid and Reliable Questionnaire The development of a valid and reliable questionnaire is vital to ensuring that the questions used are really measuring what policy makers intend to measure. For instance, if policy makers are interested in collecting information on the proportion of the population that has an account at a formal financial institution, asking a question like, “Do you have an account at a formal financial institution?� would most likely introduce large measurement errors. Respondents will either not understand what an account or formal financial institution is, or interpret it very differently because no clear definition or examples are provided. Therefore, testing questions before survey implementation is a must to reduce measurement errors, especially if a question is newly developed. The main ways of designing and testing new questions are to conduct focus groups, cogni- tive interviews, and pilot surveys. Focus groups are qualitative interviews with small groups of people invited to discuss the topic of interest. They can be a powerful tool for gaining a better understanding of how relevant and understandable existing questions are. Even more important, they can be used to inform the overall questionnaire design by gathering Review of Existing Financial Capability and Literacy Measurement Instruments 33 the general public’s view of the key concepts to be included. Cognitive interviews are inter- views with recruited volunteers who are usually interviewed in a laboratory environment. They focus on the cognitive process people use to answer survey questions and help to discover how well questions are understood or why they might fail. Pilot surveys, on the contrary, are interviews conducted in the field with the fully developed questionnaire. Usu- ally they serve the purpose of final fine-tuning of the instrument in case there are still ques- tions that are not properly understood by many respondents or even enumerators. At least one of these methods to develop a valid and reliable instrument was used by most of the surveys reviewed. In order to develop a survey instrument that is suitable to measure the concept of financial capability across different countries and income levels, the WB/RTF survey used all of these methods; focus groups to inform the design of the questionnaire and the main concepts of financial capability to be measured, cognitive interviews to test the understanding of the questions and to ensure that they have the same interpretation in different settings of income and educational attainment across and within countries, as well as pilot surveys for the final fine-tuning of the questionnaire. A completely different approach is to use mainly questions that have already been well tested. This approach—among others—was followed by OECD/INFE, which developed the whole questionnaire by drawing well-tested questions from existing surveys and adopt- ing them after several rounds of discussions with INFE country representatives. 4.4.2 What Must Be Considered for a Sound Sampling Design? Representativeness of Results Policy makers should consider three basic principles if they want to generalize the survey results to the overall population. First, representativeness is determined by the method used to select the units of the sample. As long as the units are drawn randomly, the sample is representative. Randomly selecting units from the population helps avoid selection bias (that is, having a sample with a systematically higher or lower proportion of units with cer- tain characteristics compared with the population), which allows policy makers to make inferences from the survey data. Most of the surveys reviewed as part of this study use some form of probability sampling in selecting respondents, which means that all units of the population have a non-zero and known chance to be drawn into the sample. For example, the Jump$tart Financial Literacy Survey used a stratified random sample drawn from a list of high schools in the United States provided by the U.S. Department of Educa- tion, and the U.K. FSA Financial Capability Survey employed a sampling method where census enumeration areas with 300 households on average were randomly selected from a full listing of all enumeration areas in the United Kingdom. Second, any randomly drawn sample is representative of a certain “unit.� A sample could represent a subgroup of the population, a location, or a combination of a subgroup and location. Therefore, it is important for policy makers to understand precisely what popula- tion groups are to be sampled. A “nationally representative� sample, for instance, provides a rather broad definition and could be representative of an entire country, and/or each region, and/or a combination of regions, and/or subgroups of the population. The Survey 34 Review of Existing Financial Capability and Literacy Measurement Instruments of Consumers is not only nationally representative but also representative of the four main regions of the contiguous United States, including the Northeast, the North Central, the South, and the West. To allow for accurate generalization of survey data, it is important that what the sample represents is adequately understood and explained. Finally, a sample represents the population from which the sample is drawn, and that population alone. Since a representative sample reflects the characteristics of a particular population, the data findings can be generalized only to that population. For instance, the results of a poll conducted in Russia based on a sample of mobile telephone owner, rep- resents the opinions of Russians who own mobile phones. Such data would be unlikely to represent the opinions of Russians who do not own mobile phones, or all retirees or all high school students. The U.S. HRS 2004 represents the U.S. population ages 50 or older. The Financial Literacy Survey in Portugal represents the Portuguese population resident in Portugal, including the mainland and autonomous regions, who are 16 years and older, and the Jump$tart survey represents all high school seniors from public schools listed by the U.S. Department of Education. Sample Size and Precision of the Results The right sample size depends on a number of factors, including the desired level of confi- dence and precision of the estimates. Policy makers may be interested in using the survey data to calculate a specific statistic, such as the proportion of the population with access to finance. It must be kept in mind, however, that the true value of the statistic for the popula- tion will not be equal to the statistic calculated for the randomly selected sample. Instead, the true value will lie with a certain probability (called the confidence level, typically set to 95 percent) within a range around the sample statistic (for example, ±5 percentage points). This range can be narrow or large and it can be interpreted as the level of precision. For example, if the survey has a level of precision of ±5, a confidence level of 95 percent, and 30 percent of the sample has access to finance, it can be concluded that—with 95 percent probability—the proportion of the population with access to finance is between 25 percent and 35 percent. A greater level of precision necessitates a larger sample size. Hence, deciding on an ap- propriate sample size depends on how comfortable policy makers are with the error in the estimated parameter. The 20 methods included in this review cover a wide range of sample sizes, from as low as 32 (Financial Diaries) or 200 (Fiji Fin Cap) to as high as 6,856 (Jump$tart 2008 study) or 22,000 (FinScope Nigeria 2009). Alternatively to increasing the sample size, the same level of precision can be achieved if one is willing to accept a lower level of confidence, for instance, at 95 percent instead of 99 percent. Ultimately, there is no right sample size; each sample size depends on how precise policy makers want to mea- sure their topic of interest and how confident they want to be in the results. The optimal size of the sample also depends on the specific group of the population that a survey seeks to measure. If policy makers consider measuring multiple groups, it will be important that an optimal sample size is selected for each group, to ensure that suitable inferences can be made about each. This will likely increase the overall sample size. For example, policy makers might want to determine the percentage of the unbanked in rural Review of Existing Financial Capability and Literacy Measurement Instruments 35 areas and select a sample size of 1,000. They may also want to determine the unbanked population in urban areas and select a sample of 1,000, thus increasing the total sample size to 2,000. The U.K. FSA Financial Capability survey seeks to survey the four countries of the United Kingdom; accordingly, the survey sample includes 3,100 from the English population, and 500 each from the Scottish, Welsh, and Northern Irish populations, giving a total sample size of 4,600. In some cases policy makers might find it useful to oversample a particular subgroup of the population or location. This approach allows a more detailed study and increases the precision of estimates for this subgroup or location. Oversampling involves deliberately sampling a much higher proportion of a group than the rest of the population because a random sample of the entire population might result in too few or none of the individuals in these groups being sampled. The New Zealand Financial Knowledge Survey, for example, oversamples Maori and Pacific people, in order to produce relevant analysis for these mi- nority groups, bringing the sample size to 856. Likewise, the U.S. Consumer Fraud Survey oversamples Hispanics, African Americans, American Indians, and Asian Americans and the U.K. FSA Financial Capability study oversamples areas with a non-white population of 20 percent or more. In sum, to draw conclusions about multiple groups in the population with the same level of precision, it is necessary to have multiple samples for each subgroup of the population of interest. Unit non-response is an important factor to consider in designing a sample. Some people are hard to reach or not willing to participate, which reduces the desired sample size and the precision of survey results. For the survey methods reviewed in this study, non-response rates range from as high as 84 percent (Jump$tart 2000) and 77 percent (FTC Fraud 2005) to as low as 0.5 percent (FinScope Swaziland 2011). In determining the right sample size it is therefore common to inflate the desired sample size by an assumed non-response rate based on those observed for previously conducted surveys with similar scope and complexity. For instance, if policy makers are planning to conduct a survey similar to the Jump$tart survey and wish to gather data on 1,000 high school students, they would need to sample 6,250 students. A poorly implemented sampling design can result in data that are not representative of the population at large. A serious problem related with unit non-response is that it can be selective. If those that were interviewed are systematically different from those that did not participate, the survey results will be biased. For example, if policy makers wanted to mea- sure the percentage of the population that has a bank account and notice that only those with high incomes are easy to reach and willing to participate, it is legitimate to assume that this unit non-response might bias the results because having a bank account is probably more likely within higher income groups. Strategies to increase survey participation, such as training of enumerators and several contact attempts at different times during a day and over a longer period of time, most likely yield more accurate results. 4.4.3 Which Modes of Interviewing Can Be Considered? In designing a survey, several modes of interviewing can be considered, each with distinct advantages and disadvantages. There are four widely used modes of interviewing: face- 36 Review of Existing Financial Capability and Literacy Measurement Instruments to-face, telephone, mail, and Internet-based surveys. Depending on the population being sampled, the available sampling frame, the type of questions to be asked, and the avail- able resources, one mode may be preferred over the others. Face-to-face interviews offer significant advantages in terms of coverage of the population and quantity and quality of data that can be collected. Telephone or online surveys require that the targeted population has a telephone or Internet access and that an appropriate sampling frame, such as a telephone directory or list of email addresses, is available. Mail surveys further require the ability to read and write. Including populations without telephones, Internet access, or basic literacy may be of particular interest if the survey objective is to determine levels of financial capability or inclusion, because non-covered populations are likely to be the most financially excluded or least financially knowledge- able. Face-to-face surveys usually use the most recent census data as a sampling frame, which could be a weak source of information if the census data are old or not very accurate. In such cases, random route procedures are usually applied, whereby enumerators are sent around and requested to follow specific instructions to randomly select individuals or households. Face-to-face interviewing is usually also associated with lower non-response rates and higher data quality. Choosing face-to-face interviews further allows the collection of more data of higher complexity. Compared with other data collection approaches, the main disadvantage of face-to-face surveys is the substantially higher costs. Telephone surveys are an alternative to face-to-face interviews, especially if resources are constrained, a high fraction of individuals or households with telephones can be assumed among the target population, and an appropriate sampling frame exists. For face-to-face surveys, like most of the surveys reviewed, transportation and personnel represent a sig- nificant part of the costs. If phone interviews replace face-to-face interviews, no travel costs or per diems for travel time need to be paid. None of the studies reviewed provides a detailed reporting of costs, but literature suggests that face-to-face surveys could cost between 5 and 10 times as much as telephone surveys (Groves and others 2004). Another attraction of telephone-administered surveys is their relatively high degree of efficiency. No traveling is required and it is not necessary to wait for mail or email responses. Administer- ing surveys on the phone can be problematic, though, if separate listings of landlines and mobile phones exist, or if listings of mobile phones are not available at all. Of the surveys reviewed, only the Survey of Consumers, the U.S. Consumer Fraud, and the Global Findex use this mode of interview; Findex only in countries in which telephone coverage in the overall population is higher than 80 percent. Switching the mode of interview to mail or online surveys can be a substantial cost saving. Collecting mail or online surveys can save even the cost of enumerators since the respon- dents self-administer the questionnaires. However, as outlined before, this switch can have serious effects on the quality and quantity of the collected information. Therefore, these modes of interviewing have rarely been applied among the surveys reviewed. Only one, Jump$tart, used online surveys as mode of data collection. Table 6: Overview of Survey Designs List of Surveys Standalone / Mode of Level of Sampling Unit of Target Population Sample Size Reviewed Module Interview Representativeness Frame Analysis High school seniors Self-administered / High school seniors/ Jump$tart Survey Standalone or college students Census 1535-6856 Individual online College students 18-23 population Singapore Study Standalone Face-to-face Nationally Census Adults 18-60 2023 Individual New Zealand Study Standalone Face-to-face Nationally Census Adults 18+ 856 Individual OECD/INFE Standalone Face-to-face Nationally Census Adults 18-79 535-2254 Individual Portugal Survey Standalone Face-to-face Nationally Census Adults 16+ 2000 Individual U.S. HRS Module Face-to-face Nationally Census Adults 50+ 1200 Individual Fiji FinCap Standalone Face-to-face One province Census Adults 18+ 200 Household Financial Diaries Standalone Face-to-face None representative Listing Adults 18+ 42-152 Household U.K. FSA FinCap Standalone Face-to-face Nationally Census Adults 18+ 4600 Individual WB/RTF survey Standalone Face-to-face Nationally Census Adults 18+ 1401-3009 Individual Nationally, and 4 Random digit Survey of Consumers Module Telephone Adults 18+ 1004 Household regions dialing Nationally, provincial FinScope Standalone Face-to-face Census Adults 18+ 1200-22000 Individual and urban/rural Census/ Face-to-face / Random WB Global Findex Module Nationally Adults 18+ 504-4220 Individual telephone digit dialing/ telephone lists Ghana’s FSS Standalone Face-to-face Nationally Census Individuals 15+ 1955 Household Eurobarometer 230 Module Face-to-face Nationally Random route Individuals 15+ 1000 Individual FinAccess Kenya Standalone Face-to-face Nationally Census Adults 18+ 6548 Individual Random digit U.S. Consumer Fraud Standalone Telephone Nationally Adults 18+ 2500-3888 Individual dialing WB FCCP Standalone Face-to-face Nationally Census Adults 18+ 1036-2026 Individual CP Kenya Standalone Face-to-face Nationally N/a Adults 18+ 1548 Individual FSA Consumer Standalone Face-to-face Nationally N/a Adults 18+ 2064 Individual Awareness 39 5 Translating Data into Policy: Analyzing Survey Results In order to effectively inform policy objectives, data collected through surveys must be properly analyzed and interpreted. Policy makers are often interested in assessing progress toward specific objectives in the areas of financial capability, financial inclusion, or financial consumer protection. To determine whether a program or intervention is on the way to achieving its ob- jectives, meaningful indicators need to be developed. Indicators are qualitative or quantitative variables that synthesize the relevant information collected by the survey on a specific issue. These are typically used to create benchmarks or targets and to compare progress toward these targets over time and among different socio-demographic groups of the population. 5.1 How to Develop Indicators for Setting Benchmarks and Targets and for Monitoring Progress over Time A good starting point for developing indicators is to report frequencies and proportions of peo- ple who answer individual questions in a specific way. This analysis can be done for all ques- tions. The main advantage of this approach is that no specific statistical knowledge is needed to either analyze data or understand the results. All of the reviewed surveys use this analysis, regardless of whether the survey’s main objective is to measure financial capability, inclusion, or consumer protection. The WB FCCP surveys, for instance, analyze the data by describing proportions of the population who indicate particular attitudes, like mistrust in financial institu- tions or low confidence in quickly resolving issues with financial service providers. Similarly, FinScope measures the extent to which people have access to or use financial products by utilizing indicators such as the percentage of adults that have or use transactional products, savings products, credit products, insurance products, and remittance products. Instead of focusing on single questions, data can be combined by creating composite scores or indices. Frequencies and tabulations are useful to present results about simple issues that are covered by one or two survey questions. For more complex issues, however, single questions alone cannot provide a complete picture of the topic policy makers are interested in. Therefore it is necessary to develop composite scores or indices. The simplest overall scores that can be calculated are arithmetic scores. These scores count specific answers given by each respon- dent. The simplest financial literacy scores, for instance, sum up the number of correct an- swers to quiz-like financial knowledge questions. The simplest financial inclusion scores add the number of financial products and services held by each respondent; the simplest financial consumer protection scores add the number of financial frauds or conflicts with financial ser- vice providers encountered in the past. The Jump$tart study, for example, calculates for each student the number of correct responses to 31 financial literacy test-like questions. 40 Review of Existing Financial Capability and Literacy Measurement Instruments More advanced scores weight responses according to how important the question is. Re- sponses to more or less important questions can be weighted in different ways in an overall score. An example of a weighted composite score is the approach developed by the U.S. HRS 2004 survey, which is done through a two-step weighting procedure: (1) summing up correct answers and comparing each respondent’s answers to the pool of responses, and (2) giving larger weights to the questions that fewer people answered correctly. Due to the multifaceted nature of some topics, it might not always be meaningful to construct a single composite score or index. This is particularly relevant for studying financial capability data because financial capability is an abstract concept that cannot be measured directly, but may be thought of as reflected in a combination of behaviors, skills, attitudes, and knowledge. In order to construct a valid measure of financial capability, it is advised to test with appropriate statisti- cal methods whether all concepts captured in the instrument are associated with the same underlying capability. If several distinct areas of financial capability exist that are somehow but not strongly related, a single index will not properly synthesize the information contained in the survey data. Among the surveys reviewed, the U.K. FSA Financial Capability Survey and the WB/RTF survey seek to create a standardized assessment of the concept of financial capability and employ sophisticated statistical data reduction tools, such as factor analysis, to identify the main components of financial capability. Based on the results of the factor analysis, a score is then constructed for each identified component of financial capability. 5.2 How to Identify Potential Demographic Target Groups The simplest way to identify vulnerable groups of the population for which targeted interventions can be designed is to use cross-tabulations, in which indicators are calculated by key demograph- ic variables (such as age, gender, education, household composition, employment status, and income). This can be done for indicators, which comprise replies to single variables, but also for composite scores. Again due to simplicity, all of the surveys reviewed disaggregate their indicators to determine the extent by which those vary by demographic characteristics. The U.S. Consumer Fraud Survey, for instance, uses cross-tabulations to differentiate the problems of fraud faced by different groups of the population. Similarly, FinAccess uses cross-tabulations to identify those groups within the population that are the most financially excluded. Regression techniques can improve understanding of the relationship between socio-demo- graphic characteristics and financial capability/inclusion or financial consumer protection in- dicators. Cross-tabulations, for example, could indicate that the proportion of the formally banked is higher among high-income and urban populations. However, earning high incomes and living in urban areas most likely go hand in hand. Employing regression analysis helps to identify the independent effects of each of these characteristics while holding other fac- tors constant (for example, is a high-income person living in a rural area more likely to be formally banked compared with a low-income person living in a rural area?). It could be that once effects of other factors are taken into account, the effects of some socio-demographic characteristics turn out to be insignificant. In the given example it could be that, because of the presence of financial service providers, living in urban locations is more important for be- ing formally banked than having a high income. Some of the studies reviewed use regression analysis; Ghana’s FSS survey, for instance, conducts regressions to determine the charac- teristics of individuals and households that reported lower levels of financial product usage. Review of Existing Financial Capability and Literacy Measurement Instruments 41 Another approach to identify groups that should be targeted is to segment the population. The simplest way to do so is to rank people according to their scores and to build equally sized groups. For instance, the U.S. Federal Reserve Board used the Survey of Consumers data to create four measures of financial capability: cash flow management, saving, investment, and credit management. There were between 5 and 11 practices identified under each of the four measures of financial behavior. If households reported fewer than 25 percent of the practices in a given area, they were classified as low; between 25 and 75 percent, they were classified as medium; and more than 75 percent, they were classified as high. The group with the lowest score was then analyzed in more detail (Hogarth and Hilgert 2003). Cluster analysis is a more sophisticated statistical technique that can be used to segment popula- tions; it can be used to identify subgroups of the population that show similarities in their scores. The number of distinct groups that will be formed is thereby determined by the data. The U.K. FSA Finan- cial Capability Study, the WB/RTF study, and the WB FCCP survey used cluster analysis to identify and describe populations who were more or less capable across different areas of financial capability. 5.3 How to Identify Potentially Useful Types of Intervention Baseline survey data can help identify potentially useful areas of interventions. They can be very useful in narrowing down the choices of types of interventions to consider. For instance, if the survey data identify younger populations as being more likely to save less, to engage in impulse buying, and to live beyond their means, the younger populations could be specifically targeted with financial education programs. If survey data on the other hand reveal that no specific group, but rather large parts of the population, lack knowledge of basic financial prod- ucts, mass media channels for public financial awareness campaigns could be considered. If the demand for savings accounts is particularly low due to limited trust in commercial banks, measures to strengthen the legal and regulatory framework for the banking sector should be taken. Or if survey data show that the most financially excluded live far away from the next bank branch but many of them use mobile phones or the Internet regularly, measures to increase branchless banking should be promoted. To properly address the question of which type of intervention works best and which does not, rigor- ous impact evaluations are required. For example, if policy makers want to determine if school-based financial education or comic books would be an effective way to deliver financial messages on money management to younger populations, a rigorous impact assessment would be necessary. Likewise, an impact assessment is needed to determine if mass media such as a TV soap opera is effective at all in changing financial knowledge, attitudes, and behavior. A rigorous impact evaluation would also be valuable to assess if the use of branchless banking is higher if people receive specific training to increase their familiarity with the technology behind mobile phone or Internet banking. Robust evidence on which approaches are the most effective in delivering financial education or supporting beneficial financial inclusion are so far limited. The RTF is therefore supporting a comprehensive set of impact evaluation studies to assess the efficacy of a wide range of financial capability programs in different settings. Insights to each of these projects as well as an Evalua- tion Toolkit focusing on the specific challenges of evaluating financial capability interventions in low- and middle-income environments will be made available on the RTF website (http://www. finlitedu.org), and on the WB’s Financial Inclusion and Consumer Protection Service Line website (http://responsiblefinance.worldbank.org). Appendix A: Policy Objectives & Survey Concepts Measured APPENDIX A: POLICY OBJECTIVES & SURVEY CONCEPTS MEASURED Table 1: Financial Capability Concepts Topic Concept Fund FinScope U.S. HRS Survey 2009 Global Findex FinAcess Kenya Financial Diaries Applicable Objectives Special Eurobarometer 230 Experiments (Ghana’s FSS) Portugal Financial Literacy Survey FSA Consumer Awareness Survey New Zealand Financial Knowledge LSMS/FinScope Financial Services Jump$tart Financial Literacy Survey FSA Financial Capability Study 2005 Survey of Consumers Nov/Dec 2001 Wellbeing in Rural Fijian Households Consumer Fraud in the United States OECD/INFE Financial Literacy Survey National Financial Literacy Survey Singapore Russia Financial Literacy and Education Trust Financial Capability and Consumer Protection Consumer Protection Diagnostic Study Kenya Financial Capability, Financial Competence, and Knowledge Knowledge of financial of Concepts concepts and terms FI � � � � � � � � � � � � Knowledge Knowledge of financial of Services services and products, and the and function of financial service FI � � � � � � � � � � Products providers Understanding of risks and costs associated with using FCP � � � � � � � � financial products and services Awareness and understanding Know-How/ of consumer rights and Practical protection laws, and/or existing FCP � � � � Knowledge recourse mechanisms Knowledge of how to manage money, manage credit, prepare a budget, manage risks, maintain records, pay � � � taxes, plan for the long-term, save, and/or invest Knowledge of how to use financial services and products (open and use bank accounts, make payments, obtain loans, FI � � � obtain a mortgage, transfer or receive payments or remittances, etc.) Knowledge of how and where to seek financial advice or FCP � � information Table 1: Financial Capability Concepts Topic Concept Fund FinScope U.S. HRS Survey 2009 Global Findex FinAcess Kenya Financial Diaries Applicable Objectives Special Eurobarometer 230 Experiments (Ghana’s FSS) Portugal Financial Literacy Survey FSA Consumer Awareness Survey New Zealand Financial Knowledge LSMS/FinScope Financial Services Jump$tart Financial Literacy Survey FSA Financial Capability Study 2005 Survey of Consumers Nov/Dec 2001 Wellbeing in Rural Fijian Households Consumer Fraud in the United States OECD/INFE Financial Literacy Survey National Financial Literacy Survey Singapore Russia Financial Literacy and Education Trust Financial Capability and Consumer Protection Consumer Protection Diagnostic Study Kenya Financial Capability, Financial Competence, and Understanding financial documents (investment or FI, bank statements, bills, loan FCP � � � agreements, credit card statements and agreements) Knowledge of how to resolve problems with financial FCP � � products or services FI, Skills Numeracy skills FCP � � � � � � � FI, Literacy skills FCP � � � Reasons for saving/not Attitudes saving, borrowing, lending, or FI � � � � � � � � obtaining insurance Proclivity toward spending, paying bills on time, borrowing, budgeting, saving, planning � � � � � � � � � � long-term, lending, obtaining insurance, or investing Time preferences, patience, and attitudes toward risks � � � � � � � � Attitudes toward planning for /dealing with financial � � � � emergencies Attitudes toward receiving financial information/advice � � � � � FSA Consumer Awareness Survey Consumer Protection Diagnostic Study Kenya Financial Capability and Consumer Protection � � � � Consumer Fraud in the United States � Special Eurobarometer 230 LSMS/FinScope Financial Services � Experiments (Ghana’s FSS) Global Findex � FinScope � FinAcess Kenya � � � � � Survey of Consumers Nov/Dec 2001 � � � � � � Russia Financial Literacy and Education Trust � � � � � � � Fund FSA Financial Capability Study 2005 � � � � � � � Financial Diaries � � � � Financial Capability, Financial Competence, and � � � � Wellbeing in Rural Fijian Households U.S. HRS OECD/INFE Financial Literacy Survey � � � � New Zealand Financial Knowledge � � � � Survey 2009 National Financial Literacy Survey Singapore � � � � � Jump$tart Financial Literacy Survey � Portugal Financial Literacy Survey � � � Applicable Objectives FCP FI Table 1: Financial Capability Concepts habits (paying bills, loans, and Cash and credit management choices (kinds of information management, and budgeting Saving, investment, loaning, products, demanding clarity long term (investing, saving, How people make financial for information, comparing and/or borrowing behavior drop in income, and major emergencies, unexpected seeking financial advice) (record keeping, account their household finances How people plan for the from financial advisers, used, shopping around retirement, insurance) How people deal with How people manage credit cards on time) expenditures Expenditure Concept habits) Behavior Topic Table 1: Financial Capability Concepts Topic Concept Fund FinScope U.S. HRS Survey 2009 Global Findex FinAcess Kenya Financial Diaries Applicable Objectives Special Eurobarometer 230 Experiments (Ghana’s FSS) Portugal Financial Literacy Survey FSA Consumer Awareness Survey New Zealand Financial Knowledge LSMS/FinScope Financial Services Jump$tart Financial Literacy Survey FSA Financial Capability Study 2005 Survey of Consumers Nov/Dec 2001 Wellbeing in Rural Fijian Households Consumer Fraud in the United States OECD/INFE Financial Literacy Survey National Financial Literacy Survey Singapore Russia Financial Literacy and Education Trust Financial Capability and Consumer Protection Consumer Protection Diagnostic Study Kenya Financial Capability, Financial Competence, and Sources of financial FI, information/advice FCP � � � � � Following financial and economic information � � � Reading and reviewing contractual agreements and FCP � � � other financial documents Reasons for choosing/using/ owning a financial product or service (checking account, Attitudes saving account, loan, financial � � � � service provider, insurance, etc.) Reasons for not choosing/ using/owning a financial product or service (checking account, saving account, loan, � � � � � financial service provider, insurance, etc.) Accessibility of financial service providers � � � � Preferences for receiving financial information/ � � � � � � Confidence in using financial products and engaging with FCP financial service providers Table 1: Financial Capability Concepts Topic Concept Fund FinScope U.S. HRS Survey 2009 Global Findex FinAcess Kenya Financial Diaries Applicable Objectives Special Eurobarometer 230 Experiments (Ghana’s FSS) Portugal Financial Literacy Survey FSA Consumer Awareness Survey New Zealand Financial Knowledge LSMS/FinScope Financial Services Jump$tart Financial Literacy Survey FSA Financial Capability Study 2005 Survey of Consumers Nov/Dec 2001 Wellbeing in Rural Fijian Households Consumer Fraud in the United States OECD/INFE Financial Literacy Survey National Financial Literacy Survey Singapore Russia Financial Literacy and Education Trust Financial Capability and Consumer Protection Consumer Protection Diagnostic Study Kenya Financial Capability, Financial Competence, and Types of formal and/or informal financial products/mechanisms people use and have access to (informal or formal savings Behavior and credit systems, mobile FC � � � � � � � � � � � � � � � � banking, home banking, remittance systems, insurance systems, etc.) Frequency of usage of informal and/or formal financial products/mechanisms (savings, checking, credit, FC � � � � insurance, and/or remittance products/mechanisms) How people use different types of financial products/ mechanisms (save, lend, borrow, pay bills, insurance, FC � � � � � � and/or remittance transfers, etc.) Number of financial products and services people use/ FC � � � possess Sources of information that influenced choice of financial FC � product/service Accessibility of financial service providers FC � � FSA Consumer Awareness Survey � � � Consumer Protection Diagnostic Study Kenya � � � � Financial Capability and Consumer Protection � � � � Consumer Fraud in the United States � � � Special Eurobarometer 230 � � � � LSMS/FinScope Financial Services Experiments (Ghana’s FSS) Global Findex FinScope � FinAcess Kenya � Survey of Consumers Nov/Dec 2001 Russia Financial Literacy and Education Trust Fund FSA Financial Capability Study 2005 � � � Financial Diaries Financial Capability, Financial Competence, and Wellbeing in Rural Fijian Households U.S. HRS OECD/INFE Financial Literacy Survey New Zealand Financial Knowledge Survey 2009 National Financial Literacy Survey Singapore Jump$tart Financial Literacy Survey Portugal Financial Literacy Survey � Applicable Objectives FC FC FI FI FI Table 1: Financial Capability Concepts disputes about and/or seeking How people resolve problems recourse for financial service Experiences with financial service providers, general Satisfaction with financial or for fraudulent financial financial service provider Confidence in resolving Perceptions of financial Perceptions of financial products, services, and products, services, and unsatisfactory financial Reasons for or against advisers and preferred seeking recourse from with financial service sources of financial service providers service providers Experiences with information Concept problems providers behavior fraud Experiences Behavior Attitudes Topic Review of Existing Financial Capability and Literacy Measurement Instruments 49 References Alliance for Financial Inclusion (2010). “Financial inclusion measurement for regulators: Survey Design and Implementation.� AFI. Retrieved From http://www.afi-global.org/ services/publications/en/financial-inclusion-measurement-regulators-survey-design- and-implementation Anderson, Keith B. (2004). “Consumer Fraud in the United States: An FTC Survey, Federal Trade Commission Staff Report,� Federal Trade Commission Staff Report. Retrieved from http://www.ftc.gov/reports/consumerfraud/040805confraudrpt.pdf Anderson, Keith B. (2007). “Consumer Fraud in the United States: The Second FTC Sur- vey.� Federal Trade Commission Staff Report. Atkinson, Adele and Messy, Flore-Anne, (2012), “Measuring Financial Literacy: Results of the OECD/International Network on Financial Education (INFE) Pilot Study,� OECD Working Papers on Finance, Insurance, and Private Pensions, No. 15, OECD Pub- lishing. Banco de Portugal (2011).� Survey on the Financial Literacy of the Portuguese Popula- tion (2010), Final Report.� Portugal: Banco de Portugal. Available at http://cliente- bancario.bportugal.pt/pt-PT/Publicacoes/InqueritoLiteraciaFinanceira/Documents/ Survey%20on%20the%20Financial%20Literacy%20of%20the%20Portuguese%20 Population%20(2010)%20-%20Final%20Report.pdf. Colman Brunton (2009). “2009 Financial Knowledge Survey.� Auckland: Colman Brunton. Colmar Brunton (2006). “ANZ-Retirement Commission Financial Knowledge Survey, March 2006 Research Report.� Auckland: Colman Brunton. Demirguc-Kunt, Asli and Klapper, Leora (2012), “Measuring Financial Inclusion: The Glob- al Findex Database,� The World Bank Policy Research Working Paper, No. 6025. Di Castri, Simone, Flaming, Mark, Owino, Alexander, McKee, Katharine, Jentzsch, Nicola, Maina, Bilha, Ochieng, Moses, Collins, Daryl and Ahern, Brendan (2011). “Con- sumer Protection Diagnostic Study: Financial Consumer Protection in Kenya.� Janu- ary 16, 2011. Available at SSRN: http://ssrn.com/abstract=1843279 or http://dx.doi. org/10.2139/ssrn.1843279 Eurobarometer (2005), “Public Opinion in Europe on Financial Services,� Special Euroba- rometer 230, European Commission. Financial Sector Deepening Trust (FSDT), (2009). “FinAccess National Survey 2009: Dy- namics of Kenya’s Changing Financial Landscape.� FSDT, Nairobi. Financial Sector Deepening Trust (FSDT), (2009)., “The Demand for and Barriers to Ac- cessing Financial Services in Tanzania,� FSDT, Nairobi. 50 Review of Existing Financial Capability and Literacy Measurement Instruments Financial Services Authority (2011), “Consumer Awareness of the FSA and Financial Reg- ulation,� FSA. Groves, Robert M., F J Fowler, Mick P. Couper, James M. Lepkowski, Eleanor Singer, and R Tourangeau. Survey Methodology. New York, NY: Wiley. 2004. Grifoni, A. and F. Messy (2012), “Current Status of National Strategies for Financial Edu- cation: A Comparative Analysis and Relevant Practices�, OECD Working Papers on Finance, Insurance and Private Pensions, No. 16, OECD Publishing. Hilgert, Marianne, Jeanne Hogarth, and Sondra Beverly (2003), Household Financial Man- agement: The Connection between Knowledge and Behavior, Federal Reserve Bul- letin, 309-32. Kempson, Elaine, Atkinson, Adele, McKay, Stephen, and Collard, Sharon (2006). “Levels of Financial Capability in the UK: Results of a baseline survey.� UK: Financial Ser- vices Authority. Levine, R.. (1997), “Financial Development and Economic Growth: Views and Agenda�, Journal of Economic Literature, pages 35, 688–726. Lusardi, A, and Mitchell, O (2011). “Financial Literacy around the world: An overview�, Jour- nal of Pension Economics and Finance, No.10, pages497–508. Lusardi, Annamaria, and Olivia S. Mitchell. 2011. “Financial Literacy and Planning: Implica- tions for Retirement Wellbeing,� in Olivia S. Mitchell and Annamaria Lusardi (eds.), Financial Literacy: Implications for Retirement Security and the Financial Market- place. Oxford: Oxford University Press, pp. 17–49. Mandell, Lewis (2008), “The Financial Literacy of Young American Adults: Results of the 2008 National Jump$tart Coalition Survey of High School Seniors and College Stu- dents,� The Jump$tart Coalition for Personal Financial Literacy, Washington, DC. Media Research Consultants Pte Ltd. (2005), Quantitative Research on Financial Lit- eracy Levels in Singapore, prepared for MoneySENSE Financial Education Steer- ing Committee. Retrieved: http://www.mas.gov.sg/~/media/resource/news_room/ press_releases/2005/Summary%20Report.ashx OECD, (2012). “OECD/INFE High-Level Principles on National Strategies for Financial Education, OECD 2012.� Paris: OECD. Sibley, Jonathan, (2010). “Financial Capability, Financial Competence and Wellbeing in Rural Fijian Households.� UNDP Pacific Centre, April. 51 Appendix B: Survey Reviews B.1. ANZ-Retirement Commission Financial Knowledge Survey 1. General Introduction In 2005, the New Zealand Retirement Commission, funded by the Australia and New Zealand Banking Group (ANZ), commissioned Colmar Brunton1 to undertake a study of adult financial knowledge in New Zealand. The New Zealand Financial Knowledge sur- vey was the first study to measure adult financial knowledge. Previous studies generally focused on the financial literacy and knowledge of secondary school students. The study sought to also measure investor behavior for the Ministry of Economic Development. A number of past surveys on financial knowledge had been conducted in Australia; specifi- cally, the ANZ-funded 2003 Australian financial knowledge survey, which informed the New Zealand study. 1.1. Purpose of the Survey The New Zealand Financial Knowledge study had a number of objectives: 1. To determine areas of and populations with low financial knowledge and assist educators in targeting programs in those areas 2. To develop a benchmark of financial knowledge across the adult population in order to measure trends in financial knowledge and target financial education pro- grams 3. To identify aspects of financial products and services that pose difficulty for New Zealanders in order to assist financial service providers with their communications and design 4. To inform the reform of legal mechanisms for consumer protection 5. To determine participation rates, investment behaviors, and level of sophistication of consumers and retail investors in the securities market 1 Colmar Brunton is an Australia market research agency that specializes in product, services, and social research. 52 Review of Existing Financial Capability and Literacy Measurement Instruments 2. Methodology 2.1. Questionnaire 2.1.1. Original development of questionnaire (source of questions, methods for designing new questions, etc.) To guide the research process, a steering group was organized, including ANZ representa- tives, academic and financial experts, and representatives from the Retirement Commis- sion and the Ministry of Economic Development. Based on the financial literacy frame- works used in Australian and U.K. studies, the steering committee selected a range of skills and knowledge to be measured in the New Zealand survey and developed a Financial Knowledge Framework. The survey instrument was designed based on a Financial Knowledge Framework that detailed the knowledge and skills required for a person to be financially literate at the basic and advanced level. The survey questions were designed to reflect the most important skills and knowledge under each category of the framework, and were divided into ques- tions that would be used to calculate financial knowledge scores and filter questions. 2.1.2. Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) Following work on the Financial Knowledge Framework that was established in the ques- tionnaire development stage, the final version of the instrument contained questions cover- ing the following topics (and subtopics): 1. Math, standard literacy, and understanding of financial records 2. Understanding of financial terms • Understanding financial records (such as bank statements) • Understanding of commonly used or important financial terms (for example, asset, liability, capital gain, net worth, equity, secured loan) • True/false statements relating to credit cards • Knowledge of which is better: fixed vs. variable interest rate • Knowledge of compound interest • Effect of inflation on saving • Correctly identifying New Zealand age on entitlement (retirement) 3. Financial competence • Money management and payment methods • Use of different payment methods: automatic payments, direct debits, credit cards, and checks 4. Understanding mortgages • Understanding of how to minimize interest on mortgages Review of Existing Financial Capability and Literacy Measurement Instruments 53 5. Budgeting • Describing a budget • Identifying the importance of budgeting 6. Debt management • Identifying responsibilities of a debt guarantor • Understanding debt consolidation 7. Managing risk 8. Savings • Financial holdings (term deposit, unit trust/managed fund, high-interest call ac- count, or personal retirement savings) 9. Investing • Concepts of risk vs. return and the variability of returns on investment. • Understanding that diversifying the investment portfolio reduces risk • Use of Internet banking and telephone banking • Use of home equity for investment 10. Planning for the future/retirement savings • Having financial goals and a financial plan • Having a will and an enduring power of attorney. • Financial planning for retirement 11. Consumer rights • Internet banking passwords • Identifying features of financial scams • Sharing Internet banking passwords with partner, friend, or bank staff • Reading and understanding the investment statement that explains the details about the investment before investing 12. Getting financial advice • Attitude toward financial advisers (ask about their qualifications and experience) 2.2. Sampling The study measured the financial knowledge of New Zealanders aged 18 and over. The full sample consisted of 856 New Zealanders aged 18 or over, including two ethnic booster samples (104 Maori and 96 Pacific People). The first booster sample of interviews was undertaken with Maori in order to provide sufficient sample for analysis: 181 people who identified as Maori were interviewed. (Booster interviews were done in areas where 20 t% or more of the population identified as Maori.) These were then weighted back to be representative of the population (12t%). The second booster sample was undertaken with 54 Review of Existing Financial Capability and Literacy Measurement Instruments Pacific people in order to provide sufficient sample for analysis: 133 Pacific people were interviewed. (Booster interviews were done in areas where 20 % or more of the population identified as Pacific ethnicity.) These were then weighted back to be representative of the population (5 %). 2.3. Implementation 2.3.1. Countries/regions of application The survey was implemented in New Zealand. 2.3.2. Interview method (face-to-face, telephone, CAPI, etc.) The interview method was face-to-face. 2.3.3. Response rate The response rate was 60 %. 3. Analysis 3.1. Methods used for analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) In analyzing the survey results, questions were divided between those that would feed into the financial knowledge scores and filter to knowledge questions or descriptors. Respon- dents were given one point for each correct answer. Where multiple responses were pos- sible, these were given part scores so that the overall score was in line with other questions of equal value. Each question was identified as measuring either basic or advanced knowl- edge, with the majority falling under basic and those tagged advanced being predominantly investment related questions. It was agreed that, as financial knowledge reflects each person’s circumstances and expe- riences, people should not be disadvantaged because they were not familiar with products and services they might never use or need. Therefore, only the questions testing basic knowledge were used to determine the overall financial knowledge scores. The advanced questions, relating primarily to investment, were scored separately. Having taken this ap- proach, scores were leveled across basic knowledge. Planning for the future, saving, un- derstanding mortgages, and budgeting were deemed the most important areas of financial knowledge. The basic knowledge questions had a maximum score of 58.5 and the ad- vanced knowledge questions had a maximum score of 18.5. 3.2. Key results The population was divided into four knowledge groups based on their score on the survey. Review of Existing Financial Capability and Literacy Measurement Instruments 55 Financial Knowledge Main Traits Group §§ About a third of the population §§ Were significantly more likely to be: • Female • 18–24 years, 75+ years • Maori and Pacific people • Not in paid employment – retired or home duties or social welfare beneficiary • Semi-skilled and low level occupations – laborer or manual, agricultural, or domestic worker • Primary or basic secondary school education Low • Low household income §§ Have lower levels of knowledge in all those areas that could help them manage their personal finances §§ Much more likely to find it hard to make ends meet and not manage if they had a major loss of income. §§ Less likely to save §§ Tend to make greater use of personal loans, leasing, hire purchase, or other debt on household items §§ Have low knowledge of financial concepts and how to use the range of financial products and services available. §§ Much more reliant on cash §§ Evenly split between men and women §§ Were more likely to be: • European ethnicity • In paid employment Medium §§ Were less likely to be Maori and Pacific ethnicity §§ A wide spread of household income §§ Half were regular savers §§ Weaker in their knowledge of the following areas than they were in other areas of knowledge §§ Significantly more likely to be: • Male • 40–54 years of age • European • Tertiary educated • White collar occupations High • Home owners • High household income §§ Fewer find it hard to make ends meet, feel out of control with their borrowing, or feel they would be unable to manage if they had a major loss of income §§ Much more likely to save regularly §§ Impulse buying §§ Generally scored well in most areas of financial knowledge 56 Review of Existing Financial Capability and Literacy Measurement Instruments Financial Knowledge Main Traits Group §§ About one in seven New Zealanders (15%) can be considered to have advanced financial knowledge relating to investing. §§ Were more likely than the average to be: • Male • 35–54 years of age • University educated • Professional or senior government official • Household income of $100,000+ Advanced • High savings/investments balances • European • Home owners • In households with no children/none at home §§ Exhibit strong basic knowledge of financial concepts as well as advanced financial concepts §§ In control of their borrowing and debt §§ Feel financially confident B.2. Consumer Fraud in the United States: The Second FTC Survey 1. General Introduction In 2003, the U.S. Federal Trade Commission (FTC), in cooperation with survey research firm Synovate, conducted a study of consumer experiences with specific types of fraud. In 2005, the FTC implemented a second survey that included additional questions on four new areas of fraud. Both surveys measure financial fraud, employment fraud, and product fraud. Synovate conducted telephone interviews in English and Spanish with 2,500 ran- domly selected adults in 2003 and 3,888 adults in 2005. 1.1 Purpose of the Study The FTC designed the Consumer Fraud survey for a number of reasons: 1. To enhance general understanding of consumer fraud in the United States and to gain information on the characteristic of victims. 2. To measure consumer experiences in the previous year with the specific types of fraud most frequently reported in the FTC’s Consumer Sentinel fraud complaint system. 3. To help the FTC target its education campaigns toward particular consumer groups at risks of falling victim to fraud, and to inform its law enforcement activities. Review of Existing Financial Capability and Literacy Measurement Instruments 57 2. Methodology 2.1 Questionnaire Development and Revision In designing the survey instrument, the FTC reasoned that consumers might have different notions of fraud; therefore, it constructed the surveys to measure consumer experiences with specific types of fraud. The FTC focused on those fraudulent experiences that had the greatest number of complaints in its Consumer Sentinel Database of Fraud Complaints and had led to enforcement actions. In the 2003 survey, the FTC asked consumers about 10 experiences with specific types of fraud and two general types of experiences that may indicate fraud. For the 2005 survey, however, the instrument included four additional questions on specific types of frauds. In addition, some questions from 2003 were revised to gain a better understanding of whether consumers had experienced a specific type of fraudulent action. The 2003 survey instru- ment was revised with the assistance of staff of the FTC Bureau of Consumer Protection. 2.1.2 Final structure of questionnaire The original instruments included the following specific areas of consumer fraud, including experiences with employment fraud, product fraud, and (as related to our study) financial fraud. 1. Paying an advance fee to obtain a loan or credit card that a consumer was prom- ised or guaranteed to receive, 2. Being billed for a buyers’ club membership a consumer did not agree to purchase, 3. Purchasing credit card insurance 4. Purchasing credit repair services 5. Paying money or making a purchase to receive a promised prize and then not receiving the prize or receiving a prize that was not as promised, 6. Being billed for Internet services a consumer did not agree to purchase, 7. Purchasing a membership in a pyramid scheme, 8. Being billed for information services provided either over the Internet or by pay- per-call telephone service that a consumer had not agreed to purchase, 9. Making a payment to someone who represented that as a result of making the pay- ment a consumer would receive a government job, and 10. Purchasing a business opportunity where the seller made earnings claims that were not realized or promised assistance that was not provided. 58 Review of Existing Financial Capability and Literacy Measurement Instruments The questionnaire also measured two additional areas of possible consumer fraud: 1. Paying for a product or service that a consumer does not receive 2. Being billed for a product, other than the specific products identified above, that a consumer had not agreed to purchase. The 2005 questionnaire included one additional question on financial fraud and three ques- tions on fraudulent experiences related to purchase of weight-loss products, work-at-home programs, and foreign lotteries. Below is the question on financial fraud: • Debt Consolidation: Paid money to someone who promised to help consumers pay off debts. However, seller did not in fact make it easier for consumers to pay off their debt. Seller also failed to make payments to the consumers’ creditors as promised, if the service promised to make such payments. (7 questions) In addition to the specific areas outlined above, the survey instrument included questions on how fraudulent offers were promoted, methods used to pay for fraudulent transaction, and the cost of fraudulent transactions. 2.1.1 Questions on financial fraud Of the 16 areas included in the survey instrument, the following seven areas dealt directly with the issue of fraudulent financial products and services: 1. Credit Card Insurance: Purchased insurance against the misuse of a lost or sto- len credit card. (5 questions) 2. Advanced Fee Loans: Paid an advance fee to obtain a promised or guaranteed loan or credit card; promised credit was not received. (7 questions) 3. Credit repair: Paid someone who promised to remove negative, but accurate, information from the consumer’s credit report or promised to provide information on how the consumer could establish a new credit record that would not contain negative information in the consumer’s current credit report. (7 questions) 4. Debt Consolidation: Paid money to someone who promised to help consumers pay off debts. However, seller did not in fact make it easier for consumers to pay off their debt. Seller also failed to make payments to the consumers’ creditors as promised, if the service promised to make such payments. (7 questions) 5. Pyramid Schemes: Purchased membership in a pyramid scheme; failed to earn at least half of the amount the promoter promised would be earned. (2005 survey asked about the amount of money people actually made) (7 questions) 6. Business Opportunity: Purchased a business opportunity but did not earn at least half as much as promised or did not receive promised assistance. (2005 survey asked about the amount of money people actually made) (9 questions) 7. Unauthorized billing: other products (general question): Billed for a product or service consumer did not agree to purchase, products other than those identified above. (7 questions) Review of Existing Financial Capability and Literacy Measurement Instruments 59 2.2 Sampling In total, Synovate interviewed 2,500 adults in 2003 and 3,888 adults in 2005. Respondents were selected at random using random digit dialing. The sample included adults aged 18 or over in all 50 states and the District of Columbia. A number of minority groups were oversamples, including Hispanics, African Americans, American Indians and Alaska Na- tives, and Asians. Nevertheless, weights were applied to the survey data to ensure that the results were nationally representative. 2.3 Implementation 2.3.1 Countries/regions of application The survey was implemented in the 50 U.S. states and the District of Columbia. 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) Interviews were conducted using computer-assisted telephone interviewing in both English and Spanish. 3. Analysis 3.1 Method used for analysis (methods used to construct measures of literacy/capability/inclusion, etc., and for further analysis) To determine victims of fraud, the FTC included questions about the amount of money that had been lost. Only respondents who indicated that they had experienced a specific fraudulent experience and had lost money were considered victims. 3.2 Key results The more recent 2005 survey indicated that 13.5 % of American adults (30.2 million people) had been victims of one or more of the areas of frauds measured. The results by specific financial fraud are as follows: Area of Fraud Number of Victims Percentage (millions) of Victims Credit card insurance 2.1 0.9 Advance fee loans 1.7 0.8 Credit repair 1.2 0.5 Business opportunities 0.8 0.4 Pyramid schemes 0.8 0.4 Debt consolidation 0.5 0.2 Unauthorized billing–Other products 5.2 2.4 References Keith B. Anderson (2007), Consumer Fraud in the United States: The Second FTC Survey, Federal Trade Com- mission Staff Report, Retrieved from http://www.ftc.gov/opa/2007/10/fraud.pdf Keith B. Anderson (2004), Consumer Fraud in the United States: An FTC Survey, Federal Trade Commission Staff Report, Retrieved from http://www.ftc.gov/reports/consumerfraud/040805confraudrpt.pdf 60 Review of Existing Financial Capability and Literacy Measurement Instruments B.3. Consumer Awareness Survey 1. General Introduction Since 2003, the U.K. Financial Services Authority (FSA) has implemented the annual Con- sumer Awareness Survey. The survey was designed to measures consumer awareness of financial regulation, confidence in the regulatory regime, and to gather information on certain aspects of consumer financial knowledge and behavior, in order to evaluate the FSA’s effectiveness in identifying and mitigating risks. In January 2011, TNS Research International implemented the 2011 Consumer Awareness survey on behalf of the FSA. The study included 2,064 face-to-face interviews with adults from Great Britain, and the sample was designed to be representative of the British population in terms of age, sex, social class, and region. 1.2 Purpose of the Study The purpose of the Consumer Awareness Surveys is to help the FSA assess whether it is meetings its strategic goals in the area of consumer protection. This is done by gathering information on gauge consumer confidence in the FSA and its regulatory regime, cogni- zance of financial regulation, experience with financial crime, and their financial behavior and knowledge. 2. Methodology 2.3 Questionnaire Since 2003, the survey questionnaire has evolved and the content is broadly similar to previous surveys. Older surveys have been used as a starting point and adjusted in ac- cordance with the FSA’s changing priorities. 2.4 Sampling For the 2011 survey, a two stage sampling design was employed in selecting the sample. Census enumeration districts in Great Britain were stratified by region, and 149 sampling points were then selected. The sample was designed to be representative of Great Brit- ain’s population in terms of age, sex, social class, and region. TNS Research International conducted face-to-face interviews with 2,064 adults across Great Britain. 2.3 Implementation 2.3.1 Countries/regions of application The survey was implemented across Great Britain. 2.3.2 Testing and piloting All questionnaires were pre-tested by survey researchers at TNS before the implementa- tion of the survey. 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal Review of Existing Financial Capability and Literacy Measurement Instruments 61 interviewing, etc.) Interviews were conducted face-to-face using CAPI. 3. Analysis 3.2 Method used for analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) Results of the study are weighted to the population to be representative of age, sex, region and social class. 3.2 Key results Below are the key results from the 2011 FSA Consumer Awareness Survey: Financial Knowledge Main Traits Group Financial Product About 12% of respondents had no financial products. Younger and Holdings older less affluent groups were less likely to have financial products and to be aware of the FSA and financial regulation. Awareness of FSA 36% of respondents were aware of the FSA, around the same level since 2003. Overall awareness of the FSA is highest among the 35- 44 (48%), 45-54 (53%) and 55-64 (47%) age groups, and over half of the 35-64 communities are aware of the FSA. Knowledge of the FSA is highest among those who own more risky financial products, such as direct ownership of shares (75%) and equity ISAs (64%). Knowledge of the FSA is low among those with a low risk financial product only (27% Understanding of the 33% of respondents believe that the main responsibility of a FSA’s responsibility financial watchdog should be to ensure that firms treat their customers fairly. About 17% believe that a financial watchdog should ensure that only appropriate people or firms can operate in the market, and 10% in ten thinks that such an organization prosecutes firms and individuals. 25% of respondents that a financial watchdog is there to prevent the false advertising of products and services. FSA Effectiveness 60% of respondents were either very or fairly confident that the FSA was effectively regulating the financial services industry. 62 % of those surveyed were confident that firms followed the FSA’s regulations. Problems with 11% of respondents surveyed report having a problem with a Financial Service financial services firm in the previous 12 months. 33% of these Providers problems had to do with poor customer service and slightly fewer resulting from handling of complaints. Other problems included money transfer delays (12%); confidence in advice given (9%); being refused service (8%), and pressure selling (7%). 55% of respondents were very or fairly confident that complaints would be resolved fairly, but over one in six were not confident of this outcome. 62 Review of Existing Financial Capability and Literacy Measurement Instruments B.4. Consumer Protection Diagnostic Study, Kenya 1. General Introduction In 2010, the Kenya Financial Sector Deepening (FSD Kenya) program in cooperation with the Consultative Group to Assist the Poor (CGAP) launched a diagnostic study of issues in consumer protection in order to provide the Ministry of Finance of Kenya with recommen- dations for enhancing financial education and consumer protection (FSD 2010). As part of the diagnostic study, FSD Kenya and CGAP organized 14 focus group discussions and a national survey of 1,548 consumers implemented by Synovate. Both the survey and focus groups focused specifically on consumer experiences with common financial services. 1.1. Purpose of the Study The purpose for the diagnostic study and survey was twofold: 1. To assist the government of Kenya in understanding the key issues that consum- ers face with regard to consumer protection; and 2. To provide recommendations that could inform additional diagnostic work on con- sumer protection and consumer education and protection projects. 2. Methodology 2.1. Questionnaire The diagnostic study employed a number of different methods for gathering information, including desk research, interviews with financial service providers and regulators, focus groups, and a national survey. The focus groups and survey were designed to provide direct qualitative and quantitative information on consumer experiences with financial pro- viders and consumer protection. FSD Kenya organized 14 focus groups with a total of 112 consumers from different socio- economic backgrounds. The focus group discussions were a basis for the questions in the survey and also helped the team determine how to pose certain questions. 2.2. Sampling The diagnostic study survey consisted of interviews with 1,548 adult consumers who had experience using financial services. FSD and Synovate selected a sample of 1,000 people proportionate to population size across provinces and districts. To adequately measure consumer experiences with different types of financial services, a bolster sample of 548 consumers was added to the sample. This was to ensure that at least 50 respondents had experience with each type of financial product or service the survey sought to measure. The population sample is therefore biased toward consumers that are more engaged with financial services. Review of Existing Financial Capability and Literacy Measurement Instruments 63 3. Analysis 3.1. Key results Many respondents held some form of savings with various types of informal and formal financial service providers. Respondents noted that misappropriation of money was more prevalent with informal financial products. However, while banks were trusted the most as a savings instrument, some users believed that penalties and interest fees were not properly explained. Many users (91–93 %), however, were able to effectively resolve their issues with financial institutions yet 5 % to 7 % were not able to satisfactorily resolve their problems. Few respondents had long-term investments. Those who did invest reported that they were promised a certain return from a salesperson (25 %), pressured to make an investment (14 %), or left without written documentation on the terms of the product (32 %). Forty-four % of the survey respondents reported that they had been approached to invest in a pyra- mid scheme and 8 % ultimately reported investing. Of those that did invest, 22 % did not complain because they were not certain where to field their complaints, while 40 % did not complain because they did not feel the process would be worthwhile. B.5. Financial Access in Kenya (FinAccess) 2009/2011 1. General Introduction In 2009, the Kenya Financial Sector Deepening Program (FSD) and The Financial Access Partnership (FAP)2 commissioned a survey to measure financial access levels in Kenya. The 2009 Financial Access (FinAccess 2009) survey built upon an earlier 2006 FinAccess survey. In addition to questions on financial access, the survey sought to measure consum- er financial knowledge, behavior, and capabilities. Based on the interim findings from the 2009 FinAccess survey, a more comprehensive questionnaire including more questions on financial capabilities was planned for the FinAccess 2011 survey. Synovate/Steadman Group fielded the survey with the assistance of FSD, FAP, and the Kenyan National Bureau of Statistics. A total of 6,598 interviews were conducted based on a nationally representative sample of people from all socioeconomic backgrounds in Kenya. 1.1 Purpose of the Survey The FinAccess surveys were designed and implemented for the following reasons: • To gather a baseline measure of financial access, and to track progress in govern- ment and donor-led initiatives in this area; • To inform policy makers about the main barriers to financial access and areas for necessary reforms; • To assist the private sector in reaching and providing products to market segments with limited financial access; and • To support academic research on the links between financial access, growth, and poverty reduction. 2 FAP was created from representatives of both public and private sectors to guide the work on mapping supply and demand for financial services. 64 Review of Existing Financial Capability and Literacy Measurement Instruments 2. Methodology 2.1 Questionnaire 2.1.1 Original development of questionnaire (source of questions, methods for designing new questions, etc.) FSD and the Financial Access Partnership developed the survey instrument. Survey ques- tions gather information on general demographics, product usage, money transfers, use of mobile phones and technology, livelihoods and income, savings, credit, insurance, general money matters, and expenditure patterns. To measure financial capability, the FinAccess survey included questions that represented financial capability in access diverse financial services and managing money. These ques- tions sought to capture consumer knowledge, attitudes, and behaviors. For the FinAccess 2011 instrument, a number of revised questions were included that sought to gauge actual financial knowledge and past behaviors. The FinAccess survey instrument is translated into Swahili and other major Kenyan languages, including Luo, Meru/Embu, Kissi, Luhya, Kalenjin, Kamba, Somali, Turkana, and Maasai. 2.1.2 Revisions to the questionnaire Unlike the 2006 survey instrument, the FinAccess 2009 instrument measured two more topics, effective literacy/numeracy and financial literacy. Based on both the analysis of Fi- nAccess 2009 data and the fieldwork conducted with Kenyan consumers during July 2010, the survey team revised the questionnaire for the FinAccess 2011 instrument by expanding the financial capability questions. 2.1.3 Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) The final FinAccess 2009 survey instrument was 49 pages and included the following top- ics and sections: • General demographics • Effective literacy and numeracy • Biggest risks • Financial literacy • Livelihood and income • Product usage • Money transfers • Savings • Community-based groups • Credit/loans • Insurance • Mobile phone and technology usage • Vulnerability and general psychographics • Housing conditions • Personal expenditure and minimum household income Review of Existing Financial Capability and Literacy Measurement Instruments 65 Below are specific topics on financial capability that were included in the 2009 survey and the additions made to the 2011 instrument: FinAccess 2009 Section Purpose A: Control §§ Asks questions about saving and spending behaviors. over finances §§ Includes whether the word “budgeting� is part of one’s vocabulary and whether the need for disciplined spending is part of one’s goals. It is also a comfort level that one has in knowing how well they are managing their money, or whether they feel out of control. §§ Reasons for borrowing and saving. B: Short- and §§ Identifies both attitudes and intentions about saving and how well long-term financial those intentions are carried through into action. planning §§ Indicators to examine actual financial behaviors indicating planning to cover financial needs across a range of time horizons: income and consumption smoothing in the short term, preparing to respond to emergencies/shocks and investment in the more distant future. C: Appropriate use §§ The ability to assess and judge different financial product options. of financial services §§ This includes whether respondents recognize the names of different financial services providers and whether they are using these options inadvisably. D: Ability to §§ Enquires into the parentage of monthly income spent on debt Manage Debt service, the purpose of borrowing, and the number of credit sources. FinAccess 2011: Additional questions from the analysis of FinAccess 2009 data and fieldwork conducted in 2010 Knowledge §§ Familiarity with practices of service providers (for example, paying dividends to members, providing loans). §§ Understanding key concepts about engaging with the formal financial sector. §§ Knowledge about how to resolve a problem with common financial services. Attitude §§ Planning day-to-day expenditures. §§ Discipline to stick to a budget. §§ Borrowing to buy things but don’t need to survive. §§ Planning for common emergencies. §§ Able to cope with income shocks. §§ Planning adequately for old age. §§ Saving in advance for anticipated expenditures. §§ Demanding clarity from providers (for example, asking questions before taking out loans). §§ %age of monthly expenditures spent on debt service. §§ Number of formal and informal credit sources. 66 Review of Existing Financial Capability and Literacy Measurement Instruments 2.2 Sampling 2.2.2 Unit of analysis Survey respondents were individuals aged 16 years or older. 2.2.3 Sampling strategy (sampling frame, method, sample size, etc.) The Kenya National Bureau of Statistics designed the sample based on the National Sam- ple Survey and Evaluation Programme (NASSEP IV), developed from the Population and Housing census of 1999. NASSEP IV has 1,800 clusters, of which 1,260 are rural and 540 urban. Using the NASSEP base, 650 clusters were selected for the FinAccess 2009 study us- ing a stratified three-stage design. First, clusters were selected to ensure that they were representative at national, provincial, and urban/rural levels. Second, the survey team se- lected 12 households within each cluster and targeted 10. Third, respondents within the households were selected based on a listing of all household members aged 16 years and above, using the Kish Grid-diagram. 2.3 Implementation 2.3.2 Countries/regions of application The survey was implemented in Kenya. 2.3.3 Interview method (face-to-face, telephone, CAPI, etc.) The survey was completed using face-to-face interviews. 2.3.4 Average duration of interview and total cost The average duration for the FinAccess 2009 survey was 60 minutes. 3 Analysis 3.1 Key Results The report describes various aspects of financial access and capability among different population groups. Below are some key findings: 1. About 23 % of the population aged 18 years or older was formally included in the financial system and 17.9 % of Kenyans used nonbank financial services, includ- ing mobile banking. 2. Usage of formal financial services increases with education with those with 4.9 % of those with no education using financial services compared with 70.3 % of those with tertiary education 3. About 49 % of rural respondents and 60.1 % of urban respondents used saving products. Respondents used a variety of savings products and a lot of Kenyans Review of Existing Financial Capability and Literacy Measurement Instruments 67 were aware of the concepts of saving and budgeting. 4. Most respondents did not seem to have a problem with debt but some showed signs of credit difficulties. 5. Most Kenyans were familiar with financial terms like savings account (91.2 %), budget (71.3 %), checking (65 %), insurance (52.3 %), interest (50.9 %), and pen- sion (52.7 %). Further analysis of the FinAccess data helped in broadly identifying three consumer seg- ments: 1. Struggling Excluded: The people in this category are predominately poor and rely on irregular sources of income-like remittances, small businesses, and ag- riculture. They have few financial instruments to help manage their uncertain in- comes and tend to have one, usually informal, savings mechanism. 2. Discipline Planners: People in this group are poor but strong at using financial tools for short- and long-term planning. Nevertheless, this group tends to use more informal than formal financial instruments. People in this group are from rural and urban areas and represent a mix of education levels. 3. Engaged Elite: People in this group are more educated, relatively better, and have full-time salaried positions compared with the groups above. This group is more familiar with using financial products and services; however, they are weak at using financial tools to manage their spending. In addition, people in this group tend to have a larger number of creditors. 3.2 Policy Applications Based on the findings of the FinAccess 2009 survey, the FSD and other stakeholders have outlined a number of possible strategies for addressing the needs of the three categories of consumers: the struggling excluded, the disciplined planners, and the engaged elite. 1. Struggling Excluded: Work with community based organizations and radio pro- grams develop a program that would introduce this group to easily accessible formal financial services (especially for savings), assist them with strategies for consumption smoothing, and educate them on how to use financial products to mitigate prepare for financial shocks/emergencies. 2. Disciplined Planners: Work with media and employ community workshops to introduce this group to financial regulation mechanisms, assist them with long- term planning and investments, and help them with preparing for financial shocks/ emergencies. 3. Engaged Elite: Assist this group with credit management and planning for long- term planning. Such assistance could be provided via the media, employer-based financial education workshops, or one-on-one advice. 68 Review of Existing Financial Capability and Literacy Measurement Instruments References Daryl Collins, Julie Zollmann and Bilha Maina, (2010), Financial Capability in Kenya: Findings from FinAccess 2009, Financial Sector Deepening, http://www.fsdkenya.org/pdf_documents/11-04-13_Financial_capability_in_ Kenya.pdf (Accessed June 14, 2012) Financial Sector Deepening (2010), FinAccess National Survey 2009: Dynamics of Kenya’s changing financial landscape, FinAccess, http://www.fsdkenya.org/finaccess/documents/09-06-10_FinAccess_FA09_Report.pdf (Accessed October 3, 2012) The Steadman Group Research Department (2007), Financial Access in Kenya: Results of the 2006 National Survey, FinAccess, http://www.fsdkenya.org/finaccess/documents/FinaccessReportFINALMain.pdf (Accessed October 3, 2012) B.6. Financial Capability, Financial Competence, and Wellbeing in Rural Fijian Households (Fiji Fin Cap) 1. General Introduction In 2007/08, the Reserve Bank of Fiji, in partnership with the Pacific Financial Inclusion Pro- gram3 and the Fiji Bureau of Statistics, designed and implemented a quasi-experimental survey to understand the ways in which rural Fijian villagers managed money and ac- cessed the formal financial system, and the nature of the relationship between villagers’ levels of financial competence and the wellbeing of their households. The study was devel- oped as part of an impact evaluation to determine the extent to which villagers’ participation in a United Nations Development Programme (UNDP) Pacific Center Financial Literacy Training Workshop and/or experience with the ANZ Rural Banking Service had facilitated an improvement in their household wellbeing. The National Center for Small and Micro En- terprise (NCSMED) collected data for the survey from 14 villages in the Naitasiri Province of Fiji, and interviewed 389 villagers from 200 households. 1.1. Purpose of the Study The purpose of the Fiji survey was to determine whether villagers’ participation in a fi- nancial literacy program and engagement with reliable banking services increased the financial competence and capability of household financial decision makers and whether this increased competence improved enhanced their ability to manage their household’s fi- nances, and consequently whether this improved their household wellbeing. The initiatives under consideration comprised two different programs: 1. A rural financial literacy program implemented by the UNDP Pacific Center in part- nership with NCSMED 2. A concurrent mobile vehicular banking service or Rural Banking Service program implemented by ANZ Bank The objective of the training program was to provide a means by which villagers could ac- 3 United Nations Capital Development Fund (UNCDF), United Nations Development Programme (UNDP), the European Union/Africa, Caribbean and Pacific Microfinance Framework Program (EU/ACP), the Government of Australia (AusAID) and the Government of New Zealand (NZAID) through its financial contribution to the UNDP Pacific Centre. Review of Existing Financial Capability and Literacy Measurement Instruments 69 quire the knowledge and skills required to undertake basic money management, including the management of a simple bank account to enable safe custody of savings, while at the same time providing villagers with access to financial services through the Rural Banking Service. To determine the impact of these two initiatives, the Fiji survey’s objective was to post-factor examine and compare the following elements in order to determine the impact of the two initiatives: 1. The financial competence and wellbeing of villagers who participated in either or both of the programs 2. The financial competence and wellbeing of a control group of villagers who had not participated in either or both of programs 1.2 Link Between Purpose and Chosen Approach The Fiji study is based on the premise that the management of money in the household is as important as the amount earned by the household. Therefore, the study seeks to un- derstand the relationship between financial competence and household income by looking at rural households’ ability to use financial knowledge to enhance wellbeing using present income to purchase household assets, reinvest in income generating activity, and plan for future income shocks, among other activities. The study develops a casual model of the relationship between financial literacy, financial competence, and household wellbeing by collecting data in the following areas: • managing money • making financial choices • planning ahead • getting help • attitude to money • subjective wellbeing • objective wellbeing The instrument for the Fiji study, therefore, builds on earlier financial capability studies implemented in the United Kingdom, Australia, and New Zealand, including the U.K. 2006 Financial Services Administration (FSA) financial literacy and capability survey, the Asian Development Bank (ADB) Experimental Methodology for Evaluation of Savings Product Innovations, and the International Wellbeing Group’s Personal Wellbeing Index. The team sought to choose instruments that had been used in a range of environments and that could be localized to fit the conditions in rural Fiji. Finally, in order to determine correla- tions between financial competence and wellbeing and also to specifically determine the impact of the UNDP and ANZ programs, the team opted for an ex post facto experimental research design that allow for comparison between a control and treatment group from vil- lages in the Naitisiri province that had participated in the financial capability and inclusion development interventions. 70 Review of Existing Financial Capability and Literacy Measurement Instruments 2. Methodology 2.1. Questionnaire 2.1.1 Original development of questionnaire (source of questions, methods for designing new questions, etc.) The survey team developed a questionnaire to cover the areas of financial knowledge, skill, and behavior, as well as attitudes toward money, and measures of wellbeing. The Fiji survey questionnaire was developed based on the following existing instruments: • Financial knowledge, skill, and financial behavior: Questions on financial knowledge, competency, and behavior were based on the U.K. FSA Adult Fi- nancial Capability Framework (2006) and the Financial Capability Baseline Sur- vey Questionnaire (2006). The survey team only used questions from both stud- ies that focused on basic or intermediate financial capabilities and did not include questions that covered advanced capabilities as these did not relate to the finan- cial activities of rural Fijian households. The survey team also sourced addition- al knowledge and skills from the ANZ-NZ 2006 Retirement Commission Survey. In August 2006, a group of 62 subject matter experts with considerable knowledge of the Fijian financial environment and rural financial requirements met in Suva to review the draft questions and determine a set of competencies based on the FSA Framework that were relevant for rural Fiji. Where necessary, questions were ad- justed to ensure that they were relevant for the Fijian context. The Working Group determined a set of 33 competencies that were relevant for Fiji villagers and this was used to inform the development of the survey instrument. • Attitude to money: Questions on attitudes toward money were developed using a modified version of Kent Yamauchi and David Templar’s Money Attitude Scale, which includes a 32-item Money Attitude Scale developed on the basis of four factors, power/prestige, retention/time, quality, and distrust/anxiety. During the Fiji survey’s 2006 Financial Competency Workshop, the group of experts decided not to include four questions on quality (trade-offs between price and quality) because did not ap- ply to rural villages. • Subjective wellbeing: The Fiji survey team developed its questions on subjective wellbeing based on the 2006 Personal Wellbeing Index developed by the Interna- tional Wellbeing Group. The scale comprises nine questions related to people’s sat- isfaction and feelings about themselves. • Objective wellbeing: The survey developed its questions on objective wellbeing based on the ADB’s 2003 Experimental Methodology for Evaluation of Savings Prod- uct Innovations based on household economic portfolios. • Literacy: The survey team worked with a specialist English-as-a-Second-Language teacher to develop the literacy questions. The questions were then pre-tested and adjusted to ensure that they were equivalent to an English primary school reading level. Review of Existing Financial Capability and Literacy Measurement Instruments 71 2.1.2 Revisions to the questionnaire All questions were reviewed in 2006 by a workshop of subject matter experts. A dual lan- guage Fijian schoolteacher also reviewed the questions to ensure that the instrument was comprehensible to respondents with only a primary school education. In addition, the Fiji survey team developed a set of show-cards to support key questions. The questions un- derwent multiple translations and were pre-tested to ensure clarity. A number of changes were made to the ordering and language of questions, but no questions from the original instrument were dropped. 2.1.3 Final structure of questionnaire (topics, sections, number of questions, special skip patterns) The survey instrument consisted of between 270 and 331 questions (Sibley 2009). The questionnaire included closed questions with some semistructured questions. There were two final version of the questionnaire: one version administered to men that included objec- tive wellbeing questions related to the farm/and or business, and a second version admin- istered to women that included objective wellbeing questions related to household assets, health, and age structures. Questions were organized based on the FSA (2006) survey’s groupings and knowledge and skill questions largely preceded questions on behavior, attitudes, and wellbeing. The final questionnaire included the following topics and was organized as follows: Section Subsection Number of Questions Demographics 7 Functional Literacy Test of functional literacy (numeracy and literacy 11 test). The literacy questions were designed to measure a respondent’s ability to use common arithmetic operations and to identify key information contained in brochure.4 Managing Money Managing Payments: Knowledge of different 13 forms of money and different forms of payment, and use of different forms of money and different forms of payment, including cheques, debit cards, and automatic payments to manage financial commitments. Managing Income: Ability to recognize 8 regular and unpredictable sources of money, understanding how earnings and savings are calculated, monitoring household income, differentiating between regular and unpredictable sources of money, periodically checking earning and salary for accuracy. 4 The literacy part of the functional literacy test was measured on the basis of the OECED Level One English Language Literacy requirement (Sibley 2009). The survey team could not determine a similar standard for the Fijian language. 72 Review of Existing Financial Capability and Literacy Measurement Instruments Section Subsection Number of Questions Managing Expenses: Understanding household 32 expenditures and how to manage it, ability to prioritize different needs within the constraints of limited resources, ability to differentiate between one-off expenses and regular financial commitment, ability to differentiate between essential and nonessential spending, monitoring and managing household expenditure, prioritizing expenditure, and planning for expenditure. Taxes & FNPF: Knowledge of taxation and fiscal 17 system in Fiji, ability to identify forms of taxation, understanding of why and how deductions are taken from earnings, keeping knowledge of taxation and FNPF system up to date, checking tax and deductions, and calculating and providing for future taxation. Keeping Records: Ability to keep financial 14 records, knowledge of financial records, and the need for financial records, keeping financial records, and keeping official financial records. Making Choices Saving: Knowledge of savings options, 33 understanding the importance of saving, knowledge of using an account for savings, understanding different ways and places to save; keeping money in a bank account, using methods of saving appropriate to saving goals, keeping money in a safe place, and saving money for planned future events. Investing 21 Borrowing: Understanding the implications of 37 being in debt, understanding different forms of debt and related terms and conditions, repaying loans in accordance with terms and conditions, managing debts and household cash flows. Cost of Money/Financial Terms 13 Financial Organizations: Know about and 11 understand the roles of financial organizations, selecting a financial organization based on suitability and risk. Planning Ahead Budgeting and Planning: Understanding how to 29 budget to plan and control budgets, developing and utilizing an overall household income and expenditure budget Review of Existing Financial Capability and Literacy Measurement Instruments 73 Section Subsection Number of Questions Getting Help Seeking advice 9 Attitude to Money Power/Prestige, Retention/Time, Distrust/Anxiety 38 Subjective 8 Wellbeing Objective Wellbeing Household income 9 Farm income 12 Business ownership and income 9 Total Questions 331 2.2 Sampling 2.2.1 Survey population The survey instrument was given to 400 adults from 200 households in 14 rural villages in the Naitasiri Province of Fiji. The target population for the survey was villagers from com- munities transitioning from a subsistence economy to a monetized economy. The survey focused on people who engaged little with the money economy as a core part of their daily lives. Accordingly, Naitasiri was selected as the focus of the study because the ANZ and UNDP programs were implemented there and because villagers derived their income pri- marily from agriculture and had limited access to other banking and saving services. 2.2.2 Unit of analysis The principal unit of analysis for the Fiji survey was the family as defined by the World Food Organization: “A type of household consisting of two or more persons related by blood, marriage or adoption who also satisfy the conditions of sharing the same housing unit and making common provisions for food and other essentials of living� (Sibley 2009, 102). Each household was treated as an individual unit, and within each family, the survey team selected the principal male and female financial decision makers to take the survey. Usu- ally, these principal financial actors were a married couple, husband and wife. 2.2.3 Sampling frame (sampling frame, method, sample size etc.) The survey employed a multistage cluster sample to select the households that participat- ed in the survey. Fourteen villages were selected from a total of 26 villages. All 14 villages had received Financial Literacy Training workshops two and a half years previously. Ten villages had visits from the ANZ Rural Banking Service, while four villages did not receive visits from the Rural Banking Service. The Rural Banking Service had begun work in the 10 villages two and a half years before the survey. 74 Review of Existing Financial Capability and Literacy Measurement Instruments Households in each village were mapped to the following sampling frame and then ran- domly selected from the sampling grid: Rural Banking Visits No Rural Banking Visits Financial Random sample from households Random sample from households that Literacy with an active rural banking attended financial literacy training Training account and which attended 63 households interviewed §§  financial literacy training (103 people) 48 households interviewed §§  (49 people) No Financial Random sample from households Random sample from households Literacy with a rural banking account and that did not attend financial literacy Training that did not attend financial literacy training training 61 households interviewed §§  34 households interviewed §§  (186 people) (51 people) A target of 50 households per sampling cell was established, with a minimum of 44 house- holds per cell. However, this was achieved for three of the four sampling cells. 2.3 Implementation 2.3.1 Countries/regions of application The survey was implemented in 14 rural villages in the Naitasiri Province of Fiji. 2.3.2 Testing and piloting The instrument for the Fiji Financial Capability, Financial Confidence, and Wellbeing survey was pretested with the interviewers during the interviewer-training workshop. The group reviewed the English and Fijian language versions of the survey to determine translation problems and understanding of the financial concepts asked. The survey was then pilot tested in two villages in the Naitasiri Province. The pilot test was used to determine how reluctant villagers were to answering questions and if they understood questions relating to financial constructs. The functional literacy portion of the test was pretested with a group of randomly selected 9- to 11-year-old native English Fijians. The survey team found no indications of misunderstanding of the questions or reluctance to answer the survey ques- tions (Sibley 2009). 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) The survey was administered in Fijian by trained bi-lingual interviewers using the face-to- face method. 2.3.4 Response rate The survey response rate was 95 %. Because the survey was designed to gather data on Fijian communities living in villages, the survey team conducted meetings with all members of the villages in the region of Viti Levu Fiji. All members agreed to participate in the sur- Review of Existing Financial Capability and Literacy Measurement Instruments 75 vey, and the survey team used random sampling to select households. Respondents were informed of the survey and their voluntary consent was acquired. 2.3.5 Average duration of interview and total cost The typical interview lasted between 1 hour and 45 minutes and 3 hours. 2.3.6 Key challenges and lessons learned The survey team identified the translation of the questionnaire from English to Fijian to be challenging. This was primarily because certain financial terms in English have no equiva- lent in Fijian and therefore may not have been accurately translated. Moreover, the team had to be particularly attentive to how they translated complex financial concepts into Fijian in order not to effectively answer the survey questions (Sibley 2009). 3. Analysis 3.3 Method Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) The survey data was analyzed using descriptive and inferential statistics and data reduc- tion. 3.2 Key Results The survey found a positive relationship between villagers’ level of financial literacy and their financial skills, behavior, and inclusion. The survey data also presented evidence of a positive relationship between the level of competent financial behavior of the principal financial actor and the household’s wellbeing. Households that participated in the Rural Financial Services Initiative demonstrated a greater level of financial competence across a range of aspects of the management of agricultural, business and household cash flows. Yet the survey also found that households that participated in these initiatives appeared to be already more financially competent and affluent than households that did not participate in either of the programs. • Financially Competent Households: Households in which the principal financial actors were more educated were typically more financially literate and adopted more positive financial behaviors. They were also more likely to attend the Financial Lit- eracy Training Program and open a Rural Savings account. Households in which the principal financial actors were finally competent, they were more likely to plan for future expenditures and effectively manage household cash flows. They are more attentive to income and expenditure and as a result they are more likely to have a cash surplus to reinvest in their farms or businesses. These households are also more likely to save regularly using a formal banking account. • Less Financially Competent Households: Families in which the principal finan- cial actors were less financially competent: they had low levels of financial literacy and showed negative financial behaviors. These households were more likely to underestimate household expenditures while overestimating income, and they 76 Review of Existing Financial Capability and Literacy Measurement Instruments were less likely to plan for the future and effectively manage household cash flows. In addition, the households are less likely to have businesses to augment income, have surplus cash, and reinvest in farm or business activities. 3.3 Policy Application The Fiji study points to the value of financial competency and literacy for the wellbeing of households. The study also found evidence to support the argument that the extension of existing institutional retail financial services is likely to be more successful in reduc- ing levels of financial exclusion than the promotion of poverty-lending based microfinance schemes. The findings also suggest accurate targeting of training programs to enhance financial knowledge and skills are required (Sibley 2009). Partnerships with governments, the private sector, and development partners can be formed to help enhance financial lit- eracy and provide greater financial inclusion. 3.4 Potential for Replication (over time and in different contexts) The Fiji survey seeks to measure the link between financial knowledge and behavior and the link between financial behavior and wellbeing. Because the Fiji survey was designed as an impact evaluation, the instrument for the survey can be used as the basis for the development of a similar quasiexperimental survey to measure changes of behavior over time or among groups. The survey instrument can also measure behavior and knowledge at a given time depending on the objectives of the particular survey. B.7. World Bank Financial Capability and Consumer Protection Surveys 1. General Introduction The World Bank’s Financial Inclusion and Consumer Protection Service Line has been assessing existing financial consumer protection (and financial literacy) frameworks in 18 countries, by reviewing their laws, regulations and practices and by comparing them to international good practices. Based on these diagnostic reviews, the World Bank provided recommendations on ways to improve financial Consumer Protection and Financial Capa- bility. To have a useful baseline assessment against which the macro impact and effective- ness of financial consumer protection reforms and financial education interventions can be assessed, the World Bank also conducted six nationally representative Financial Capabil- ity and Consumer Protection (WB FCCP) surveys. 1.1. Purpose of the Surveys The main purpose of the WB FCCP surveys of the World Bank is to provide critical in- formation on (1) the level of satisfaction of individuals with services offered by financial institutions; (2) the level of awareness of consumer rights (deposit insurance) and exist- ing recourse mechanisms (for example, Ombudsman); (3) the level of trust in regulatory/ supervisory agencies and in financial service providers; and (4) the level of confidence that conflicts with financial institutions will be resolved by existing redress mechanisms in a timely and just manner. Review of Existing Financial Capability and Literacy Measurement Instruments 77 From a policy perspective, this information is pivotal to strengthen the existing consumer protection framework and to promote financial inclusion by building consumers’ trust in financial institutions and encouraging first-time consumers to access financial services. Because financial literacy is an important part of consumer protection, the WB FCCP sur- vey also includes core questions relating to financial literacy. The concept of “financial literacy� is measured by questions which assess individuals understanding of basic eco- nomic concepts (for example, inflation, compound interest, money illusion), their patterns of money management, their usage of financial products, and so on. Therefore, the World Bank’s WB FCCP surveys also assist in the development, impact evaluation, and monitor- ing of any future financial literacy initiatives by enabling policy makers to (1) identify the most vulnerable groups of the population, who would benefit the most from any financial literacy and consumer empowerment programs; (2) ascertain specific areas where people have the greatest lack of knowledge and the topics they would like to receive more informa- tion on; and (3) provide a benchmark in estimating the impact of interventions in the next three to five years. 2. Methodology 2.1. Original Development of the Questionnaire The first WB FCCP instrument was developed and administered in 2008 in Russia. It is based on the input of researchers and the outcomes of four focus groups. The research- ers provided the normative concepts of financial literacy and consumer protection and designed questions to measure these concepts. If possible, existing questions from other surveys have been used. This applies to the majority of the financial literacy questions. To measure the consumer protection concepts, new questions had to be designed. These concepts and questions where then tested with focus groups. Two of the focus groups were conducted in Moscow, the remaining ones in Volgograd Oblast. In each city, one focus group consisted of participants younger than 35, whereas the other one comprised participants aged 35 and more. The final version of the first WB FCCP instrument included 54 questions. 2.2. Final Structure of the Questionnaire Though the WB FCCP instrument used in different countries developed over time (for example, the number of questions increased, the order or wording of some questions changed), the overall structure of the questionnaire remained unchanged: Most of the questions had to be answered by all respondents. If possible, skips were avoided. To be able to benchmark with other countries, the main topics covered stayed the same: “con- sumer protection,� “financial literacy,� “managing household finances,� “planning ahead,� “using and choosing financial products and services,� and “being and staying informed.� In addition, to be able to identify the most vulnerable groups of the population, all WB FCCP instruments also captured the respondent’s sociodemographic information. A more detailed description of the topics is provided in table A.3.1: 78 Review of Existing Financial Capability and Literacy Measurement Instruments Table A.3.1: Structure of WB FCCP Instrument Topics Purpose Consumer protection §§ Provide critical information on the level of satisfaction with services offered by financial institutions. §§ Collect information on people’s awareness of consumer protection rights (deposit insurance) and of existing recourse agencies (for example, Ombudsman). §§ Gain knowledge about the level of trust people have in regulatory and supervisory institutions and financial service providers. §§ Determine people’s level of confidence that occurring conflicts with financial service providers will be resolved in a timely and just manner by existing recourse mechanisms. Financial literacy §§ Assess the respondent’s understanding of basic economic concepts like inflation, compound interest, and money illusion, as well as their numeracy skills. §§ Understand people’s perception of risks associated with various financial products. §§ Gather information on individual’s self-assessed level of financial literacy. Managing household §§ Determine how people manage their household finances. This finances comprises their ability to plan income against spending, to keep track, and to make ends meet. Planning ahead §§ Learn more about people’s savings behavior and their strategies to cope with unexpected expenses/income drops. Using and choosing §§ Learn more about the decision-making process upfront when financial products and a specific financial choice is made by asking individuals if they services shop around, which kind of information they use, and the criteria by which they opt for a specific product. §§ Gather information on the kind of financial products and services individuals use and have access to. Being and staying §§ Ascertain if people follow specific price trends and, if they do, informed which sources of information they therefore use. §§ Determine the topics regarding various aspects of money management respondents would like to learn more about in the course of a financial training. Sociodemographics §§ Collect information on age, gender, education, employment status, location, and income. Review of Existing Financial Capability and Literacy Measurement Instruments 79 3. Sampling All World Bank WB FCCP surveys are nationally representative and have individuals as units of analysis. The target population is equivalent in every country and comprises the entire civilian, noninstitutionalized population aged 18 or older. To fulfill the basic require- ment of a scientifically sound survey, in each country a probability sample selection mode was applied. Typically, the most recent population census data was used as a sampling frame, but alternatives have been used in case no up-to-date and reliable census was available (in Romania and in Bosnia and Herzegovina, voting registers were used instead). The stratification criteria most commonly used were geography and population size. To re- duce travel costs and to increase interviewing efficiency, clustering was most of the times achieved through three sampling stages. The first stage of sampling consisted of adminis- trative geographic areas (primary sampling units, or PSU), which were usually drawn pro- portional to size. If a list of dwellings or households within each PSU was available, house- holds were randomly selected from it at the second stage. If such information was not accessible, households were randomly selected by random walk methods (for example, Bosnia and Herzegovina, Bulgaria). Finally, within each selected household respondents were randomly drawn out of the pool of eligible household members aged 18 and older, either through Kish grid or the latest birthday method. 4. Implementation 4.1. Countries/regions of application The WB FCCP instrument has been used in six countries, mostly in Europe and Central Asia. Besides Russia, where the instrument was tested the first time, it has also been used in Azerbaijan, Bulgaria, Bosnia and Herzegovina, Romania, and most recently in West Bank and Gaza. 4.2. Interview method (face-to-face, telephone, CAPI, etc.) Because of the relatively simple structure and nature of the questionnaires (short with simple skip patterns), the hardware costs as well as time and efforts needed to program the instrument into a computer-assisted version, the mode of all WB FCCP surveys was Paper-and-Pencil-Interviewing (PAPI) (that is, face-to-face interviews administered by enu- merators using paper and pencil). 4.3. Response rate A key criterion of any survey is the response rate achieved. To keep nonresponse rates low in each country the implementing survey company was requested to adequately train and supervise the enumerators. As a result, the response rates for most of the WB FCCP surveys were remarkably high and range from 60.5 % in Bosnia and Herzegovina to 95 % in Bulgaria and Romania. The lower response rate for Bosnia and Herzegovina is in line with those witnessed by other surveys (for example, LSMS). As a postconflict country, the expected lower level of trust might explain the lower response rate. As shown in table A.3.2, the sample sizes of successfully conducted interviews range from 1,036 in Bosnia and Herzegovina to 2,026 in Romania. 80 Review of Existing Financial Capability and Literacy Measurement Instruments Table A.3.2: Overview of WB FCCP Surveys Country Year Sample Size No. of Questions Azerbaijan 2009 1,207 65 Bosnia and Herzegovina 2011 1,036 114 Bulgaria 2010 1,618 129 Romania 2010 2,026 109 Russia 2008 1,600 54 Russia 2009 1,366 36 West Bank and Gaza 2011 2,022 109 4.4. Average Duration of Interview and Total Cost Table A.3.2 also illustrates the number of questions asked in the course of each WB FCCP survey: During the WB FCCP interviews in Russia and Azerbaijan, only up to half of the questions asked during the following WB FCCP surveys needed to be answered. There- fore, the average interview duration in both countries was only around 30 minutes. For Bosnia and Herzegovina, Bulgaria, Romania, and West Bank and Gaza, an interview last- ed on average about 45 minutes. 5. Analysis 5.1. Methods Most of the analysis, based on the collected WB FCCP data, has been descriptive (that is, describing proportions and characteristics of people who severely mistrust financial insti- tutions, who are not satisfied with financial products they use, who are not confident that any conflicts with financial service providers would be solved in a quick and just manner by existing recourse mechanisms, or the average number of correctly answered quiz-like questions on inflation, compound interest, and the like) for different subgroups of the popu- lation. From the data collected in Bosnia and Herzegovina, Bulgaria, and Romania, more sophisticated statistical data reduction tools (for example, factor analysis) have been used to identify key dimensions of financial literacy and to construct indices of financial literacy. Furthermore, cluster analysis was conducted to classify individuals with similar financial literacy scores. 5.2. Key results Similar result patterns emerged in each of the six World Bank WB FCCP survey countries: (1) Large parts of the population are either unsatisfied or not able to assess the quality of products offered by financial institutions; (2) Citizen awareness of consumer rights is lim- ited; (3) The level of trust in regulatory/supervisory institutions is moderate, whereas the level of mistrust of financial service providers is high; (4) People’s confidence that conflicts Review of Existing Financial Capability and Literacy Measurement Instruments 81 with financial institutions will be resolved in a timely and just manner by existing recourse mechanisms is low; (4) Prevailing levels of financial literacy are low within and across all countries; and (5) The most vulnerable and illiterate population groups are women, the unemployed, the uneducated, the elderly, and people living in rural areas. 5.3. Policy application The information collected with this WB FCCP instrument is highly relevant from a policy perspective. It provides the basis for any interventions to strengthen the existing consumer protection framework and to promote financial inclusion. For instance, improving the ef- fectiveness of existing recourse mechanisms or any other interventions will increase the overall low consumer trust and encourage first-time consumers to access financial ser- vices. Furthermore, it helps to identify the most vulnerable and illiterate segments of the population and to empower them by launching education programs which cover the topics they have the greatest lack of knowledge and would like to receive training on. 5.4. Potential for replication (over time and in different contexts) The WB FCCP instrument presented here has mostly been tested in middle-income coun- tries. The application in the context of low-income countries needs some adjustments, especially against the backdrop of low levels of financial inclusion or educational attain- ment of the majority of the population. This shortcoming has already been addressed by the World Bank’s Financial Inclusion and Consumer Protection Service Line: Leveraging on a diagnostic tool developed by the Russia Financial and Education Trust Fund, which was specifically designed to measure financial capability in low- and middle-income en- vironments, their survey designers developed a new WB FCCP instrument. Figure A.3.1 sets out the structure of the new instrument. This instrument is set to be used soon in 10 additional countries, including Mongolia, Morocco, Mozambique, and Tajikistan. Figure A.3.1: Structure of the New Instrument Explanatory questions on all Key motivations Financial literacy household members (e.g. impulsivity) (age, education, etc.) Screening Using and choosing Core explanatory questions financial products, questions (selecting person Consumer (e.g. income) for interview) protection Day-to-day money Planning for management the future Concluding (e.g. making (e.g. provisions for questions ends meet) unexpected events) 82 Review of Existing Financial Capability and Literacy Measurement Instruments B.8. Financial Diaries 1. General Introduction to the Method 1.1. Purpose of the Survey One of the least remarked-on problems of living on US$2 dollars a day is that the individu- als and families faced with this situation do not literally get that amount each day. Two dol- lars a day is only an average over time. Some days they make more and some days they make less. Often, they get no income at all. How does one manage one’s money if there is so little of it? These are practical questions that confront billions every day. Though these questions about the financial practices of the poor are fundamental, they are surprisingly hard to answer and existing data sources offer limited insights. Given this knowledge gap and accumulating questions, several years ago a series of de- tailed, year-long studies were launched to shed light on how families live on so little. The studies uncovered the nuanced financial lives of these households as they seek to “man- age� their money by saving when they can and borrowing when they need to. The Financial Diaries methodology that developed from this exercise is geared toward un- derstanding how poor households manage their cash flows at a fine-grained, quantitative level. Because many of these cash flows are small, they require a sensitive instrument to capture them. The Financial Diaries methodology is such an instrument, allowing research- ers to discover the crucial importance of financial tools for poor people. Through a series of questionnaires and interviews containing a combination of closed-end and open-ended questionnaires every two weeks over 14 months, interviewers spend significant time with the participating households learning about their money-management methods in minute detail. 1.2. Link Between Purpose and Chosen Approach The basic concept behind the Financial Diaries methodology is that finance (and money management) is the relationship between time and money, and to understand it fully, time and money must be observed together. The best description of this methodology is to call it a “diary,� a term that appropriately conveys the sense that the researcher is tracking in- timate details of financial management over time. The methodology intends to capture the richness and complexity of poor people’s financial lives while being systematic enough in its data collection to prevent it from being dismissed as a set of mere “anecdotes.� The Financial Diaries are not diaries in the literal sense; not only are many of the diarists illiterate, but the detail of information collected is far beyond what households have the patience or time to keep track of themselves. Instead of relying on the householders to write the diaries themselves, the Financial Diaries methodology uses a team of skilled in- terviewers to record the transactions and the comments during home visits that take place at roughly 14-day intervals, for a time period of anywhere from six months to a year. The intensity of this approach and high levels of household activity require a focus on col- lecting a depth of information on a single household rather than a breadth across many. Review of Existing Financial Capability and Literacy Measurement Instruments 83 The household interviews are “deep-dives� that aim to learn how and when income flows in and how and when it is spent. Looking at poor households almost as one might look at a small business, household-level balance sheets and cash-flow statements are created, to focus sharply on the financial behavior and the money borrowed and repaid, lent and recovered, and saved and withdrew, along with the associated costs. The understanding of these choices is enriched by the real-time commentary of the householders themselves. Interviewers listen to what respondents have to say about their financial lives: why they did what they did, what was hard and what was easy, and how successful they felt they had been. The tools of corporate finance balance sheets and cash-flow statements offer the structure with which to understand what it takes, day by day, for poor households to live on so little. The frame-after-frame views reveal much greater levels of financial activity than large sur- veys usually show, and much more active management of finances. 2. Methodology 2.1. Survey Instrument The Financial Diaries methodology is driven by a series of questionnaires that evolve over the lifetime of conducting the diary with the specific respondent household. In the Finan- cial Diaries methodology, establishing trust and familiar interactions with the participating households is as important as the questionnaires themselves. Made up of open and closed questions, the questionnaires allow researchers to generate a complete set of daily house- hold cash flows over the time period of the study. (See figure A.7.1 for an illustration of the process.) 2.1.1. A system of questionnaires and interviews, evolving over time Broadly, the Financial Diaries methodology relies on four sets of questionnaires for data collection: I. Initial questionnaires: In the enrollment stage, newly recruited households are taken through three “initial questionnaires.� II. First diary questionnaire: The information gathered from the three initial ques- tionnaires, is used to produce a household specific “first diary questionnaire.� III. Diary questionnaires (ongoing): The responses from the first diary question- naire are used to create the next questionnaire, which then gets continually up- dated and produced as the households are seen from fortnight to fortnight. As new financial instruments, new jobs, new people and other changes happen, “change� questionnaires are incorporated to update the on-going diary questionnaire IV. Optional modules: Additional in-depth questionnaires to probe topics of particular interest beyond cash flow. 84 Review of Existing Financial Capability and Literacy Measurement Instruments Figure A.8.1: The Financial Diaries Process Recruitment Enrollment Usually 1st initial 2nd initial 3rd initial purposive to questionnaire: questionnaire: questionnaire: maximize range Living Income sources Details of of HHs of interest standards, HH for all HH financial for the study roster, physical members instruments for assets (1 hour) all HH members (1 hour) (Two 1- hr interviews) Use initial data to generate ongoing diary questionnaires Open or close financial instruments as needed Ongoing Record major questionnaires household events with supplements Record changes in (ex: death, birth, (on edge) as needed; income sources wedding, etc.) automatically updates diary questionnaire Record major Change household acquisition, loss, roster as or sale of physical individuals leave assets and join Initial questionnaires Following recruitment, households are taken through three “initial questionnaires� to be- gin getting to know the household through the collection of information. The three initial questionnaires are structured to gather information on (1) household demographics, (2) physical assets, (3) typical income and expenditure patterns, (4) historical and current employment, and (5) current and previous use of financial instruments. Identifiers such as year of birth and gender are taken with the person’s first name only, and are only collected on the first initial questionnaire. Financial instruments, formal and informal, are predefined and each has its own ques- tionnaire that defines different aspects of the instrument. These instruments may include checking accounts, savings accounts, savings clubs, payday loans, pawn shops, bank loans, loans from friends or employers, money orders, check cashing services, or remit- tances. Each existing financial instrument receives its own financial device code against which cash flows will be captured over time. Review of Existing Financial Capability and Literacy Measurement Instruments 85 These three interviews not only allow the household to become more comfortable with the fieldworkers, but are also used to create an initial balance sheet position, as well as a typical monthly cash flow statement. During this time, interviewers also become aware of households that may become uncooperative or unavailable for interviews and may need to be replaced with other households. First diary questionnaire and ongoing diary questionnaires Upon completion of the three initial questionnaires, the households are interviewed every other week for 14 months, capturing every cash flow that comes into and out of the house- hold, including income, expenditure, changes in physical assets, and servicing and initiat- ing financial instruments. The content from the three initial questionnaires is combined to form the first diary ques- tionnaire, and the responses from the first diary questionnaire are used to create the next questionnaire, which then is continually updated and produced as the households are seen from fortnight to fortnight. Each questionnaire is specific to the household being surveyed. It is crafted using data from the first initial questionnaires and updated as understanding of the households cash- flows and tools develops. This highly tailored questionnaire is used to both prompt memory, aid data collection and to save the respondents time and patience. For example, if the interviewer knows that the household does not have loans, they will not waste time ask- ing whether they paid their loan off. As new financial instruments, new jobs, new people, and other changes happen, “change� questionnaires are used to update the ongoing diary questionnaire. The ongoing questionnaire includes an open-ended “journal� page to probe qualitative questions. Optional modules For the Financial Diaries being conducted in the United States in 2012, a new component has been developed for the Financial Diaries toolkit in the form of six to eight optional “mod- ules.� These additional in-depth questionnaires probe topics of particular interest for some of the diary interviews. For example, topics include a nutrition module, a health module, a financial education module, and a module to measure psychological propensities such as risk aversion, patience, and fatalism. The optional modules are aimed at understanding the interplay between the financial lives of the household and other important aspects of their lives. The optional qualitative modules can be developed as relevant to the country context and pressing themes. 2.1.2. Inherent and constant updates to the diary questionnaire The purpose and high frequency of data collection warrants the use of ongoing diary ques- tionnaires that are bespoke to each household (and potentially to each household mem- ber) and therefore are revised as new information is discovered about each household, or as their circumstances change. The methodology therefore is inherently designed to offer an opportunity to update the questionnaire after each interview to incorporate new informa- tion to pursue in the next interview. 86 Review of Existing Financial Capability and Literacy Measurement Instruments This revision process is enabled by a specially conceived and built database and the con- sistent bi-weekly capture of data. Data is inputted by fieldworkers after the interviews and diary questionnaires, specific to each household, will be generated. Each week the respon- dents are also asked if they did anything new in the past two weeks (that is, opened a new bank account or if they stopped a financial device). Each new financial device is captured on a specific form and cash flows generated by that device are captured thereafter. If a financial device is closed, a “close form� is captured and cash flows on that device are not captured thereafter. They are also asked if a major event happened; if a person joined or left the household; if a new or casual job is joined or left; or if a new physical asset is bought, sold, or stolen. As mentioned previously, each week fieldworkers also fill in journals, noting various obser- vations, events, or comments made by respondents that are not captured elsewhere in the diary questionnaire. Extenuating circumstances, concerns, and non-cash-flow events that go beyond the questionnaire but yield important information that might explain some of the cash flow patterns observed. Ongoing data checks A key feature of the database and the ongoing diary system is the ease with which the data can be checked. Of particular importance is the report that tracks the sources and uses of funds in and out of the household from the time of the most recent interview to the time of the previous interview. Allowing for cash in hand in both interviews, it should be that the sources of fund (for example, income, taking a loan, drawing money from a bank account) should equal the uses of funds (for example, paying for food, putting money under the mattress, repaying a loan). A summary report of sources less uses is run for each area. Any household that has an excess or deficit of sources versus uses of funds greater than $30 will be examined individually to detect where the imbalance is coming from. If the im- balance is coming from an input error, the data is corrected. If the imbalance seems to be coming from a lack of reporting, the fieldworker is informed and given details on which to follow up in the next interview. In this way, data is continuously checked and the interview questions are revised to make all the information is captured from a household. 2.2. Sampling 2.2.1.  Survey population The appropriate target survey population is determined based on the research needs for each new project. Ordinarily, the target survey population is low-income or poor communi- ties. This may be determined by a national survey, international poverty lines, or participa- tory wealth ranking, or a combination of these tools, amongst others. 2.2.2.  Unit of analysis The Financial Diaries have adopted the household as the unit of analysis, but information on income, expenditures, financial flows, and physical assets are gathered by household member, so the data can be broken down into adult person-specific information. It is worth noting that Financial Diaries interviewers are made aware that situations arise Review of Existing Financial Capability and Literacy Measurement Instruments 87 where all members of the household are not aware of certain financial issues or where members are concealing information from each other, and treat the relationship with great sensitivity. Interviewers make considerable effort to establish a friendly relationship with everyone in the household and to allow a comfortable environment that would encourage respondents to be open. To help enable this environment, interviewers are recruited speak the local dialect and are not too distant in class and background from the people they in- terview. 2.2.3.  Sampling strategy The sampling strategy and size across research sites varies based on the specific objec- tives of the research and available resources. Because of their limited size, samples are nonrepresentative of the communities or countries from which they are drawn. There is flexibility in the sampling strategy to meet research needs and to represent specific groups of interest. There is also a relatively inclusive, community-driven approach employed by the sampling methodology, to ensure transparency and trust in this sensitive relationship. The sampling strategy has varied across the Diaries research to date. Three examples of sampling methodology used in the original Diaries in Bangladesh, South Africa, and India, along with the latest Diaries projects in Kenya and the United States, provide a feel for the size, goals, and strategies employed. In the cross-country Diaries conducted in India, Bangladesh, and South Africa, and cap- tured in Portfolios of the Poor, the sampling determined using measures of participatory wealth ranking and ensuring representation across rural and urban areas. The sample size in Bangladesh and India was 42 and 48 respectively, while in South Africa the sample was larger at 152. In all three countries there was an interest in working with both urban and rural households, to capture any variations in financial behavior caused by the differences in economy and environment of the two kinds of location. The choice of communities was guided by those shown to be poor by a national survey, but it was also largely dictated by the practicalities of places where fieldworkers could reliably travel on a fortnightly basis. The sampling design was influenced by the understanding that poverty is a dynamic condi- tion, where at any one time there are people moving up or down the poverty ladder. Pov- erty is also relative. For both these reasons, we needed to distinguish between degrees of poverty and include a mixed range of households with different social-economic profiles. The design also took advantage of the reality that, in poor communities, people often know their neighbors well. This allowed for “wealth ranking� to determine the selection of house- holds. This technique relies on comparing key informants’ estimates of the relative wealth of their neighbors to compile a ranked list of households from the most to the least wealthy. With these wealth rankings for each area, a sample was drawn from the poorest, middle, and wealthiest in each area. Though precise conditions varied by location and country, researchers defined three different levels of wealth in each context: poor, upper poor, and non-poor. In choosing the households, researchers did not explicitly measure levels of poverty or objectively rank houses, as the resources needed to do this rigorously would have reduced the levels of information acquired about microfinance. For this research, knowing which 88 Review of Existing Financial Capability and Literacy Measurement Instruments of these three categories the households belonged to, and whether they were moving up, moving down, or stable, was sufficient. For the U.S. Financial Diaries (currently ongoing), the sampling design is more purposive than those in the previous diaries and employs a “snowballing� technique. The sampling frame for the U.S. Financial Diaries is households that fall into the definition of “low-in- come� as determined by average income levels in each geography of the sample. The sample is drawn with an eye toward ensuring diversity in relation to characteristics such as race, ethnicity, household and family structure, and industry of employment. To achieve this, the main sampling methodology is a combination of purposive and respondent-driven sampling techniques that are commonly employed in qualitative social science research.5 Sampling takes place in two stages. First, researchers draw on the expertise of individuals working at local universities, social service and community based organizations to iden- tify and approach “typical� households in the population. This purposive approach helps ensure that the sample is broadly representative of the local population and does not sys- tematically exclude subgroups. Further, using local partners within each site to identify and approach households is critical to establishing trust from the outset. Second, these “seed� households are asked to refer researchers to one or two other households. Respondents are encouraged to refer households researchers might not be able to find through com- munity organizations, and are asked to identify at least one household “like them� and a second household who differs from them in some way that might make their financial situ- ation different (for example, many children, live-in parent, seasonal worker). Local area estimates derived from the 2010 Decennial census and the 2010 American Community Survey are also used to guide the sampling procedure and to ensure that households from meaningful subgroups within the population are not systematically ex- cluded. Given the sample size (about 60 or 90 households per site) the primary objective is to craft a sample that is substantively representative of the diversity of households present in each community. In the upcoming Financial Diaries survey in Kenya, 300 households will be surveyed across five areas of Kenya, with each site consisting of 60 households. Within each site, screening interviews will be conducted to select households to participate in a purposive manner. The aim is to incorporate a diversity of living situations, livelihood patterns, utilization of some particular financial instruments (for example, savings groups), and wealth levels (though no truly rich households will be included) into the sample. Potential households will also be assessed under overarching criteria, such as interest in the study and likelihood of commit- ting to the entire research period. Data will be recorded for entire households through the research, mainly interacting through a key informant. In this study, there is an emphasis on understanding Kenya’s youth. To bring in this perspective, the sampling will try to ensure that 20 % of key informants (typi- cally the head or spouse of the head of household) are below the age of 30. This would mean approximately 12 youth respondents in every site of 60 respondents. 5 See Singleton et al. (1999). For an example of this methodology in a comparable qualitative study, see Edin and Lein (1997). Review of Existing Financial Capability and Literacy Measurement Instruments 89 2.3 Implementation 2.3.1 Countries/regions of application To date, Financial Diaries research has been completed in Bangladesh, India, and South Africa. Financial Diaries research is currently ongoing in India, Kenya, Mexico, South Af- rica, and the United States. The methodology is applicable across countries and can be adapted for the local context and issues. 2.3.2 Testing and piloting In each country, all survey instruments of the diary questionnaire process are rigorously tested to pick up important differences in the initial questionnaires as well as the diary questionnaires. The initial questionnaires would change based on important socioeconomic factors such as levels of schooling, household structures, religions, incomes, important physical assets, as well as available formal and informal financial tools and behaviors. The diary question- naire would change with respect to all of those items, but also be tested for the appropriate level of granularity of expenditures, different transaction types that can be done on financial instruments (for example, deposits, fees, interest charged), ways that cash flow can be made (that is, with cash, by check, with M-Pesa), and other questions about what is hap- pening generally in the households and the physical and emotional health of the members. Each Financial Diaries project is made specific to each country, but collects data on a structure that makes cross-country comparisons possible. 2.3.3 Interview method The Financial Diaries are a unique form of survey in that questionnaire and interview meth- od are closely intertwined because the high frequency with which clients are interviewed face-to-face. The interview strategy centers on slowly gaining trust and maintaining it. Ini- tial interviews allow researchers to get to know the household with relatively gentle ques- tions, such as the ages and education of its members. Following interviews venture into slightly more intrusive areas of income and livelihoods, and then eventually into financial information. This preparation sets the basic framework for the financial mechanisms of the household, but it does not ensure perfect information from the beginning. Data must often be reworked and revisited in subsequent interviews, with the process taking on the qualities of solving a puzzle. Challenges may arise where the information encountered may not fit the con- ceptual categories interviewers are trained to use, or where complex financial behavior or relationships lead interviewees to tell only half the story to even the most trusted outsiders. The strength of the Diaries approach is that it can, over time, break down much of this reti- cence and confusion. For this reason, most of the focus during the fortnightly interviews is on gaps and imbalances, which are followed up in subsequent interviews. Interviewers are encouraged to persevere until they can trace the full chains of transactions. In addition to the phased design of the questionnaires, efforts are made to establish a friendly relation- 90 Review of Existing Financial Capability and Literacy Measurement Instruments ship with everyone in the household and to allow a comfortable environment that would encourage respondents to be open. To help enable this environment, the Diaries rely on researchers who speak the local dialect and are not too distant in class and background from the people they interview. Interviewers are ever mindful of the sensitive nature of the relationship and trained to listen empathetically to respondents who are distraught with events in their lives, while trying to maintain the role of an observer. They take care not to offer advice or judgment, and to try not to interfere or burden the households. Interviews often took place while the inter- viewees got on with their everyday work, cooking lunch or feeding the cow, and were often interrupted by other visitors. 2.3.4 Response and attrition rate Unlike many household surveys, the Financial Diaries provide many opportunities, at the start of fieldwork, to replace households or find new ones if response rates are low. A long recruitment phase can be built into the process, and households can replaced even throughout the initial questionnaires. A more problematic issue, however, is attrition. With such small sample sizes, losing house- holds during the course of the study year is particularly detrimental. Experienced and sen- sitive field staff can help, as often households grow to enjoy the interviews because they bond with the interviewer. However, attrition can happen: in South Africa, 15 % dropped out during the course of the year. For that reason, we have learned that it is far better to take more time to recruit carefully, focusing on households that seem interested in the research and that are likely to continue on with it than trying to force a household to stick it out. 2.3.5 Average duration of interview and total cost The duration of each interview varies depending of the stage of the diaries process (for example, enrollment, initial questionnaire, ongoing questionnaire). The average duration of interviews ranges from one to two hours for each initial questionnaire. Subsequent diary questionnaires are shorter, lasting 15 to 30 minutes depending on the size and complexity of the households. The total cost of conducting Financial Diaries research varies depending on the location and sample size. For large-scale samples (that is, 300 households), in an expensive place to do research could cost several millions of dollars. Smaller samples of 90 and 60 house- holds, particularly in lower cost places, would cost a fraction of that amount—several hun- dred thousand dollars. 2.3.6 Key challenges and lessons learned Through the different implementations of the Financial Diaries methodology, several chal- lenges and lessons learned have been identified. One of the major strengths of the Diaries method when compared with questionnaire-based surveys is exposed by carefully working back through the cash flows of each household. In the more sophisticated version of the Diaries created in South Africa, it was possible to Review of Existing Financial Capability and Literacy Measurement Instruments 91 formalize the process by which the diaries questionnaires accurately captured the full set of cash flows. For each questionnaire period (which usually covered two weeks), the sources of funds (income plus financial inflows into the household) were measured against the uses of funds (expenditures plus financial outflows from the household) with an adjustment of cash on hand at the beginning and end of the interview period. The difference is called the “margin of error,� which represents the amount by which the household had over- or un- derreported cash flows. This is used as a means of tracking data quality. Fieldworkers are then provided with immediate feedback to follow up in the next interview, greatly improving the quality of data and decreasing the margin of error. Thus, the Diaries methodology has improved accuracy. Interviewers are also encouraged to explore the emotions that accompanied the transac- tions to elicit comments on the different devices used and to estimate the degree to which the householders see financial activity as an important or as a trivial part of their lives. Especially striking comments are recorded verbatim. The result is a simultaneous mixed method a means of capturing both quantitative and qualitative data in the moment, across time. The limitations of the diary method are the mirror image of its strengths: above all, the number of households that are worked with are too small to represent whole populations. There is also a doubt as to whether participating in the Diaries changes the behavior of some respondents. In some cases it may have done so. However, without a different type of study design, it is difficult to tease out exactly how much of an influence the Diaries process may have. The sharp focus of the Financial Diaries on the users of financial services and devices also means the data has less to say directly about the providers of those financial services. The Diaries do not help answer the questions raised in some of the current microfinance debates about sustainability and the role of subsidies. However, the data do bring a fresh, much-needed perspective to debates on understanding of microfinance services and de- vices from the clients’ viewpoint. 3. Analysis  3.1. Methods Used for Analysis Both quantitative and qualitative methods can be used to analyze the data from the diaries. Although the aspect of the Financial Diaries documented in “Portfolios of the Poor� focused on the qualitative dimension of the data, the data can also be analyzed quantitatively to measure the impact of financial inclusion policies and to look deeper into the measurement of income, consumption and financial usage. The Financial Diaries process generates a complete set of daily cash flows over the time period studied. These cash flows are completely disaggregated, so that, for example, we do not just collect the value spent on food over a two week period, but rather capture each individual purchase. To each of these daily cash flows, a number of “comment codes� can be added. This tagging is done for all incomes, expenditures, and financial movements for all the households, daily, over time. This allows researchers to determine, with a high de- 92 Review of Existing Financial Capability and Literacy Measurement Instruments gree of detail, what portion of household and individual level cash flows are done with cash versus non-cash and to calculate the transaction cost implications of different transactions from the consumer perspective. The tool that makes this fine-grained data collection possible is a specially built database application that was developed and used for the original 180-household 2004 South Afri- can Financial Diaries, as well as a revisit to the same sample of households in 2009. The addition of new countries have made for new iterations of the database that allow for flex- ibility based on each country, but also constructed in a way that makes cross-country data comparison simple and robust. Using the data collected through the questionnaires, various template reports have been constructed to begin to develop hypothesis around the various research questions. For each household, we will track both the number of financial instruments used, as well as the monthly charge in such use. Two composite reports are produced for each household based on this information that breaks down number and turnover by type of dwelling, in- come, livelihood (whether at least one person has a regular job vs. not) and dependency ratio (number of working adults/number of non-working adults + children). These reports are also broken down into categories based on formal/informal and savings/credit/insur- ance. The composite reports will be produced monthly to pick up trends. We also track the balance sheet of each household. Balance sheets are calculated on a rolling two week basis. A summary sheet for all households in an area against household characteristics is then reported with net worth, change in net worth from the previous pe- riod, and a flag that signals a change in net worth greater than a particular amount. 3.2. Key Results To date, the Financial Diaries have played a critical role in uncovering the nuanced finan- cial lives of poor households in different parts of the globe. Despite the obvious differences between sites and across countries there were some striking commonalities in the opportu- nities and challenges in the financial lives of Diary households in the varied environments. Savings: Almost every household where we have conducted the Diaries, even the poor- est, had both savings and debt of some sort. No household used fewer than four types of instrument during the year. Small, irregular, unpredictable: Of all the commonalities, the most fundamental is that the households are coping with incomes that are not just low, but also irregular and unpre- dictable, and that too few financial instruments are available to effectively manage these uneven flows. Poor households face three reinforcing challenges: low incomes, irregularity and unpredictability, and a lack of tools. Together, these three challenges make it difficult to take on basic cash-flow management to transform irregular income flows into a depend- able resource to meet daily needs. Coping with risk: The households covered in the diaries live lives that are far more un- certain than those in better-off circumstances. The diarists are, as a group, less healthy, live in neighborhoods with weaker security, and face income volatility tied to the swings of Review of Existing Financial Capability and Literacy Measurement Instruments 93 local supply and demand, no matter whether they are employed or self-employed or are small-scale entrepreneurs. Those sources of uncertainty pile on top of others. Dealing with emergencies could derail families with little in reserve. Raising lump-sums: Diary households face the challenges of paying rent, education, busi- ness investments, and the like. Each of these events requires chunks of cash at a single moment. Seizing opportunities and paying for big-ticket expenses by accumulating usefully large sums of money is a significant challenge. The portfolio perspective: Poor households often hold their tiny total savings in a range of instruments with different mixes of characteristics, including an insurance savings plan that requires fixed monthly premiums. Similarly, the three drivers of need may cause the poor to approach different kinds of lenders who offer loans that vary in value, term, price, repay- ment structure, and availability This is not to suggest that poor households are blessed with an abundance of choice when they are deciding where to place savings or where to seek a loan: unfortunately, that is almost never the case. But to the extent that they have choice, they exercise it. 3.3. Policy Application Financial Diaries research offers both quantitative and qualitative lessons for policy, de- pending on the nature of the analysis. To date, the focus has been on providing a deeper, illustrative understanding of the financial lives of the poor. Fragments of data from the Diaries eventually resolved into yearlong movie reels that has changed the understanding of many potential stakeholders. The frame-after-frame views have revealed much greater levels of financial activity than large surveys usually show, and much more active manage- ment of finances. Without the pieces, it would have been easy to imagine poor households would be unso- phisticated about their finances because they are only partially literate, or would be unable to save in a disciplined way because they are so poor. Arguments and assumptions that poor households are especially eager for loans to run a small business, will fall rapidly into deep debt if they do take loans, or that they always demand rock-bottom prices, have been easily accepted. The Diaries have demonstrated that all of those assumptions are right some of the time, but they are wrong much of the time. Uncorrected, they can mislead businesses that plan strategies to work with poor households and misdirect policy makers who design interventions to hasten their escape from poverty. On the quantitative side, data and lessons from the Financial Diaries have been used to try and improve the collection of financial data more generally (Collins, 2009)Information on financial flows is difficult to collect since flows can be tiny and transitory; the structure of informal financial services is not well understood, so the door is opened to misunderstand- ings between researcher and respondent, and respondents are naturally sensitive about revealing too much. The Financial Diaries data, a unique, high-frequency panel dataset, has enabled analysis to interrogate a number of commonly held assumptions about finan- cial data collection. Furthermore, data from the Financial Diaries in South Africa have also been used to dem- 94 Review of Existing Financial Capability and Literacy Measurement Instruments onstrate how policies such as basic bank accounts, direct deposits, and lowering transac- tion costs may support a shift the savings behavior of poor households. There is potential for additional quantitative analysis using data from the diaries to try and answer specific policy or research questions. 3.4. Potential for Replication The Financial Diaries methodology is replicable and is now powered by a new database application that is flexible and allows for the collection of data across countries despite a bespoke approach. The database also allows for easier comparison across countries where the diaries are undertaken, providing the ability to compare financial management across different populations. B.9. FinScope 1. General Introduction FinMark Trust6 (www.finmark.org.za) has been commission by the Russia Financial Liter- acy and Education Trust Fund to describe the FinScope methodology as part of a broader review of existing financial capability and literacy measurement instruments untaken by the Trust. The FinScope survey is a research tool that was developed by FinMark Trust. It is a nation- ally representative survey of how individuals source their incomes, and how they manage their financial lives. It also provides insight into attitudes, awareness and perceptions re- garding financial products and services. This paper summarizes the purpose of the sur- vey, the methodology, including the questionnaire, sampling approach, and implementation process, as well as the analytical framework, and potential for replication. 1.1 Purpose of the Survey There are two types of surveys: the FinScope Consumer survey and the FinScope MSME (micro, small, and medium business) survey. While FinScope Consumer looks at financial inclusion of the adult population, FinScope MSME focuses on small business owners and their financial service needs. Accordingly, the objectives of the FinScope Consumer survey include the following: • To measure the levels of financial inclusion (that is, the proportion of the population using financial products and services – both formal and informal) • To describe the landscape of access (that is, the type of products and services used by financially included individuals) • To identify the drivers of, and barriers to the usage of financial products and services • To stimulate evidence-based dialogue that will ultimately lead to effective public and private sector interventions that will increase and deepen financial inclusion strategies 6 Established with initial funding from the U.K.’s Department for International Development, FinMark Trust is an independent trust whose business is controlled by seven trustees from countries in Southern Africa. FinMark Trust’s purpose is “making financial markets work for the poor, by promoting financial inclusion and regional financial integration.� It does this by conducting research to identify systematic constraints that prevent financial markets from reaching out to these consumers and by advocating for change on the basis of research findings. Thus, FinMark Trust plays a catalytic role, driven by its purpose to start processes of change that ultimately lead to the development of inclusive financial systems that can benefit all consumers. Review of Existing Financial Capability and Literacy Measurement Instruments 95 The objectives of the FinScope MSME survey are as follows: • To assess the size and scope of micro, small, and medium enterprises (MSME) • To describe the levels and landscape of access to financial products and services (both formal and informal) • To identify the most binding constraints to MSME’s development and growth with a focus on access to financial markets • To identify and describe different market segments with specific development needs in order to stimulate segment related innovation • To propose recommendations regarding financial assistance to MSMEs and finan- cial policies 1.2 Link Between Purpose and Chosen Approach To design effective financial inclusion strategies, evidence-based country-specific infor- mation is required to establish baseline data, track progress over time, and to identify the most vulnerable population groups to enable appropriate targeting of policy and regulatory interventions. As such, the FinScope survey looks at the broader concept of financial capability includ- ing savings, borrowings, insurance, and remittance. The study looks at a range of factors such as perceptions, attitudes, and behavior that are associated with financial decision making. In addition, the survey seeks to determine levels of usage of financial products and services, as well as barriers to and drivers of usage. These efforts are combined with demographic data and economic status of the individual and the household. The survey is complemented by a literature review that provides the background to the fi- nancial services sector in the country from both a supply and demand perspective together with in-depth interviews in the form of focus group discussions to provide context to the survey results. 2. Methodology 2.1. Questionnaire 2.1.1. Original development of questionnaire FinMark Trust initiated the FinScope programme in 2002, following a full review of existing surveys, as well as several stakeholder workshops, with the aim of developing a compre- hensive survey tool to measure levels of financial inclusion in South Africa. The FinScope Consumer pilot study was launched in South Africa in October 2002. As financial service providers and policy makers found the survey informative and useful, this resulted in the continuation of the program in South Africa as well as an expansion into other countries. To date, FinScope has been conducted in 16 countries: Botswana, Ghana, Kenya, Lesotho, Malawi, Mozambique, Namibia, Nigeria, Pakistan, Rwanda, South Africa, Swaziland, Tanzania, Uganda, Zambia, and Zimbabwe. 96 Review of Existing Financial Capability and Literacy Measurement Instruments 2.1.2. Revisions to the questionnaire By design, it is intended to involve a range of stakeholders in a syndicate, thereby enrich- ing the survey tools through a process of cross-cutting learning and sharing of information. The base questionnaire is continuously revised and contextualized for each country. The questionnaire design is guided by FinMark Trust’s research team and lead by a Steering Committee which usually includes members of the national Ministry of Finance (which also acts as the host ministry), various regulators and other government departments, the Central Bank / Reserve Bank, the National Statistics Agency, bank associations, consumer council, as well as the private sector (for example, banks, insurance companies). During this process, the team ensures that changes to the survey content do not affect the FinScope indicators which are used for trend analysis and tracking changes in the financial system over time. The final questionnaire is signed off by all relevant parties involved and is then translated and back-translated from English into the relevant languages, depending on the country and region/province. The questionnaire is used for the training of enumerators and is completed by a detailed training manual. The questionnaire is then piloted internally and in field. Based on the feedback from the supervisors, the questionnaire is revised again before the final version is signed off and ready to go into field. 2.1.3. Final structure of the questionnaire The final structure of the questionnaire varies from country to country. The FinScope Con- sumer survey Namibia 2011, conducted in December 2011 to January 2012 has the larg- est number of financial capability questions, as outlined below. The Namibia survey tool includes the following broad sections. Review of Existing Financial Capability and Literacy Measurement Instruments 97 Table 1: Structure of Questionnaire Section Topic Number of Questions Introduction 10 A Household information and demographics 38 Household structure, gender, age, marital status, education, decision making, health, housing, land ownership, water and sanitation, energy, access to technology, identification documents B Farming 14 Agricultural activities, livestock, farming as business activity C Income and expenditure 13 Sources of income, personal monthly/annual income, expenditures, payment mechanisms D Access to infrastructure 2 Accessibility to facilities, perceptions regarding infrastructure E Financial capability 43 Financial commitments, coping mechanisms if money runs out, perceptions regarding finances, record keeping, financial behavior, awareness of financial products and institutions, financial knowledge, consumer rights, sources of information F Money management: saving 7 Definition of saving, incidence, reasons for not saving, institution, usage of saving products/mechanisms G Money management: borrowing 10 Incidence, reasons for not borrowing, institution, usage of credit/ loan products/mechanisms H Money management: risk and risk mitigation 5 Risks and coping strategies, incidence, reasons for not having an insurance cover, usage of insurance products/mechanisms I Money management: remittance 14 Incidence of sending/receiving money, locations, frequency, amount J Bank penetration 13 Incidence, usage of banking products/services, perceptions regarding banking products/services, incl. microfinance K Informal products 7 Saving clubs, other mechanisms, perceptions regarding informal mechanisms L General 6 Range of statements, household assets 98 Review of Existing Financial Capability and Literacy Measurement Instruments 2.2. Sampling 2.2.1. Survey population The survey population refers to a representative sample of the bankable adult population in a country including both wealthy and poor, individuals residing in urban and rural areas, women and men. “Bankable� refers to the age at which an individual can open a bank ac- count, which in most countries is 18 years and older. 2.2.2. Unit of analysis The FinScope survey is an individual survey and thus the ultimate unit of analysis is the individual, although some additional household information is collected. 2.2.3. Sampling strategy Sampling method: A multistage sampling approach is used which could include an initial listing exercise for the purpose of weighting: • Random selection of rural and urban Primary Sampling Units (usually Enumerator Areas, EAs) based on probability proportionate to population size (PPS) • Random selection of Sampling Start Points (SSPs) • Random selection of households (usually 8 to 10 qualifier households per selected EA) • Random selection of respondents within the selected households using the Kish Grid Sample frame: The sample frame (= total bankable population for the FinScope Consumer survey and small business owners for the FinScope MSME survey) is usually based on the latest official Census data. In addition, a listing exercise is conducted to determine the cor- rect weights, that is all households and its members / all small business owners are listed in the selected EAs. Sample size: The sample size depends on the adult population size / total number of small business owners found in the selected EAs (= sampling frame). The sample is usually de- signed to provide reliable estimates at national, provincial and urban/rural levels. Hence, the sample size can vary considerably. The National Statistics Agency is usually respon- sible for drawing the final sample. A detailed example of the sampling strategy is included in the Namibia Case Study. 2.3. Implementation 2.3.1. Countries/regions of application To date, FinScope Consumer has been conducted or initiated in 16 countries (figure 1), while FinScope MSME (right) has been conducted or initiated in 7 countries as illustrated in the map below: Review of Existing Financial Capability and Literacy Measurement Instruments 99 Figure1: Countries of Application Nigeria Ethiopia Sierra Leone Cote D’Ivoire Gahna Uganda Kenya Ruwanda Burundi Tanzania Tanzania Zambia Malawi Zambia Malawi Namibia Zimbabwe Zimbabwe Botswana Mozambique Mozambique Swaziland South Africa Lesotho South Africa First cycle Completed Repeat cycle Underway Potential first cycle Using standardized methodology and tools for determining financial inclusion, FinMark Trust is able to benchmark data across countries and regions thus allowing for data com- parability. 2.3.2. Training and piloting FinScope follows a standard training procedure. The enumerators usually undergo ten days of training. The questionnaire is tested internally (mock interviews) and through field pilots. 2.3.3. Interview method Interviews are conducted face to face. The interviewer reads out the questions and catego- ries and in addition, show cards are used to assist with the interview process. 2.3.4. Response rate Considering that the interviews are conducted face-to-face, the response rate is 100 % (including where substitutions have been necessary). 2.3.5. Average duration of interview and total cost The interview usually takes about 75 minutes. The survey costs highly depend on the sam- ple size as well as other factors such as physical size of the country, accessibility issues and hence associated travel costs, cost for accommodation and food within the country, etc. Accordingly, the survey cost ranges, from US$180,000 to about US$370,000. 100 Review of Existing Financial Capability and Literacy Measurement Instruments 2.3.6. Key challenges and lessons learned Challenges mainly relate to capacity of partners, as well as political and logistical issues. The following table outlines some of the key challenges and lessons learnt. Key Challenges Lessons Learned Design §§ Commitment and cooperation §§ Communication is key, especially face- of all stakeholders involved to-face meetings which require several (National Statics Agency, site visits, as well as signed letters from research house, Host Ministry, government officials confirming support etc.) §§ Identification and selection of §§ Terms of Reference need to be circulated qualified research house widely and through different channels §§ Getting suitable references is helpful §§ Finding a balance between §§ Stakeholder involved need to be clear research agendas of different on aims and objectives of this survey – stakeholders, ensuring the expectations need to be managed carefully questionnaire does not become §§ Core questions in the questionnaire too long, sign off questionnaire needs to remain the same to ensure §§ Obtaining sign-off comparability §§ Length needs to be managed carefully Fieldwork §§ The postal system is not §§ Introduction letters need to be hand reliable, most people do not delivered have access to email §§ Printing of maps – including §§ Relationship with the National Statistics substitution EAs Agency needs to be managed carefully – maps need to be obtained in advance §§ Fieldwork over the holiday §§ No fieldwork should be planned over season (Christmas and New the holiday season as many people are Year’s Eve) traveling and would be unavailable §§ Fieldwork over the rainy §§ Fieldwork over the rainy season is not season, which might affect ideal and might require more time and accessibility in some countries additional logistical support §§ Entry into commercial farms §§ Negotiations might be easier through farm associations §§ Quality control and back- §§ Quality control is vital and require checks numerous field visits, as well as checks at each stage of the §§ Skills and capacity both §§ Constant careful monitoring of data nationally and with the statistics quality office in particular Review of Existing Financial Capability and Literacy Measurement Instruments 101 3. Analysis 3.1. Methods Used for Analysis Financial capability is an important component of financial inclusion. Thus, FinMark Trust first looks at the overall financial inclusion landscape within a country. Defining financial inclusion: The concept “financial inclusion� is core to the FinScope methodology. Based on financial product usage, the bankable population is firstly seg- mented into two groups: the financially excluded and the financially included: Financially excluded = adults who do not have/use any financial products and/or services— if borrowing, they rely onl on friends/family; and if saving, they save at home Total adult population = minimum age defined by the age at which individuals can enter into a legal financial transaction in their own capacity Financially included = adults who have/ use financial products and/or services — formal and/or informal The financially excluded segment refers to individuals who manage their financial lives without the use of any financial products or mechanisms external to their personal rela- tionships. To further understand financial inclusion, the financially included segment of the population is taken through a further step of segmentation. As the financially included seg- ment of the population comprises individuals who have/use formal and/or informal financial products and mechanisms, this second step in the segmentation seeks to identify— • Those individuals who have or use products or services from financial institutions that are regulated through an act of law (formal financial institutions) – the formally served segment of the population; • Those individuals who have or use products or services from financial institutions that are not regulated (informal financial institutions and mechanisms) and/or use community based organisations/mechanisms to save or borrow money – the infor- mally served segment; and • Those individuals who have or use both formal and informal products and services. 102 Review of Existing Financial Capability and Literacy Measurement Instruments Formally served = adults who have/use financial products and/ or services provided by a financial Financially included = institution (bank and/or non-bank) adults who have/use financial products and/ or services — formal and/or informal Informally served = adults who have/use financial products and/or services which are not regulated, e.g. cooperatives, farmers associations, savings clubs/ groups, private money lenders The next step in the segmentation seeks to better understand or unpack the formally served segment of the population, such as individuals who have or use products or services from financial institutions that are regulated through an Act of law (formal financial institutions). This step further segments the formally served population into: • Those individuals who have or use products or services from licensed commercial banks that are regulated by the central/reserve bank – the banked population; • Those individuals who have or use products or services from financial institutions that are regulated through acts of law but which are not commercial banks. Those individuals who have or use products or services from such institutions, comprise the “Served by Other Formal financial institutions� segment of the population (referred to as “Other Formal� segment); and • Those individuals who have or use products or services from both commercial banks and other formal financial institutions. Banked = adults who have/use financial products and/or services provided by a commercial bank regulated by the central bank Formally served Served by other formal financial institutions = adults who have/use financial products and/or services provided by regulated non-bank formal financial institutions, insurance companies, retail credit providers, remittance service providers) Review of Existing Financial Capability and Literacy Measurement Instruments 103 Finally the segmentation process looks at the overlaps between the different population segments allowing for a better understanding of the following population segments: • Those individuals who have or use only bank products and services; • Those individuals who have or use bank and other formal products and services; • Those individuals who have or use bank and informal products and services; • Those individuals who have or use bank and other formal and informal products and services; • Those individuals who have or use only other formal products and services; • Those individuals who have or use only informal products and services; and • Those individuals who have or use other formal and informal products and services. Financial inclusion Formally served Have/use only bank products Have/use bank AND non-bank formal Informally served products Have/use Have/use bank AND bank products non-bank formal Have/use AND informal products AND only non- products informal products bank formal products Have/use non-bank Have/use formal products only informal AND informal products products FinScope indicators: To reflect on the FinScope objectives, the FinScope approach uses a specific set of indicators developed for this purpose: • The Financial Access Strand • The Financial Access Landscape • The Access Frontier The Financial Access Strand is used to enable comparison of levels of financial inclusion across countries/market segments. In calculating the Access Strand, a hierarchical ap- proach is used in order to depict the following points: • The %age of adults who are banked (banked) – identifying adults using commer- cial bank products. This is not necessarily exclusive usage; these individuals could also be using financial products from other formal financial institutions or informal products as well as bank products; 104 Review of Existing Financial Capability and Literacy Measurement Instruments • The %age of adults who are formally served but who are not banked (other formal) – identifying adults using financial products from formal financial institutions which are not commercial banks such as microfinance institutions or insurance com- panies. This excludes bank usage but not informal usage; these individuals could also be using informal products; • The %age of adults who are not formally served but who are informally served (informal only) – adults using informal financial products or mechanisms only. This is exclusive informal usage and does not include individuals who are within the banked or other formal categories of the access strand that also use informal services; • The %age of adults who are excluded/unserved – adults using no financial prod- ucts to manage their financial lives, neither formal nor informal, and depend only on family/friends for borrowing and save at home if they save. The Financial Access Landscape is used to illustrate the extent to which individuals have or use financial products and services. The spider diagram depicts the %age of adults that have or use— • Transactional products/services • Savings products/services • Credit products/services • Insurance products/services • Remittance products/services. The Access Frontier tool was developed to understand market development, providing a perspective of individuals who can be served by the market over time. It typically estimates the maximum %age of people who can access a particular product under certain terms and conditions. Financial capability: FinMark Trust views the core components of financial capability as follows: • Awareness: Determines the level of consciousness of financial services. It includes specific questions (for example, which financial products are you aware of?). • Knowledge: Determines the level of information and skills which have been acquired through experience or education regarding financial services. Includes several rating questions on knowledge (for example, on banking products, insurance products). • Perceptions and attitudes: Follows awareness and/or knowledge, and determines the settled way of thinking or feeling about financial services. It includes a battery of attitudinal statements to agree or disagree with. • Behavior: Follows the above and determines the way in which one acts with respect to financial services. Assesses the %age of the bankable population that have/use financial products/services (for example, banked, formal other, informal) The Consumer Financial Vulnerability Index (CFVI): The CFVI is published twice a year in South Africa by the Bureau of Market Research (BMR), in collaboration with Fin- Review of Existing Financial Capability and Literacy Measurement Instruments 105 Mark Trust, using FinScope data. The CFVI covers the latest trends in consumer financial vulnerability, as gathered from interviews with key informants as well as the responses obtained following the inclusion of key CFVI questions in the FinScope South Africa study. The overall CFVI and subindices are based on a 10-point scale where 0 indicates total financial security and 10 indicates total financial vulnerability. 0–1.99 2.0–3.99 4.0–5.99 6.0–7.99 8.0–10 Financially very Financially Somewhat Financially Financially secure secure financially vulnerable very vulnerable vulnerable 3.2. Key Results Key results for each survey are available at www.finmarktrust.org.za and www.finscope. co.za. Country comparison of levels of access where FinScope surveys have been com- pleted: RSA ‘11 63 5 5 27 Namibia ’11 62 3 4 31 Swaziland ‘11 44 6 13 37 Boswana ’09 41 18 8 33 Lesotho ‘11 38 23 20 19 Ghana ’10 34 7 15 44 Nigeria ‘10 30 6 17 47 Zimbabwe ’11 24 14 22 40 Kenya .09 23 18 26 33 Uganda ‘10 21 7 42 30 Malawi ’08 19 7 19 55 Rwanda ‘08 14 7 26 53 Zambia ’09 14 9 14 63 Tanania ‘09 12 4 28 56 Mozambique ’09 12 1 9 78 0 10 20 30 40 50 60 70 80 90 100% Banked Other formal non-bank informal only Excluded 3.3. Policy Application The FinScope results are used by various stakeholders as a catalyst to stimulate evidence- based dialogue that will ultimately lead to effective consumer financial empowerment strat- egies and policies, impacting regulation as well as prompting public and private sector interventions. 106 Review of Existing Financial Capability and Literacy Measurement Instruments 3.4. Potential for Replication FinScope has a high potential for replication, both in different geographies (countries and regions) as well as over time. The FinScope South Africa, for example, has been con- ducted on a yearly basis since 2002. 4. Conclusion Although the FinScope survey is primarily used as a determination of financial access and inclusion, it is adaptable for as a measure of financial capability. Case Study: FinScope Consumer Survey Namibia 2011 The following section outlines the approach for the FinScope Consumer survey Namibia 2011. Sampling approach The sample for Namibia was designed as a representative cross-section of all individuals, 16 years and older in Namibia. The goal was to give every individual 16 years and older an equal and known chance of selection for interview. This objective was reached by (a) applying random selection methods at every stage of sampling and (b) applying sampling with probability proportionate to population size (PPS). A randomly selected sample of 1,200 cases allows inferences to the national population with an average margin of sampling error of no more than plus or minus three %age points, at a confidence level of 95 %. The latest official census data was used as the sampling frame. The Central Bureau of Sta- tistics (CBS), Namibia created and maintains the sampling frame for Namibia. This frame is based on the 2001 Population and Housing Census. The frame was updated in 2006 for high growth urban areas that resulted from urban migration. The sampling frame is strati- fied by 13 regions, and within regions by urban/rural locality. The total national population was 1,830,330 (see table 1 hereafter) based on the 2001 Population and Housing Census and 1,991,746 as projected in 2006. Review of Existing Financial Capability and Literacy Measurement Instruments 107 Table 1 Population by STratum Region 2001 Census Population Total Rural Urban Caprivi 79,826 57,692 22,134 Erongo 107,663 21,572 86,091 Hardcap 68,249 37,105 31,144 Karas 69,329 31,730 37,599 Kavango 202,694 165,730 36,964 Khomas 250,262 16,733 233,529 Kunene 68,735 51,731 17,004 Ohangwena 228,384 225,570 2,814 Omaheke 68,039 54,138 13,856 Omusati 228,842 226,202 2,640 Oshana 161,916 112,019 49,897 Oshikoto 161,007 146,078 14,929 Otjozondjupa 135,384 80,373 55,011 Namibia 1,830,330 1,226,718 603,612 Table 2 shows the number of interviews conducted in each stratum, as well as the number of primary sampling units (PSUs) that were originally selected in each stratum. A clustered, multistage probability sample design was employed. Multistage sampling was used to select geographically defined sampling units of decreasing size. The sampling process for both urban and rural areas consisted of four stages: • Random selection of PSUs (Enumerator Areas) • Random selection of Sampling Start Points (SSPs) • Random selection of households • Random selection of respondents within the selected households The random selection of PSUs was based on probability-proportional-to-size principles (PPS). Since the census bureau does not have household maps or listings available, each field team conducted a household listing exercise in each PSU prior to randomly selecting eight households per PSU by dividing the total number of households listed by eight, to determine the sampling interval. A table of random numbers was used to select a starting point on each household listing sheet and the sampling interval was employed to select the eight households within each PSU. If a selected household refused the interview, or needed substitution because the selected respondent did not return to the house on the same day, or did not honour the interview after the third call, a subsequent household was 108 Review of Existing Financial Capability and Literacy Measurement Instruments Table 2 Sample Selected Region Number of interviews Number of PSUs per stratum selected per stratum Urban Rural Total Urban Rural Total Caprivi 16 56 72 2 7 9 Erongo 80 24 104 10 3 13 Hardcap 32 40 72 4 5 9 Karas 32 40 72 4 5 9 Kavango 24 80 104 3 10 13 Khomas 128 16 144 16 2 18 Kunene 24 48 72 3 6 9 Ohangwena 8 96 104 1 12 13 Omaheke 16 48 64 2 6 8 Omusati 8 104 112 1 13 14 Oshana 32 64 96 4 8 12 Oshikoto 8 88 96 1 11 12 Otjozondjupa 40 48 88 5 6 11 448 752 1200 56 94 150 selected using the sampling interval, counting from the household after the last household selected for interviewing. Each enumerator was assigned two households for interviewing. Using this method eliminated the need for selection of households by the enumerators. At each household, enumerators used a Kish grid to randomly select an individual for inter- viewing. During respondent interviews, enumerators also enquired about informal financial service providers in the area. These were noted and communicated to the supervisor. The supervisor was then required to conduct a semistructured interview with the providers that were identified and willing to participate in the survey. Training and piloting A 10-day training workshop was scheduled and 25 enumerators were invited to attend. At- tendees were evaluated based on their performance during the training workshop and the 20 enumerators who performed best were selected for the data collection activities. From these; the best four enumerators were identified as supervisors. The piloting was under- taken in areas outside the FinScope Namibia 2011 sample in the Khomas region. Thirty- two English interviews and 12 local language interviews were completed during the pilot phase. The questionnaire was pretested for language, comprehension, length of adminis- tration and the effectiveness of the questionnaire to collect the data required for analysis. Additionally, local language questionnaires were tested for consistency with the English questionnaire. All processes and procedures for data collection purposes were also as- sessed. It included how survey teams located PSUs and identified PSU boundaries on the Review of Existing Financial Capability and Literacy Measurement Instruments 109 respective PSU maps. It furthermore assessed the household listing exercise and the se- lection of households and participants for the survey. The training agenda was as follows: • Day 1: introductions, background and introduction to the study, familiarization of English questionnaire • Day 2: familiarisation of local language translations, checking of local translations, corrections suggested to translations, general questionnaire Q&A • Day 3: listing methodology, sampling and respondent selection, general listing, sam- pling, selection Q&A • Day 4: practice sessions (enumerators doing mock-interviews), debrief on practice interviews • Day 5: practice sessions (enumerators doing mock-interviews), debrief on practice interviews, planning: pilot testing • Day 6: supervisor training (quality control, listing, sampling selection, data collection planning, general Q&A regarding supervision duties) • Day 7: pilot testing in Windhoek • Day 8: debrief on pilot testing, suggestions for changes to questionnaire, practice sessions (enumerators doing mock-interviews), debrief on practice interviews • Day 9: data collection planning, familiarisation with PSUs to be visited • Day 10: final debriefing on training, revision, finalisation of staff contracts Each day’s session started at 9 a.m. and ended at 4 p.m. On conclusion of the training workshop, four data collection teams were identified. Each team consisted of one supervi- sor and four enumerators. A total of 20 field staff were employed for the survey. Data collection The survey introduction: Prior to embarking on data collection activities, introduction letters were sent to all Regional Councillors and the respective Chief Regional Officers (CROs). Public service announcements were sent to public and commercial radio stations in Namibia. Furthermore, the survey introduction letter was submitted to the Namibian Agricultural Union (NAU) which then published an announcement of the survey in their weekly newsletter. The NAU further sent letters to its members, informing them about the survey. In addition, the introduction letter was translated in each of the questionnaire languages and each team member carried a laminated hard copy of the introduction letter with them at all times. Data collection activities: Prior to fieldwork deployment, each supervisor received a de- tailed travel and work plan. This aided supervisors in planning and rolling their daily data collection activities, and assisted the office in tracking each team’s progress throughout the data collection period. Upon arrival in each region, teams were required to visit the of- fices of the Regional Councillors. Teams had to introduce themselves and announce their presence in the region. In many instances, Regional Councillors informed Constituency Councillors who in turn informed village headmen and women. In some instances, the survey was also announced on the local radio stations within the constituencies again. As 110 Review of Existing Financial Capability and Literacy Measurement Instruments additional courtesy, teams visited village headmen and women upon arrival in the respec- tive villages to announce their presence and to introduce themselves. Some of the village headmen and women also sent messengers to their respective households within the vil- lage to further announce the survey. B.10. U.K. FSA Measuring Financial Capability: An Exploratory Study (2005) 1. General introduction to the method As the independent body that regulates the financial services industry in the UK, the Fi- nancial Services Authority (FSA) initiated a national Financial Capability Strategy in 2004 to understand the current state of financial capability in the UK and to improve consumers’ understanding of the financial services industry. The FSA subsequently asked the Personal Finance Research Centre at the University of Bristol to carry out an exploratory qualitative study to design a questionnaire that could be used to measure levels of financial capabil- ity based on a theoretical investigation of the concept of Financial Capability in addition to an intensive qualitative research. The British Market Research Bureau (BMRB) and the Financial Services Authority (FSA) commented on the process of developing the question- naire from being primarily a qualitative guide to a structured questionnaire for use with CAPI software7. The consultancy firm Ethnos also contributed with insight that enabled the questionnaire to be more relevant to black and minority ethnic respondents. 1.1 Purpose of the survey The FSA baseline survey was commissioned to achieve two main objectives: 1. Collect robust and reliable data that will enable measurement of financial capability in the UK population 2. Deliver data that allows reliable sub-analysis of financial capability for England, Scotland, Wales and Northern Ireland. The baseline survey was aimed to identify a set of indicators of financial capability across the following dimensions: • Finding out how people manage their money • Understanding how much people plan ahead for both expected and unexpected expenses • Capturing people’s understanding of different products, and whether they choose those which are suitable for their needs • Identifying whether people have access to independent advice; and • Checking what people do if they have a dispute or complaint about a financial trans- action. The survey was run as a baseline in 2005 with the aim of repeating it in future years in order to monitor people’s understanding of financial matters over time and assess the ef- fectiveness of any measures taken to enhance their understanding. 7 CAPI (Computer Assisted Personal Interview) is an interviewing technique where either the respondent or the interviewer sits at a computer terminal and enters the answers. Review of Existing Financial Capability and Literacy Measurement Instruments 111 1.2 Link between purpose and chosen approach The conceptual model of financial capability used on the FSA survey drew on the Adult financial capability framework that was developed earlier by the FSA and the Basic Skills Agency in 2004. The model that emerged identifies three key elements that determine financial capability and, consequently, affects overall financial behavior of individuals: knowledge and understanding, skills, confidence and attitudes. These elements are influ- enced by a person’s experience and circumstances, and by their personality. As a result of extensive qualitative research of the current definitions and concepts of financial capability, the research team established the basis for the initial discussions in the focus groups. The researchers were then able to develop and refine questionnaires that were consistent with the conceptual model but which used approaches that were rooted in people’s understand- ing of what financial capability is all about. In conclusion, it was evident that people conceived financial capability in behavioral terms, suggesting that a baseline survey should be structured accordingly. 2. Questionnaire (3 pages) • Original development of questionnaire (source of questions, methods for designing new questions, etc.) The framework of developing and testing the questionnaire can be summarized in the fol- lowing five stages; the first two represent developing and testing the conceptual framework assumptions, while the remaining three were undertaken for testing the developed ques- tionnaires and piloting them in the field: 1. A literature and research review to help develop a model of financial capability and to review questions used in other surveys. 2. Eight focus groups held in three different locations to explore people’s perceptions of financial capability and to identify ways of capturing financial capability in a survey. 3. A first wave of depth interviews with people who had participated in the focus groups to develop the content of the questionnaire. 4. A second wave of semi-structured interviews to provide a cognitive test of the questionnaire. 5. Two further waves of interviews to test the questionnaire8. Discussions in the focus groups reflected that financial capability encompassed four be- havioral domains: • Managing money: being able to live within one’s means. • Planning ahead: coping with unexpected events and making provision for expected ones in the long term. 8 Further testing was undertaken with people from black and ethnic minority communities in a separate but linked study undertaken by Ethnos Research and Consultancy. 112 Review of Existing Financial Capability and Literacy Measurement Instruments • Making choices: being aware of the financial products offered, and being able to choose the most appropriate to an individual’s circumstances. • Getting help: self-reliance and/or using third-party advisers. Considerable time was spent designing questions within the first domain to ensure that it included questions that were appropriate to people at the income extremes, that it identi- fied and accommodated people with limited responsibility for money management. This section of the questionnaire covered: record keeping and awareness of personal finances, planning for ‘lumpy’ expenditure, and living within one’s means. As for the second domain, it was important to distinguish, among better-off people, between those who have planned for expected and unexpected events and those who have adequate provision by default rather than by design. Likewise it shows the importance of being able to identify people on low incomes who would plan if they had enough money, and those who would not. The questions in this section covered how people would deal with: an unexpected drop in income, an unexpected expense, anticipated expenses, and retirement. The third domain of questions begins with a general section, which assesses basic knowl- edge about financial products, attitudes towards risk, confidence about selecting appro- priate financial products, and the extent of trust of financial advisers. The questions are tailored to accommodate different levels of involvement with the financial services market. This section collects information about how people: monitor the products that they hold and how they choose which products are most appropriate. The fourth and last section covers the steps that people take to ensure that they are well informed as well as their use of advisers. It explores how people: keep up with changes, use information and advice, and deal with complaints. 2.2.1 Revisions to the questionnaire The questionnaire was developed through two waves of interviews with 33 people: a wave of depth interviews to refine the content of the questionnaire and a wave of cognitive test- ing interviews using the structured questionnaire. The questionnaire was also tested with 15 people from black and ethnic minority communities. The draft questionnaire was refined by two further waves of testing, each involving 100 people. Some questions were added as a result of the pilot stage. For example, the mortgage sec- tion of the questionnaire asked about the value of property owned by those who currently held a mortgage product. As a result of the pilot, it was thought desirable to also find out the value of property owned by those who did not currently hold a mortgage. An additional question was inserted into the demographic section, with a filter, which excluded people who had already been asked this in the mortgage section. It was also thought necessary to specifically ask people about informal savings and debts. In particular, people from ethnic minorities and those on lower incomes were thought more likely to have savings held by, or owe money to, family members, friends, or other people in their community. On the other hand, some questions were thought to be less relevant than others to the core objectives of the research and, hence, were deleted. Because of the need to make Review of Existing Financial Capability and Literacy Measurement Instruments 113 space for new questions while ensuring that the overall questionnaire length did not place an unreasonable burden on some respondents, these questions were also deleted from the questionnaire. An example of that are questions specifically asking about the type of interest rate on current mortgage holdings (such as whether fixed, variable or capped). The greatest number of changes, however, involved the alteration of existing questions. These included altering question text, inserting clarifying statements, adding or removing question codes, splitting questions out into two separate questions, putting the answer list on a showcard, or moving the questions to more appropriate sections of the questionnaire to improve the overall flow. The last type of changes was technical (or so-called non-textual) where computer code was developed by BMRB to ensure the efficiency and accuracy of data collection during the interviews using the CAPI software program. 3.4.1. Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) Section Purpose A: Personal §§ Provide demographic information on the person selected randomly for Information the interview as well as their partner, if applicable, including questions about: age (or age group), number of children (age range 1-20), education level, and employment status. §§ The eligibility for being selected is to be 18 years old or older. §§ The questions in this section are required for routing and question wording later in the questionnaire. B: Money §§ Most of the questions are addressed to respondent and partner (if Management applicable). §§ Many questions have additional probing or prompting parts if needed to assist respondent in answering. §§ Provide detailed information on how people manage their bank accounts, their record keeping habits, how accurately they know their account balance, how they manage overdraws, and how they manage their money in order to pay for ‘lumpy’ expenses such as car insurance or utility bills. §§ One self-reporting question on how people would describe their attitude towards money management in general (e.g.; impulse purchases, latency in paying bills, saving vs. spending, etc) C: Planning §§ Covers topics on how well people plan ahead or would plan ahead if Ahead they had the necessary funds to do so. §§ Focuses on how people would deal with an unexpected drop in income, an unexpected expense, anticipated expenses and retirement. §§ Contains a number of attitude statements in regard to long-term financial planning. §§ Prompting and probing may be used if necessary §§ One self-reporting question on how people would describe their attitude towards planning ahead in general. 114 Review of Existing Financial Capability and Literacy Measurement Instruments Section Purpose D: Making §§ Provides information on the use of financial advisers, attitudes Choices towards advice, which types of financial products the respondent currently holds, and any active purchases they have made in the last five years. If they have taken out any financial products in the last five years they will be asked which ones they personally played an active role in taking out. §§ One self-reporting question on how people would describe their attitude towards financial advisers and taking out financial products §§ These questions, which are asked using two sets of shuffle cards, determine which of the follow-up sections respondents will be asked. E, F, G, H, J, §§ Obtain information on the value and type of holdings, and the amount and K: Product of money paid into or drawn out of holdings. Respondents were asked Holdings about each of the following categories of product held: mortgages, protection insurance, savings, investments, loans and credit cards. E, F, G, H, J, §§ Capture the sources of information respondents used and whether and K: Product they shopped around for their most recent purchase of prioritized Purchases products, up to a maximum of two. Within product categories, respondents were asked about the product type purchased most recently. §§ Where the respondent had made an active purchase across more than two product categories, the categories to be followed up were prioritized in the following order: investments, mortgages, protection insurance, credit cards, loans and credit agreements, general insurance, savings products and current accounts. M: Getting Help §§ Questions were asked about information sources used to monitor economic and financial matters in the economy generally. §§ Questions were also asked about whether or not people have experienced dissatisfaction with product and service providers, and the course of action, if any, they took as a result. N: Money Quiz §§ A series of questions instrumented to test respondents’ knowledge of financial products, their ability to read simple graphs, and their understanding of percentages. Z: Final §§ This asked about respondents’ religion, ethnicity, income, highest Demographics qualification held, tenure of household, etc. Sampling • Survey population Population data was obtained from the Labor Force Survey June-August 2005 (derived from Census 2001 projections). The FSA survey involves interviewing a representative sample of adults aged 18 and over in the UK using a random location sample design. Da- tasets were obtained from the DWP Family Resources Survey. • Unit of analysis The unit of analysis is taken to be an adult aged 18 and over in the UK. • Sampling strategy (sampling frame, method, sample size, etc.) A sophisticated form of quota sampling called “Random location sampling� method was Review of Existing Financial Capability and Literacy Measurement Instruments 115 used to select areas to conduct interviews. Firstly, the random selection of small areas (combinations of 2001 census output areas (COAs), comprising 300 households on average) was undertaken by BMRB. Areas were selected from a full listing of all areas in the country, stratified by region and neighborhood type. Sample points were then selected with probability proportional to size. This process was conducted separately for the four countries of the UK, in order to provide 3,100 inter- views representative of the English population, and 500 interviews representative of each of the Scottish, Welsh and Northern Irish populations, giving 4,600 interviews in total. All addresses in the area were issued to the interviewer. Interviewers were able to contact all of these addresses if necessary in an attempt to obtain interviews, but were given no choice in terms of where to work, since they must interview within a few streets. At the second stage, interviewers were asked to conduct a fixed number of interviews per sample point. In selecting individuals for interview, interviewers applied quota-style controls to ensure that representativeness was maximized in each sampling point across gender and age classes. The controls were based on four age categories and working status within gender. These controls are based on the propensity to be available for interview, therefore minimizing the risk that the interviews would be skewed towards an unrepresentative group. In addition to the main survey, a boost survey was conducted in which the sample frame com- prised sample units based on COAs with a non-white population of 20% or more (61% of the Great Britain non-white population was represented). COAs that had been selected for the main survey were excluded from the boost sample frame, to eliminate the possibility of inter- viewing in the same household on each survey. Quotas were set on age (three categories), sex and working status (two categories). Wherever possible, interviewers from ethnic minori- ties conducted the boost sample fieldwork, leading to 50 out of 84 boost sample points (60%) being allocated to Black or Asian interviewers. In order to preserve the integrity of survey responses it was decided that the survey population would be restricted to English speakers. Implementation • Countries/regions of application (Information on this section included in 2.2.3) • Testing and piloting Two pilot stages took place, and during each stage 100 interviews were conducted by BMRB interviewers. A range of neighborhood types was chosen for the pilot stages, and interviewing procedures replicated those that would be used in the main stage. Represen- tative quotas were applied, respondents were not pre-recruited, and were offered the same £10 incentive as in the main fieldwork stage to help make the pilots as realistic as possible. Using two pilot stages was particularly beneficial for a project of this size and complexity, because it enabled changes that were made as a result of the first pilot to be checked in the field before the main survey was conducted. The two pilots proved valuable in providing information about how questions worked with the general public, and how people responded to the survey. Learning from the pilot also enabled alterations to be made to the computer questionnaire and the fieldwork docu- 116 Review of Existing Financial Capability and Literacy Measurement Instruments ments, making the interview easier to administer and data-collection more accurate. • Interview method (face-to-face, telephone, CAPI, etc.) The questionnaire was designed to be administered in face-to-face computer-assisted in- terviews using the CAPI software. As a result of the pilot studies, BMRB enhanced the software with additional features to make it user-friendlier and more efficient in collecting accurate data by performing quantitative crosschecking of values entered. • Response rate Country/Survey Type Target Achieved England (main survey) 3100 3318 Wales (main survey) 500 551 Scotland (main survey) 500 524 Northern Ireland (main survey) 500 512 Ethnic minority (boost survey) 400 423 Total 5000 5328 Table 1: Number of target interviews vs. achieved interviews in each country by survey type. • Average duration of interview and total cost Quartile Average Length (in minutes) Quartile 1 24 Quartile 2 34 Quartile 3 45 Quartile 4 64 Average total length (once outliers are removed) 42 Table 2: Average timings of the four quartiles of interview questions • Key challenges and lessons learned Most of the challenges lied in the questionnaire design stage. It is a fairly difficult process to measure individuals’ financial capability in the kind of structured questionnaire that is required Review of Existing Financial Capability and Literacy Measurement Instruments 117 by a quantitative survey. Key challenges can be summarized in the following three points: • Interview length: The survey needs to address a wide range of factors that are not amenable to simple measures. It became evident in the interviews that assess- ing financial capability is a complex process, which needs to be tailored to specific circumstances and cannot be captured in a short interview. None of pilot interviews lasted less than an hour, while a number took more than two hours to complete. • Collecting information from people who live in widely different circumstances: At one extreme, people living on very low incomes with little or no opportunity to save or plan for the future and limited engagement with financial service products need to be assessed primarily in terms of their ability to manage their finances day to day. They need to be asked about plans that are realistic in their circumstances and prod- ucts that they have experienced. At the other extreme, someone with an income far in excess of their needs, who has money to invest and experience of a wide range of financial products including some complex investment ones, needs to be assessed quite differently. In this case, greater weight has to be given to their ability to plan for the future, and to choose and use appropriate products wisely. Failure to modify the interview in this way could lead to the false conclusion that poor people are finan- cially incapable and that financial capability increases with income. This has proved a considerable challenge. Designing a questionnaire for the majority of people was relatively straightforward. • Capturing the financial capability of people who rely on someone else to make their financial decisions: This includes people who rely on their partner to manage the household finances and people who rely on a financial adviser. The challenge was to find out what they would be capable of if they had to do it themselves. To circumvent this challenge, questions throughout the questionnaire were asked to ascertain who is mainly responsible for different aspects of the household finances. And, although most questions relate to the individual’s behavior, questions on in- come, saving and borrowing, and the ability to make ends meet are asked about the family as a whole. So, for individuals who rely on their partner to manage the family budget or make financial provision for the future questions were asked to assess the outcome at the family level to assess whether or not they are wise to rely on their partner in this way. • Making the questionnaire culturally sensitive: It needs to take account of ways that specific ethnic groups might handle their affairs that differ from others at their age or income level. These include language and literacy difficulties, the extent to which there is gender segmentation in financial responsibilities, and the need to take account of the complexities of money management and planning ahead in extended families. As noted above, the questionnaire was tested separately for these cultural differences. 3. Analysis Methods used for analysis (methods used to construct measures of literacy/capability/in- clusion etc., and for further analysis) The analysis of survey data was done with two goals in mind: 118 Review of Existing Financial Capability and Literacy Measurement Instruments 1. Creating a scoring mechanism to identify people’s relative strengths and weak- nesses in the four financial capability domains 2. Describing the types of people most likely to display higher or lower levels of finan- cial capability. Data was analyzed separately by each domain under the presumption that individuals who do well in one of the four domains will not necessarily do well in all the others. The nature of the questions indicated that it would be most appropriate to use factor analy- sis to indicate levels of consistency in the ways that survey questions were answered and to create a financial capability score. This approach is well suited to the types of inter- related questions used in the survey as it makes use of many different pieces of informa- tion about each person. The initial investigatory analysis helped to identify those questions that might be most confidently added to or discarded from a score of financial capability for each domain. The decision about which questions to use in the scoring was based on a combination of statistical evidence and the findings of the conceptual phase of this re- search project. The factor score from the factor analysis reflects a particular combination and weighting of the questions used to derive that factor. In its raw form this score has an average value of zero, with values typically ranging from +3 to -3, depending on the patterns of people’s answers to the key questions. In the case of our analysis of financial capability, the factor score represents the responses of each individual across a range of questions, taking into account the relative importance of each question. Values were scaled to vary between 0 and 100 with no representation of pass and fail thresholds. The factor analyses across the four domains have created five sepa- rate scores for each respondent. The first two scores relate to different aspects of the first domain, ‘managing money’, while the final three scores each relate to separate domains, ‘planning ahead’, ‘choosing products’ and ‘staying informed’. The second stage in the analysis was to find a way to identify people at risk of having particularly low levels of financial capability. To that end, Cluster Analysis was used to iden- tify groups of respondents (or clusters) with similar patterns of financial-capability scores across the domains. Once the clusters were identified the analysis drew on demographic data to identify common characteristics within groups. Key results The analysis involved studying the overall distribution of scores in each domain as well as the variation in scores by key personal characteristics. Cluster analysis To gain a better understanding of the characteristics underlying the range of financial capa- bility scores, the statistical technique of Cluster Analysis was employed. Cluster groups were identified by the cluster analysis according to their average factor scores compared with the overall averages. The clusters were then labeled according to the numbers of areas of weakness in the four financial capability domains. Within these Review of Existing Financial Capability and Literacy Measurement Instruments 119 clusters, particular individuals have scored more or less than the group average. Further analysis of the cluster groups enabled the researchers to identify the following eleven groups (or clusters) based on a general description of the typical person within each cluster: Cluster % of Number of Characteristics Sample Weak Areas A 36 0 §§ The most financially capable §§ Scored well above average on all factors except keeping track §§ Higher incomes and high levels of product holding §§ Slightly older than average §§ Included a disproportionate number of couples with no dependent children. Bi 13 1 §§ Adept at making ends meet, achieving the highest score §§ Scored well on planning ahead, but below average on staying informed §§ Had below-average incomes §§ %62 women §§ Less likely to be parents with dependent children. Bii 9 1 §§ Scored very poorly on keeping track §§ Average scores for planning ahead (which was surprising given their high incomes and high levels of product holding) §§ Good at choosing financial products and staying informed. Ci 4 2 §§ Scored very badly on keeping track §§ Poor at making ends meet §§ Good at planning ahead §§ High incomes and high levels of product holding §§ Indications of living beyond their means (not making ends meet despite having relatively high Incomes) §§ Had the highest proportion of couples and parents with children. Cii 4 2 §§ Poor on planning ahead and making ends meet §§ Considered to have been ‘living for the day’ §§ Good at keeping track and staying informed about financial matters §§ Young compared with the sample §§ More of them had children §§ Average Incomes and below-average levels of product holding. 120 Review of Existing Financial Capability and Literacy Measurement Instruments Cluster % of Number of Characteristics Sample Weak Areas Di 3 3 §§ Poor scores at choosing financial products and staying informed §§ Poor scores at keeping track of their finances but good at making ends meet §§ Above average age §§ Below average incomes and levels of product holding §§ More women but few had dependent children. Dii 3 3 §§ Successful at making ends meet §§ Did fairly well with keeping track but weak at planning ahead, staying informed and choosing financial products (probably due to very low incomes and levels of product holding) §§ Average age of 48 (included few parents with dependent children). Diii 7 3 §§ Good at staying informed, but bad at making ends meet and planning ahead. §§ The youngest of all groups (average age of 34) §§ Likely to be single §§ Low levels of product holding and below average their incomes. Ei 16 4 §§ Good at keeping track §§ Very low score on planning ahead, staying informed and choosing products. §§ Include many people who would be considered financially excluded. §§ Younger people §§ Lowest levels of income §§ Disproportionate number of women, single people, and parents with children. Eii 2 4 §§ Slightly above-average scores for keeping track §§ Taking positive steps with regard to planning ahead §§ Average levels of income §§ Large proportions of couples and parents with children. Fi 3 5 §§ Scored below average on all five aspects of financial capability §§ Young (average age 36) §§ Included equal numbers of single people and couples. §§ Lower than average incomes and levels of product holding, but not the lowest of all the groups. Review of Existing Financial Capability and Literacy Measurement Instruments 121 The general conclusion to be drawn from the cluster analysis is that individuals may be particularly capable in one or more areas, but lack skills or experience in others. Policy application One of the key outcomes from the FSA survey project is the production of a new database containing the answers to several thousand questions provided by over 5,000 respon- dents. The resulting report has analyzed that data, combining in it various ways to create new information. There are important groups of people within the data whose circumstanc- es could be explored in greater detail. Further analysis could look separately at the differ- ent countries within the UK, and consider whether there are important regional differences in capability. The experience of groups with lower financial capability could be unpacked in greater detail, or groups that are relatively unusual (highly capable, younger people and less capable, older people) could be investigated in greater detail to consider if there are important lessons to learn. Since the state plays a role in regulating the relevant financial markets in the study areas, it is important to revisit the measures of financial capability identified in this report, to con- sider how far the areas of strengths and weakness are changing, and the groups requiring most attention. Potential for replication (over time and in different contexts) By providing a new source of information to be made publicly available, researchers, aca- demics and policy makers are able to address new lines of enquiry and look at people’s financial lives in greater detail. Ultimately, the FSA survey was designed to be able to track changes in people’s financial capability over time by analyzing data through methods of longitudinal data analysis. Moreover, the FSA baseline survey methodology in designing the questionnaire was adopted by the Russia Financial Literacy and Education Trust Fund managed by the World Bank after tailoring it to suit the context of measuring financial capability in developing countries. B.11. Global Findex Gallup World Poll 1. General Introduction As part of an effort to provide better data to inform financial inclusion policies and research, the World Bank’s Development Research Group, with a grant from the Bill & Melinda Gates Foundation, developed the Global Financial Inclusion Index (Global Findex) questionnaire to measure people’s financial access and use of financial products across countries and over time. Gallup, Inc. fielded the questions in more than 100 countries in 2011 as part of its annual Gallup World Poll. Gallup used a randomly selected, nationally representative sample to select at least 1,000 adults in each country to interview for the survey. The Glob- al Findex survey provides cross-country, time-series data on individuals’ use of financial services that can be disaggregated not only by country but also by a number of different demographic characteristics, thus providing policy makers with a useful way to identify specific population groups excluded from formal financial systems. The survey database 122 Review of Existing Financial Capability and Literacy Measurement Instruments can be used in the development of targeted policies on financial inclusion and to track the effects of such policies over time. 1.1. Purpose of the Study The World Bank developed the Global Findex to provide comparable cross-country data on financial inclusion from the perspective of users of financial services. The survey’s objective is to measure the extent to which different groups, including the poor, women, and youth, use financial services and/or are excluded from the formal financial sector. The survey looks at financial inclusion along four indicators: account penetration, saving, borrowing, and manag- ing risks. The 2011 survey was fielded as a baseline survey, and the World Bank anticipates conducting future surveys to support the tracking over time of policies for financial inclusion. 1.2. Link Between Purpose and Chosen Approach According to the World Bank, the Global Findex is designed to fill the gap in the study of financial inclusion by collecting systematic data about the global reach of the financial sec- tor (Demirguc-Kunt and Klapper 2012). To this end, the World Bank developed questions to support the development of four main indicators on how people around the world save, borrow, make payments, and manage risks. To allow for cross-country comparison, the Global Findex used identical questions in all countries and Gallup Inc. fielded the 2011 questions as part of its annual World Poll in 148 countries using a consistent, sampling methodology in all countries. Gallup used a multi- stage area frame design and a Random-Digit Dial (RDD) to randomly select a nationally representative sample for each country, and in each country at least 1,000 people were interviewed. To allow for data comparison over time, the World Bank plans to collect annu- ally the main indicators on the use of formal accounts and borrowing, and will collect on a triennial basis the full set of indicators on financial inclusion. 2. Methodology 2.1. Questionnaire The Global Findex questionnaire includes 23 questions designed to gather information across the following four main indicators of financial inclusion: Account Eleven questions measure account penetration. To guide respondents, Penetration the Global Findex defines an account as an individual or joint account at a formal financial institution that can be used to save money, to make or receive payments, or to receive wages and remittances. The questionnaire thus includes questions that measure individual or joint ownership of formal accounts, which also comprises those having a debit or ATM account tied to an account. In addition, the questionnaire asks questions on how people use their accounts, barriers to account use, the use of formal accounts to receive wages, remittances, and payments from the government, how people deposit and withdraw money, the use of mobile money, and reasons for why people do not have an account, among others. Review of Existing Financial Capability and Literacy Measurement Instruments 123 Saving Behavior Four questions that focus on saving behavior. These questions explore the use of formal accounts for savings and also the use of informal saving methods (including, community based saving methods). In addition, there are questions on reasons for savings.1 Borrowing Five questions focus on borrowing. These include questions on informal and formal sources of borrowing, the use of credit cards, and purposes of borrowing. Insurance Three questions look at managing risk, including questions on the use Products/ of insurance for health care and agriculture. Managing Risk 1 The World Bank asked questions on the main purpose of loans only to those in developing countries because Gallup limits the time for telephone interviews in high-income countries. This, therefore, limits the number of ques- tions added to the core questionnaire for phone interviews. The questionnaire was translated into and executed in 142 languages to ensure national representation in 148 economies. Questionnaires were translated into the major languages of each country. Gallup’s translation process includes two independent translations, after which survey personnel adjudicate the differences and select the best translation. Gallup core questionnaire includes an average of 16 indexes on a range of social, political, and economic issues, and includes questions that are tailored to each region. 2.2. Sampling The Global Findex data provides universally comparable data on financial inclusion, using identical questionnaires and consistent sampling and methodology, for 148 economies around the world. 2.2.1 Survey population The target population is the entire civilian noninstitutionalized, population ages 15 and older in 148 developing and high-income countries. 2.2.2 Unit of analysis The Global Findex measures financial inclusion at the individual-level. This is done to mea- sure differences in economic empowerment among different groups, especially women, youth, and the poor. 2.2.3 Sampling frame (sampling frame, method, sample size, etc.) Gallup employed two different interview methods in implementing the Global Findex ques- tionnaire, phone interviews and face-to-face interviews. Accordingly, Gallup used two dif- ferent methods for selecting sample frames for the survey. Face-to-face interviews: In the developing world, including much of Latin America, East- ern Europe, nearly all of Asia, the Middle East, and Africa, an area frame design is used for face-to-face interviewing. In countries where face-to-face surveys were conducted, Gallup used a multistage area probability sample. In the first stage it identified the census listings 124 Review of Existing Financial Capability and Literacy Measurement Instruments of Primary Sampling Units (PSUs) consisting of clusters of households. PSUs were then stratified by population size and/or geography, and clustering was achieved through one or more stages of sampling. The samples include but rural and urban areas. In cases where population information was available, Gallup selected samples based on probabilities pro- portional to population size; otherwise it used simple random sampling. Random route procedures are used to select sampled households. Unless an outright refusal occurs, interviewers make up to three attempts to survey the sampled household. To increase the probability of contact and completion, attempts are made at different times of the day, and where possible on different days. If an interview cannot be obtained at the initial sampled household, the house to the left is selected as a substitute. Interview re- spondents are randomly selected within the selected households through the Kish grid or latest birthday method. In economies where cultural restrictions dictate gender matching, respondents are randomly selected using the Kish grid from among all eligible adults of the interviewer’s gender. Telephone interviews: In countries where telephone interviewing was employed, Gallup used Random-Digit-Dial (RDD) or a nationally representative list of phone numbers. In select countries where cell phone penetration is high, a dual sampling frame was used. At least three attempts are made to reach a person in each household. An RDD telephone survey design is used in countries where 80 % or more of the population has landline phones. Overall, the survey included 150,000 interviews in total, and at least 1,000 interviews were conducted in each country. In some countries Gallup collected oversamples for major cit- ies or for areas of particular interests. In China and Russia, samples of at least 2,000 were collected, while in some rare cases, samples of between 500 and 1,000 were selected. 2.3 Implementation 2.3.1 Countries/regions of application The Global Findex questionnaire was implemented in 148 countries, including high-, mid- dle-, and low-income countries. The regional and income group classifications were those used by the World Bank. However, some countries were excluded because of security risks, large expatriate communities, or an inconsistent methodology. These countries in- clude Algeria, Bahrain, the Central African Republic, Iran, Madagascar, Somalia, and the United Arab Emirates. 2.3.2 Testing and piloting Gallup piloted the Global Findex questionnaire in 20 countries across all regions. 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) In countries where less than 80 % of the population has telephone coverage, surveys were conducted face-to-face. In countries where more than 80 % of the population had land- lines, a telephone survey was conducted. Review of Existing Financial Capability and Literacy Measurement Instruments 125 2.3.4 Average duration of interview and total cost The face-to-face interviews for the entire survey, including the Findex module, lasted ap- proximately one hour, while telephone interviews were about 30 minutes long. In most countries, Gallup completed the fieldwork within two to four weeks. 2.3.5 Key challenges and lessons learned One of the things the Global Findex found in testing its questionnaire was that most people found it difficult to differentiate between banks, credit unions, and microfinance institutions. Accordingly, the World Bank does not differentiate between these types of formal financial institutions in its questionnaire. In some countries, The World Bank found that respondents were shocked by some ques- tions that were considered personal, especially in Mexico and Zimbabwe, where security was a high concern, and in countries like Cameroon, Italy, and Portugal, where personal finances are regarded as private. In some countries like Afghanistan, Cambodia, Chad, and rural Ukraine, respondents found some of the topics in the questionnaire unfamiliar or complex, which made questions difficult to answer. The survey team had tried to address this before the survey was implemented by including definitions. 3. Analysis The World Bank created a publically accessible database to share the Global Findex data. The Global Findex dashboards show data disaggregated by income, gender, age, educa- tion levels, country, region, urban or rural geography, and World Bank lending. The data- base also includes graphs and tables presenting the latest data and trends. 3.1. Method Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) The World Bank collected information on more than 40 indicators of financial practices around the world which it then organized into four main indicators of financial inclusion: account penetration, saving, borrowing, and managing risk. 1. Account Penetration indicators focus on the %age of adults that used formal ac- counts, the %age of adults that use a formal account to receive payments from the government, an employer, or family members living elsewhere, the process of us- ing formal accounts, the purpose of accounts, payments, barriers to account use, and alternatives to formal accounts. 2. Savings indicators focus on the %age of adults that saved in formal financial insti- tution in the past 12 months, the %age of adults who have saved using a commu- nity-based savings method in the past 12 months, and the reasons people save. 3. Borrowing indicators focus on the %age of adults that have credit cards, the sourc- es of borrowing, and the purpose of borrowing. 126 Review of Existing Financial Capability and Literacy Measurement Instruments 4. The managing risk indicators focus on the %age of adults with health or medical insurance, and the %age of adults working in farming, forestry, or fishing that have purchased crop, rainfall, or livestock insurance over the past year. In addition to presenting the data on account penetration indicators, the World Bank con- ducted a regression analysis to determine the sources of variations in account penetration around the world. Using proxies such as GDP per capita, spending per student on primary education, income levels, country Gini coefficients, age group, and sex the Global Findex determine the effects of country income, social inequality, education levels, individual in- come, age, and gender in explaining variations on account penetration around the world. 3.2 Key Results Below are summaries of the key findings of the Global Findex. Account §§ Worldwide 50 % of adults reported having an account at a formal Penetration financial institution Indicators • Eighty-nine % of people in high-income countries have an account • Forty-one % in developing countries have an account • Thirty-seven % of women have an account, as compared to 46 % of men • The Middle East has lowest account penetration with 18 % of adults reporting a formal account • In Cambodia, the Democratic Republic of Congo, Kyrgyzstan, Turkmenistan, and the Republic of Yemen, 95 % of adult population do not own a formal account • Mobile bank penetration is less than 5 % in developing countries §§ The regression analysis found that: • National income explains 70 % of variation among the world’s economies in the share of adults with a formal account. • At the individual level for low income, household income plays an important part in explaining variation in account penetration- worldwide only 22 % of those living on less than US$2 a day have an account (from Gallup’s household data on monthly income, converted into dollars) • A strong correlation exists between inequality and account penetration), lower inequality the higher account penetration the lower the inequality in the use of formal accounts by the richest income quintile and the poorest income quintile • A strong correlation exists between high levels of education and high account penetration, and a strong correlation between those ages 24–64 and high account penetration §§ Twenty-five % of adults did not own an account because they found account ownership too expensive, while most respondents (30%) did not have enough money to use one Saving Behavior §§ Worldwide, 22 % of adults reported having saved in a formal financial Indicators institution in the past 12 months §§ Thirty-six % of adults reported having saved or put aside money in the past 12 months §§ Forty-six % of adults with an account reported having saved or set aside money §§ Twelve % of account holders saved using alternative methods Review of Existing Financial Capability and Literacy Measurement Instruments 127 Borrowing §§ Half of adults in developed countries have credit cards Indicators §§ Seven % of adults in developing countries had credit cards §§ Worldwide, 30 % of adults reported borrowing money in the past 12 months §§ In all regions, except high income countries, most adults report borrowing money from friends and family, while in high income economies formal financial institutions are the most commonly used source of loans Managing Risk §§ Seventeen % of people in developing countries reported having paid Indicators for health/ medical insurance §§ Of those working in agriculture, forestry, or fishing, only 6 % had crop, rainfall, or livestock insurance 3.3 Policy Application By collecting indicators on various aspects of financial literacy and on an individual basis, the Global Findex offers a detailed look at financial inclusion around the world. The data offers policy makers information on the %age of people who own an account, how people make use of accounts and formal financial institutions, the major barriers to financial in- clusion, and the differences in financial inclusion by demographic characteristics. These indicators can help inform the development of policies and since the survey is designed to capture data over time, it also offers policy makers a tool for benchmarking policies on financial inclusion and measuring the expansion of financial products. In addition, since the Global Findex collects data across countries, it can also help policy makers comparatively study the effects of different financial inclusion policies across countries. 3.4. Potential for Replication (over time and in different contexts) Since the Global Findex was designed to capture cross-country data on financial literacy, it can be replicated in other countries excluded from its initial survey provided a consis- tent methodology is followed. In addition, since the Findex survey is designed to measure longitudinal changes in financial inclusion, it allows for replication over time as well as across countries. Additionally, the Global Findex questionnaire can be added to country or regional efforts to measure financial inclusion over time and to compare levels of financial inclusion across countries. B.12. Jump$tart Personal Financial Survey 1. General Introduction Since 1997, the Jump$tart Coalition has conducted a biennial Personal Financial Survey to measure the levels of financial literacy of U.S. high school seniors. The surveys were developed and managed by Lewis Mandell, currently of the School of Management at SUNY Buffalo. In 2008, the Jump$tart Coalition introduced its first ever national financial literacy survey for college students. The coalition is an umbrella organization of 180 corpo- rations, government agencies, and foundations, and 47 affiliated state coalitions that work on improving financial literacy for youth from kindergarten through college. This is done 128 Review of Existing Financial Capability and Literacy Measurement Instruments through advocacy, research, and the development of financial education standards and educational resources. The survey questionnaire is in the form of a multiple choice “test� that was developed by a team of financial educators. The questionnaire has served as the basis for a number of other surveys of financial literacy and financial behavior, including surveys for the Federal Reserve Board. In 2010, the Jump$tart Coalition announced the development of a Version 2 of the Per- sonal Financial Survey to be administered by a new firm, Learning Point Associates. The survey was piloted in 2010 but has yet to be implemented. The results of the original Jump$tart surveys, however, have informed the work of the Jump$tart Coalition and other government agencies, including the Federal Reserve Board and the U.S. Department of Education. 1.1. Purpose of the Study The original objective of the Jump$tart Personal Financial Surveys was to determine the extent of knowledge young people have about personal finance to determine how capable they are of navigating today’s complex economy. The results of the surveys were to guide the coalition in developing effective strategies for promoting and increasing financial edu- cation for students. In 2008, Jump$tart introduced college survey to find out how financial literacy develops as youth gain additional years of education and grow older. 1.2. Link Between Purpose and Chosen Approach The key objective of the Jump$tart surveys is to determine the levels of financial literacy of American high school seniors in order to better tailor financial education programs and activities. The idea for a biennial survey began after the initial 1997 baseline survey, which demonstrated that of the 1,532 high school students given the instrument, only 10.2 % were able to answer 75 % of the survey questions correctly. The Jump$tart Coalition then decided to administer the Personal Financial Survey every two years to measure progress toward increasing the financial literacy of U.S. high school students (Lewis 2008). To this end, Jump$tart randomly samples more than 1,100 high school seniors nationwide and administers the survey questionnaire in the form of a test. To allow for longitudinal comparison, largely the same questions are used for each survey. High school seniors serve as the survey population for a number of reasons: (1) senior year of high school is considered the last year of basic schooling required for all young Americans and therefore, the last chance for educators to encourage students to study any particular subject, (2) at 18, young people begin making financial decisions that could significantly affect their lives, and (3) the high school setting provides a good setting where randomly selected students can be encouraged to participate in a lengthy survey (Lewis 2008; Lewis and Schmid Klein 2007). 2. Methodology 2.1. Questionnaire 2.1.1 Original development of questionnaire (source of questions, methods for Review of Existing Financial Capability and Literacy Measurement Instruments 129 designing new questions, etc.) The questionnaire for the Jump$tart Coalition’s survey was developed by a committee of financial educators, including economist, in the form of a multiple-choice test. Questions were designed to evaluate financial literacy across four main areas, including income, money management, spending and credit, and saving and investing. These areas of finan- cial literacy were developed by 20 professionals from education, government, and financial services organizations as part of the Jump$tart Coalition as part of its national standards for financial education. 2.1.2 Revisions to the questionnaire The test questions for each survey are generally identical, but Jump$tart reorders the questions and makes a few cosmetic changes to prevent teachers from teaching the quiz. Cosmetic changes include changing the names of characters in the mini-case question included in the questionnaire. In addition to these changes, the Jump$tart Coalition has made a few revisions to some questions in order to reflect new financial regulations and changes in financial markets. 2.1.3 Final structure of questionnaire (topics, sections, number of questions, special skip patterns) The questionnaire for high school students is divided into two parts. The first part is a test that contains 31 multiple-choice questions to assess the respondent’s financial literacy, and the second part of the questionnaire gathers the respondent’s demographic and social information, including gender, ethnicity, family income, employment history, and educa- tional aspirations. The 31-question test is designed to gather information across these four main areas of financial literacy: Income The questionnaire has seven questions that measure the respondent’s ability to identify important sources of income and to understand how economic conditions and career, education, and skills affect income. It also comprises questions to test the respondent’s understanding of the effects of taxes, employee benefits, and government transfers on disposable income. Money There are five questions related to money management in the Management questionnaire. These questions assess the respondent’s ability to determine how limited financial resources affect personal decisions, the opportunity cost of financial decisions, and the importance of taking responsibility for personal financial decisions. The questions test respondent’s ability to plan for saving, investing, earning, and spending. In addition, these questions test respondent knowledge of the effects of inflation on spending and investing decisions, how insurance and other risk-management strategies protect against financial loss, and the availability of money management tools at financial institutions. 130 Review of Existing Financial Capability and Literacy Measurement Instruments Saving and The questionnaire includes eight questions on saving and investing. Investing These items focus on the reason for saving and investing, the relationship between saving and investing, how to buy and sell investments, sources of investment information, and how investors are protected. In addition, questions focus on factors that affect the rate of return on investment, and the risk, return, and liquidity of investment alternatives. Spending and The questionnaire consists of 11 questions that measure the Credit respondent’s knowledge of spending and credit. These questions evaluate the respondent’s ability to compare the benefits and cost of spending decisions, choose products and services, analyze the benefits and cost of consumer credit, and compare the advantages and disadvantages of different payment methods. Questions also focus on the respondent’s knowledge of the buyers and sellers under consumer protection laws, the factors that affect creditworthiness, the purpose of credit records, ways to avoid or correct credit problems, and the sources of consumer credit (McKenzie 2008). Unlike the high school survey, the 2008 Jump$tart first ever survey for college students consisted of 56 questions. Although the 31-question test was identical to that in the high school questionnaire, in addition to standard demographic questions, the college ques- tionnaire also included questions on financial behavior, such as credit card use, checking account balancing habits, incidence of insufficient funds, the incurrence of debt, and tax preparation. The Jump$tart Coalition added questions on financial behavior because since college students are legally adults and most have considerable experience with financial products, it is possible to measure their financial behavior and compare it to their financial literacy (McKenzie 2009; Mandell 2008). 2.2 Sampling 2.2.1 Survey Population The survey population is all public high school seniors in the United States. For the 2008 college survey, the target population was full-time U.S. college students between the ages of 18 and 23. 2.2.2 Unit of analysis The individual is the unit of analysis. 2.2.3 Sampling frame (sampling frame, method, sample size, etc.) The Jump$tart Coalition has developed two separate samplings methods for the high school and college samples. High School Sample: The coalition uses a stratified random sample to select high school students for the survey. The sample is drawn from a list of all public high schools in the United States provided by the U.S. Department of Education, and it is then stratified by state to ensure geographic representation. To minimize cost, the students are clustered by Review of Existing Financial Capability and Literacy Measurement Instruments 131 high school. The sample is deigned to so that a school’s probability of selection is propor- tional to the size of their senior class. To allow schools of all sizes a chance of selection, public schools within each state are rank ordered by the numbers of their seniors and school selection is based on a random start number and predetermined national sampling interval. States that wanted comparative state specific data were oversampled (40 schools per state) with the provision that results would be supplied if ten or more schools within their state participated in the survey. To minimize content knowledge bias, participants for the survey excluded seniors in honors, business, or economic classes. To further random- ize the process, schools were asked to administer the test to the class that took place closest to 10 a.m. Until 2004, participating teachers were rewarded with a U.S. savings bond, but now Jump$tart offers teachers a gift certificate to buy school supplies. Some participating teachers refuse the offer. College Sample: For the 2008 survey, Jump$tart collected a sample of 1,030 full time college students from a national compensated panel list maintained by Survey Sampling International. The coalition drew a subset of students from the panel that were between ages 18 and 23, and who listed themselves as students. Those in the sample who were not currently full-time college students were eliminated through a screening test. 2.3 Implementation 2.3.1 Countries/regions of application The Jump$tart survey was implemented in the four major U.S. regions: Northeast, Mid- west, South, and West. 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) The high school surveys were self-completed in the classroom, while the college surveys were completed online. 2.3.4 Response Rate Below are response rates for the Jump$tart high school surveys. The rates are based on the cluster sample of schools that participated in the surveys. All students at schools that participated in the survey completed the survey questionnaire. Year Response Rate Number of Students Interviewed 1997 43.6% 1,535 2000 18.3% 723 2002 21.3% 4,024 2004 15.8% 4,074 2006 17.6% 5,775 2008 21% 6,856 132 Review of Existing Financial Capability and Literacy Measurement Instruments 3. Analysis 3.1. Method Used for Analysis To measure the level of financial literacy of the survey participants, the Jump$tart Coali- tion calculates the %age of correct responses to the 31 financial literacy test. Generally, the coalition considers those students who correctly answer at least 70 % of the ques- tions, financially literate. The score of 70 % was chosen because it was considered the nationally accepted minimum %age-passing grade for students. Individual scores are averaged and used to create mean scores for high school and college students. These mean scores are compared to those of past Jump$tart surveys to determine longitudinal trends in financial literacy. In addition, the coalition typically analyzes the data by disaggregating mean scores by specific demographic characteristics like family income, race, gender, employment history, and degree of personal finance education. Scores are also disaggregated by the four main areas of financial literacy, and mean scores are developed for income, money manage- ment, saving and investment, and credit and spending. The coalition also calculates the %age of students that answer each individual question on the questionnaire correctly. For the college survey, mean scores were disaggregated by types of financial behavior and years of college education. 3.2 Key Results High School Students In the analysis of its latest survey data, the Jump$tart Coalition found that the financial literacy of high school students in 2008 was lower than when the surveys began in 1997. The mean score for the 2008 survey was 48 %, compared with 57.3 % in 1997, 51.9 % in 2000, 50.2 % in 2002, 52.3 % in 2004, and 52.4 % in 2006. Broken down by demographics, for the 2008 survey, male, Caucasian, and students from families with incomes of $80,000 and over, and whose families owned homes did relatively better on the test. When the 2008 scores were disaggregated by the four areas of financial literacy disaggre- gate scores, high school students did best on income (scored 56.1 %) and credit (44.1 %) and spending questions (50.5 %).9 They, however, scored worse on money management (40.9 %) and saving questions (43.2 %). When compared to demographic characteristics, high school students with bank accounts, paid employment, higher income families, and whose families owned homes, generally had significantly higher financial literacy in these four areas. College Students For the 2008 survey, the mean score for college students on the financial literacy test was 62.2 %, approximately 15 points higher than the mean score for high school students. In addition, scores improved for every year of college with seniors average a score of 64.8 %, 9 The credit and spending questions were scored separately. Review of Existing Financial Capability and Literacy Measurement Instruments 133 juniors scoring 62.1 %, sophomores scoring 61 %, and freshmen averaging a score of 59.3 %. Among the college students, Caucasian college students scored 63.3 % while Hispan- ics answered 59.8 % and African Americans answered 55.3 % of the questions correctly. By financial behavior, students with four or more credit cards, who prepared their taxes on their own, who had their own checking account, who generally paid the minimum balance on credit cards, and who expected $50,000 or more in debt after graduation did better on the financial literacy test. College students do better in income, spending, and credit as- pects of the test, and do worse on the money management and saving questions. Overall, the 2008 survey found that college students were more financially literate than high school students. Although financial literacy increases with each year of education, the survey found that the financial literacy scores of both high school seniors and college stu- dents appear to be unaffected by having taken a full-semester course in personal finance in high school. By contrast, having played a stock market game in class significantly raised the financial literacy scores of both groups. 3.3 Policy Application The Jump$tart Personal Financial Literacy surveys have been used to inform a range of financial education activities implemented by the Jump$tart Coalition, including advo- cacy campaigns, the development education materials, and the creation of standards and benchmarks for financial education. In addition, Jump$tart has used the work of the sur- veys to inform its work with more than a hundred private, nonprofit, academic, and govern- ment institutions on conducting financial education programs and offering financial educa- tion materials to the youth and other groups. 3.4. Potential for Replication (over time and in different contexts) The Jump$tart Coalition’s questionnaire have been used to develop a number of other survey questionnaires, including questions for the Federal Reserve Board’s 2001 financial literacy and capability survey fielded as part of the U.S. Survey of Consumers (Hogarth and Hilgert,2003). The questionnaire can therefore be replicated by other organizations interested in testing the financial literacy of not only the youth but also adults. In addition, the survey can be implemented over time to monitor changes in levels of financial literacy for the chosen target population. The coalition plans to continue conducting the biannual high school and college financial literacy survey. References President’s Advisory Council on Financial Literacy (2008), “2008 Annual Report to the President,� The Depart- ment of the Treasury, http://www.jumpstart.org/assets/files/PACFL_ANNUAL_REPORT_1-16-09.pdf (Accessed May 15, 2012) The Jump$tart Coalition for Personal Financial Literacy (2007), “ National Standards In K-12 Personal Finance Education,� The Jump$tart Coalition for Personal Financial Literacy, Washington, DC. http://www.jumpstart.org/ survey.html. 134 Review of Existing Financial Capability and Literacy Measurement Instruments B.13. Livings Standards Measurement Survey (LSMS) /FinScope Fi- nancial Services Experiments (Ghana’s FSS) (LSMS/FinScope Gha- na) 1. General Introduction to the Method Recent attempts to collect data on financial sector breadth have been using both demand- and supply-side approaches. On the supply side, measures of the outreach of the financial sector often focus on the number of accounts of providers of financial services. A limitation of those data is that they are derived only from information about banks, which while im- portant or even dominant providers of financial services in many countries are not the full story. The accounts-based approach provides little information about the account holders themselves, and thus about the nature of financial exclusion in a given country. However, the demand-side approach focuses on interviewing users and potential users of financial services. These demand-side efforts to measure outreach are based on surveys of individuals and households using (1) stand-alone surveys that focus only on access to financial services, which tend to be relatively expensive but produce rich data sets and a detailed portrait of access, and (2) a small module of questions on financial usage and ac- cess that is embedded within a larger survey that is designed to cover another topic (for example, surveys of household expenditures or labor market participation) or multiple top- ics as in the Livings Standards Measurement Surveys (LSMS). The marginal cost of the modules is much lower than that of stand-alone surveys, but they yield data that are much less rich. Neither approach has produced comparable financial usage data that could be used to monitor the situation in a given country over time, or to compare outreach across a large number of countries. Because the stand-alone surveys are costly, they tend not to be re- peated at regular intervals, and when a stand-alone financial survey is eventually repeated, there is no guarantee that the sampling frame and questions will be the same as in the previous survey. Because the modules of financial questions are placed within a survey de- signed for a different purpose, they tend not to be given high priority, and comparability of data across surveys occurs largely by chance. A recent summary of the financial informa- tion generated in the LSMS shows that only a handful of basic questions about accounts and loans are asked in most modules, and even those are often asked in different ways, making the validity of comparisons across surveys indefinite (Gasparini et al. 2004). This study was part of a larger effort in the World Bank Development Research Group, Fi- nance and Private Sector Team, & Poverty and Inequality Team to understand household access to financial services and improving survey methods with focus on demand-side factors affecting household financial usage. 1.1. Purpose of the Survey The study reports results from a randomized experiment in Ghana to test whether the identity of the respondent and the inclusion of product-specific cues in questions affect the reported rates of household usage of financial services. Review of Existing Financial Capability and Literacy Measurement Instruments 135 Although there have been a few stand-alone efforts to measure household financial usage, the most advanced one is that by the FinMark Trust, which has deployed its FinScope sur- vey in a number of developing countries, primarily in Africa. Finscope surveys are designed to provide nationally representative information on individuals’ use of financial services, including detailed inquiries about specific types of financial products. These questions are also supplemented by others regarding the respondent’s attitudes toward financial institu- tions, risk, and coping strategies in times of economic hardship, among other issues. By contrast, the most comprehensive effort to use the modular approach to measure us- age, the LSMS, tends to ask broad, generic questions about “credit� or “accounts� or deal- ings with types of institutions such as banks. Another important difference between the FinScope and LSMS approaches is that the LSMS finance modules track household usage of financial services, whereas FinScope randomly selects individuals from the population to provide information only on their own use. In light of those two different approaches, the purpose of this paper is to— 1. Provide evidence from a randomized experiment that tests whether measured us- age of financial services is similar when respondents are asked detailed product- based questions (the FinScope approach) as opposed to more generic, institution- based questions (the LSMS approach). 2. Test whether the household financial usage information provided by the household head or a randomly selected informant is as accurate as that provided by a full enumeration. 1.2. Link Between Purpose and Chosen Approach The accounts-based and survey-based measures of usage of financial services are not substitutes for one another, but recent research has shown that there is a robust statistical link between the two (Beck, Demirgüc-Kunt, Peria 2007; Honohan 2008). That is, a regres- sion model can be constructed from the more readily available accounts-based information that can be used to generate reasonably accurate estimates of the harder-to-collect survey based data. Scaling up collection of data on usage of financial services to ensure accuracy and comparability across countries and over time would therefore require a survey-based approach. 2. Questionnaire 2.1. Original Development of Questionnaire (source of questions, methods for designing new questions, etc.) To provide some insights into the tradeoffs involved in the area of collection financial data in household surveys, the research team developed an experiment to test whether re- ported use of financial services is affected by either the choice of respondent or the format in which questions on financial services are posed. Respondents selected to provide financial service use data in surveys are often the head 136 Review of Existing Financial Capability and Literacy Measurement Instruments of household (however defined). This person is asked to provide information on house- hold use of loans and savings and, more rarely, insurance. This is the way that LSMS surveys were typically carried out. However, there are concerns that not all individual financial service use in the household is known to the head of household and thus, the data collected in this manner may understate overall use of services and, consequently underestimate specific types of services or use by certain categories of household mem- bers. Reviews of both savings and credit issues in LSMS surveys recommended the al- ternative method of using direct informants to collect savings and credit behavior (Kochar 2000; Scott 2000). In this way, data are collected from all adults in the household about their own use. On the other hand, in the FinMark surveys a third option for respondent selection arises. In these surveys, a random adult is selected for interviewing. This is a strategy to ensure that the pool of respondents is a probability sample of all adults in the country: a sample of household heads would be inappropriate for this purpose. Although FinMark surveys do not collect household financial service data, it would be easiest to have the randomly selected adult (already being interviewed) provide financial data for the household. Of con- cern would be whether this strategy would provide data of similar quality to that garnered from full enumeration of adults or even selecting the head of household to provide the information. A second key dimension on which surveys vary is the way in which questions are asked. For ease of implementation, lower costs, and low respondent burden, a short set of ques- tions that aggregate individual items into more global questions is preferred. However, research in other areas has shown that such questions may lead to accidental omissions or memory lapses, thus lowering reported incidence or use. For financial service use questions, the most disaggregated level would be to ask respon- dents about each and every service available, from ATM cards to health insurance, to in- formal saving associations to formal bank loans. This is the approach taken in the FinMark surveys. Such an approach should prevent accidental omission of service use. It does, however, increase the burden of the interview, which can lead to lower data quality. The opposite approach, more similar to that taken by LSMS surveys, is to ask about fi- nancial service-use at an aggregate level with a focus more on relationships with types of financial service providers than the specific products used. This approach is simpler and less expensive to implement. The experiment carried out in Ghana explicitly tests the ef- fect on reported financial service use of changing the respondent and changing the set of questions asked. To develop and implement the experiment, the research team worked with the national statistical office of Ghana (Ghana Statistical Service, or GSS). The GSS carries out a se- ries of national household surveys and other statistical activities. The most comprehensive of household surveys, the Ghana Living Standards Survey (GLSS), is an LSMS survey designed to provide information on multiple aspects of living conditions: human capital, productive activities, consumption, and access to and use of public services. Review of Existing Financial Capability and Literacy Measurement Instruments 137 The survey has been implemented five times in the country since the 1980s, with the one used in this study being carried out in 2005-06 (GLSS5). By revisiting a sub-sam- ple of the GLSS5 households, the study was able to take advantage of the information already collected. More data could be collected on financial issues in the Financial Service Survey (FSS) as questionnaire was not needed for gathering data on other characteristics of the household or individuals. Additionally, a more complex design was possible as interviewers only needed to be trained on financial questions and data collection. 2.2. Revisions to the Questionnaire The questionnaires were developed in close collaboration with the GSS and were adapted from both the LSMS and FinScope surveys. In addition to their own expe- riences in other surveys, the GSS called upon various experts and sources in the country to determine the comprehensive list of financial services that existed and the range of service providers. To adapt the original questionnaire modules to the context of Ghana, the GSS staff worked to determine the best terminology to use and how to minimize translation problems. 2.3. Final Structure of the Questionnaire (topics, sections, number of questions, special skip patterns) No details available. 3. Sampling 3.1. Survey Population The survey uses a national probability sample of households stratified by region and urban/ rural areas developed to present results at the level of the three ecological zones of the country (GSS 2006). The selected enumeration areas (and households) were distributed throughout the country as seen in table A.12.1. 138 Review of Existing Financial Capability and Literacy Measurement Instruments Table A.12.1: Enumeration Areas from the GLSS5 Used for FSS Sample Regions GLSS5 Cycle 10 GLSS5 Cycle 11 Total Urban EAs Rural EAs Urban EAs Rural EAs Northern 1 5 - 6 12 Upper East 1 2 - 3 6 Upper West - 3 - 3 6 Ashanti 3 6 3 6 18 Eastern 2 4 2 4 12 Brong Ahafo 2 4 - 6 12 Volta - 6 - 6 12 Western 4 2 3 3 12 Central 6 - 4 2 12 Greater Accra 6 3 8 1 18 Total 25 35 20 40 120 Note: The GLSS5 households included in the FSS were taken from the two interviewing cycles (10 and 11) of the GLSS5 closest in time to the fielding of the experiment. This was done to minimize the chances that a household might have changed significantly between the time of the GLSS5 and the FSS. 3.2. Unit of Analysis The unit of analysis is the household where usage of financial service is measured through asking the head of household, a randomly selected informant, or a full enumeration of household members financial usage. Only individuals ages 15 or older were included in the full enumeration or were selected as random respondents. 3.3. Sampling Strategy (sampling frame, method, sample size, etc.) The original framework for the experiment was a three-by-two matrix, with three types of respondents (head, randomly selected adult, and full enumeration) and two types of ques- tionnaires (product-based and institution-based). A simplified and more feasible design was drawn up to allow for making such basic comparisons. These comparisons focus on two issues. 1. The quality of household usage information provided by informants versus a full enumeration. 2. Data quality obtained using a product-based questionnaire versus an institution- based one. Review of Existing Financial Capability and Literacy Measurement Instruments 139 Three different questionnaires were fielded with the second and third questionnaires con- taining more than one treatment (see table A.12.2). Table A.12.2: Questionnaires Administered Respondent Product Institutional Head of household --------- Group 3 n = 659 households Randomly selected adult Group 2 Group 3 n = 643 households n = 659 households All adults (15 and older) Group 1 Group 2 n = 653 households n = 643 households 1118 individuals 1568 individuals Note: Each group represents a different questionnaire, where “n� is the number of households, household heads, randomly selected adults, or individuals who were administered the questionnaire. Once the sample design was complete, households were randomly assigned to one of three groups. Each of these groups of households were administered a different question- naire. In households where one of the treatments was for a randomly selected adult to be interviewed, interviewers used Kish tables to make that selection in the field. 4. Implementation 4.1. Countries/Regions of Application (Information already mentioned in 1.1) 4.2. Testing and Piloting It was mentioned that extensive piloting of questionnaire instruments were undergone (both at GSS and in the field) but no further details were mentioned about the particulars of the pilot. 4.3. Interview Method (face-to-face, telephone, CAPI, etc.) Trained interviewers were employed to conduct face-to-face interviews, but there was no information mentioned on the tools that were being used (pencil-paper, CAPI, etc.). 4.4. Response Rate For mobility reasons, out of the 2,291 households re-visited, a total of 335 could not be r- interviewed (for a turnout of 85 %). 4.5. Average Duration of Interview and Total Cost 140 Review of Existing Financial Capability and Literacy Measurement Instruments Table A.12.3: Time Costs (in minutes) of Administering the FSS Survey Variations Does respondent have/hold the Questionnaire Type financial product? 1 2 3 Banked No 41.52 32.15 35.92 (468) (475) (479) Yes 52.75 36.09 42.76 (187) 158 (178) Formal Credit No 44.12 33.15 37.50 (613) (621) (634) Yes 53.50 32.08 45.22 (42) (12) (23) Insurance No 41.23 32.95 37.02 (495) (582) (579) Yes 55.55 35.18 43.32 (160) 51 (78) Note: Figures represent means and number of observations (in brackets). 4.6. Key Challenges and Lessons Learned Cull and Scott (2009) hypothesized that the basic characteristics of households in the FSS to match those in the GLSS 5 and for there to be no significant differences among the three groups of households administered the different FSS questionnaires. However, the result- ing data revealed a different image: 1. For variables related to location, the FSS sample seems to deviate from that of the GLSS by being more rural, having more households engaged in agriculture and being more likely to be located in the Coastal and Forest Zones of the country. This resulted from using simple random sampling of enumeration areas for the FSS, which fails to take into account the greater population found in urban enumera- tion areas that was captured in the original probability proportional to size sample. Even though the effects were not strong, the sample somewhat over-represents the rural population. 2. Between the time that households were interviewed for the GLSS 5 and when re- interviews of these households were attempted for the FSS, some households had moved or dissolved and could not be re-interviewed. Others chose not to respond. Of the 2291 households re-visited, a total of 335 could not be re-interviewed. Sys- tematic differences that might exist between the households that could and could not be re-interviewed could be problematic. To explore this issue, a probit model was run where the dependent variable takes on the value of one if the household Review of Existing Financial Capability and Literacy Measurement Instruments 141 was not re-interviewed and zero otherwise. Results show that rural households were less likely to be lost between rounds, which reflects the lower mobility of rural households compared to their urban counterparts. 3. Households with younger heads are more likely to have not been re-interviewed. These may be more mobile households. 4. Households with heads working at the time of the GLSS5 were slightly more likely to be re-interviewed. In conclusion, there is some evidence that the actual sample of households in the FSS underrepresents more mobile households than in the GLSS5. 5. Analysis The aim of the analysis was to reports usage rates across financial products for product- versus institution-based questions and household usage rates provided via full enumera- tion versus an informant. A regression was then employed to test whether certain types of individuals and households are responsible for the under-reporting of access that was found for some questionnaire formats that have been used. 5.1. Methods Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) 5.1.1. Construction of Indicators Seven different indicators of the use of financial services for product-based questions were calculated using subsets of questions. For example, a respondent was considered banked if responded yes to any of the following questions: 1. Do you currently have an ATM card? 2. Do you currently have a debit card? 3. Do you currently have a Savings Plus account? 4. Do you currently have a current account (checking)? 5. Do you currently have a savings account at a bank? 6. Do you currently have a PostBank account or a post office savings account? 7. Do you currently have a bank loan? 8. Do you currently have a bank overdraft facility? The other six indicators that were constructed measured for the following financial products: • Indirect access to a bank account (using someone else’s account) • Formal non-bank savings • Formal credit • Informal savings 142 Review of Existing Financial Capability and Literacy Measurement Instruments • Informal credit • Insurance Similarly, for the institution-based questions addressed to a household informant (either the head of household or a randomly selected adult that is not the head), the seven indica- tors and the survey questions from which they derive are as follows: 1. Banked: Some people like to keep their money in an account with a bank. Do you or any member of your household have a bank account? 2. Indirect access to an account: Do you or other members of your household per- form banking transactions using someone else’s account? 3. Formal non-bank savings: Now think of all the ways that you and members of your household save money. We are not talking about investing in a business or buying land, but only about where you or other household members put their money to use later. Have you or anyone in your household used an institution such as a credit union or a savings association to save money in the past 12 months? 4. Formal credit: Many people borrow money to buy things on credit. Have you or any other member of your household used an institution such as a credit union, savings association, or bank to borrow money or to buy on credit in the past 12 months? 5. Informal savings: Have you or any other household member used a Susu, wel- fare scheme, or other savings club to save money in the past 12 months? 6. Informal credit: Have you or any member of your household used a Susu, welfare scheme, or savings club to borrow money in the past 12 months? 7. Insurance: Many people insure themselves and their possessions against un- expected circumstances. Have you or any member of your household used an institution to insure yourselves (life, health) or property (household goods, house, vehicle, and the like) in the past 12 months? That is, do you or anyone in the household have any long- or short-term insurance policies with any institution? Finally, for the full enumeration treatments, the same questions were asked but with re- spect to individual use. For example, for the banked indicator, the question would be: “Do you have a bank account?� Reponses are aggregated across all members of the house- hold to arrive at a collective measure of household usage. In other words, if one member of the household reports having a bank account, then the whole household is considered banked for the full enumeration treatments. 5.1.2. Indicators Results For five of the seven indicators (banked, indirect access, formal non-bank savings, informal savings, and informal credit) household usage rates are almost identical when the head of household is the informant or when a full enumeration is undertaken. For formal credit, Review of Existing Financial Capability and Literacy Measurement Instruments 143 usage rates reported by the head of household are slightly higher than those from the full enumeration treatments. Generally speaking, the head of household reports information that is similar to that generated by full enumeration. 5.1.3. Regressions Myriad regression sets were conducted for different purposes: (1) test whether the differ- ences across treatment groups hold up when other factors that could affect usage are con- trolled for; (2) identify the characteristics of the individuals and households that reported lower levels of usage for institution-based versus product-based questions; (3) examine the household characteristics of the randomly selected informants who reported lower household usage rates than those obtained from the head of household or the full enu- meration of individual usage; (4) describe individual use of financial services and compare product- versus institution-based questions; and (5) three sets of regressions were run to make sure that supply-side effects (the presence of providers of financial services) are not driving the differences in reported usage across treatments that we reported above, we run three additional sets of regressions. To describe household usage of financial services, the analysis used a probit model that included such specifications as age, rural/urban status, household size, age of household head, highest grade of household head, household share of agricultural and other forms of employment, and whether the respondent is the head of household or a randomly selected informant. 5.1.4. Regression results Though the significance levels are reduced when additional factors that affect usage are controlled for, the same qualitative patterns emerge: the head of household and a full enumeration produce similar household usage rates, but a randomly selected (non-head) informant produces lower usage rates for services from formal providers. To get a better understanding of whether particular household characteristics are driving the relatively low usage rates reported by random informants, control variables were inter- acted with the treatment variables. 5.1.5. Regression results For the indirect access indicator, there is a marginally significant difference between ran- dom informants and head of household informants, but there is not a significant difference between random informants and a full enumeration. The determinants of being banked are similar across the three treatments. There is a positive relationship between the house- hold’s share of agricultural workers and being banked for the treatments that used a ran- dom informant, suggesting that family members that work together in an agricultural set- ting are knowledgeable about the use of banking services by other household members. Finally, younger, poorly informed household members appeared to be largely responsible for the relatively low usage of banking services reported by random informants. For the regressions that describe individual use of financial services and compare product- 144 Review of Existing Financial Capability and Literacy Measurement Instruments versus institution-based questions, the following individual characteristics were added to the household characteristics that were already used in the previous regressions: the num- ber of grades completed by the respondent and two dummy variables indicating whether he/she is numerate and employed. 5.1.6. Regression results The results from the full interaction specifications for individual use of financial services do not provide firm conclusion on the types of individuals that benefit most from product- based cues when reporting on their personal use of informal savings, formal credit, and insurance. All respondents appear to benefit from these cues. In the last set of regressions that controlled for supply-side effects, the first set uses the travel time (in minutes) to the nearest bank as a measure of the local availability of financial services. The second set uses dummy variables corresponding to Ghana’s ten regions, while the third one includes dummy variables for each enumeration area from which ob- servations were drawn. 5.1.7. Regressions Results As in the base regressions, household usage is similar for full enumeration and when the head of household is the informant, and reported usage of banking services is significantly lower for the random informant. For individual usage, results are also similar to the base re- gressions. Reported usage is significantly higher for product-based questions for informal savings, formal credit, and insurance. This pattern holds for the limited sample when we include the question on travel time to the nearest bank, and for the broader sample when we include region or enumeration area fixed effects. It seems unlikely therefore that the omission of supply-side variables from our base regressions could be driving our results. 5.2. Key Results In conclusion, the regression analysis shows that rates of household usage are almost identical when the head reports on behalf of the household and when the rate is tabulated from a full enumeration of household use. Randomly selected informants (non-heads of the household) provide a less complete summary of household use of financial services than the other two methods. The findings also show that for credit from formal institutions, infor- mal sources of savings, and insurance, usage rates are higher when questions are asked about specific financial products rather than about the respondent’s dealings with types of financial institutions. More specifically, product-based questions appear to be important to gain a full understanding of usage of insurance, formal credit, and informal savings, but do not appear necessary for more basic services such as bank accounts, formal savings, and informal credit. 5.3. Policy Application Comparisons between product- vs. institution-based questionnaires and using them to ad- dress heads of household (vs. full enumeration and randomly selected informants) provide vital indicators on the cost-benefit analyses of rolling out such experimental surveys in other developing countries. In other words, the evidence that the head of household is able Review of Existing Financial Capability and Literacy Measurement Instruments 145 to provide similar data to that obtained from full enumeration for most products has posi- tive implications for the feasibility of expanding data collection on financial service use to other countries. Also, although the FinScope approach focuses on individual usage, the results regarding informants can also indicate the feasibility of including a short module on household use of financial services in those surveys. In this way, the FinScope surveys could be used as a cost-effective vehicle for gathering data on household use of financial services that could be compared with that gathered in other countries via LSMS modules or other non- FinScope stand-alone surveys. Therefore, the results achieved from this study provide a practical guidance on how to generate comparable financial usage data across countries via different versions of survey questionnaires and for different financial products. 5.4. Potential for Replication (over time and in different contexts) The research team has already experimented with the same survey design in Timor-Leste and Jamaica. B.14. National Financial Literacy Survey, MoneySense, Singapore 1. General Introduction In 2003, the Monetary Authority of Singapore (MAS) launched the MoneySENSE national financial education program to coordinate public and private sector initiatives in enhancing financial literacy in Singapore. MoneySENSE’s programs focus on enhancing financial lit- eracy in three areas: basic money management, financial planning, and investment know- how. To inform its programs and understand the level of financial literacy in Singapore, in 2005, MoneySENSE’s Financial Education Steering Committee1 (FESC) commissioned Media Research Consultants to implement the first national financial literacy survey. The survey measured the financial literacy of 2,023 Singaporean citizens and permanent resi- dents between the ages of 18 and 60 years old. 1.1. Purpose of the Study MoneySENSE had as its purpose a number of reasons for implementing the National Fi- nancial Literacy survey, including: 1. To measure how Singaporeans manage their finances, and their knowledge and understanding of common financial products and services 2. To assess Singaporean attitudes to financial matters 3. To identify gaps in financial knowledge and consumer education, and to determine the most effective delivery channels for financial education programs; and 4. To measure Singaporean awareness of MoneySENSE’s national financial educa- tion program 146 Review of Existing Financial Capability and Literacy Measurement Instruments 2. Methodology 2.1. Questionnaire 2.4.1 Original development of questionnaire (source of questions, methods for designing new questions, etc.) To develop the original questionnaire, FESC, in coordination with Media Research Consul- tants Pte. Ltd., organized six focus groups with industry partners from the MoneySENSE Industry Working Group. Based on the discussions of the focus groups, a survey question- naire was developed along with a framework for scoring survey responses. The survey questionnaire sought to measure financial literacy with regard to knowledge, actions, and attitudes across three key areas: 1. Basic money management, including budgeting, spending, saving, borrowing, and managing credit 2. Financial and retirement planning 3. Investment know-how The questionnaire also includes questions designed to measure Singaporean awareness of the MoneySENSE’s financial literacy program. 2.2.1. Final structure of questionnaire (topics, sections, number of questions, special skip patterns) Below is a description of the key concepts covered in the survey questionnaire. In addition to questions on attitudes and actions, the survey instrument included a Knowledge Quiz with five questions on basic money management, seven questions on financial planning, and five questions on investments. Areas Concepts Actions §§ How people manage their money • Monitoring spending • Spending patterns • Lending, borrowing, and saving • Loan repayment §§ What long-term and retirement plans Singaporeans have made • Long-term savings patterns and behavior • Planning for emergencies §§ What steps have Singaporeans taken to manage their investments • Investment planning • Factors considered when investing • Monitoring investments Review of Existing Financial Capability and Literacy Measurement Instruments 147 Areas Concepts Knowledge §§ Knowledge and understanding of using financial products, including loans, insurance, and investment products §§ Knowledge of basic money management, financial planning, investments Attitudes §§ Consumer attitudes toward— • Financial advice • Savings, spending, and paying bills • Levels of emergency savings • Investing • Insurance §§ Reasons for/for not • Planning • Lending • Budget management • Investing 2.2 Sampling 2.2.1 Survey Population The survey measured the financial literacy of 2,023 Singaporean citizens and permanent residents between the ages of 18 and 60 years old. 2.3 Implementation 2.3.1 Countries/regions of application The survey was implemented in Singapore. 2.3.2 Testing and piloting The survey instrument was tested across a wide cross-section of the population, and was reviewed by community bodies, industry associations, grassroots organizations, and aca- demic organizations. 2.3.3 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) The surveys were administered through face-to-face interviews. 3. Analysis 3.1. Method Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) Based on the responses to the action and knowledge questions, a Financial Literacy Score was developed for the Singaporean population across the three areas measured in the sur- vey. Each response was given a weighted score based on how closely it correlated with the preferred outcomes, and then scores for all responses were aggregated and normalized across a 0 to 100 scale. Scores on attitudes, however, were not included in the Financial Lit- eracy Score as FESAC defines financial literacy as “the ability of individuals to make informed judgments and take effective decisions in managing their finances� (Media Research 2005). Given their Financial Literacy Scores, respondents were assigned to one of five financial 148 Review of Existing Financial Capability and Literacy Measurement Instruments literacy quintiles, with the Level 1 quintile holding the 20 % with the lowest average scores, through to the Level 5 quintile, holding the 20 % with the highest average scores. Demograph- ic characteristics were then analyzed for each quintile. The quintiles were divided by the fol- lowing scores: Level 1 scores below 55 %, Level 2 between 56 % and 65 %, Level 3 between 66 % and 71 %, Level 4 between 72 % and 79 %, and Level 5 between 80 % and 100 %. 3.2 Key results The results of the survey indicated an average Financial Literacy Score of 67 % with the highest score achieved for basic money management (74 %) and much lower scores for financial planning (62 %) and investment know-how (58 %). Overall, most Singaporeans save and monitor their spending, and engage in some financial planning, as well as pay- ing careful attention to their investments. Nevertheless, many Singaporeans fail to plan adequately for retirement and emergency needs, and are not knowledgeable of certain financial products. B.15. OECD/INFE Measuring Financial Literacy 1. General introduction to the method In 2008, the OECD International Network on Financial Education established a Financial Literacy Measurement expert sub-group to address the lack of internationally-comparable data on levels of financial literacy. The sub-group commissioned a stock-taking exercise to identify the type and range of financial literacy measurement tools being used around the world. Good practices were identified, and guidance was developed on the most appropri- ate methods to measure financial literacy at the national level.10 The stock-take also informed the INFE of the types of questions that could be included in an internationally representative survey. The OECD secretariat, with the support and guidance of the expert subgroup on financial literacy measurement, developed a survey instrument (questionnaire) based on these recommendations. The survey instrument was subsequently applied in 14 countries. 1.1 Purpose of the survey The survey instrument was created to measure financial literacy levels at an international level, and to create basic indicators of financial inclusion. Specifically, the goals of the de- velopment project were to develop a questionnaire that could help public authorities and other organisations to collect data that will enable: 1. an initial measure of financial literacy to identify national levels of financial literacy, provide a baseline and set benchmarks for national strategies or particular pro- grammes; 2. a description of levels of financial literacy in terms of key socio-demographic groups and explanatory variables that will enable policymakers to identify the needs of the population, the groups with the greatest needs and the gaps in provision; 10 See for instance Kempson, E. (2009), “Framework for the Development of Financial Literacy Baseline Surveys: A First International Comparative Analysis�, OECD Working Papers on Finance, Insurance and Private Pensions, No. 1, OECD Publishing. http://dx.doi.org/10.1787/5kmddpz7m9zq-en Review of Existing Financial Capability and Literacy Measurement Instruments 149 3. the opportunity to conduct repeat measures of financial literacy to identify change over time; and 4. a comparison of levels of financial literacy across countries. 5. The questionnaire is specifically designed to be applicable across people of dif- ferent education and income levels as well as in markets with different levels of financial inclusion. 1.2 Link between purpose and chosen approach The survey design and recommended approach to data collection were informed by exist- ing national surveys and the recommendation of the INFE expert subgroup, INFE mem- bers and expert advisors. They drew on good practice at a national level and a detailed comparison of approaches taken in different countries. 2. Methodology 2.1 Questionnaire 2.1.1. Original development of questionnaire (source of questions, methods for designing new questions, etc.) The questionnaire was developed to capture levels of financial literacy as defined by the OECD INFE: ‘A combination of awareness, knowledge, skill, attitude and behaviour necessary to make sound financial decisions and ultimately achieve individual financial wellbeing.’ Questions were identified that were closely aligned with the following criteria: • in line with good practice for question design; • could be ‘scored’, with replies ranked from most to least capable; • common to a number of surveys; • equally applicable across all sections of the population; • reflect the concept being measured. In some cases, it was possible to identify questions that met some of the criteria but need- ed editing. When this was necessary, a range of procedures were used to ensure that the question met the requirements of the project.11 Questions were also circulated to INFE members, and a range of experts, including policy experts, researchers and statisticians. The individual questions were then combined into a draft survey tool, and again circulated for comment. The draft included potential filters and instructions to the interviewer, in order to provide a full picture of the finished item. 11 In particular, good practice was followed based on Fowler, Floyd J. Jr. (1995) Improving Survey Questions, Ap- plied Social Research Method Series, Vol. 38, Thousand Oaks, London, New Delhi: Sage Publications. 150 Review of Existing Financial Capability and Literacy Measurement Instruments 2.1.2 Revisions to the questionnaire After various edits and refinements, the INFE members were invited to pilot the questionnaire. The pilot data was submitted to the OECD for analysis along with feedback from participating countries, and this information was used to guide additional fine-tuning of the questionnaire. 2.1.3 Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) The questionnaire captures information about each individual’s behaviour, attitudes and knowledge relating to day to day financial management, long term planning and choosing and using products. It also gathers socio-demographic information. The questionnaire begins with simple socio-demographic questions including one that can be used to slightly edit the wording of follow-up questions. Based on the recommenda- tions of the consultant report and good practice elsewhere, the more sensitive socio-demo- graphic questions come at the end of the questionnaire. Similarly, more sensitive financial literacy questions appear after less sensitive topics such as budgeting. After a few questions on keeping track of finances, financial inclusion is the next substan- tive topic covered by the survey. The questions capture data about the awareness, use and recent purchase of financial products. We defined ‘recent’ as ‘in the last 2 years’ as our experts told us that a longer time horizon was impossible for some people to recall, and a shorter time period would capture too few people for analysis. Countries that piloted the survey had the opportunity to modify a suggested list of product types in the questions about products awareness and holding. This ensures that all countries have internationally comparable data whilst also having information of relevance to their specific markets. The questionnaire then addresses short and longer term money management before fin- ishing the substantive financial literacy section with a series of questions testing financial knowledge. The questions contained within the survey instrument are designed to be applicable across the widest possible portion of the population. Filters are kept to a minimum in order to col- lect the same information from each participant. Guidance is also provided to countries that wish to include the financial literacy survey within larger questionnaires: • If the core questions will be added to a larger survey, they should be grouped with other questions that address similar topics. • If additional questions are going to be added to the core questions they should either be placed after the financial literacy questions and before the socio-demographic questions, or grouped with similar topics within the financial literacy questions – this will depend on the topics to be covered. • If the core questions are used without any additional questions, the question order should be retained. • In all cases, it is important to remember to avoid providing any information that might influence responses to the core questions or provide the answer to particular ques- Review of Existing Financial Capability and Literacy Measurement Instruments 151 tions, whether through discussion with the respondent before beginning the survey, or through additional questions The survey instrument includes instructions for the interviewer, and where necessary for the authority commissioning the survey. It also includes a code-frame in order to facilitate the collection and sharing of comparable data. More details are available in the question- naire and guidance notes.12 2.2 Sampling The OECD/INFE instrument is a tool available for countries to use. The guidelines men- tioned above provide indications on the target population, unit of analysis and sample size, but the implementation is conducted by national survey agencies commissioned by national authorities. The annex to the Atkinson and Messy, 201213 reports survey details in each country (i.e. target population, number of observations, interview method, use of supplementary questions, etc.). 2.2.2 Survey population In most cases the countries sampled adults aged 18+. Where individuals younger than this were interviewed they have been deleted from the pilot dataset. Some countries set an up- per age limit but this was never less than 70. 2.2.3 Unit of analysis Individuals were sampled. Countries were requested to ensure that necessary weights were created if the sample frame was of households. 2.2.4 Sampling strategy (sampling frame, method, sample size, etc.) During the piloting process, countries were given guidance on the size of the sample and the target response rate but they were advised to seek local experts/agencies to implement the survey and to draw an appropriate sample. Our recommendation concerning sample size was that each country has a minimum achieved sample size of 1,000 participants per country. 2.3 Implementation 2.3.2 Countries/regions of application Thirteen countries were formally part of the piloting process: Armenia, Czech Republic, Estonia, Germany, Hungary, Ireland, Malaysia, Norway, Peru, Poland, South Africa and the UK originally agreed to pilot the survey in late 2010. Albania piloted the questionnaire in 2011, following the agreed methodology. BVI used the questionnaire independently, pre- pared their own analysis and then submitted the data to the OECD. Other countries have signaled their intention to use the questionnaire in the next year. 12 OECD INFE (2011) Measuring Financial Literacy: Core Questionnaire in Measuring Financial Literacy: Ques- tionnaire and Guidance Notes for conducting an Internationally Comparable Survey of Financial literacy. Paris: OECD. http://www.oecd.org/dataoecd/44/53/49319977.pdf 13 Atkinson, A. and F. Messy (2012), “Measuring Financial Literacy: Results of the OECD / International Network on Financial Education (INFE) Pilot Study�, OECD Working Papers on Finance, Insurance and Private Pensions, No. 15, OECD Publishing. http://dx.doi.org/10.1787/5k9csfs90fr4-en 152 Review of Existing Financial Capability and Literacy Measurement Instruments 2.3.3 Testing and piloting Countries translated and tested the questionnaire for national use. The OECD analysed the extent to which the survey instrument provided a valid tool for data collection at the international level through the Pilot process. 2.3.4 Interview method (face-to-face, telephone, CAPI, etc.) Our recommendation was to undertake personal interviews- that is face-to-face or tele- phone interviews. One country used web-based delivery and tested a sample of questions by CATI. 2.3.5 Response rate The response rate differed by country. Information, where available, is reported it in the annex to Atkinson and Messy (2012). Note that response rates are not collected for quota samples. 2.3.6 Average duration of interview and total cost Countries were not required to share information on duration and cost, which will depend on the number of additional questions incorporated, and other national factors. 2.3.7 Key challenges and lessons learned Based on our experience, the main challenges have been: • Addressing the fact that whilst most topics are relevant across the globe, the product and financial markets vary widely in each country; • Co-ordinating a data collection and reporting period across countries, whilst being sensitive to national concerns; • collecting and managing data from various sources held in various formats and not always coded in a uniform way. • Gathering complete information from the fieldwork companies about the sample, fieldwork etc. The main lessons that we learnt so far have been: • There is considerable appetite for a measure of financial literacy: countries want to measure financial literacy across their population and most are aware of a suitable survey agency to collect the data, but finding the resources and expertise to analyse the resulting data is a challenge for some authorities; • countries do not like long questionnaires, particularly because they almost always want to add some other questions; • it is possible to be contextually flexible and still create an international measure (e.g., our savings indicator are structured in such a way that individuals may be considered to be saving whether they are doing so informally or investing in stocks and shares). Review of Existing Financial Capability and Literacy Measurement Instruments 153 Other more basic (but nevertheless important) lessons: • it is really important to code ‘do not know’ and ‘no response’ separately during data collection; • it is necessary to emphasise the need to check terminology even when the question- naire is to be used in English – for example the product types listed may not exist in all English speaking countries; • it is also important to define the terminology used in the questionnaire (e.g., terms like ‘budget’). 3. Analysis 3.1 Methods used for analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) The approach taken to creating measures of financial literacy reflects the definition given above. In particular we have separate indicators for behaviour, attitudes and knowledge. Measures of financial literacy were constructed by identifying the number of financially liter- ate behaviours, the extent to which individuals had long-term preferences and the number of correct responses they gave to the knowledge questions. These have the advantage of being objectively designated- we know whether a set of questions ares capturing knowl- edge or behaviour, whereas we may need to rely on factor analysis to identify what concept a set of questions are reflecting. We analysed all the national datasets following a common methodology. We reported the frequencies of individual questions by country. The data has been analysed through tabu- lations of the financial literacy score and its components, cross-tabulations of financial literacy with key socio-demographic variables, and multivariate analysis. We also identified ‘high scores’ on each of the three components, and undertook analysis by the number of high scores achieved. 3.2 Key results We refer to the above mentioned results report (Atkinson and Messy, 2012). 3.3 Policy application Governments need to know the overall level of financial literacy within their country, and identify which aspects cause the most difficulties, and which groups face the largest chal- lenges: in particular, this is an important component of the design and implementation of a national strategy on financial education. Governments look to other countries for ideas and examples- as exemplified through the INFE and through the OECD PISA. The internationally applicable questionnaire allows governments to benchmark their country against others and to seek information to under- stand the causes of any differences. 154 Review of Existing Financial Capability and Literacy Measurement Instruments The data gathered through the survey instrument also permits analysis of financial inclu- sion and the association between financial inclusion and financial literacy. As financial edu- cation is seen as an important tool in improving financial inclusion, there is considerable value in being able to analyse these two issues simultaneously. Furthermore, the data has already been invaluable in informing the work of the OECD INFE Expert Subgroup on Empowering Women through Financial Awareness and Educa- tion, and will continue to be used to inform the work of the INFE. 3.4 Potential for replication (over time and in different contexts) The survey was designed explicitly to be replicated across countries and time. Following the pilot process, the questionnaire was edited, and is now freely available online for wide- spread use. B.16. Survey on the Financial Literacy of the Portuguese Population 1. General introduction to the method In 2008, Banco de Portugal began developing the Survey on the Financial Literacy of the Portuguese Population when it assumed the supervision of credit institutions. The survey was designed as part of the Bank’s strategy to promote financial literacy in conjunction with supervising banks. The survey measures financial literacy through a number of di- mensions, including attitudes, behavior, and knowledge of financial matters. Through the survey, the Banco de Portugal sought to identify knowledge gaps and groups with limited financial knowledge in order to define national financial education priorities. 1.1 Purpose of the survey Banco de Portugal employed the Schagen (1997) definition of financial literacy and it sought to measure three dimensions, financial knowledge, behavior, and attitudes. In par- ticular, the survey sought to accomplish the following: • Measure the level of financial inclusion in Portugal by looking at consumer financial access and usage; • Assess consumer knowledge of banking products and their account and money management behavior; • Gather information on how consumers resolve disagreements with financial institu- tions; and • Assess how consumers choose financial products and make other financial deci- sions. 1.2 Link between purpose and chosen approach In preparation for the survey project, Banco de Portugal conducted a review of existing financial literacy surveys, including surveys implemented in Australia, the Netherlands, and the United Kingdom. Based in part on this review, Banco de Portugal delimited the main objectives of the financial literacy survey while also taking into accounts its institutional objective to regulate the Portuguese banking sector. Review of Existing Financial Capability and Literacy Measurement Instruments 155 2. Methodology 2.1 Questionnaire Development Banco de Portugal commissioned a survey research firm, Eurosondagem, to help in the development and implementation of the financial literacy survey. Together, the two institu- tions worked on the questionnaire design. Eurosondagem presented an initial draft of the survey instrument, and then Banco de Portugal reviewed the draft to better adapt it to the project’s main objectives. In order to assess the financial literacy of the Portuguese population in terms of financial behavior, attitudes and knowledge, a questionnaire was structured with 94 multiple choice questions on the following topics: financial inclusion, bank account management, planning expenses and savings, choice of banking products, choice and knowledge of sources of information and financial understanding. 2.1.1 Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) According to original research objectives, the survey was organized into the following groups of questions: 1. Profile of the interviewee (Group A with 11 questions) – questions to define the placement of the interviewee in the intended sample, according to stratification variables relative to gender, age, geographic location, employment situation and schooling level 2. Financial inclusion (Group B with 8 questions) – questions to appraise the use of the banking system: number and frequency of use of demand deposit account(s) and, when applicable, reasons for not having a bank account 3. Bank account management (Group C with 19 questions) – questions to assess the form of utilization and monitoring of movements and balance of the demand account (use of debit cards, use of home-banking, reading of bank account state- ments, use of overdrafts) and the reasons for the choice of the bank 4. Planning expenses and savings (Group D with 8 questions) – questions on habits regarding the planning of expenses and savings 5. Choice of banking products (Group E with 30 questions) – questions on the main banking products held by families: term deposits and other savings products, mortgage loans, consumer credit and credit cards. The questions concerned the process of choice of products, the knowledge of their characteristics and the read- ing of pre-contractual and contractual information; 6. Choice and knowledge of information sources (Group F with 7 questions) – questions to assess the information that the population would like to obtain on banking products and through which means, as well as knowledge on the ways to 156 Review of Existing Financial Capability and Literacy Measurement Instruments obtain assistance in the event of disagreement with credit institutions or in the case of difficulty in meeting the loan installments 7. Financial understanding (Group G with 11 questions) – questions to find out the knowledge of the population on various financial concepts. Not all the ques- tions were posed to all interviewees, because some of the replies of the interview- ee to previous questions constrained the questions that were made subsequently. There were, therefore, different paths for the progression of the questionnaire. 2.2 Sampling 2.2.1 Survey population The survey population was the adult population of Portugal (mainland and autonomous regions), including the following regions: North, Centre, Lisbon and Tejo Valley, Alentejo, Algarve, Autonomous Region of Madeira, Autonomous Region of the Azores. 2.2.2 Unit of analysis The unit of analysis was individuals aged 16 years and older. 2.2.3 Sampling strategy (sampling frame, method, sample size, etc.) Banco de Portugal in collaboration with Eurosondagem selected a sample of 2,000 indi- viduals so as to enable estimating, by statistical inference, the results with an average error of 2.2 percent and probability of 95 percent to the entire population. Of the 2,000 interview- ees, 52 percent were female and 58 percent are part of the active population. Approximately half are aged between 25 and 54 years old, and 13 percent are over the age of 70. Approximately a quarter of the interviews were conducted in the North of the country and 36 percent in the Centre and in Lisbon and Tejo Valley. A little over half of the respon- dents (53 percent) have compulsory schooling (current 9th year) and only 12 percent have a university degree. 2.3 Implementation 2.3.1 Countries/regions of application The survey was implemented across Portugal including in the following areas: North, Cen- tre, Lisbon and Tejo Valley, Alentejo, Algarve, Autonomous Region of Madeira, Autono- mous Region of the Azores. 2.3.2 Testing and piloting The questionnaire was piloted in the two biggest Portuguese cities, Lisbon and Oporto, with 60 interviewees. The interviewees were selected in accordance with the stratification designed for the final survey. Following the pilot test, the wording of a number of questions was modified and some questions were replaced, in order to ensure that the questionnaire was clear and easily understandable. Review of Existing Financial Capability and Literacy Measurement Instruments 157 2.3.3 Interview method (face-to-face, telephone, CAPI, etc.) Interviews were conducted face-to-face. 2.3.4 Response rate No information available. 2.3.5 Average duration of interview and total cost The interviews conducted during the pilot test had an average duration of about 45 min- utes, with a minimum duration of 15 minutes and a maximum duration of 60 minutes. 3. Analysis (1-2 pages) 3.1 Methods used for analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) Global Financial Literacy Index The construction of the Global Financial Literacy Index is based on the selection of ques- tions that are considered “timeless�, which might be asked again in a few years time, in order to repeat its calculation and allow for the assessment of the evolution of the financial literacy of the Portuguese population. For its construction, 57 questions considered rel- evant were selected. This group includes questions that assess financial knowledge, behavior and attitudes, since the definition of financial literacy covers these three components. The answers to the selected questions were classified on the scale {-2, -1, 0, 1, 2} 2 so as to produce a global indicator of financial literacy. However, the distribution of the scores in the questions is not always the same because, due to their importance in the degree of financial literacy and the variability of the admissible answers, not all were capable of containing answers with the maximum score of “+2� and/or minimum of “-2�. For the questions on financial understanding, the maximum score was given to the correct answer and the minimum to all the other answer options. Regarding questions that assess financial behavior and attitudes, the classification of the answers on the scale referred above was based on the behavior/attitude that was considered most appropriate and re- vealed highest capacity of financial assessment. For the construction of this indicator ques- tions such as those relative to the socio-economic characterization of the interviewee, to the possession of a given financial product and those not susceptible of a valuation were not selected. For each interviewee, an index was calculated which results from the arithmetic sum of the scores of their answers, divided by the number of questions posed to the interviewee. Thus, the index of each interviewee takes into consideration only the questions that were answered, enabling a comparison between the indices of interviewees with a different number of answers. 158 Review of Existing Financial Capability and Literacy Measurement Instruments After the calculation of the index of each interviewee, it was found that the lowest value ob- served is -1.18 and the maximum is 1.29. However, because the interviewees did not neces- sarily answer the same questions, the individual indices are not all at the same scale, and so they were transformed to a scale of 0 to 1006. The values expressed in this scale constitute a Global Financial Literacy Index (Global FLI) comparable between interviewees. Partial Financial Literacy Indices In order to complement the analysis of the Global FLI, Partial Indices were calculated, one for each of the 6 groups of thematic areas of the questionnaire: Financial inclusion (Group B); Bank account management (Group C); Planning of expenses and saving (Group D); Choice of banking products (Group E); Knowledge of sources of information (Group F); and Financial understanding (Group G). The calculation methodology of these Partial Indices follows that used for the Global FLI; for each interviewee, the sum of the scores obtained in the questions of this group was di- vided by the number of questions posed to the individual. Then, the Index was transformed to the scale of 0 to 100, according to the minimum and maximum values of each individual. Therefore, 6 Partial Financial Literacy Indices were obtained for each interviewee. 3.2 Key results The results of the Survey on the Financial Literacy of the Portuguese Population show that: • The majority of the respondent population presents a level of financial literacy above the average observed in the answers to the different questions of the questionnaire. • Access to the banking system is widespread and the use of electronic means of pay- ment is very frequent, results that compare favorably with other developed countries. • In spite of the generally positive results concerning financial literacy, certain popula- tion groups stand out as having particular deficiencies, which correspond simulta- neously to low levels of schooling and income. The more elderly population shows results that are greatly below the average, and this is also the age group that records lower schooling levels. Other groups also present relatively low levels of financial literacy, in particular the young and unemployed. • There is evidence of important gaps in the understanding of basic financial concepts and in knowledge of the main sources of information. • The survey reflects a relation of trust in the credit institutions, shown by the ma- jority of bank customers, namely through the importance attributed to the advice received at the bank branch and the preference for receiving information through credit institutions. • Banco de Portugal is the second most important source used to access information on banking products, in spite of the Bank Customer Website still being unknown to almost all the respondents. Banco de Portugal is also recognized as the first entity to resort to in the case of a dispute with a credit institution. Review of Existing Financial Capability and Literacy Measurement Instruments 159 Population Groups with Higher Levels of Financial Literacy • Associated to individuals with an undergraduate or graduate degree, and those who are in the 24 to 59 year old age group. • Individuals with higher income levels, which is in accordance with the existing posi- tive correlation between schooling level and income level. • Respondents with employment • Individuals who hold banking products (bank account, term deposit, mortgage loan, consumer credit and credit card) in general present better financial literacy results. • Users of home-banking services also show higher indicators of financial literacy. Population Groups with Lower Levels of Financial Literacy • The more elderly population (aged over 70) and those without primary education. • The population with lower schooling and income levels in which women are predominant • The younger population and the unemployed • Individuals who do not have a bank account – and who are, therefore, excluded from the banking system – also present low levels of financial literacy. • People who show a negative attitude in relation to the importance of planning the household budget. 3.3 Potential for replication (over time and in different contexts) Banco de Portugal plans to implement a follow-up survey in planned for 2015. The ob- jective is to evaluate the impact of the National Plan for Financial Education on financial literacy levels in Portugal. References Banco de Portugal (2011), Survey on the Financial Literacy of the Portuguese Population (2010), Final Report. Available at http://clientebancario.bportugal.pt/pt-PT/Publicacoes/InqueritoLiteraciaFinanceira/Documents/ Survey%20on%20the%20Financial%20Literacy%20of%20the%20Portuguese%20Population%20(2010)%20 -%20Final%20Report.pdf Schagen, S. (1997). The evaluation of Natwest Face 2 Face with Finance, National Foundation for Educational Research. B.17. World Bank Russia Financial Literacy and Education Trust Fund Development of the World Bank Financial Capability Survey 1. Objectives The financial capability measurement work managed by the World Bank has the objective to develop new methods for assessing levels of financial capability in a way that is relevant for low and middle income environments and that can be used consistently across coun- tries to conduct international comparisons. The measurement instrument developed by the Trust Fund will provide a diagnostic tool that policy makers can use to identify the key areas of behavior, skills, attitudes and knowledge that are related with financial capability 160 Review of Existing Financial Capability and Literacy Measurement Instruments and need improvement, and to determine which groups of the population are the most or least capable, in order to inform the design of targeted interventions in the area of financial education and financial capability enhancement. In order to define an appropriate conceptual framework and to develop a survey instrument that is suitable for use across different countries and income levels, the World Bank has followed a rigorous process involving both qualitative and quantitative research methods to understand how the concept of financial capability can be measured in these settings. For this purpose, the Trust Fund has involved an international team of experts in financial capability and in questionnaire design, and it has supported research projects in a group of eleven low and middle-income countries (Armenia, Colombia, Lebanon, Malawi, Mexico, Namibia, Papua New Guinea, Tanzania, Turkey, Uruguay, and Zambia) to help develop and test the methodology. The final output of the activities will include a survey questionnaire and related operational guidance for survey implementation, documented data sets from the surveys carried out in the pilot countries, and a detailed report on methodology development supported by analy- sis of the survey data. The questionnaire has already been developed and fieldwork has been completed in the participating countries. One additional country, Nigeria, is expected to join the group of participating countries and to complete data collection by November 2012, and reports and data analysis will be completed in the fall. This note provides a short overview of the methodological approach and process followed by the World Bank for developing the survey questionnaire. 2. Methodological approach 2.1. Designing the questionnaire The key objective is to develop and test a new instrument that is applicable across different income levels and different cultures. This is a particularly challenging task since there is still very limited evidence on what matters for financial capability in developing countries, and the existing surveys carried out in high-income countries may not be entirely relevant, because they rely heavily on questions about the use of financial services or about knowl- edge of financial concepts that are difficult to understand for people with lower educational attainments and limited access to these services. In order to develop a new questionnaire, the conceptual framework for financial capability needs to be tailored to these settings through an extensive preliminary research phase to identify the key aspects to be covered in the survey. This was done by following the same approach adopted in the UK Financial Capability Study done by the Financial Services Authority (FSA, 2005). The key feature of this approach is that, similar to the psychometric literature for measuring intelligence or personality traits (Spearman, 1904), it assumes that financial capability like other types of abilities cannot be measured directly, but it is possible to identify manifestations of this capability through behaviors, attitudes, skills and knowl- edge. There are two main ways to identify these manifestations: one is to let the research- ers determine what kind of behaviors, attitudes, skills and knowledge can be considered as indicative of financial capability according to economic theory, and the other is to determine Review of Existing Financial Capability and Literacy Measurement Instruments 161 this empirically, for example by conducting qualitative research in the environment of inter- est to see how the general public defines a financially capable person. Since little is known about these issues in the context of developing countries, the World Bank has adopted this second, more exploratory approach. Similarly to the FSA study, focus group discussions in the participating countries were used to identify behaviors, attitudes, skills and knowledge that indicate financial capability or incapability. These would form the core topics to be included in the questionnaire. The use of qualitative research methods to identify empirical indicators for instrument develop- ment has been used extensively in psychology and in the marketing research literature (Churchill, 1979), where qualitative methods such as focus groups where originally intro- duced (Calder, 1977). More recently, these techniques have been used in health care re- search (Pett et al., 2003) and indicated in official guidelines for the construction of patient- reported outcome measures (FDA, 2006). The next step after the analysis of the focus groups was the identification of existing questions (where available) or the design of new questions about the key concepts emerged from the focus groups. The expert team consulted a review of 26 existing national and international sur- veys that had been compiled for the OECD under the same Trust Fund, in order to include as many questions as possible that had already been tried and tested. The relatively small num- ber of gaps that were left were filled by the expert team who collectively designed questions to be tested. The main criteria to select and design the questions were that the questions should: • capture a key topic identified in the focus groups; • work across countries and apply to the whole population (be culturally neutral and income neutral); • be unambiguous; • discriminate between more capable and less capable people; • avoid scales based on value judgments; • allow the use of different statistical tools for data analysis, including principal com- ponent analysis. Two rounds of cognitive interviews were then used to test alternative questions for the same topic, alternative wordings for the same question, and to identify appropriate pre- coded replies. The questionnaire developed at the end of this process was then tested in quantitative pilot surveys with about 100-200 individuals per country. This additional phase was used to refine the questionnaire, to test the respondent selection and fieldwork protocols, and to conduct some preliminary analysis. The next section presents the analytical approach for constructing measures of financial capability, and section 8 describes the implementation and key lessons learned for each phase of the questionnaire development process. Section 4 presents the structure and content of the questionnaire, and section 5 reviews the sampling guidelines provided for survey implementation. 162 Review of Existing Financial Capability and Literacy Measurement Instruments 2.2. Developing measures of financial capability Like other types of abilities, skills or personality traits, financial capability is a quite abstract concept that cannot be measured directly, but may be thought of as reflected in a com- bination of behaviors, skills, attitudes, and knowledge. The objective of this project is to construct measures of financial capability that take into account this multi-faceted nature. Quite consistently across countries, the focus groups have indicated a set of behaviors, attitudes and skills that denote a financially capable (or incapable) person. In order to construct a valid measure of financial capability, it is necessary to test with appropriate statistical methods whether all these concepts are associated with the same underlying capability, or if there exist more than one domain of financial capability, that are somehow related but cannot be described by a single index. Results from the UK study showed that there were four key domains of financial capability (managing money, planning for the fu- ture, choosing products and staying informed), and that it was not meaningful to construct a single measure of financial capability. Whether this is also true for the countries involved in the Trust Fund research project is an empirical question that will be addressed in the analytical stage. Principal component analysis will be used to identify the key domains of financial capability and to construct scores of financial capability in each domain. By exploiting the correlation structure of a group of observed variables, this method identifies common underlying (and unobserved) factors that can explain the variation of these variables. Once the analysis is completed, a score can be constructed from the observed variables as a measure of the underlying factor (or domain of financial capability, in this case), by weighting each variable based on the correlation between that variable and the underlying factor. Principal compo- nent analysis is also helpful in discriminating between questions that are highly correlated with the underlying factors, and those that are not, and it can therefore be used to reduce the number of questions required to measure capability in each domain. Once the principal component analysis is completed, it will be possible to analyze the distribution of financial capability scores in the population. Cross tabulations and standard regression techniques will be used to understand how financial capability scores vary with socio-demographic characteristics such as age, gender, education, household composi- tion, employment status, income, etc. Further analysis can then be done to understand how capabilities in each domain are related to each other (for example, if respondents who are better at day-to-day money management tend to be also better in choosing financial products). In particular, a second-order principal component analysis can be done to as- sess whether it is actually possible to construct a single score for overall financial capability. Cluster analysis will then be used to segment the population into groups of individuals who have financial capability scores similar to other individuals in their group and different from individuals in other groups. The method involves calculating a measure of similarity between individuals, based on differences in the scores. Then, individuals that have similar scores on the domains being considered are grouped together. The number of groups that will be formed depends on the distribution of the scores and on the relationship between the domains. For example, if a population is composed of only very incapable people and Review of Existing Financial Capability and Literacy Measurement Instruments 163 very capable people across all aspects of financial capability (i.e. a person is either very good at every aspect of financial capability or very bad at all of them), we will likely find two clusters. Real-case scenarios will generally be more complex. By segmenting the popu- lation, the study will provide useful information for policy makers interested in targeting financial education interventions. 3. Questionnaire Development The questionnaire developed by the Trust Fund is the result of a long and complex devel- opment process, in order to ensure its applicability in a diversity of settings. The guiding principle for the development of the questionnaire was that it should contain questions that are relevant in the setting of developing countries, that are meaningful and easily under- stood across income and education levels, and that can be used to construct a measure of financial capability that takes into account all the relevant aspects. The qualitative phase of questionnaire development was carried out in eight countries: three in Latin America (Colombia, Mexico and Uruguay), four in Africa (Malawi, Namibia, Tanzania, Zambia), and one in the Pacific Islands (Papua New Guinea). The questionnaire used in the quantitative pilot testing therefore reflects lessons learned from focus groups and cognitive testing of the questions in these countries. For the quantitative survey pilots, three new countries joined the group from the Middle East and Central Europe (Armenia, Lebanon and Turkey). As of October 2012, the full-scale survey data collection is com- pleted in Armenia, Colombia, Lebanon, Mexico, Turkey and Uruguay, while Papua New Guinea has conducted a pilot and a new country, Nigeria, is planning to implement the survey as a module attached to a larger survey. 3.1. Focus groups 3.1.1. Objectives The first step for questionnaire development was the identification of behaviors, attitudes, skills and knowledge that are associated with financial capability by individuals living in low- or middle-income environments. This was done through a qualitative research tool known as focus groups: these are a particular type of interview that is conducted by a facili- tator with a small group of people who are invited to discuss the topic of interest. The par- ticipants were asked to describe a financially capable (or incapable) person with minimal intervention by the facilitator, whose role was to get the conversation started and to make sure that every person actively participated in the discussion, without suggesting specific topics or expressing personal views. 3.1.2. Implementation Between 6 and 13 focus groups were conducted in each of the eight participating countries (for a total of 70 focus groups), with different groups of individuals (selected based on age, gender, income, and type of location – rural or urban). The topic guide for facilitators conducting the focus group discussions was developed by the team of experts and presented at a workshop with the country teams, where the teams 164 Review of Existing Financial Capability and Literacy Measurement Instruments were also provided with some training on the style of facilitation that should be used, the type of people to be invited as participants, etc. The original topic guide proposed at the workshop was slightly modified following discussions at the workshop, because some of the country teams were concerned about using too abstract concepts and felt that the dis- cussions needed some minimal framing. For this reason, the final guide started by asking respondents to describe a financially capable or incapable person to get their spontaneous replies, and it then included more specific probes in the areas of money management, plan- ning for the future, selecting and using financial products, getting information and advice. All the focus groups were recorded and transcribed to facilitate their analysis. For each country, the first focus group transcript was reviewed by the lead expert who noted the main concepts emerged from the discussion onto a thematic grid. This early review helped to detect any problems in the way the focus groups were being conducted and to provide additional guidance to the country teams. The local teams then completed the grids with results from the other focus groups. The focus group transcripts were then allocated to the experts for review against the the- matic grids sent by the country teams, and a master grid was produced by evaluating the results across countries. The master grid listed the key components mentioned by the fo- cus groups and their frequency. Two main areas emerged in describing financially capable and incapable people: day-to-day money management and planning for the future. These results were presented at a second workshop with the country teams where initial sugges- tions for questions around the proposed topics were also presented by the expert team. 3.1.3. Key lessons learned The focus groups have provided surprising results by showing many similarities among the participating countries and even compared to higher-income countries like the UK. The most notable difference from high-income countries was that there was very limited mention of use or knowledge of financial products and their characteristics as an aspect in- dicating financial capability. It also emerged that psychological characteristics such as self- discipline, impulsivity, avoiding peer pressure, or action orientation were frequently used by focus group participants when describing financially capable (or incapable) people. The key concepts emerging from the focus groups, which constituted the core of the question- naire, were focused on the areas of day-to-day money management (budgeting, keeping track of expenses, not overspending, prioritizing expenses, borrowing within affordable limits, etc.) and planning for the future (trying to save, planning for the children’s future, having strategies to cover expenses in old age, etc.). Understanding the context of decision making proved very important. Although the objec- tive of the project is to measure the financial capability of individuals, it is necessary to take into account the role that the individual has within the household (for example, if the person is only responsible for her own money, or is also responsible for managing the household’s money). Review of Existing Financial Capability and Literacy Measurement Instruments 165 3.2. Cognitive interviews 3.2.1. Objectives After identifying the key concepts to be included in the questionnaire based on the results of the focus groups, the expert team selected existing questions on these topics from other surveys and designed new questions where necessary. All the questions needed to be tested to ensure that they met the criteria defined at the beginning of the process (ap- plicable to the whole population, income neutral, culturally neutral, discriminating between capable and incapable, etc.), that they had the same meaning across the countries, and that they could be understood and answered across education levels. This qualitative cognitive testing was done by conducting in-depth interviews, in which probes are used after the questions to assess the respondent’s understanding of the ques- tion wording. This is also a useful tool to make sure that relevant topics that may arise during the interview are adequately covered in the questionnaire, and that appropriate pre- coded answers are provided. Several questions were initially kept open in order to identify the key answers to be included in the pre-codes. 3.2.2. Implementation Two rounds of progressively more structured in-depth interviews were carried out (about 15 per country in each round, for a total of 117 interviews in the first round and 111 in the second round). After each round, the Trust Fund team and the experts reviewed the inter- view transcripts and the feedback provided by the country teams, to identify the questions, wording and pre-codes that worked best. Interviewers were requested to provide written notes or transcripts of each interview, in ad- dition to their feedback on questions that were not understood, that produced inconsistent replies, that were not relevant or did not capture the respondent’s level of financial capabil- ity, etc. This feedback, combined with debrief sessions organized between the Trust Fund, the experts and the country teams, informed the subsequent revisions of the questionnaire and the drafting of its pilot version. 3.2.3. Key lessons learned The in-depth interviews stage was very helpful to identify problems with the structure and wording of some questions, particularly those that were taken from existing surveys car- ried out in developed countries. Respondents with lower education levels had difficulties to understand long questions or questions containing negative forms, to remember some of the pre-codes that were read out after a question, or to use response scales that were not binary (for example, a 5-point agree/disagree scale). To the extent possible, the questions were therefore revised to be short, with a simple construct, and requiring a yes/no type of answer. Since the construction of composite indices through principal component analysis requires answers that are not binary, many questions were split into two questions that will need to be combined in the analysis (for example: Q1: “Do you agree with…?� YES/NO, followed by Q2: “And do you agree/disagree strongly or only to some extent?� STRONGLY/ TO SOME EXTENT). As a result of the in-depth interviews, the number of questions was reduced by selecting the questions that worked best for each concept. 166 Review of Existing Financial Capability and Literacy Measurement Instruments The cognitive testing also stressed the need to customize the wording of some questions to the background of the respondent. The main element affecting how the question should be asked was the role played by the respondent in managing the household’s finances: since the object of study is how people deal with the resources they are responsible for, the wording of the questions needed to be slightly adjusted depending on what the respondent was responsible for. For example, people who are only responsible for their own expenses would be asked whether they themselves had money left over after they had paid for nec- essary items, while respondents managing their household’s resources would be asked whether their household had any money left over. Relatively abstract concepts such as “managing� could not be used as they were not well understood by lower educated respondents: for this reason, in some cases multiple ques- tions had to be used instead of a single question to make sure that every practical aspect related with the abstract concept was explored (in the case of “managing�: planning, mak- ing decisions, being responsible for, etc.). Testing results showed that in some instances asking two questions instead of one took less time because the questions were readily un- derstood by the respondent, without the need for further clarification from the interviewer. 4. Questionnaire structure The survey questionnaire developed by the Trust Fund for the national surveys is composed by three parts: 1) an individual questionnaire to be completed by the respondent randomly selected within the household; 2) a questionnaire to be completed by one knowledgeable person for each enumeration area; 3) a questionnaire completed by each interviewer. The core individual questionnaire is divided into several sections with the following purposes: Section Purpose R: Household Roster §§ Provide information on the type of household the respondent lives in, i.e. to provide data that will allow various typologies based on demographic composition, education levels, activity levels, and decision making. §§ Provide the criteria for determining which household members are eligible to be respondents for the survey, this number of eligible is then used in the Kish table to generate a random selection of the respondent. Note the criteria for being eligible to be a respondent is that the person be 18 years of age or older. A: Role in managing §§ Collect information directly from the respondent about his or her money role in the household and own personal spending. Depending on the answers to these questions, the respondent will then be asked a different set of follow up questions. §§ Identify individuals who really play no role in financial or spending decisions: these people will not be interviewed as we have no means to measure their financial capability. B: Day-to-day money §§ Understand how people manage their day-to-day money. This management includes planning spending, spending on food and other necessary items, keeping track of spending, borrowing, general money management. Review of Existing Financial Capability and Literacy Measurement Instruments 167 Section Purpose C: Planning §§ Understand whether people plan for future expenditures, including for: known expenditures, unexpected expenditures or emergencies, old age, for their children. D: Financial products §§ Understand how people choose financial products, whether they check the features, terms and conditions before buying financial products, whether they look for information before buying products and whether they seek advice or information before making financial decisions. §§ Obtain a broad indication of the level of financial inclusion of the individual, by asking which financial products the respondent holds. E: Motivations §§ Capture underlying motivations that influence the way people behave. This includes questions on time preferences, impulsivity, action orientation. F: Sources of income §§ Obtain information on the variations in income that the individual respondent and his or her household faces. To do this, information is collected for all sources of income and then on how total income varies throughout the year. G: General questions §§ Understand whether the respondent seeks information or advice before making important financial decisions. §§ Understand if the respondent would like to have more information about general aspects of money management discussed during the interview, and if so what type of information he/she would like to have. NOTE: Sections B, C and D have two different versions: one for people who are being asked about the money they manage both personally and for the household (these are on WHITE pages), and one for people who are being asked only about the management of their own personal money (these are on GREEN pages). The questionnaire completed at the enumeration area level will provide additional explana- tory variables about living standards and available infrastructures in the area. The interviewer questionnaire collects information about basic socio-demographic character- istics of the interviewers and their professional experience. These data, appropriately matched to the main dataset, will be useful in analyzing non-response or other data quality issues. Optional sections that interested countries may add include: financial literacy, banking, financial inclusion, credit cards, remittances, financial intermediaries, consumer protec- tion. All these sections were taken from existing surveys. The questionnaire developed by the Trust Fund is designed to be used in face-to-face paper-and-pencil interviews. When possible, however, computer-assisted personal inter- views are very helpful to improve data quality because quality checks can be embedded in the CAPI software and prompt the interviewer when there are inconsistencies in the responses to different questions, or to ensure that answers are in the appropriate range, etc. Due to the cost of the hardware and in some cases to safety considerations, this type of interview was conducted in only two of the participating countries (Mexico and Uruguay). 168 Review of Existing Financial Capability and Literacy Measurement Instruments 5. Survey implementation As of October 2012, the survey fieldwork is completed in all the participating countries (ex- cept for Nigeria). The research teams of the participating countries have developed their own sampling strategies, with the following broad recommendations provided by the Trust Fund: 1. The survey is a survey of individuals. 2. The sample should be representative of the national adult population (aged 18+). To be eligible for the interview, the respondent must participate in the household’s financial or spending decisions and/or be at least partly responsible for own spend- ing. 3. Probability sampling must be used (the most commonly used method was strati- fied random sampling) and related standard techniques should be used to deter- mine sample size. 4. Both urban areas and rural areas should be domains of study. 5. It is generally assumed that the sampling frame is a list of dwellings (a sampling frame of individuals was not available in any of the participating countries). 6. The respondent must be randomly selected within the household that is contacted (typically using a Kish table). The initial questions of the survey are designed to de- termine whether the respondent has any role in managing money and is therefore eligible for the interview. 7. The definition of household is country-specific, in line with use by national statisti- cal offices. References Calder, B.J. (1977), “Focus Groups and the Nature of Qualitative Marketing Research�, Journal of Marketing Research, 14, 353–364. Churchill, G.A. (1979), “A Paradigm for Developing Better Measures of Marketing Constructs�, Journal of Market- ing Research, 16, 64–73. FDA (2006), Guidance for Industry: Patient-Reported Outcome Measures. FSA (2005), “Measuring financial capability: An exploratory study�, UK Financial Services Authority, Consumer Research 37. Pett, M., Lackey, N.R., and Sullivan, J.J. (2003), Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Sage Publications. Spearman, C. (1904), “‘General intelligence’ objectively determined and measured�, American Journal of Psy- chology, 16, 201–293. Review of Existing Financial Capability and Literacy Measurement Instruments 169 B.18. Special Eurobarometer 230: Public Opinion in Europe on Finan- cial Services 1. General Introduction In 2005, the Directorate-General Health and Consumer Protection of the European Com- mission commissioned a Eurobarometer module (Special Eurobarometer 230) to measure consumer ownership and usage of financial services and products. The survey followed upon earlier Eurobarometer surveys in 2003 and 2004 that measured consumer financial access and usage, financial priorities and attitudes, and experiences with cross-border financial services. TNS Opinion & Social implemented the special module as part of the 2005 Standard Euro- barometer wave 63.2. The Standard Eurobarometer is a longitudinal study that has been implemented since 1973 to measure and compare attitudes and experiences on political, social, and economic trends within the European Union. The Standard Eurobarometer is conducted annually in two waves, in the spring and in the autumn. The Special Eurobarom- eter 230 module was implemented in the 25 member states of the European Union, and interviews were conducted with consumers aged 15 years old and over. 1.1. Purpose of the Study The objective of the Special Eurobarometer 230 was to measure European ownership and use of financial products and services and to assess Europeans’ views of finances and their financial priorities. In addition, the survey sought to gather information on consumer experiences with cross border financial services, their perceptions of financial institutions, and their preferences for payment methods. 2. Methodology 2.1. Questionnaire The module consisted of 40 questions: 17 questions on consumer demographics and 23 questions that measured the following: • Financial priorities • Attitudes toward their finances and financial services • Ownership of certain types of financial services • Experience with cross-border financial services • Methods of payments used/preferred in home country and another member Euro- pean Union country • Usage of financial services and products • Bank and credit card usage • Feelings about owning/using financial services and products • How financial decisions are made • Attitudes toward resolving disputes with financial institutions 170 Review of Existing Financial Capability and Literacy Measurement Instruments 2.2. Sampling The survey populations were citizens of the 25 European Union member states in 2005 ages 15 and over. The survey consisted of interviews with about 1,000 people per country, except in Germany (1,500), Luxembourg (600), and United Kingdom (1,300). TNS Opinion & Social utilized a multistage, random probability sample design, with probability propor- tional to size and population. The Standard Eurobarometer surveys are designed to be nationally representative, including metropolitan, urban, and rural areas. Households were selected using random route procedures, and within each household, respondents were selected at random according to the “closest birthday rule.� 2.3 Implementation 2.3.1 Countries/regions of application The Special Eurobarometer 230 module was implemented in the 25 EU countries: Austria, Belgium, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hun- gary, Ireland, Italy, Latvia, Lithuanian Luxemburg, Malta, Netherlands, Poland, Portugal, Slovakia, Slovenia, Spain, Sweden, Republic of Cyprus, and United Kingdom. 2.3.2 Interview method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) Interviews were conducted face-to-face in the national language of the country. Where available, Computer Assisted Personal Interview (CAPI) was employed. 3. Analysis 3.1. Method Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) For each of the topics measured in the module, the data was tabulated and analyzed for the European Union as a whole, by individual countries, and by sociodemographic variables. 3.2. Key Results The module found that on average 72 % of Europeans owned banks or credit cards, however while in the Netherlands, Sweden, and Finland about 90 % of citizens own bank or credit cards, in Cyprus, Greece, and Poland the majority did not own bank or credit cards. In terms of payment, Europeans tends to make payments in cash for goods or services purchased in their national territory although there is a rising trend in those favoring electronic payments. The results for measuring how financial decisions are made reveal that 90 % of Europeans rely on themselves to make financial decisions but 72 % expect financial institutions to pro- vide them with financial advice. However, the level of trust for financial service providers is low (46 %). A large majority of Europeans also believe it is difficult to win a dispute with a financial service institution. References European Commission (2012), Eurobarometer Surveys: Methodology-Instrument Description, http://ec.europa. eu/public_opinion/description_en.htm (Accessed on September 20, 2012) Review of Existing Financial Capability and Literacy Measurement Instruments 171 B.19. U.S. Survey of Consumers, November/December 2001 1. General Introduction to the Method The U.S. Survey of Consumers is a monthly survey implemented by the University of Michigan’s Survey Research Center since 1946. The survey measures monthly changes in consumer confidence across three broad areas: personal finances, business conditions, and future buying plans. For the November and December 2001 Survey of Consumers, the Federal Reserve Board developed a module to attach to the questionnaire to measure financial knowledge and financial behavior. With its module, the Board sought to test the implicit assumption that financial education changes financial behavior by determining the direct link between financial knowledge and financial behavior. The results of the Federal Reserve Board’s module were used to inform discussion papers for community educators and policy makers on how to effectively target financial education programs (Hogarth, Hil- gert, and Beverly 2003). 1.1. Purpose of Survey The Federal Reserve Board’s November and December 2001 Survey of Consumers ques- tionnaire had as its objective three ends: 1. To determine the connection between financial knowledge and financial behavior in order to test the effects of financial education; 2. To identify the most effective methods for providing financial education; and 3. To inform guidance to policy makers, educators, and community groups on how to effectively target financial education programs. 1.2. Link Between Purpose and Chosen Approach The rationale for the Board’s module was based on increasing efforts in the United States over the last few decades to provide financial education to citizens. Implicit in the increased focus on financial education in the United States is the assumption that financial education can improve citizen financial knowledge and thus motivate and encourage citizens to prac- tice sound financial behaviors. The Federal Reserve Board, therefore, sought to directly determine and test the link between financial knowledge and financial behavior using a multivariate framework (Hogarth and Hilgert 2003). To this end, the Board crafted a number of questions regarding household financial knowl- edge, experience, behaviors, learning experiences, and learning preferences to include in the monthly Survey of Consumers. The Board’s module was developed based on its qualitative analysis of previous studies and on past surveys like the Consumer Federation of America-American Express Surveys, the American Savings Education Council youth survey, the Jump$tart Coalition’s biannual survey of high school seniors, and Money 2000 surveys. The module questions were designed in large part to match knowledge of financial man- agement to specific financial behaviors, and to determine how households gathered fi- 172 Review of Existing Financial Capability and Literacy Measurement Instruments nancial information. The November and December 2001 surveys included 28 true or false questions that measured financial knowledge, 18 questions on financial management be- havior, and questions related to the use of 13 different financial products. To obtain a sample that could generate reliable results and represent the U.S. population, the Survey Research Center collected data from 1,004 adult respondents from the contigu- ous United States. The Board used a multivariate framework to analyze the survey data from the survey. In their analysis, the team combined the 18 different financial management practices and 13 different financial products to form four measures of financial behavior: cash flow management, saving, investment and credit management. A regression analysis was used to find correlations between these behaviors and levels of financial knowledge. 2. Methodology The Federal Reserve Board’s methodology for developing its questionnaire for the Novem- ber and December 2001 Survey of Consumers consisted of a review of previous studies on the links between financial knowledge and financial behavior, and a review of other surveys dealing with these issues. The questionnaire included a financial knowledge quiz, ques- tions on specific financial practices, inquiries into consumer experiences with a number of financial products, and questions about how consumers obtain or prefer to obtain financial knowledge. Experts in financial education reviewed the draft questionnaire and the Uni- versity of Michigan’s Survey Research Center (SRC) conducted two pretests of the survey questions in November and one pretest in December. 2.1. Questionnaire 2.1.1. Original development of questionnaire (source of questions, methods for designing new questions etc.) The questionnaire for the U.S. Surveys of Consumers needed to cover detailed informa- tion about consumer financial knowledge, behavior, and actual and preferred sources of financial information. To develop the questionnaire, staff members of the Federal Reserve Board worked with colleagues in the U.S. Department of Agriculture’s Cooperative State Research, Education, and Extension services to craft additional questions for the Survey of Consumers. The additional questions included inquiries into financial behaviors such as managing bills and planning, saving patterns and behavior, credit management, and investment, among others. Questions were drawn from independent research but also from experiences with other surveys on financial education and financial behavior. Past surveys that served as models included the Consumer Federation of America-American Express Surveys, the American Savings Education Council youth survey, the Jump$tart Coalition’s biannual sur- vey of high school seniors, and Money 2000 surveys. Since the Survey of Consumers is administered by telephone, the Federal Reserve Board used a true-false-uncertain format to develop the questions related to financial knowledge or the knowledge quiz instead of a multiple-choice format. Questions on financial knowl- edge covered the topics of cash flow management, general credit management, saving, Review of Existing Financial Capability and Literacy Measurement Instruments 173 investment, mortgages, and wide-ranging financial management topics. The questions on financial behavior included a yes-no format and the questions on effective and preferred financial knowledge sources used a multiple-choice format. 2.2. Revisions to the Questionnaire Once the questions were drafted, they were shared with a set of researchers who had worked in the area of financial education. The researchers helped review the questions and provided additional guidance. Further revisions were made to the questionnaire in consultation with the staff at the Survey Research Center. 2.3. Final Structure of the Questionnaire (topics, sections, number of questions, special skip patterns) The questions developed by the Federal Reserve Board were divided into five parts and included in the core questionnaire for the Survey of Consumers. The five parts included: 28 questions on house hold financial knowledge or a knowledge quiz, an assessment of experiences with 13 financial products, an assessment of 18 financial behaviors, questions on ways respondents learned about financial management and, questions on how respon- dents would prefer to learn to about financial management. The knowledge quiz included true or false questions that covered cash flow management, general credit management, saving, investment, mortgages, and a broad category of other financial-management top- ics. The financial management behavior questions ranged from very basic money manage- ment skills to more sophisticated ones: • Pay all bills on time • Have a financial record-keeping system or track expenses • Pay credit card balances in full each month • Have savings account • Have an investment account • Save or invest money out of paycheck • Save for long-term goals • Have money spread over different types of investments In addition, the questionnaire asked about the use of financial products such as credit cards, mortgages, home equity loans, and investments: • Have a credit card • Compare offers before applying for deposits • Have mutual funds • Have bonds • Own a home • Bought a home • Refinanced mortgage or loan for home improvements 174 Review of Existing Financial Capability and Literacy Measurement Instruments • Put money into retirement plans such as an IRA • Have 401(k) plan or company pension plan The past two sections of the questionnaire on financial literacy and behavior dealt with sources and preferred sources of knowledge on financial management. The questionnaire included three multiple choice questions on where households obtained their financial knowledge, including questions on the most important way to learn about personal financ- es and the most effective way to learn to manage money. Finally, the questionnaire asked six multiple-choice questions about how households would prefer to learn about financial management. 3. Sampling 3.1. Survey Population Data was collected from households in 48 continental U.S. states and the District of Colum- bia. The sample was statistically designed to be representative of all American households excluding those in Alaska and Hawaii. Respondents were all adults 18 years or older. One question, however, about holding a 401(k) plan or company pension plan was asked only of respondents between the ages of 18 and 65 years old. 3.2. Unit of Analysis The unit of analysis for the Survey of Consumers was the household. Within each house- hold SRC uses probability methods to randomly selected one individual over the age of 18 to be interviewed using a respondent selection table. The SRC defines a household as a group of people living together who are related by marriage, blood, or adoption, or any individual living alone or with a person or people to whom the individual is not related. 3.3. Sampling Strategy (sampling frame, method, sample size etc.) Since the Survey of Consumers is an ongoing nationally representative monthly survey, to measure the study of change, the SRC uses a rotating panel sample design. In a panel design, a sample of persons (a panel) is interviewed over several survey rounds for a specified period of time. New panels are created at designated time periods, and stag- gered into the sample. For the November 2001 sample, an independent cross-section sample of households was drawn and 60 % of this new sample was interviewed with 40 % of the sample from a previ- ous month. Likewise for December, the sample included 60 % new respondents, and 40 % of respondents being interviewed for a second time. The total sample size for the Novem- ber and December 2011 Survey of Consumers was 1,004 respondents. The monthly Survey of Consumers samples are stratified, one-stage, equal probability samples of telephone households in the contiguous United States. SRC used Random Digit Dialing (RDD) telephone sampling to determine the sample frame for the November and December 2001 Survey of Consumers. The RDD procedure is a one-stage digit list assisted design in which selection probability is assigned based on the number of possible Review of Existing Financial Capability and Literacy Measurement Instruments 175 phone number that can be formed from the first 8 digits of phone numbers for a defined group of Area Codes by adding the set of numbers “00� to “99� to create 10-digit phone numbers. The RDD design provides for an equal probability sample of all telephone house- holds. For the Survey of Consumers, the sample was stratified by urbanity to and to reflect the four U.S. main regions: Northeast, the North Central, South, and West. 4. Implementation 4.1. Countries/Regions of Application The Survey of Consumers was implemented in the Northeast, the North Central, the South, and the West. In total, SRC interviewed respondents in 48 states and the District of Co- lumbia. The states of Hawaii and Alaska were excluded from the sample and as such, no respondents were interviewed. 4.2. Testing and Piloting The SRC tested the November and December 2001 Survey of Consumers that included the Federal Reserve Board questionnaires under “final state� conditions. The pretest re- spondents were selected to fit all the eligibility criteria of the actual study. All interviewers were trained in conducting telephone-interviews, and studied the questionnaire instruc- tions before beginning the practice interviews. SRC used experienced interviewers and supervisory personnel to monitor the pretest ques- tionnaire. Since the Federal Reserve Board added new questions on financial knowledge and behavior to the November 2001 survey, SRC pretested the questionnaire twice. Qual- ity control teams reviewed the practice interviews and provided feedback to the Board.14 4.3. Interview Method (face-to-face, telephone, CAPI-computer assisted personal interviewing, etc.) The SRC conducted interviews for the November and December 2001 surveys by tele- phone. To reach all the sample population, interviews were conducted during the daytime, in the evening, and on the weekends. The SRC conducted interviews only with the desig- nated respondent; no substitutions were taken even when the respondent was not avail- able during the survey period. For quality control, the SRC conducts telephone validation checks on a small subset of all the completed interviews to guard against potential falsification of interviews. Interview- ing stations were acoustically isolated and supervisors monitored how interviewers asked questions to respondents. Interviews were reviewed as they were completed. 5. Analysis 5.1. Method Used for Analysis (methods used to construct measures of literacy/capability/inclusion etc., and for further analysis) 14 Richard T. Curtin, “Surveys of Consumers,� University of Michigan, Survey Research Center (http://www.sca. isr.umich.edu/documents.php?c=i). 176 Review of Existing Financial Capability and Literacy Measurement Instruments In analyzing the results of the November and December 2001 Survey of Consumers, the Federal Reserve Board used frequency tables to present the results on financial knowl- edge and created an index score to measure different levels of financial behavior. The Board then used multivariate analysis to determine correlations between specific financial knowledge questions and reported financial behavior. To measure patterns of financial behavior, the Federal Reserve Board combined the 18 different financial management questions and 13 questions about financial products to cre- ate four measures of financial behavior: cash flow management, saving, investment, and credit management. To characterize the extent of a household’s participation in each type of financial management behavior, the Board created an index in which levels of cash-flow management, saving, investment, and credit management were scored/classified as “low,� “medium,� or “high.� Between five and 11 practices were identified under each of the four measures of financial behavior. If households reported fewer than 25 % of the practices in a given area, they were classified as low, between 25 % and 75 %, they were classified as medium, and more than 70 % they were classified as high. For each measure of financial management behavior, the Federal Reserve Board deter- mined if there was a key component essential to the measure and if a respondent didn’t practice this behavior, they were automatically categorized in the “low� group. For example, the Board determined that “paying bills on time� was an essential component of cash-flow management, therefore, respondents who did not practice this behavior were automatically placed in the low index group for cash-flow management. The Board also controlled for conditional variables; therefore, those who did not interact with a specific financial product were not expected to practice behaviors associated with the product. For instance, house- holds who did not own credit cards were not expected to pay credit card balances in full each month (Hogarth and Hilgert 2003). The Federal Reserve Board also conducted a descriptive analysis within each financial behavior index to determine the demographic and economic characteristics of the house- holds with high, medium, and low scores for each financial behavior index. Each of the indexes were modeled using a multivariate framework to control for socioeconomic and demographic characteristics within the multivariate analysis, the researchers included age, marital status, gender, ethnicity and income. After comparing and contrasting the corre- lates of each of the four financial behavior indices, the Board combined the information from the four indices to analyze the patterns of financial behavior. To determine levels of financial knowledge, the Federal Reserve Board tabulated the re- sults of the knowledge quiz for each household. In addition, the Board used a frequency table to compare the level of financial knowledge of households within and across each financial behavior index. To determine the relationship between financial knowledge and financial behavior while holding all other variables constant, the Board performed logistic regression analysis. The results of the regression analysis were then used to predict a household’s propensity to score in the low, medium, or high groups on the cash-flow man- agement, saving, investment, and credit management indexes. Review of Existing Financial Capability and Literacy Measurement Instruments 177 5.2. Key Results In its analysis, the Federal Reserve Board found a statistically significant correlation be- tween certain types of financial knowledge and specific financial practices. • For cash flow management, the data analysis showed that using the index methodol- ogy, 12 % of the sample was categorized as low on the cash flow index, 22 % as me- dium, and 66 % as high. In determining the correlation between financial knowledge and financial behavior, the Federal Reserve Board found that households classified as low on the cash flow management index had lower financial knowledge regarding cash flow management (55 %) than did households classified as medium (66 %) or high (69 %). • For credit management, the Federal Reserve Board found that households with low credit management scores had low financial knowledge scores related to credit man- agement of 47 %, compared to 61 % and 66 % for households in the medium and high indexes respectively. Households with higher knowledge of credit management had a higher chance of being in the high index group for credit management. • In analyzing saving behavior, the data showed that 10 % of households scored low, 35 % scored medium, and 54 % scored high. Households in the low savings index scored 67 % on the knowledge quiz, on the medium index scored 77 %, and the high index scored 86 %. Households with a higher saving score had a higher propensity to be in high saving score index. • Finally, for investment, the survey 19 % of households scored high on investment practices, 44 % scored medium, and 37 % scored low on investment behavior. The correlation between financial knowledge scores and financial behavior indices was significant. Financial knowledge scores regarding investment for those in the low index was 50 %, 67 % for those in the medium index, and 80 % for those in the high index. Households with high index scores in the area of cash flow management, credit manage- ment, saving, and investment, reported learning a lot about financial management from friends and family compared to households in the low and medium index groups for finan- cial behavior. Moreover, households in the high financial behavior index group were more likely to prefer learning about financial practices from the Internet than those in the medium and low index groups. Overall, however, households preferred obtaining financial knowl- edge through the media, including television, radio, magazines, and newspapers. 5.3. Policy Application The intended results of the Federal Reserve Board’s questionnaire for the Survey of Con- sumers were to provide a clear picture of the links between financial knowledge and finan- cial education, in an effort to help educators, policy makers, businesses, and community development educators effectively target their financial education programs. The survey data can be used to inform the design of new financial education programs or better im- prove existing financial education initiatives. 178 Review of Existing Financial Capability and Literacy Measurement Instruments 5.4. Potential for Replication (over time and in different contexts) For organizations and countries interested in a cost-effective way to collect data on specific financial literacy and capability questions the Federal Reserve Board’s Survey of Consum- ers offers a replicable example. Rather than commissioning a stand-alone survey, the use of an existing survey can help produce useful data on financial literacy and capability. In addition, some of the questions regarding financial knowledge and behavior can be used over time and in different contexts, depending on how a country or organization defines financial knowledge and financial behavior. Importantly, organizations must note that many of the questions used in this survey are more likely appropriate for respondents in devel- oped countries with well-established financial institutions. Questions on credit card use, the possession of checking accounts, and retirement saving plans might not be appropriate for a developing country context. Nevertheless, categorizing specific financial behaviors to create indexes and score levels can be replicated. Comparing score levels of levels of financial knowledge can also help analyze data to provide insight into links between financial knowledge and financial behav- ior. B.20. U.S. Health and Retirement Study 2004 Annamaria Lusardi The George Washington University School of Business 1. General Introduction to the Method 1.1 Purpose of the Methodology It is increasingly important to assess individuals’ level of financial literacy. However, explor- ing how individuals process economic information and how they make informed financial decisions remains a complex task. The purpose of our methodology was to outline a set of questions which could overcome subjectivity and provide reliable measures of respondent understanding of key financial literacy concepts. The questions were originally designed for, and included in, an experimental financial literacy module that was part of the 2004 U.S. Health and Retirement Study (HRS). They were subsequently incorporated into several other surveys both in the United States and in other countries, with the survey results documenting that financial illiteracy is widespread in both developed and developing countries. 1.2 Link Between Purpose and Chosen Approach To design the financial literacy questions, we relied on economic models of saving and portfolio choice to identify crucial concepts that are indispensable for making everyday financial decisions, namely: interest compounding; inflation; and risk diversification. The financial literacy questions also had to comply with four basic principles: simplicity, rel- evance, brevity, and capacity to differentiate among levels of financial knowledge. In adher- ing to these requirements, our set of financial literacy questions has become a benchmark for financial literacy measurement and is now used by a wide variety of analysts and re- searchers to measure levels of financial literacy across countries. Review of Existing Financial Capability and Literacy Measurement Instruments 179 2. Methodology 2.1 Questionnaire 2.1.1 Original development of questionnaire (source of questions, methods for designing new questions, etc.) In designing the questions we followed four basic principles (Lusardi and Mitchell 2011b): 1. Simplicity. We aimed to measure understanding of basic financial concepts, that is, the abc of finance. 2. Relevance. Questions had to relate to concepts pertinent to peoples’ day-to-day financial decisions over the life cycle; moreover, they had to capture general rather than context-specific ideas. 3. Brevity. Few representative surveys can devote much time to financial literacy topics, so the number of questions had to be kept to a minimum in order to secure widespread adoption. 4. Capacity to differentiate. We needed questions that allow for differentiation be- tween financial knowledge levels, so as to compare respondents by their scores on a common set of questions. The principle of simplicity was extended to the calculation process and answer choice. Ac- cordingly, we did not require respondents to engage in difficult calculations; rather, we test- ed their ability to carry out elementary computations relating to basic financial concepts. We maintained simplicity in answer choice by providing a short list of possible answers (three to four choices). Answer choices for questions requiring calculations were phrased using “more than� or “less than� terminology, so that approximations or rule-of-thumb cal- culations could equally reach the correct answer. For each question, respondents were given the opportunity to indicate that they did not know the answer or that they would pre- fer not to answer. This is a particularly important aspect of the question framing process because it prevents respondents from being forced to pick a numeric answer, creating unnecessary biases. This also allows researchers to better discriminate among different levels of financial knowledge. Ease of calculation and simplicity of answer choice are characteristics which made these questions suitable for surveys using varied types of interviews (for example, telephone, computer-assisted, face-to-face). Below, we list the three questions that were included in the 2004 HRS, together with the given set of possible answers: 1) Suppose you had $100 in a savings account and the interest rate was 2 % per year. After 5 years, how much do you think you would have in the account if you left the money to grow? A. More than $102 B. Exactly $102 180 Review of Existing Financial Capability and Literacy Measurement Instruments C. Less than $102 D. Do not know E. Refuse to answer 2) Imagine that the interest rate on your savings account was 1 % per year and inflation was 2 % per year. After 1 year, how much would you be able to buy with the money in this account? A. More than today B. Exactly the same C. Less than today D. Do not know E. Refuse to answer 3) Please tell me whether this statement is true or false. “Buying a single company’s stock usually provides a safer return than a stock mutual fund.� A. True B. False C. Do not know D. Refuse to answer The first question seeks to assess whether the respondent can carry out simple interest compounding calculations. The main goal of this question is to assess basic knowledge. The second question provides a measure to assess understanding of inflation, again in the context of a simple financial decision. This question is a recurrent, real world situation. The third question tests a respondent’s knowledge of risk diversification; it is a joint test of knowledge about stocks and stock mutual funds, and of risk diversification, since the answer to this question depends on knowing what a stock is and that a mutual fund is com- posed of many stocks. This question is particularly interesting if we take into consideration that employees are increasingly in charge of retirement planning and other crucial financial decisions. With this perspective, it is important to assess whether they possess sound knowledge of risk diversification. 2.1.2 Revisions to the questionnaire As we did not develop an entire survey but just the few specific questions, this section is not applicable to our case. 2.1.3 Final structure of the questionnaire (topics, sections, number of questions, special skip patterns) As we did not develop an entire survey but just the few specific questions, this section is not applicable to our case. Review of Existing Financial Capability and Literacy Measurement Instruments 181 2.2 Sampling A few key issues should be highlighted with reference to sampling techniques: A. Our three questions are not targeted to a specific subgroup; rather, they can be administered to the entire population; B. The questions measure financial literacy at an individual level. Nonetheless, these questions are particularly suitable for use in surveys where the unit of analysis is the individual responsible for financial decision-making within the household; C. The three questions are simple and short enough to be included in both limited- sample surveys and long census-like questionnaires. Note that these questions have been administered in several national surveys through different sampling techniques. The detailed sampling techniques and explanations are de- scribed in each of these survey’s methodological abstracts. 2.3 Implementation 2.3.1 Countries and regions of application The three financial literacy questions were first implemented in the United States in a spe- cial module of the 2004 HRS. The HRS financial literacy module covered more than 1,200 respondents ages 50+ (Lusardi and Mitchell 2011c). These questions were subsequently added to— • Wave 11 of the National Longitudinal Survey of Youth (NLSY) for 2007–08, covering respondents aged 23 to 28 (Lusardi, Mitchell, and Curto 2010). • The American Life Panel in 2008, an Internet-based panel data set (Lusardi and Mitchell 2009). • The U.S. Financial Capability Survey (FINRA 2009).15 The questions have been fielded in Germany, Italy, Japan, the Netherlands, New Zealand, Russia, and Sweden (Lusardi and Mitchell 2011b).16 These questions have also been add- ed to a few surveys in developing countries. Financial literacy questions similar to the ones used in the Financial Capability Study have been used in Mexico and Chile (Hastings and Tejeda-Ashton 2008; Hastings and Mitchell 2011; Behrman et al. 2010), India and Indone- sia (Cole, Sampson, and Zia 2011), and among a sample of entrepreneurs in Sri Lanka (de Mel, McKenzie, and Woodruff 2011). 15 The National Financial Capability Study consists of three linked surveys: (1) National Survey: a nationally pro- jectable telephone survey of 1,488 American adults; (2) State-by-State Survey: an online survey of approximately 28,000 American adults (roughly 500 per state, plus the District of Columbia); (3) Military Survey: an online survey of 800 military service members and spouses. This report is based upon data from the State-by-State survey, ad- ministered to respondents between June and October 2009. Data collection and survey instrument design were supported by the FINRA Investor Education Foundation. More information about this survey is provided at http:// www.finrafoundation.org/resources/research/p120478. 16 A full review of this material is available in the special issue on financial literacy published by the Journal of Pension Economics and Finance, October 2011, Vol.10. 182 Review of Existing Financial Capability and Literacy Measurement Instruments Across the various countries, these questions have done a good job in measuring peoples’ levels of financial knowledge. What is important to note is that, notwithstanding the large differences in these countries, findings regarding financial literacy are pretty similar across the population and among subgroups. 2.3.2 Testing and piloting Since these questions have been administered in several national surveys, the testing and piloting techniques are specific to each of these surveys. Information can be found in each survey’s methodological description. As we did not develop an entire survey but just the few specific questions, other elements of testing and piloting are not applicable to our case. 2.3.3 Interview method Different methods have been used across countries, as described below, and results have been robust across different interview methods: • Telephone interviews (for example, FINRA National Survey; see FINRA 2009) • Computer-assisted personal interviews (for example, FINRA State-by-State Survey) • Paper-and-pencil format (for example, SAVE data in Germany; see Bucher-Koenen and Lusardi 2011) • Face-to-face interviews (for example, ANZ/Retirement Commission Financial Knowl- edge Survey; see Crossan, Feslier, and Hurnard 2011) 2.3.4 Response Rate The response rate to the three questions is generally high in both limited-sample surveys and national surveys. To gather full information on survey response rates, please refer to each survey’s statistical abstract. 2.3.5 Average duration of interview and total cost As we did not develop an entire survey but just the few specific questions, this section is not applicable to our case. 2.3.6 Key challenges and lessons learned The key challenges for the development of reliable financial literacy questions relate to the following three aspects: A. Measurement error issues B. Questions’ adaptability to different contexts C. Questions’ adaptability to different time periods A. Measurement error issues Measurement error is a recurrent issue in every type of financial literacy measure. In par- ticular, our questions may be subject to guessing or question misinterpretation, especially Review of Existing Financial Capability and Literacy Measurement Instruments 183 during phone interviews. To evaluate to what extent measurement error can be a threat to the robustness of the data, in some cases we inverted the question wording, randomly asking two groups of respondents the same question but with inverted wording (Lusardi and Mitchell 2011c). For example, when asking about risk diversification, we asked half the group using format (a) and the other half format (b): (a) Buying a company stock usually provides a safer return than a stock mutual fund. True or false? (b) Buying a stock mutual fund usually provides a safer return than a company stock. True or false? Results show that answers are sensitive to how the question is worded. For example, in the Dutch DNB Household Survey (Van Rooij, Lusardi, and Alessie 2011), only a few re- spondents answer correctly when the question is whether “buying a stock mutual fund usu- ally provides a safer return than a company stock.� However, when the same question is asked in inverted order, correct responses double. This proves that some respondents who answer correctly do not actually understand the questions and do not know what stocks, bonds, and mutual funds are. Empirical analyses should therefore take into account that these measures may be noisy proxies of true financial knowledge levels, especially when the financial literacy concepts under analysis are difficult ones. B. Questions’ adaptability to different contexts The questions were first designed to be adopted in the United States; as a consequence, they assess specific aspects which were deemed to be important in that particular coun- try. Despite the fact that these questions have proven successful in surveys conducted in other countries, automatic adoption is not recommended. Rather, countries’ characteristics should be carefully assessed before incorporating the questions into new surveys. C. Questions’ adaptability to different time periods When new financial literacy measurement tools are being developed, it is important to re- member that a crucial aspect of this type of analysis is providing replicability of the analysis in different contexts and time periods. Hence, the adoption of complicated questions which are time-bound to a given financial system architecture would likely impede performance of a follow-up study. In this respect, we note that simple financial literacy concepts are the ones which are less likely to become obsolete, and therefore more likely to be useful. What we have learned from the experience of different countries is that while these three questions are simple and limited in number, they are able to differentiate among individual levels of financial literacy. Also, research has shown surprisingly similar levels of literacy in different countries, irrespective of the degree of economic development. Similarities apply also to differences in financial literacy among different social groups (for example, age, sex, education). 184 Review of Existing Financial Capability and Literacy Measurement Instruments 3. Analysis 3.1 Methods Used for the Analysis (methods used to construct measures of literacy, capability, inclusion, and further analysis) The data from the three financial literacy questions have been analyzed via multiple meth- odologies. The data is flexible enough to allow for the construction of several different financial literacy indexes. For example, it is possible to— • Construct dummy variables for each correct question: The construction of dummy variables provides a direct way to assess each question’s effect and weight on the outcome variable; • Construct basic and advanced financial literacy measures: By combining the respon- dents’ scores on the different questions, it is possible to differentiate among high and low levels of literacy. • Differentiate between numeracy and knowledge of basic financial concepts: This is made possible by dealing separately with the scores on the calculation-based ques- tions (interest rate and inflation) and the risk diversification question. The scores on the three financial literacy questions are readily interpretable and are suit- able for use both in descriptive statistics and econometric analysis. Moreover, indexes can be constructed just by summing the number of correct answers to the different questions. A more sophisticated approach is to relate each answer’s score to the weight of its explana- tory power (PRIDIT approach). This approach involves a two-step weighting procedure: the first step compares each respondent’s answers to the pool of responses; the second step assigns larger weights to the questions which fewer people answered correctly (Behrman, Mitchell et al. 2010). The weighted index obtained through PRIDIT has proved to be highly correlated to the “rough� index. 3.2 Key Results The results of multiple research analyses in different contexts have highlighted the follow- ing three main findings: 1. The level of financial literacy is low among the population in both developed and developing countries. For example, in the United States only a third of respondents is able to correctly answer all three basic financial literacy questions (Lusardi and Mitchell 2011a); 2. Financial literacy differs widely across education groups. These differences are similar across countries. Further, financial literacy is particularly low among the young, the old, women, and the less-educated; 3. In all countries financial literacy can be linked to specific financial behaviors. For example, Lusardi and Mitchell (2011a) have shown that individuals who score higher on the financial literacy questions are much more likely to plan for retirement. Review of Existing Financial Capability and Literacy Measurement Instruments 185 3.3 Policy Application Policy applications can be derived from the research findings described above. For ex- ample, successful policies in the field of financial literacy would: 1. Add financial education in schools. Schools are ideal places in which to educate the young, who—in all countries—display little financial knowledge. 2. Target the most vulnerable groups. As described previously, the young, the old, women, and the less-educated all display very low levels of financial knowledge. 3. Provide help in the various stages of life cycle planning. 3.4 Final Remarks on Replication (over time and in different contexts) The questions described above have been designed to reflect the priorities of the U.S. economy, hence it would be unsuitable to apply them directly to a completely different context. However, when the context is not too different, these questions can be reliable measurement instruments. The analyses carried out in India and Indonesia have provided insightful and important findings. Lastly, with regard to consistency over time, we note that these questions have been incorporated into national surveys only since 2004; conse- quently it is difficult to conduct tests of temporal consistency. References Behrman, J., O. S. Mitchell, C. Soo, and D. Bravo. 2010. Financial literacy, schooling, and wealth accumulation. NBER Working Papers No. 16452. Bucher-Koenen, T., and A. Lusardi. 2011. “Financial Literacy and Retirement Planning in Germany,� Journal of Pension Economics and Finance 10(4): 565–584. Cole, S., T. Sampson, and B. Zia. 2011. “Prices or Knowledge? What Drives Demand for Financial Services in Emerging Markets?� The Journal of Finance 66: 1933–1967. Crossan, Diana, David Feslier, and Roger Hurnard. 2011. “Financial literacy and retirement planning in New Zea- land.� Journal of Pension Economics and Finance 10(4): 619–635 De Mel, S., D. McKenzie, and C. Woodruff. 2011. “Getting Credit to High Return Microentrepreneurs: The Results of an Information Intervention� World Bank Economic Review 25(3): 456–485. FINRA Investor Foundation. 2009. Financial Capability in the United States: National Survey—Executive Sum- mary. As of May 16th, 2012: http://www.finrafoundation.org/web/groups/foundation/@foundation/documents/foundation/p120535.pdf Hastings, J., and L. Tejeda-Ashton. 2008. Financial literacy, information, and demand elasticity: Survey and ex- perimental evidence from Mexico. NBER Working Paper 14538. Hastings, J., and O. S. Mitchell. 2011. How financial literacy and impatience shape retirement wealth and invest- ment behaviors. NBER Working Paper 16740. Lusardi, A., and O. S. Mitchell. 2009. How ordinary consumers make complex economic decisions: Financial literacy and retirement readiness NBER Working Paper 15350. Lusardi, A., and O. S. Mitchell. 2011a. “Financial literacy and planning: Implications for retirement wellbeing.� In Financial Literacy: Implications for Retirement Security and the Financial Marketplace, edited by Olivia S. Mitchell and Annamaria Lusardi, Oxford University Press. Lusardi, A., and O. S. Mitchell. 2011b. “Financial literacy and retirement planning in the United States.� Journal of Pension Economics and Finance 10(4): 509–525 Lusardi, A., and O. S. Mitchell. 2011c. “Financial literacy around the world: an overview.� Journal of Pension Eco- nomics and Finance 10(4): 497–508 186 Review of Existing Financial Capability and Literacy Measurement Instruments Lusardi, A., O. S. Mitchell, and V. Curto. 2010. “Financial literacy among the young.� Journal of Consumer Affairs 44(2): 358–380. Van Rooij, M., A. Lusardi, and R. Alessie. 2011. “Financial literacy and stock market participation.� Journal of Financial Economics 101(2): 449–472. 187 Appendix C: Survey Bibliography ANZ-Retirement Commission Financial Knowledge Survey Main Reference(s) Colman Brunton (2009). “2009 Financial Knowledge Survey.� Colman Brunton, Auckland. Questionnaire http://www.financialliteracy.org.nz/sites/default/files/webfm/Research/r-fksurvey-2009.pdf Similar Studies: The Social Research Centre (2011). “Adult Financial Literacy in Australia: Full Report of the Results from the 2011 ANZ Survey.� The Social Research Centre, Mel- bourne. Retrieved from http://www.anz.com/resources/f/9/f9fc9800493e8ac695c3d7fc8cff90cd/2011-Adult- Financial-Literacy-Full.pdf.pdf?CACHEID=f9fc9800493e8ac695c3d7fc8cff90cd Consumer Fraud in the United States: The Second FTC Survey Main Reference(s) Anderson, Keith B. (2007). “Consumer Fraud in the United States: The Second FTC Sur- vey.� Federal Trade Commission Staff Report. Questionnaire http://www.ftc.gov/opa/2007/10/fraud.pdf Similar Studies American Association of Retired Persons (1999). “Consumer Behavior, Experiences and Attitudes: A Comparison by Age Group.� Washington DC: American Association of Retired Persons. http://assets.aarp.org/rgcenter/consume/d16907_behavior.pdf American Association of Retired Persons, (2003). “Consumer Experience Survey: Insights on Consumer Credit Behavior, Fraud and Financial Planning.� Washington DC: American Association of Retired Persons. http://assets.aarp.org/rgcenter/consume/cons_exp.pdf American Association of Retired Persons; Princeton Survey Research Associates. (1996). Telemarketing fraud victimization of older Americans: An AARP study. Washington, DC: American Association of Retired Persons. AARP. (2003). Off the hook: Reducing participation in telemarketing fraud. Washington, DC: American Association of Retired Persons. 188 Review of Existing Financial Capability and Literacy Measurement Instruments Consumer Protection Diagnostic Study Kenya Main Reference(s) Di Castri, Simone, Flaming, Mark, Owino, Alexander, McKee, Katharine, Jentzsch, Nic- ola, Maina, Bilha, Ochieng, Moses, Collins, Daryl and Ahern, Brendan (2011). “Consum- er Protection Diagnostic Study: Financial Consumer Protection in Kenya.� January 16, 2011. Available at SSRN: http://ssrn.com/abstract=1843279 or http://dx.doi.org/10.2139/ ssrn.1843279 Similar Studies Consultative Group to Assist the Poor (CGAP), (2009). “CGAP Consumer Protection Policy Di- agnostic Study Malaysia.� Retrieved: http://www.cgap.org/sites/default/files/CGAP-Consumer- Protection-Policy-Diagnostic-Report-Malaysia-2009.pdf Consultative Group to Assist the Poor (CGAP), (2009). CGAP Consumer Protection Policy Di- agnostic Study Cambodia. Retrieved: http://www.cgap.org/sites/default/files/CGAP-Consumer- Protection-Policy-Diagnostic-Report-Cambodia-2009.pdf Consultative Group to Assist the Poor (CGAP), (2010). “CGAP Consumer Protection Policy Diagnostic Study India.� Retrieved: http://www.cgap.org/sites/default/files/CGAP-Consumer- Protection-Policy-Diagnostic-Report-India-2010.pdf Financial Access in Kenya (FinAccess) 2009 Main Reference(s) Financial Sector Deepening Trust (FSDT), (2009). “FinAccess National Survey 2009: Dy- namics of Kenya’s Changing Financial Landscape.� FSDT, Nairobi. Questionnaire http://www.fsdkenya.org/finaccess/documents/09-06-10_FinAccess_FA09_Report.pdf Similar Studies Ladipo, Modupe (2012). “Access to Financial Services in Nigeria 2012 Survey: Key Findings,�Enhancing Financial Innovation & Access (EFInA), 2012. http://www.efina.org. ng/publications/ Financial Capability, Financial Competence, and Wellbeing in Rural Fi- jian Households (Fiji Fin Cap) Main Reference(s) Sibley, Jonathan, (2010). “Financial Capability, Financial Competence and Wellbeing in Rural Fijian Households.� UNDP Pacific Centre, April. Financial Diaries Main Reference(s) Collins, D.,Morduch, J., Rutherford, S., Ruthven, O. (2009), Portfolios of the Poor: How the World’s Poor Live on $2 a Day, Princeton: Princeton University Press. Financial Services Authority (UK FSA) Consumer Awareness Survey Main Reference(s) Financial Services Authority (2011), “Consumer Awareness of the FSA and Financial Reg- ulation,� FSA. Review of Existing Financial Capability and Literacy Measurement Instruments 189 Financial Services Authority (UK FSA) Measuring Financial Capability Main Reference(s) Kempson, Elaine, Atkinson, Adele, McKay, Stephen, and Collard, Sharon (2006). “Levels of Financial Capability in the UK: Results of a baseline survey.� UK: Financial Services Authority. Questionnaire http://www.pfrc.bris.ac.uk/publications/Reports/Fincap_baseline_questionnaire_06.pdf Similar Studies Keeney, Mary J. & O’Donnell, Nuala, 2009. “Financial Capability: New Evidence for Ire- land,� Research Technical Papers 1/RT/09, Central Bank of Ireland. McKay, Stephen, (2011). “Understanding Financial Capability in Canada: Analysis of the Canadian Financial Capability Survey,� Research Paper prepared by the Task Force on Financial Literacy, University of Birmingham. Pirmin Fessler & Martin Schürz & Karin Wagner & Beat Weber, 2007. “Financial Capability of Austrian Households,� Monetary Policy & the Economy, Oesterreichische Nationalbank (Austrian Central Bank), issue 3, pages 50-67, November. Australian Government, Financial Literacy Foundation, (2007). “Financial Literacy: Aus- tralians Understanding of Money.� Australian Government, Financial Literacy Foundation, http://www.financialliteracy.gov.au/media/209293/australians-understanding-money.pdf PattiChiari Consortium and The European House Ambrosetti (2008), “Financial Education in Italy: The first scientific measurement of financial culture in Italy,� The European House- Ambrosetti. Finra Investor Education Foundation, (2009). “ Financial Capability in the United States: National Survey-Executive Summary,� Finra. CentiQ, (2008). “Summary of Financial Insight Among the Dutch,� CentiQ. http://www. wijzeringeldzaken.nl/media/13191/summary_financial_insight_amoung_the_dutch.pdf Financial Regulator, (2007). “Financial Capability in Ireland An Overview.� Ireland: Finan- cial Regulator, Central Bank of Ireland. FinScope Main Reference(s) Financial Sector Deepening Trust (FSDT), (2009)., “The Demand for and Barriers to Ac- cessing Financial Services in Tanzania,� FSDT, Nairobi. Questionnaire http://dl.dropbox.com/u/4464339/TZ%20FSDT%202009%20qnr.pdf 190 Review of Existing Financial Capability and Literacy Measurement Instruments Global Findex Main Reference(s) Demirguc-Kunt, Asli and Klapper, Leora (2012), “Measuring Financial Inclusion: The Glob- al Findex Database,� The World Bank Policy Research Working Paper, No. 6025. Questionnaire http://www-wds.worldbank.org/external/default/WDSContentServer/IW3P/IB/2012/04/19/0 00158349_20120419083611/Rendered/PDF/WPS6025.pdf Similar Studies Federal Deposit Insurance Corporation, (2012). “2011 FDIC National Survey of the Un- banked and Underbanked Households,� FDIC. National Banking and Securities Commission (CNBV), (2012). “Measuring Financial Capa- bility in Mexico,� Presented in Cartagena, Colombia, November. Jump$tart Survey Main Reference(s) Mandell, Lewis (2008), “The Financial Literacy of Young American Adults: Results of the 2008 National Jump$tart Coalition Survey of High School Seniors and College Students,� The Jump$tart Coalition for Personal Financial Literacy, Washington, DC. Questionnaire http://www.jumpstart.org/assets/files/2008SurveyBook.pdf Similar Studies Chen, H., & Volpe, R. (1998). “An analysis of personal financial literacy among college students.� Financial Services Review 7(2), 107–28 Junior Achievement Young Enterprise Europe. “Financial Literacy: A Survey of JA-YE Par- ticipants,� Junior Achievement and Citi Foundation, http://www.qualmedia.net/may09/files/ financial_literacy.pdf. Junior Achievement USA, (2012).�Teens and Personal Finance,� Junior Achievement USA, http://www.ja.org/docload/2012-Teens-Personal-Finance.pdf. Thaden, L. L. & Rookey, B. D. (2004, October). “College students’ financial literacy: Dead or alive?.� Paper presented at Inequalities Seminar, Pullman, WA. OECD/INFE Measuring Financial Literacy Main Reference(s) Atkinson, Adele and Messy, Flore-Anne, (2012), “Measuring Financial Literacy: Results of the OECD/International Network on Financial Education (INFE) Pilot Study,� OECD Work- ing Papers on Finance, Insurance, and Private Pensions, No. 15, OECD Publishing. Questionnaire http://www.oecd.org/finance/financialeducation/49319977.pdf Review of Existing Financial Capability and Literacy Measurement Instruments 191 Singapore National Financial Literacy Survey Main Reference(s) Media Research Consultants Pte Ltd. (2005), Quantitative Research on Financial Literacy Levels in Singapore, prepared for MoneySENSE Financial Education Steering Committee. Retrieved: http://www.mas.gov.sg/~/media/resource/news_room/press_releases/2005/ Summary%20Report.ashx Special Eurobarometer 230 Main Reference(s) Eurobarometer (2005), “Public Opinion in Europe on Financial Services�, Special Euroba- rometer 230, European Commission. Questionnaire http://ec.europa.eu/consumers/cons_int/fina_serv/cons_experiences/report_eurobaro meter63-2_en.pdf Survey on the Financial Literacy of the Portuguese Population Main Reference(s) Banco de Portugal (2011).� Survey on the Financial Literacy of the Portuguese Population (2010), Final Report.� Portugal: Banco de Portugal. Available at http://clientebancario.bportugal.pt/pt-PT/Publicacoes/InqueritoLiteraciaFinanceira/ Documents/Survey%20on%20the%20Financial%20Literacy%20of%20the%20 Portuguese%20Population%20(2010)%20-%20Final%20Report.pdf Questionnaire http://clientebancario.bportugal.pt/pt-PT/Publicacoes/InqueritoLiteraciaFinanceira/ Documents/Survey%20on%20the%20Financial%20Literacy%20of%20the%20 Portuguese%20Population%20(2010)%20-%20Final%20Report.pdf U.S. Health and Retirement Study (HRS) 2004 Main Reference Lusardi, Annamaria, and Olivia S. Mitchell. 2011. “Financial Literacy and Planning: Implications for Retirement Wellbeing,� in Olivia S. Mitchell and Annamaria Lusardi (eds.), Financial Literacy: Implications for Retirement Security and the Financial Marketplace. Oxford: Oxford University Press, pp. 17–49. Similar Studies Alessie, Rob & Van Rooij, Maarten & Lusardi, Annamaria, (2011). “Financial literacy and retirement preparation in the Netherlands,� Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(04), pages 527-545, October. Almenberg, Johan & Säve-Söderbergh, Jenny, 2011. “Financial literacy and retirement planning in Sweden,� Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(04), pages 585-598, October. Bucher-Koenen, Tabea, and Lusardi, Anamaria, (2011). “Financial Literacy and Retirement Planning in Germany.� Journal of Pension Economics and Finance 10(4), 565-584. 192 Review of Existing Financial Capability and Literacy Measurement Instruments Crossan, Diana, David Feslier, and Roger Hurnard, (2011). “Financial literacy and retire- ment planning in New Zealand.� Journal of Pension Economics and Finance, 10, pages 619-635. Fornero, Elsa & Monticone, Chiara, 2011. “Financial literacy and pension plan participa- tion in Italy,� Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(04), pages 547-564, October. Klapper, Leora & Panos, Georgios A., (2011). “Financial literacy and retirement planning: the Russian case,� Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(04), pages 599-618, October. Sekita, Shizuka, 2011. “Financial literacy and retirement planning in Japan,� Journal of Pension Economics and Finance, Cambridge University Press, vol. 10(04), pages 637- 656, October. U.S. Survey of Consumers Nov/Dec 2001 Main Reference(s) Hilgert, Marianne, Jeanne Hogarth, and Sondra Beverly (2003), Household Financial Man- agement: The Connection between Knowledge and Behavior, Federal Reserve Bulletin, 309-32. World Bank Financial Capability and Consumer Protection Surveys (WB FCCP) Main Reference(s) Zottel, Siegfried and Perotti, Valeria (2013), “Mongolia. Financial Capability and Consumer Pro- tection Survey 2012�, The World Bank, forthcoming. http://responsiblefinance.worldbank.org. World Bank Living Standards Measurement Survey /FinScope Finan- cial Services Experiments (Ghana FSS) Main Reference(s) Cull, Scott (2009), “Measuring household usage of financial services : does it matter how or whom you ask?� Policy Research Working Paper WPS5048, The World Bank. http://go.worldbank.org/M4QCI5EII0 World Bank/Russia Financial Literacy and Education Trust Fund Fi- nancial Capability Survey Main Reference(s) Holzmann, Mulaj and Perotti (2013) “Financial Literacy and Education in Low and Middle Income Countries: Measurement and Effectiveness. Knowledge Management Report on the Russia Trust Fund on Financial Literacy & Education�, The World Bank, forthcoming. Questionnaire http://www.finlitedu.org/measurement/wb/