April 2010 Number 155 www.worldbank.org/enbreve 56684 A regular series of notes highlighting recent lessons emerging from the operational and analytical program of the World Bank`s Latin America and Caribbean Region. Innovations in Poverty Measurement: Seven Design Lessons on the Use of Advisory Committees to Improve Official Poverty Estimates in Peru By Renos Vakis and Matthias Schmidt Since 2007 the World Bank has supported Between 2004 and 2007, no annual official statistics on poverty INEI with non-lending technical assistance existed in Peru. This was due to serious problems with the household survey used to estimate poverty and implemented by the National (NLTA) in fine-tuning and implementing Statistical Institute (INEI). Many of the technical issues arose from the AC model. The support has included changes in the survey instrument, data collection practices and participation in the AC meetings, technical delays in updating poverty measurement methodologies. This put assistance, capacity building, and advice on hold poverty analyses for more than three years, time during on dissemination activities, all with the which little was known about changes in poverty, inequality and goal of developing consensus on how to the effectiveness of policies to reduce poverty. As a result, the resolve the main methodological issues public's confidence in INEI's analysis was low. Within this context, and improve policy dialogue.1 in March 2007, INEI requested technical assistance from the World Bank to solve the methodological and data collection issues. This participatory approach of a task typically implemented solely by the Instead of providing direct technical assistance to INEI (as was government has been a unique and usually done in the past for such requests), the World Bank team proposed the creation of an external Advisory Committee (AC) innovative accomplishment. Allowing whose role would be to: (i) discuss, advise and reach a consensus on external experts and academics to discuss, the best methodology that could be used to improve and produce debate, and influence the appropriate comparable poverty estimates in Peru; and (ii) oversee the work methodology to measure official poverty of an inter-institutional technical team (composed of INEI, WB statistics has proven to be a success. It and others) who would implement the AC's recommendations. provided the space for dialogue and The AC was composed of experts on poverty issues from the consensus building, just in-time technical public sector (Ministry of Economics and Finance and the Central advice to INEI, and a new concept for Bank), local academia (Universidad Católica, Universidad del quality control of national statistics. As Pacífico, Grupo de Análisis para el Desarrollo, Universidad San a result, the methodological issues in Martín), and international organizations (Institut de recherche poverty measurement have been resolved pour le développement, the Inter-American Development Bank, and improved. INEI's capacity has been the United Nations Economic Commission for Latin America and the Caribbean, and the World Bank). This inter-institutional strengthened and dissemination efforts approach was aimed at helping INEI to create an environment of have facilitated widespread public access openness and transparency necessary to improve its credibility of the information produced. In addition, and relationship with its clients and the public. media and public perceptions of the credibility of these figures improved, 1 A description of the technical components of the work can be found at: www.worldbank.org/lacpovertymeasurement 1 moving the country debate towards the role of public perceived as an endorsement of the existing government policy in poverty reduction. The approach is being regime and its policies. Finally, by defining a narrow, replicated in a number of countries in the region, and INEI technical focus for each meeting, it allowed the work to be is currently exploring ways to expand the model to other confined to methodological issues, which sharpened the areas (labor market indicators, growth estimates). Finally, profile of the AC's mission and eventually helped the AC to the External Advisory Committee on Poverty has been resist pressures to grant membership to individuals with formalized through a presidential decree in December no technical expertise. 2009, something that will allow sustaining its role as an independent committee providing advice on poverty Lesson 2: Transfer knowledge locally methodology to INEI in the coming years. (and ensure ownership) Based on this experience, this note presents a number of The AC model allowed INEI's technical staff and external key lessons on elements of this process that facilitated experts to discuss, debate, and influence the poverty its success with the objective of informing the design of methodology. At the beginning of the process, the similar models elsewhere and in the future. involvement of the external members varied in intensity (most attended the AC meetings but were less actively involved with the data and technical analysis). While Lesson 1: Get the right people, set up this was useful, it was clear that at times the technical clear roles, and keep it technical discussions could have been more efficient and deeper if members had been more "hands-on." Over time, the AC After agreeing with INEI, an original invitation by the World agreed on the need for a more systematic involvement Bank to the potential external members was instrumental by the external members. This led to the creation of sub- in creating a concrete work plan and the committee's committees on various topics with the aim of allowing organizational structure. The World Bank's ability to lead members to focus on their areas of expertise and to this in the beginning facilitated a number of key goals and interact directly and "bilaterally" with the INEI staff before overcame various challenges. First, it ensured that external reporting to the full AC.2 This improved both ownership members included individuals with relevant expertise and and the quality of the technical advice given to INEI and has reputation on poverty methodologies (the majority of given even the most vocal critics of the poverty numbers which had been vocal critics of INEI's capacity to produce the opportunity to contribute constructively. It has also quality data). Second, under World Bank coordination, this strengthened the skills of the technical staff of INEI, who initial phase clarified the committee's role as advisory and have been working side to side with the external members not executing. This meant that INEI would be ultimately held to implement the new ideas and suggestions. accountable for any final outputs (the poverty estimates), something that reduced the concerns of external members who feared that participation in the committee would be Lesson 3: It takes time, so invest appropriately (in quantity and time) A sufficiently long-term framework was important for sustained engagement with policy and technical institutions, especially in the absence of an accompanying operation. A three-year ongoing relationship, with frequent opportunities for interaction, with local partners was essential to building rapport and establishing a working relationship and trust among the AC members. This relationship also improved communication and the quality of the technical discussions and outputs. Having the necessary resources to keep the engagement over this period was therefore crucial and was facilitated by the management team's flexibility to extend the work beyond 1 As of January 2010, there were at least 10 subcommittees with AC members supporting INEI with advice in their areas of expertise (updating caloric diet requirements, improving the construction of price deflators, sampling issues, etc.) 2 its original fiscal-year limits. Given the high fixed costs of starting up such models or providing one-off technical support, the AC model seems to have been a more effective approach from a cost-benefit point of view as it allowed the space for technical assistance and helped develop in- house skills at the Statistical Office in a sustainable way. Lesson 4: Delink short- from long-term objectives The continuous, multi-year engagement with INEI and the AC model allowed the work to be delinked from external short-term goals and pressures, and at the same time to increase its effectiveness and quality. For example, the work to improve the methodology was separated from similar to a company's "Board," but with a more narrow INEI's need to report annual poverty estimates to the focus.3 public. This was done by creating two parallel work plans: the first aimed at ensuring (short-term) consensus about Two factors seem to have aided these discussions. First, the existing poverty methodology and comparability over INEI benefited by the accompanying support of the World time, while the second (long-term) focused on improving a Bank and the AC members, as it gave a strong signal to the number of pending issues with the existing methodology public about the credibility of the process and the poverty (e.g. updating caloric requirements, demographic changes numbers. In this sense, INEI was eager to showcase the due to a new census). This meant that INEI implemented transparency of the process and the consensus building the AC's recommendation to keep using an updated, work around the poverty figures and find ways to solidify technically robust poverty methodology to report official it. Second, the set-up (see lesson 1) ensured that the AC poverty estimates to the public, while continuing updating was only there to offer advice and guidance and was not and improving the methodology to ensure that all existing to be held accountable for the poverty numbers. This limitations were addressed. In addition to avoiding reduced concerns by the external AC members who did pressures to compromise the technical work due to short- not want to the AC to be seen as a political instrument.4 term datelines, this approach also minimized confusing These factors made it easier to discuss how to move the the public with presenting too many updates on the AC's initial informal character to a more formal entity. methodology, which could have also been misinterpreted as politically driven, thus politicizing the AC and putting While this process is still not complete, the high public pressure on its technical focus. visibility of the AC model has led to a presidential decree, formally institutionalizing the Poverty Advisory Committee. Lesson 5: Think about sustainability While this is only a first step, it has strengthened the AC early on model and has renewed the need to explore additional legislative measures to institutionalize the process. In the absence of a legally binding mandate for the committee, ensuring lasting impacts and sustainability beyond the life of the NLTA was always at risk. Lesson 6: Improve transparency Institutionalizing an official AC role required a clear link to INEI legal structure of. Committee members discussed Transparency was emphasized from the start, not only how to formalize the AC early. They raised the possibility of with the public, but also within the AC and INEI. Improving creating bilateral or other types of formal, legally binding, communication about the AC work, related materials and non-binding agreements between INEI and each such as data, methodological briefs, and programs used institution represented in the AC. Another suggestion was to construct poverty estimates were published on INEI's using INEI's legal institutional status to give the AC a role website. This allowed interested parties to replicate 3 Peruvian law requires agencies like INEI to have a "board" that oversees its work. 4 The fact that the poverty numbers were not controversial - large declines in poverty during a period of record growth ­ also facilitated the media and public's acceptance of the process and its technical merits. 3 the procedures and verify the poverty findings. This team to provide more in- depth analytical products (such built confidence in the AC and INEI's work and arguably as a series of studies on understanding the poverty trends) contributed to the wide media interest and acceptance. and better coordinate with INEI and the AC members. The interaction with the media has been critical as it has slowly moved the debate away from defending the credibility Lesson 7: Support constructive dialogue of the poverty numbers to discussing the actual poverty with the media trends and policy implications. Figure 1 summarizes public perceptions of INEI's poverty numbers based on a review INEI publishes poverty numbers once a year (usually in May). of press articles and news programs covering poverty since During these events, the AC has pursued a dissemination 2006. As it can be seen from this qualitative review, the shift strategy of active interaction with the media through press in the content of the media coverage (from questioning releases, presentations, and interviews, with the objective the numbers to focusing on the actual trends) highlights of educating the public about poverty trends. This the importance of integrating a strong dissemination and engagement benefitted from the long timeframe of the communication strategy in such activities. World Bank's technical assistance, which also allowed the Figure 1. Public perceptions of poverty numbers over time 2006 April 2007 July 2007 May 2008 May 2009 World Bank poverty Advisory Committee First set of INEI announces 2007 INEI announces 2008 assessment is on Poverty is formed recommendations results. Only 3 of results. Media calls published but only and work begins. implemented. INEI 172 media articles methodology used covers trends up to announces comparable discuss concerns with "at the scientific 2004 due to the lack poverty estimates for methodology. frontier...". of quality data for 2004-2006. Media focus 2005. is divided between the credibility of the numbers and their interpretation. 2004 2005 2006 2007 2008 2009 About the Authors Renos Vakis is a Senior Economist with LCSPP and Matthias Schmidt was a consultant with LCSPP. Disclaimer: The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Executive Directors of the International Bank for Reconstruction and Development / The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. About "en breve"... "en breve" is produced by the Knowledge and Learning Team of the Operations 4 Services Department of the Latin America and the Caribbean Region of The World Bank - http://www.worldbank.org/lac Visit the entire "en breve" collection at: www.worldbank.org/enbreve