45664 SHIFTING THE FOCUS TO "QUALITY AT EXIT" ­ AN EFFECTIVE APPROACH TO IMPROVING THE BUSINESS ENVIRONMENT AT THE SUBNATIONAL LEVEL LE DUY BINH , THOMAS FINKEL, GTZ Impact monitoring and quality assurance has always been an uphill task for donors, implementing agencies, and recipient agencies. In the last few decades, we have witnessed the evolution of impact monitoring work with lots of new methodologies and concepts, resulting in thick documents that are sometimes difficult to digest. This paper does not aim to describe this evolution. Rather, it focuses on how the GTZ SME Development Program in Vietnam has adopted a new approach in impact monitoring and the challenges that we are facing. It also describes first results and achievements, as well as key lessons drawn out of the process. I. THE GTZ SME DEVELOPMENT PROGRAM IN VIETNAM Vietnam is a rising economy with discernible economic achievements in the last few years. Since 2000, the private sector in Vietnam has grown remarkably and is becoming increasingly important for the national economy. However, constraints facing the private sector are numerous, at both national and subnational level. The GTZ SME Development Program aims to improve the competitiveness of private SMEs in Vietnam by contributing to a favorable private sector policy framework. It also supports creating an enabling business and investment environment at the provincial level and fostering local economic development. Additionally, it helps enterprises to integrate into domestic and international value chains and supports the upgrading of business services. The Program started in 2005, replacing a four-phase SME Promotion Project implemented by GTZ since 1994. The Program has a budget of EUR 8.3 million for a four- year phase. II. ANALYSIS OF KEY ISSUES OR PROBLEMS For a long time, quality assurance within technical assistance projects was oriented heavily on the quality of the inputs provided, or in other words, on "the quality at entry." Many GTZ projects and interventions in Vietnam, particularly during the 1990s, were designed, implemented, and monitored with this focus being the bedrock. The success of the projects was measured mostly in terms of the number of consultant days contributed, training courses organized, participants in workshops, or other types of inputs (with much emphasis laid on the quantitative aspects). This approach failed to bring about the best desirable results and, in many cases, sent the wrong messages to project managers as well as partners. As a result, the effectiveness and impact of the projects were limited. Right at the formulation of the SME Development Program, this challenge was recognized and timely action was taken. A decisive shift toward results-based management and measuring impact was undertaken. However, this is always easier said than done. Despite many guidelines, examples, and case studies that were available, it has been an uphill learning process. SmartLessons, May 2007 1 III. DESCRIPTION OF OUTCOME The SME Development Program, besides its interventions at the national level, also provides support to four provinces: An Giang, Dak Lak, Hung Yen, and Quang Nam. Despite discernible economic success, economic governance and competitiveness of the four provinces remain serious concerns, with their business and investment climates lagging far behind the most progressive provinces of the country. In consideration of the actual context and needs for assistance in each province, separate action plans and intervention strategies have been developed. However, all of the provincial action plans are founded on the following three pillars: 1. Improving the provincial regulatory framework through improved governance capability, strengthening the capacity for implementing business laws and regulations. 2. Assisting local stakeholders in selected provinces in implementing essential promotional policies and initiatives, for example in investment promotion, public private dialogue, promotion of business start-up, etc. 3. Strengthening business and cooperative relationships between the stakeholders (production, processing, trade, and services) of selected value chains. The three pillars cover a wide range of activities, but they are strongly Key Lessons Learned interrelated. Thus, coordinated implementation and monitoring of impacts are manageable. Often, it is easier to measure impacts of · A change in the mindset of GTZ staffs subnational-level interventions than national-level ones. With the and partner organizations is a average population size of approximately 1 million, 2,000 formally precondition for success. registered enterprises and 10,000 household businesses in each · The shift must be reflected in all binding province, it appears easier to measure the impact of the intervention documents under the Program, ranging and track the impact as far down as the enterprise level. As compared from the framework cooperation with the national level, it is also easier to implement baseline studies agreement to a simple TOR for a and to maintain a database for impact measurement. consultant. · Internal monitoring is redesigned on the This represents both opportunities and challenges for the GTZ SME basis of the shift of focus, with clear and Program, particularly in the context that it is focusing increasingly on logical impact chains developed. the "quality at exit." On the one hand, the Program can document and · Impact monitoring must be every staff member's task. demonstrate its impact and outputs in a plausible manner. On the · Don't be too obsessed with the other hand, it still remains a challenge for the Program management "attribution gap." and its partners to identify the right mix of interventions that produce · Set up a well-structured database to really tangible and measurable impact, despite the fact that the support the illustration of the attribution gap at the subnational level is apparently narrower. But still, achievement of impacts. the fact that an impact (or no impact) of a given intervention by the Program can be traced more easily helps to better set the intervention strategythat is, if the impact monitoring system is used and managed as a knowledge management instrument, where learning of project staff and partners is the key aspect. This focus on learning from successes and failures of interventions and trying to continuously improve intervention strategies is what our impact monitoring system and the "quality at exit" approach is all about. An important lesson here was trying to make all Program members understand the importance, the need, and the beauty of the new approach. This was achieved through training, a short-term assignment with an international consultant, and learning by doing on the job. The whole approach and its concepts were clarified in a simple and understandable manner. The knowledge was then transferred to partners at both the national and subnational levelthose who will design and implement the Program's activities later on.1 The shift in the approach is also reflected in all binding documents between the Program and the partners, as well as in TORs for consultant missions or other activities. In order to better steer this shift in thinking and acting, the Program, local governments, and other stakeholders worked out different impact chains, using them as the navigator for the designing, implementing, and monitoring of every single activity in the Program. For the subnational interventions, the impact chains are consolidated in an abbreviated manner in the diagram below. It does not provide a full account of the output and the use of the output which the Program creates. Those examples given in the diagram are for illustrative purpose. 1The ownership of Vietnamese partners in implementing the Program's activities is quite high. SmartLessons, May 2007 2 GTZ SME Development Program ­ Improvement of Subnational Business Environment INCREASE IN # OF ENTERPRISES AND EMPLOYMENT INCREASE IN INVESTMENT AND EXPORTS IMPACT HIGHER RANKING OF FOUR PROVINCES IN PROVINCIAL TIME AND COST FOR REGISTERING AN ENTERPRISE COMPETITIVENESS INDEX REDUCED ST Local governance capacity Good programs and In selected value chains, improved, effective business law initiatives for local economy institutional framework implementation, good lessons development designed and improved, better access to and practices learned, fed back to BENEFI implemented, provincial business services, business national level and replicated in image improved, increased linkage strengthened, value CTE other provinces services for investment, and added to local products, business start-ups available significant increase in income. DIR ·Business registration at provinces ·Economic reforms in four ·Strategy action plans for value are now on-line, computerized, provinces are effectively chain development in selected OF /STUPT ES reducing registration time by 30% Implemented products formulated and VIC ·Local government is addressing ·Innovative instruments for implemented USE problems raised by business. promotion of investment and ·Improved advocacy for SME is OU SER ·With technical advice offered, business start-ups are carried out and support services local government streamlining implemented for selected products are offered ·Business ·Training ·Institutional · Investment ·Subsector ·Market registration courses on capacity for and start-up dialogue and interventions procedures business laws stakeholders initiatives strategy for specific UTSP reformed and ·Technical strengthened implemented development BDS services computerized advice on law ·Forum on ·Locational ·Strengthe- ·Partnerships OUT ·Policy implement- local marketing ning of the with business dialogues plat- tation economy program institutional community forms set up established implemented framework (PPP) REGULATORY FRAMEWORK LOCAL ECONOMIC DEVELOPMENT VALUE CHAIN DEVELOPMENT As mentioned, the impact chain diagram is used to guide the design and implementation of every single activity. In addition, impact monitoring has become the task of every Program member without having a full-time impact monitoring staff. The advantage is that this requires an impact-oriented mentality that will navigate and govern everybody's behavior and performance. Each time an activity is discussed or proposed, every partner agency will naturally ask the following questions: (i) what is the output to be produced? (ii) how will it be used? and (iii) how will it contribute to the indirect benefit and the objective of the Program? The same questions are also asked when the Program staff reviews or discusses any proposed activity with the partner and with the managers. A full-time impact monitoring staff would be desirable for allowing more time on crucial data collection, but the disadvantage would be that everybody would lose the direct obligation to do impact monitoring, and would become less obliged to integrate the concept in their daily work. The absence of a full-time impact monitoring staff has not affected the integrity of the M&E process or of the data collected. The Program's internal monitoring system will be cross-checked and supported by independent evaluators and consultants. In addition, though not full-time, one technical advisor in the Program is tasked with monitoring the whole M&E process. As a quality controller, she is tasked with obliging every staff member to follow the M&E rule, reviewing all reports and data. She performs this role by holding talks with every staff member for at least two hours on M&E every two months and by facilitating and documenting the knowledge management meetings which are held once every year. The Program's internal monitoring system has been re-engineered to reflect the shift to "quality at exit." Regular knowledge management meetings are held to review the achievement of milestones and indicators of impacts, SmartLessons, May 2007 3 and to see whether the logic of the impact chains is still correct. Reports both by GTZ and partners are prepared with an increased focus on this logical chain of "output ­ use of output ­ indirect benefit and final impact." There is a strong argument that the indirect benefit or impact (e.g., contribution to the competitiveness of Vietnam private SMEs) cannot easily be attributed to the Program's interventions. Despite this, it is the shared belief in the Program that if one or a series of activities/ interventions is implemented, producing a good product which is used by the national partners, and yielding tangible and measurable indirect benefits (e.g. a reduction in business registration time, an increase in the turnover of a selected value chain, etc.), this will result in the desired impacts. In reality, impacts are quite difficult to measure, but they are quite feasible to observe at the provincial or subnational level as long as we think about them right at the outset. So don't be too obsessed with the attribution gap! Don't let the attribution gap let you down! Work out a plausible system of impact chains and implement activities persistently along this line to produce desirable outputs and direct benefits. Then you are contributing significantly to generating impacts. What has not worked very well is the setting up of a systemic and well-structured database to support the illustration of the achievements in indirect benefits and impact. At the outset, the database is structured according to the impact chains and the indicators system developed at the advice of an international consultant. However, the limited availability of data statistics in Vietnam and the already heavy workload of the Program's staff members have hinder the completion of the database. To address this issue, everybody will be charged with updating the database with data and information most relevant to his or her area of work. The selection of indicators to reflect the achievement in terms of output, use of output, or benefits has also not been very satisfactory. Some of the indicators turned out to be irrelevant or impossible to measure. Some of the indicators are very vulnerable to the attribution gap trap or too abstract to be measured, thus making it difficult to measure the impact of the Program's intervention. For instance, "selected sub-indices of the Vietnam's Competitiveness Index are improved by the end of the Program" is far less relevant, useful, and measurable than "the time and cost spent by local authorities and business in the four selected provinces on business registration being reduced by 30 percent by the end of the Program." Therefore, adjustments to the indicator system needed to be made, and this required negotiation with the client, the German Ministry for Economic Development and Cooperation (BMZ). In summary, though the Program is still in the learning process, we are not hesitant to state that our shift to "quality at exit" has been a good choice. The implementation process has had more success than failure. Lessons have been drawn from both what worked and what did not work. They definitely will be the basis for strengthening and improving our impact measurement approach in the years to come. IV. REFERENCES 1. http://www.sme-gtz.org.vn 2. http://www.businessenvironment.org 3. http://www.publicprivatedialogue.org About the Authors Le Duy Binh started his career at the State Bank of Vietnam in 1994 in the field of credit and finance. From 1999 to 2001, he worked at the National Economic University as a trainer and a researcher. During this time, he also worked as a senior consultant for the Japan Bank for International Cooperation, with the focus on private sector and SME research. Since 2001, he has been an advisor to the GTZ SME Development Program in Vietnam, with a focus on investment climate, business environment, and SME policy. binh.sme@gtz-vietnam.com.vn. Thomas Finkel started his career working in Small and Medium Enterprise (SME) development projects. In 1999 he changed continents and worked for GTZ's new Public Private Partnership (PPP) office in Germany. In 2003 he moved to Hanoi as senior technical advisor in an SME development project of GTZ in Vietnam and is still working as project manager for GTZ's PPP program in the region. Since December 2005 he has been in charge of GTZ's SME Development Program in Vietnam in the capacity of Chief Technical Advisor. thomas.finkel@gtz.de. DISCLAIMER IFC SmartLessons is an awards program to share lessons learned in development-oriented advisory services and investment operations. The findings, interpretations, and conclusions expressed in this paper are those of the author(s) and do not necessarily reflect the views of IFC or its partner organizations, the Executive Directors of The World Bank or the governments they represent. IFC does not assume any responsibility for the completeness or accuracy of the information contained in this document. Please see the terms and conditions at www.ifc.org/smartlessons or contact the program at smartlessons@ifc.org. SmartLessons, May 2007 4