82723 Assessment Framework to Monitor and Evaluate e- Government Procurement Systems in India 6/30/2013 A Report by The World Bank Page | 0 Acknowledgement This report is the result of team work of multiple expert resources over the last few years. Special Thanks goes to Felipe Goya (South Asia Regional Procurement Manager, The World Bank) for his support, review, guidance and advice. Abduljabbar Hasan Al-Qathab (Head of Procurement Team, The World Bank, New Delhi) for providing valuable feedback during conceptualization of the study. Ashish Bhateja (Senior Procurement Specialist, The World Bank) provided detailed feedback on this report. This report was prepared under the guidance of Task Team Leader Swayamsiddha Mohanty (Procurement Specialist, The World Bank), who played an active role in preparation of the concept and leading the study. Dr. Ramanathan Somasundaram (e-Procurement Consultant, The World Bank) was the Bank’s Consultant for the analytical work and drafting this document. Page | 1 Executive Summary Many Government agencies have sought to extend use of e-Tendering system implemented in their organization to handle procurement in World Bank funded projects. The World Bank has developed a procedure to assess e-Tendering systems for compliance to certain guidelines laid down by Multilateral Development Banks (MDB). Since 2008, the World Bank has assessed a total of 21 installations and 6 different e-Tendering systems in India. Going forward, the World Bank prefers to work with Government of India in development of a robust mechanism for assessment of e-Tendering systems (i.e.) instead of independently assessing e-Tendering systems as it has done till date. The Government of India has already established a set-up under Standardization, Quality and Testing Certification (STQC) for assessment of e-Procurement systems. STQC has developed “Guidelines for Compliance to Quality Requirements of eProcurement systems� to assess the essential quality characteristics of e-Procurement system covering Security, Transparency & Functionality and has assessed and certified a few e-Procurement applications. From its assessment experience, the World Bank has found that the e-Procurement applications deployed in many of the e-Procurement installations assessed by the Bank were STQC certified, yet there were several issues with the e- Procurement systems. A set of 36 critical issues identified by the World Bank from its assessment of 21 installations are explained in this report, a few of which are listed below: a) Issues pertaining to issuance of “No Objection� by the World Bank a. Confidentiality of Bid Documents compromised b. Bid proposals shall not be encrypted using bidder’s key c. Disallow mandatory submission of originals before bid submission date d. Recovery Point Objective (RPO) and Recovery Time Objective (RTO) undefined e. Lack of commitment on record keeping f. System malfunction procedure undefined g. Procurement of Digital Signature Certificate by foreign bidders b) Important issues primarily from the view point of Government of India a. Adoption of form based systems b. Serious usability issues c. Electronic systems should replace manual systems and not co-exist d. e-Procurement servers should be under government control e. Weak Project Management Unit (PMU) f. De-duplicated vendor database g. Lack of adoption of item codification standard h. Central Public Procurement Portal (CPPP) as a one stop shop i. Need for mechanism to address vendor lock in and exit management j. Standardize the encryption methodology k. Need to define e-Procurement Enterprise Architecture l. Need for e-Procurement rules Further, the e-Procurement guideline prepared by STQC is reviewed in detail to: (i) Identify e-Tendering requirements of MDB not already covered in STQC guideline (ii) Suggest improvements to STQC guidelines such that it is more specific, actionable, decisive and trend-setting. Specifically, the following improvements are suggested: Page | 2 a. Assess e-Procurement installation and not e-Procurement application software b. Supply driven and not demand driven assessment c. Consolidate and classify the requirements in a certain logical sequence d. Make it as a check-list e. Be as specific as possible f. Visualize the larger picture g. Establish benchmarks h. Introduce competition by grading and ranking e-Procurement installations A new Concept Model for assessment of e-Procurement system is proposed as an evolved version of the existing STQC’s guideline to address the 36 critical issues and the improvements suggested to the STQC’s guideline. The Concept model identifies the subjects to be covered in e-Procurement assessment and suggests weightage allocation to each of the identified subjects. A total of 33 subject areas are identified under 4 layers viz. Data, Application, Infrastructure and Process. Setting the benchmarks by itself may not spur government agencies to achieve better by improving the quality of their e-Procurement installation. Hence, it is suggested that each installation is graded during the assessment as per weightage allocated to various sections of the e-Procurement assessment framework. The assessment will seek to evaluate not only the application but also other aspects of the installation such as adoption of the standards and the policies, data governance, project management, adoption (functional and geographical coverage) and legal framework. The grades assigned as per the assessment framework will be consolidated to arrive at a single mark as per which e-Procurement installations will be ranked. An annual report on assessment of e-Procurement systems in India will be published ranking the various e-Procurement installations. A key assumption herein is that a sense of competition will get introduced amongst the various e-Procurement installations and Government agencies will compete and seek to obtain better rank by upgrading their installations vis-à-vis the laid down e-Procurement guidelines. A new version of e-Procurement guidelines has to be released on a periodic basis to set new benchmarks and upgraded metrics. As on date, there are about 15 adequately large e-Procurement installations. This number would eventually increase to about 50 installations in the years to come. Hence, Government of India (GoI) needs to develop strong institutional capabilities required to assess multiple e-Procurement installations. An adequately large sized team is required to assess the many e-Procurement installations year on year. Such a team would need to have strong expert resources from technology and public procurement domains. The Government of India has to decide on whether to develop the full team of resources in- house or to outsource the assessment work to 3rd party agencies. The outsourcing of work while it is adopted shall be restricted to routine and well defined activities such as verification of compliances to laid-down process and technology standards. Key activities such as visualizing the larger picture and preparation of the Annual e-Procurement Assessment Report shall necessarily be prepared by a core establishment set-up within the Government. The Government may choose to segregate the routine e-Procurement assessment work and policy development work and assign them to two different organizations. In which case, an e-Procurement policy making authority has to be constituted. It is recommended that the cost of setting up and Page | 3 managing the institutional framework for assessment of e-Procurement systems is borne by the Central Government as expenditure incurred towards National Policy Making. Preparation of this report is an initial step taken by the Bank in working with GoI towards development of a robust framework for assessment of e-GP system. The Bank looks forward to working closely with GoI in achieving the following key Outcomes: a) An e-Procurement installation certified by STQC or a designated Government of India institution need not be assessed by the World Bank separately. A STQC or GoI certified e-Procurement installation can be used as such to process tenders in World Bank funded projects. b) A robust framework for assessment of e-Procurement installation is put in place by the GoI. The e-Procurement guidelines will continuously evolve in sync with developments in the e- Procurement space. Further, a designated GoI institution will assess and certify e-Procurement installations in India at regular intervals c) The e-Procurement assessment guidelines developed by STQC will be evolved to focus on the broader e-Procurement landscape as a whole addressing standardization, exit management and integration requirements From its assessment experience, the World Bank has learned that there is a genuine requirement to rank and grade e-Procurement installations and State Governments and Public Sector Undertakings (PSU) are likely to welcome such an initiative. Government agencies implementing e-Procurement systems in general lack the requisite capabilities to objectively assess their systems vis-à-vis well established standards. Herein, it is suggested that Government of India builds strong capabilities at the National level at first to assess and guide development of e-Procurement systems. Given the Federal establishment in India, imposing standards de jure on the State Governments will not work. Hence, it is suggested herein that government agencies are encouraged to adopt certain specified standards (e.g. UNSPSC code set, PAN verification during supplier registration and publication in CPPC) by allocation of marks. Standards in the proposed model will thus evolve as de facto instead of the de jure. A six-step iterative procedure is proposed to operationalize the assessment of e-Procurement installations as given below: (i) Develop assessment guidelines (i.e. the desired state) (ii) Prepare assessment list (i.e. e-Procurement installations) for the year (iii) Conduct assessment of e-Procurement installations vis-à-vis the developed guidelines (iv) Grade and rank the assessed e-Procurement installations (v) Publish Annual e-Procurement assessment report (vi) Revise assessment guidelines for the subsequent year The assessment thus conducted year-on-year will enable Government to reflect upon its policy on e- Procurement and also encourage government agencies to update their e-Procurement systems to comply with the laid down policy. The setting up and maintenance of the institutional establishment – required to develop e-Procurement assessment guidelines and to assess the many e-Procurement installations – can be done with minimal (i.e. in relation to the overall public procurement spend) investments. It is expected that this initiative will catalyze the development of robust e-Procurement systems and enable inter-networking of the many systems through adoption of standards. The Value for Money (VfM) from investments made in this initiative will be quite high if this expectation is realized. Page | 4 This six-step procedure and the institutional establishment required to undertake this assessment is jointly referred to as the Assessment Framework. The Assessment framework explained in this report could be used as a vehicle to implement policy in other federated e-Governance systems such as e-District and State Data Center (SDC). If e-District Mission Mode Project (MMP) is taken as an example, Government needs to define e-District specific assessment guidelines (i.e. e-District Policy) and assess e-District installations year on year to verify for compliance vis-à-vis the laid down guidelines and prepare e-District Annual Assessment Report. Thus, the Assessment Framework concept explained herein has wider application in implementation of e- Governance policy. Page | 5 Table of Contents ACKNOWLEDGEMENT ............................................................................................................................................ 1 EXECUTIVE SUMMARY ........................................................................................................................................... 2 1 INTRODUCTION ............................................................................................................................................. 9 1.1 APPROACH ADOPTED BY THE WORLD BANK TO ASSESS E-GP SYSTEMS........................................................................9 1.2 NEED FOR A ROBUST FRAMEWORK TO ASSESS E-GP SYSTEMS IN INDIA .....................................................................10 1.3 REPORT OBJECTIVES AND ENVISAGED OUTCOMES .................................................................................................12 1.4 METHODOLOGY ..............................................................................................................................................12 1.5 CHAPTER OVERVIEW........................................................................................................................................15 2 E-PROCUREMENT INSTALLATIONS ASSESSED BY THE WORLD BANK IN INDIA ............................................. 16 3 CRITICAL ISSUES IDENTIFIED FROM ASSESSMENT OF E-PROCUREMENT INSTALLATIONS IN INDIA .............. 18 3.1 BROAD CLASSIFICATION OF E-PROCUREMENT APPLICATIONS IN INDIA .......................................................................19 3.2 ISSUES PERTAINING TO ISSUANCE OF “NO OBJECTION� ..........................................................................................19 3.2.1 Security ..................................................................................................................................................19 3.2.2 Transparency .........................................................................................................................................22 3.2.3 Functionality ..........................................................................................................................................23 3.2.4 Data Governance ...................................................................................................................................23 3.2.5 Policy ......................................................................................................................................................24 3.3 IMPORTANT ISSUES FROM THE VIEW POINT OF GOVERNMENT OF INDIA.....................................................................26 3.3.1 Functionality ..........................................................................................................................................26 3.3.2 Policy ......................................................................................................................................................28 3.3.3 Security ..................................................................................................................................................29 3.3.4 Project Management .............................................................................................................................29 3.3.5 Standardization......................................................................................................................................30 3.3.6 System Architecture ...............................................................................................................................38 3.3.7 Legal Framework ...................................................................................................................................41 4 REVIEW OF STQC’S E-PROCUREMENT GUIDELINES ...................................................................................... 44 4.1 AN OVERVIEW OF STQC’S E-PROCUREMENT GUIDELINES .......................................................................................45 4.2 FITMENT OF MDB’S E-TENDERING GUIDELINE WITH THAT OF THE STQC ...................................................................46 4.3 IMPROVEMENTS SUGGESTED TO STQC’S E-PROCUREMENT GUIDELINE .....................................................................50 4.3.1 Assess e-Procurement Installation and Not e-Procurement Application Software ................................50 4.3.2 Supply Driven and Not Demand Driven Assessment ..............................................................................50 4.3.3 Consolidate and Classify the Requirements in a Certain Logical Sequence ...........................................51 4.3.4 Make it as a Check-list ...........................................................................................................................52 4.3.5 Be as Specific as Possible .......................................................................................................................52 4.3.6 Visualize the Larger Picture....................................................................................................................53 4.3.7 Establish Benchmarks ............................................................................................................................54 4.3.8 Introduce Competition by Grading and Ranking e-Procurement Installations ......................................54 5 CONCEPT MODEL OF THE PROPOSED E-PROCUREMENT ASSESSMENT GUIDELINE ...................................... 56 5.1 TABULAR FORMAT ADOPTED TO DEFINE REQUIREMENTS ........................................................................................56 5.2 CLASSIFICATION OF ASSESSMENT REQUIREMENTS..................................................................................................58 5.3 CONCEPT MODEL OF THE PROPOSED ASSESSMENT GUIDELINE .................................................................................58 6 INSTITUTIONAL ESTABLISHMENT REQUIRED FOR THE ASSESSMENT OF E-PROCUREMENT INSTALLATIONS 61 7 SIX STEP PROCEDURE TO OPERATIONALIZE THE ASSESSMENT OF E-PROCUREMENT INSTALLATIONS ......... 63 Page | 6 7.1 STEP ONE: DEVELOP ASSESSMENT GUIDELINES .....................................................................................................63 7.2 STEP TWO: PREPARE ASSESSMENT LIST FOR THE YEAR ...........................................................................................64 7.3 STEP THREE: CONDUCT ASSESSMENT ..................................................................................................................65 7.4 STEP FOUR: GRADE AND RANK SYSTEMS .............................................................................................................65 7.5 STEP FIVE: PUBLISH ANNUAL ASSESSMENT REPORT ...............................................................................................65 7.6 STEP SIX: FEEDBACK ........................................................................................................................................65 8 WORKABILITY OF THE PROPOSED SOLUTION .............................................................................................. 66 9 ANNEXURE .................................................................................................................................................. 68 9.1 E-PROCUREMENT RELATED SECTIONS IN DRAFT PUBLIC PROCUREMENT BILL 2012 ..................................................... 68 9.1.1 Section 7: Determination of Need for Procurement ...............................................................................68 9.1.2 Section 14: Registration of Bidders ........................................................................................................68 9.1.3 Section 18: Pre-bid Clarifications ...........................................................................................................68 9.1.4 Section 22: Exclusion of Bids ..................................................................................................................68 9.1.5 Section 25: Award of Contract ...............................................................................................................68 9.1.6 Section 29, Method of Procurement ......................................................................................................68 9.1.7 Section 30: Open Competitive Bidding ...................................................................................................68 9.1.8 Section 31: Limited Competitive Bidding ...............................................................................................69 9.1.9 Section 38: Central Public Procurement Portal ......................................................................................69 9.1.10 Section 56: Power of Central Government to make Rules .................................................................69 Page | 7 List of Figures FIGURE 1: OVERVIEW OF THE ASSESSMENTS UNDERTAKEN 13 FIGURE 2: LIST OF E-PROCUREMENT INSTALLATIONS ASSESSED BY THE WORLD BANK 16 FIGURE 3: E-GP MAP OF INDIA 17 FIGURE 4: OVERVIEW OF THE ISSUES IDENTIFIED DURING THE ASSESSMENT 18 FIGURE 5: NATIONAL DATABASE ON SUPPLIER'S PERFORMANCE 32 FIGURE 6: SNIPPET FROM A PROCUREMENT ANALYTICS REPORT 35 FIGURE 7: NATIONAL DATABASE OF REGISTERED SUPPLIERS 36 FIGURE 8: E-GP POSITION MAP TAKEN FROM A SURVEY CONDUCTED BY THE ADB 39 FIGURE 9: FITMENT OF MDB'S E-TENDERING GUIDELINE WITH THAT OF THE STQC 49 FIGURE 10: TABULAR FRAMEWORK ADOPTED TO DEFINE REQUIREMENTS 57 FIGURE 11: CONCEPT MODEL OF THE PROPOSED ASSESSMENT GUIDELINE 60 FIGURE 12: SIX STEP PROCEDURE TO OPERATIONALIZE THE ASSESSMENT OF E-PROCUREMENT INSTALLATIONS 64 FIGURE 13: WORKABILITY OF THE PROPOSED SOLUTION 66 Page | 8 1 Introduction The use of e-Tendering by Government agencies in India started around the year 2000. After more than a decade, the use of e-Tendering is prevalent amongst Government agencies across the landscape in India. Few of the e-Tendering installations1 in India are quite large with 10000+ registered vendors / contractors. Some State governments such as Andhra Pradesh, Karnataka and Gujarat have achieved success in establishing a State-wide e-Tendering platform, wherein 100+ procurement entities use a single instance of software to process their tenders. In State-wide systems, a vendor can register once and participate in any of the tenders advertised in the platform. The use of e-Tendering is mandated by some government agencies in recent years. 1.1 Approach Adopted by the World Bank to Assess e-GP Systems Many Government agencies have sought to extend use of e-Tendering system implemented in their organization to handle procurement in World Bank funded projects. The World Bank – just as the Asian Development Bank (ADB) – has developed a procedure to assess e-Tendering systems for compliance to certain guidelines laid down by Multilateral Development Banks (MDB). An overview of the assessment procedure is given below: a) Government agency provides compliance statement to MDB’s e-Tendering guidelines b) A Mission from the World Bank makes an on-site visit to interact with the Government agency and e-Tendering Application Service Provider (ASP) to better understand the compliance statement and is provided with demonstration of the e-Tendering system c) A draft of the assessment report along with key findings is prepared by the World Bank team and shared with the Government for information and further action (as required). d) The Government agency makes certain changes to its system – as required – in an effort to comply with MDB’s e-Tendering guidelines e) When the System is found satisfactory, Government agency is authorized to use its e-Tendering platform to process tenders in World Bank funded projects The assessment of e-Tendering system by the World Bank is a short term engagement. A key fundamental assumption of this assessment is that integrity and security related issues will have been audited (i.e. by verification of key activities in the file / database level, where the data is stored) in detail by a 3rd party agency. A report by the audit agency and / or a declaration from the assesse stating that there aren’t audit issues will be taken as valid prima facie. The Mission studies the system to the extent possible – to verify compliance to e-Tendering assessment requirements – during the short on-site visit. The e-Tendering systems are assessed as per this procedure and “No Objection� is provided for use of the system to handle World Bank tenders when there are no pending issues. The World Bank started assessment of e-Tendering installations in 2008. A total of 21 installations and 5 different e-Tendering solutions have been assessed till date, of which “No Objection� is provided to 10 installations. The World Bank now insists on use of e-Tendering – where possible – for newly initiated projects. As many as 10 out of the 21 installations were assessed by the World Bank during the year 2012 – 2013. The assessment experiences based on which this report is prepared covers almost all the 1 The word “Installation� and “system� are interchangeably used in this report. Both refer to the larger e - Procurement eco-system and not just the application software. Page | 9 large e-Tendering installations and the most prominent e-Tendering solutions in India. To that extent, key issues detailed in this report would be quite valid. From this assessment experience, the World Bank has acquired an intuitive understanding of the issues and challenges typically found in e-Tendering installations. In the installations assessed and cleared by the World Bank, efforts have been taken to address the identified issues by: a) Standard Bid Document (SBD) clauses to be inserted in tender documents b) Modifying the functionality in e-Tendering software 1.2 Need for a Robust Framework to Assess e-GP Systems in India The assessment of e-Tendering installations is not just a one-time activity. Instead, efforts have to be taken to monitor the various e-Tendering installations at regular intervals and verify if the installations confirm to certain laid down guidelines. Such monitoring is absolutely essential given that procurement in excess of tens of thousands of crores of Rupees (i.e. in Billions of USD) is routed through such systems. As the use of e-Tendering is now quite common in India, the World Bank prefers to work with Government of India in development of a robust mechanism for assessment of e-Tendering installations (i.e.) instead of independently assessing the systems as it has done till date. In this regard, it is noted that Standardization, Quality and Testing Certification (STQC), an office attached to the Department of Electronic and Information Technology (DeITY) has developed “Guidelines for Compliance to Quality Requirements of eProcurement systems� specifically to assess the essential quality characteristics of e- Procurement2 system covering Security, Transparency & Functionality. Refer page 12 of the STQC guidelines, wherein two types of assessment service provided by STQC are identified viz.: a) “Only for the e-Procurement application�; assessment will cover requirements defined under part 1 (i.e. application focused) b) ‘The Complete e-Procurement System�; assessment will cover requirements defined under both part 1 (application focused) and part 2 (installation focused) The preparation of e-Procurement guidelines and certification of e-Procurement application by STQC has not translated into foolproof installations. The applications deployed in many of the 21 e- Procurement installations assessed by the World Bank were STQC certified (i.e. only for the e- Procurement application). Yet, critical and glaring issues were identified during assessment of those installations, possibly because the installations were not certified as per requirements defined in part 2 of STQC guidelines. For example: a) Recovery Point Objective (RPO) is not defined in most of the installations (i.e.) quantum of data loss in case of disaster is undefined. Often, Government officials tend to enquire the ASP during the assessment in front of the Mission about data backup procedures put in place. 2 e-Procurement� system has multiple modules including Indent Management, e -Tendering, e-Auctions, Contract Management and Catalogue Management. In this document, the term e-Procurement is used interchangeably with e-Tendering from this point onwards. Most of the installations in India have implemented only the e-Tendering module. Rest of the components is yet to be used. Page | 10 b) Some of the installations are most flexible on critical aspects of tendering. For example, an option is provided to government officers to extend bid submission timeline after expiry of the due date and time for bid submission. Many of the e-Procurement solutions are designed to provide such flexibility to specific departments by way of configurations. Though a single instance of software is shared by multiple procurement entities, each procurement entity gets a different flavor (i.e. combination of functionality) of the system as per its preference. c) Policy to address system malfunction is not put in place for any of the installations assessed by the World Bank. Systems do malfunction – due to heavy work load or some hardware or software issues – in which case a decision on whether to extend bid submission timeline for the affected tenders is taken in an arbitrary manner. Many bidders would not have been able to participate in tenders due to system malfunction. Since a System malfunction policy is not put in place, the aggrieved bidders will not have a provision vis-à-vis they can rightfully claim extension of bid submission timeline. All the key issues identified during the assessment undertaken by the World Bank will be explained later in this report. A key area of concern is that most government agencies lack the expertise to independently evaluate e- Tendering systems that they use. Neither do they engage services of a qualified 3rd party agency to audit their installation on a periodic basis. They tend to work on the assumption that things are working fine, actively market uptake of e-Procurement and seek a complete and total shift towards e-Bid submission. If this status quo is left unchecked, there is bound to be audit issues on account of bad data governance or for adoption of inappropriate e-Tendering procedures. The World Bank on its part has issued “No Objection� to certain installations when certain basic tenets of transparency and security as defined in the MDB guidelines are adhered to. Also, the Bank has relied on the compliance statement to MDB’s e-Tendering guidelines provided by the assesse. From the Government of India’s view point, the assessment requirements ought to focus on the broader e-Procurement landscape as a whole and look to: a) Standardize key aspects in procurement such as bid advertisement, supplier identification, supplier registration, award notice, invoice, item codification, work experience certificate b) Develop a standard framework to facilitate exit management and transition from one e- Procurement application to another c) Integration of e-Procurement system with external systems such as: a. Banking – to receive and refund tender fee and Earnest Money Deposit (Bid Security) in electronic cash and Bank Guarantee formats b. Income Tax – PAN identity information for individuals and companies c. Ministry of Corporate Affairs (MCA) – to obtain financial statements of bidding companies d. Treasury – to obtain budget information and to pass on bill payment details e. Human Resource Management System (HRMS) – to obtain information about transfer of officers Page | 11 1.3 Report Objectives and Envisaged Outcomes Going forward, the World Bank is keen to work with Government of India (GoI) in development of a robust framework for assessment of the more broadly defined e-Procurement system. Two key components of the framework are: a) Guidelines for assessment of e-Procurement installations b) Institution required to assess e-Procurement installations at routine intervals (e.g. 6 months) to verify compliance to e-Procurement guidelines The Objectives of this report are: a) Explain in detail the key issues identified by the World Bank from the assessment of e- Procurement installations done till date in India b) Suggest certain refinements to the “Guidelines for Compliance to Quality Requirements of eProcurement systems� already prepared by STQC c) Broadly define the institutional framework required for assessment of e-Procurement systems Preparation of this report is an initial step taken by the Bank in working with GoI towards development of a robust framework for assessment of e-GP system. The Banks looks forward to working closely with GoI in achieving the following key Outcomes: a) An e-Procurement installation certified by STQC or a designated Government of India institution need not be assessed by the World Bank separately. A STQC or GoI certified e-Procurement installation can be used as such to process tenders in World Bank funded projects. b) A robust framework for assessment of e-Procurement installation is put in place by the GoI. The e-Procurement guidelines will continuously evolve in sync with developments in the e- Procurement space. Further, GoI institution will assess and certify e-Procurement installations in India at regular intervals c) The e-Procurement assessment guidelines developed by STQC will be evolved to focus on the broader e-Procurement landscape as a whole addressing standardization, exit management and integration requirements 1.4 Methodology This report is prepared based on the knowledge acquired from assessment of 21 e-Procurement installations in India. Critical issues identified by the Bank during this assessment are identified and explained herein. These issues are presented in the report under two categories as given below: a) Issues pertaining to issuance of “No Objection� by the World Bank a. Security b. Transparency c. Functionality d. Data governance e. Policy b) Important issues primarily from the view point of Government of India a. Functionality b. Policy c. Security d. Project Management e. Standardization Page | 12 f. System Architecture g. Legal Framework Figure 1: Overview of the Assessments Undertaken A total of 36 issues are identified and each issue is explained in detail in this report. The prevalence of the issues varies as given below: a) Few of the issues identified are specific to certain installations (e.g. confidentiality of bid documents –bidders participating in a tender can view bid documents of all bidders participating in the tender) b) Few others are specific to certain software (e.g. system requiring upload of bid documents as 1 MB files) and c) Some are prevalent across many installations (e.g. Anti-virus scan of uploaded documents and Recovery Point Objective (RPO) undefined) The lack of well-defined National e-GP policy is hurting the development of e-GP in India, as explained under the issues such as “De-duplicated vendor database� and “Lack of adoption of item codification standard�. The e-GP eco-system as a whole is evaluated and improper development of the eco-system is explained under issues such as “Weak Project Management Unit� and “Electronic Systems should Replace Manual Systems and Not Co-Exist�. The list of 36 issues is fairly exhaustive given the large sample size (i.e. 21 installations) from which they are drawn. In reality however, there could be issues which did not come to the attention of the Bank during its assessment. In other words, this list in exhaustive but need not be complete. Further, an issue does not always carry a negative connotation, instead can refer to a desired future state. The exact installation wherein an issue was found is not specified in this report because the intent of this report is not to show a particular installation in negative light. Instead, the intent is to inform the policy makers about the issues in existence. Page | 13 The assessment guidelines and framework existing in India to assess e-Procurement systems requires certain improvements, as explained under: a) Assess e-Procurement installation and not e-Procurement software b) Supply driven and not demand driven assessment c) Consolidate and classify the requirements in a certain logical sequence d) Make it as a check-list e) Be as specific as possible f) Visualize the larger picture g) Establish benchmarks h) Introduce competition by grading and ranking e-Procurement systems In addition to the above, a fitment analysis of MDB’s e-Tendering guideline with that of the STQC’s e- Procurement guideline is done to identify the specific MDB requirements not included in the STQC’s guidelines. The Bank seeks inclusion of these listed MDB requirements in the next version of STQC’s e- Procurement guideline. A Concept Model of e-Procurement assessment guideline is proposed herein to address all the 36 issues explained in this report. These issues are fit within the four layers of Quality and Security evaluation model of STQC: 1. Data 2. Application 3. Infrastructure and 4. Process In other words, the STQC’s four layer model is enriched with e-Procurement specific evaluation criteria drawn from the issues identified during the assessment. The assessment of e-Procurement installations done vis-à-vis the concept model will specifically verify compliance to the known issues. Indeed, the list of known issues will evolve with new assessment experiences. Consequently, the concept model of e- Procurement assessment guideline will evolve with time. A 1000 point grading system is defined, wherein each heading in the concept model is assigned certain weightage. The weightage allocation proposed in the concept model is indicative and the same has to be mutually agreed and approved by a designated government authority. The Concept model only identifies the topics to be assessed but not the detailed requirements underneath each topic and the metrics to be used to assess each requirement has to be defined. A Tabular format proposed to define the requirements in detail is illustrated in this report. It is understood that the assessment guideline will be complete only when all the requirements are detailed and the metrics well defined. This detailed e-Procurement assessment guideline will be submitted as a separate document. An overview of the institutional establishment required to assess e-Procurement installations for compliance to the assessment guidelines is explained in this report. Page | 14 1.5 Chapter Overview The e-Procurement installations assessed along with the solution details and its assessment status is listed in the next Chapter. The 36 issues identified by the World Bank from its experience in assessment of e-Procurement installations are explained in detail in Chapter Three. In the next Chapter, e- Procurement guidelines prepared by STQC are reviewed in detail and certain suggestions are provided to improve this STQC guideline. Further, the list of MDB’s e-Tendering requirement which do not find a reference in STQC guideline is listed. The Concept model of the proposed e-Procurement assessment guideline is explained and a snapshot view of the Tabular format to be used to detail the requirements is provided in Chapter Five. An overview of the institutional framework required for assessment of e- Procurement installations is provided in Chapter Six. Next, a six-step iterative procedure is proposed to operationalize the assessment of e-Procurement installations. The need for such an assessment framework and workability of the proposed solution in the Federal set-up are explained as concluding remarks in the last Chapter of this report. Page | 15 2 e-Procurement Installations Assessed by the World Bank in India The list of e-Procurement installations assessed by the World Bank in India as on May 2013 is provided in the Table below: S.no. Installation Details (Name of the Year Solution Status Bank funded Project uner which assessment was done) Karnataka (agency specific e-GP system Antares – “No Objection� given for the 1 implemented in Bangalore Water Supply 2008 Tenderwizard State-wide e-GP system & Sewerage Board) Madhya Pradesh (GoMP’s State wide e- Wipro - 2 2009 Assessment report submitted GP system) (MPWSRP) NexTenders Himachal Pradesh (Agency specific e-GP 3 system implemented in Satluj Jal Vidyut 2009 Tenderwizard / ITI Assessment report submitted Nigam) Karnataka (GoK’s State wide e-GP system) 4 2011 HP India Sales “No Objection� given (KSHIP) 5 Multi-state assessment for PMGSY-RRP-II 2011 NIC “No Objection� given “No Objection� given for the 6 Assam (ASRP & AACP) 2011 NIC State-wide e-GP system NHAI Lucknow-Muzaffarpur National 7 2011 Tenderwizard / ITI “No Objection� given Highway Project C1 India replaced 8 Andhra Pradesh, (APSRP) 2012 Assessment report submitted by Vayam Andhra Pradesh, (APMDP; NCRMP & 9 2012 NIC “No Objection� given APRWSSP) 10 Odisha (OCTMP; OSRP) 2012 NIC Assessment report submitted 11 Madhya Pradesh (MPWSRP; DRIP & HP-II) 2012 NIC “No Objection� given “No Objection� given for the 12 Rajasthan, (RRSMP) 2012 NIC State-wide e-GP system 13 Kerala, (KSRP) 2012 NIC Assessment report submitted 14 Maharashtra (MWSIP) 2012 SIFY - NexTenders Assessment report submitted 15 Gujarat, (GHSP) 2012 n(Code) Assessment report submitted Tenderwizard / 16 Bihar, (BKFRP) 2012 Assessment report submitted BELTRON “No Objection� given for the 17 Punjab (PRWSSP) 2012 Tenderwizard / ITI State-wide e-GP system 18 MORTH, (NHIIP) 2013 Tenderwizard / ITI “No Objection� given Puducherry, (Coastal Disaster Risk 19 2013 Tenderwizard/ ITI Assessment report submitted Reduction Project) Electronic Tender / 20 Power Grid Corporation, (PSIP) 2013 Assessment report submitted TCIL 21 Uttar Pradesh, (UP Roads Project) 2013 NIC Assessment report submitted Figure 2: List of e-Procurement Installations Assessed by the World Bank The installations for which “No Objection� is provided can be used to process tenders in World Bank funded projects subject to adherence of the following by the project implementation agency: a) Use of e-Procurement specific Standard Bid Document (SBD) prescribed by the Bank b) Configuration of tenders in the approved e-Procurement system as prescribed by the Bank Page | 16 In 11 out of the 21 assessments, Bank submitted the assessment report but the assessment is pending closure on account of: a) Assesse decided not to make the changes sought by the Bank or b) Assesse is working on the changes sought by the Bank c) Government agency sought to use a different e-Procurement installation It is to be noted that the Bank’s interpretation of MDB’s e-Tendering guidelines evolved with time and from the assessment experiences. To the extent possible, the Bank addressed concerns with the assessed systems by way of Standard Bid Document (SBD) clauses. Change in software is sought only in cases where installed version of the assessed software had certain critical issues. The assessment did not make a judgment on usability of e-Tendering platform and neither it ranked or qualified the assessed platform. Figure 3: e-GP Map of India as on May 31, 2013 Page | 17 3 Critical Issues Identified from Assessment of e-Procurement Installations in India Broadly, critical issues identified by the Bank from assessment of e-Procurement installations in India can be classified under: c) Issues pertaining to issuance of “No Objection� by the World Bank d) Important issues primarily from the view point of Government of India The primary focus of the assessment undertaken by the Bank is to verify whether the installation complies with the requirements laid down in MDB’s e-Tendering guidelines. The critical issues identified for issuance of “No Objection� are categorized under: a) Security b) Transparency c) Functionality d) Data governance e) Policy There are other areas which are critical especially from the view point of Government of India. The Bank typically identifies those issues in its assessment report only as a suggestion. In all, 36 have been identified as shown in the Figure below: Figure 4: Overview of the Issues Identified during the Assessment Page | 18 3.1 Broad Classification of e-Procurement Applications in India e-Procurement application software in India can be broadly classified under: a) Fully attachment based and b) Form based with provision to include file attachments All bid documents including financial bids are uploaded by bidders as file attachments in attachment based system. In such systems, commercial bid formats are defined in a locked Excel sheet and uploaded as file attachment during tender publication. Bidders responding to the tender are required to download the Excel sheet, fill out and upload the same in a specific slot provided in the bid submission module. Certain validations in-built in the application seek to verify whether the Excel file uploaded by bidders is the same as the file uploaded in the tender. The Excel sheet thus uploaded is encrypted during bid submission and subsequently decrypted during bid opening. A script in-built in the system will compare the decrypted excel sheets and prepare a comparative chart of commercial bids received in response to the tender. The attachment based application software is not designed to know about the items procured, procurement quantity, estimated cost, bid quotes received and the awarded price. In Form based application software, item description and quantity are captured online in a form and stored in the database of e-Procurement application. Bidders are required to fill out their quotes in online form. The software will allow bidders to submit their bids only when all mandatory fields in the online form are filled out. A comparative chart is auto-generated based on the rates keyed in by the bidders in online form. Information about the items procured, procurement quantity, estimated cost, bid quotes received and the awarded price is stored in the database. Thus, the application software has the capability to generate analytical reports on procurement spend. In both the software, detailed technical bid documents are uploaded by bidders as file attachments. These file attachments are downloaded after bid opening and evaluated by procurement officials just as it is done in manual tendering. 3.2 Issues Pertaining to Issuance of “No Objection� 3.2.1 Security a) Confidentiality of Bid Documents: As per Section 5.1 of MDB’s e-Tendering guidelines, “There shall be security arrangements to ensure confidentiality (i.e. protect privacy by allowing only authorized persons access to the content at the authorized time) … of bids/proposals in electronic format.� Refer Page 73 of STQC guidelines under the heading Data Protection: “Adequate and reasonable security practices and procedures are in place to protect confidentiality and integrity of the users data and credentials�. In many of the e-GP installations, bidders participating in a tender are provided unrestricted access to the entire set of technical bid documents submitted by competitors responding to the tender. The World Bank views this practice as breach of confidentiality since technical proposals are submitted in confidence to the Government and the proposals may contain certain proprietary information (e.g. design). Many in the vendor community would not even be aware that technical proposals uploaded by them in certain e-Tendering installations can be freely viewed and downloaded by their competition over the Internet. As per e-Procurement Application Service Providers (ASP), this functionality is Page | 19 provided in response to demand by government agencies. Implementation of this functionality is sought by Government agencies in an effort to enhance transparency in Government Procurement. As per Section 28 of the Draft Public Procurement Bill 2012: “(1) Notwithstanding anything contained in this Act or any other law for the time being in force, a procuring entity shall not disclose any information, if such disclosure, in its opinion, is likely to—… (c) affect the intellectual property rights or legitimate commercial interests of bidders or violate any pre-existing contractual obligations on confidentiality; (d) affect the legitimate commercial interests or the intellectual property rights or violate any pre- existing contractual obligations on confidentiality of the procuring entity…. (2) Except as otherwise provided in this Act, a procuring entity shall treat all communications with bidders related to the procurement process in such manner as to avoid their disclosure to competing bidders or to any other person not authorised to have access to such information.� Many e-Procurement installations are violating the provisions stated above. b) Original Encrypted Bid Document shall not be replaced by Decrypted Bid Document: As per 5.3 of MDB’s e-Tendering guideline: “Copies taken and decrypted for bid evaluation purposes shall not affect the integrity of the original record�. In certain e-Procurement installations, the encrypted bid submitted by bidders is replaced by the decrypted document after the Tender Opening Event. In such cases, integrity of the original record is entirely lost. If hash value of the bid (i.e. the item description and rate quoted) encrypted using the bidder’s private key (i.e. digital signature) is not kept stored separately, Government will not at all have the capability to prove/disprove veracity of the decrypted commercial quotes subsequent to Tender Opening. The STQC guideline is silent on this requirement. c) Message Definition in Digitally Signed and Submitted Bids: Ideally, the Government should get actively involved in defining the content to be digitally signed during bid submission. In practice however, definition of the message which is hashed and encrypted using bidder’s private key (i.e. digital signature) is not known. If an ASP generates a hash of the time stamp or any other content instead of the commercial quote submitted bidders, it will not be possible to address veracity claims submitted by bidders subsequent to commercial bid opening. Hence, it is important for the Government to get involved in defining the exact content (i.e. xml schema definition) to be digitally signed during bid submission. It will be best if the .xml schema definition could be defined as a standard to be adopted consistently by all e-Procurement ASP’s. d) Anti-virus Scan of Uploaded Documents: Refer section 5.2 of MDB’s e-Tendering guideline: “Bids/proposals submitted online shall be virus scanned by the Contracting Authority before being uploaded and accepted into the online bid box, and where this causes a bid to be rejected the bidder/consultant shall be notified immediately.� The MDB requirement on virus scanning can be implemented provided documents are scanned for virus in real-time and result of this scanning (i.e. for every uploaded document) is informed back to the e-Procurement application for it to take a decision on acceptance of the document. The real-time scanning of documents induces a slight delay in the file uploading activity and it adds to the implementation costs as well. Certain e-Procurement ASP’s implemented real-time scanning of uploaded documents for Anti-virus, which they later discontinued because the Anti-virus software in some cases wrongly recognized encrypted documents as virus signature and triggered rejection of those documents. When a Page | 20 document gets wrongly rejected as virus ridden, the concerned bidder would not be able to participate in the tender (for no fault of its own) and it causes operational inconvenience to the e- Procurement Application Service Provider (ASP). Anti-virus solutions tend to have their own definitions of virus signatures, due to which the Anti-virus solution installed in the e-Procurement server could recognize a document as virus ridden which the bidder’s machine could regard the same as a valid normal file. It is difficult to draft Standard Bid Document clauses to handle such scenarios. Few e-Procurement Application Service Providers claim to have implemented real-time virus scanning of documents. However, the Bank has not been able to verify authenticity / veracity of such claims during the assessment (i.e. upload a sample virus document to test how the system behaves). Typically, the e-Procurement ASP’s have taken some precautions to protect data in their servers from virus attack. The Bank in the recent years has given “No Objection� to certain e-Procurement installations on the condition that a workable solution to the Anti-virus requirement will be identified. e) Audit by 3rd Party Agency: Refer section 9.2 of MDB’s e-Tendering guideline: “There shall be no outstanding audit issues that represent material risk to the integrity or security of any project�. The Mission from World Bank typically spends about 3 days on-site to assess an e-Procurement installation. In this short time, the World Bank team can assess and judge the application software only from visual demonstration of the functionalities and only to the extent functionalities could be demonstrated within the short time. A key fundamental assumption of assessment by the World Bank is that integrity and security related issues will have been audited (i.e. by verification of key activities in the file / database level, where the data is stored) in detail by a 3rd party agency. Specifically, it is assumed that accuracy and correctness of declarations provided by the assesse such as those given below was verified by the 3rd party audit agency: (i) Commercial bids are stored in encrypted form in database or file system through-out till they are decrypted (ii) Audit trails are generated for all key events and a copy of the audit trails is securely maintained (iii) Data is backed up frequently and such back-ups are stored in a secured location and integrity of such back-ups is tested on a time to time basis (iv) Uploaded files are actually subject to real time virus scanning before they are accepted (v) The original encrypted content (either in database or in file system) is not replaced by decrypted content As such, the World Bank accepts the report by 3rd party audit agency and / or a declaration from the assesse stating that there aren’t audit issues as valid prima facie. The comfort factor while accepting this audit report would be much higher when it is known that the particular e-Procurement installation was verified for compliance to a set of certain well-laid down requirements. Also, as new versions of software are released and installed on a regular basis, it is important that e-Procurement installations are subject to a periodic audit (e.g. once in 6 months). Page | 21 3.2.2 Transparency a) Bid Proposals shall not be Encrypted Using Bidder’s Key: In certain e-Procurement installations, bids are encrypted using key supplied by the bidder. This approach is provided as “Guidance and Recommended Practices� in page 24 of STQC guidelines: “Encrypting the bid document first with public key of the bidder and then by the public key of tendering organization. The bid document may then be decrypted by the private key of the authorized official of tendering organization and then by the private key of bidder… The implementation of this system, however, would require physical presence of the bidder who encrypted the bid at the time of submission of bid. Preferably the person of bidding organization should be same who has signed the bid by digital signature. There are logistic issues with this approach.�. The bids encrypted using key supplied by bidders can be decrypted only when bidders are present either in person or online during the Tender Opening Event (ToE). However, a bidder for mala-fide or bona-fide reasons may choose not to provide the key required for bid opening. In which case buyer representatives designated to open the tender will be required to decide on whether to reject the delinquent / defaulting bidder. In the manual system, a bidder is disallowed from withdrawing its bid after expiry of the bid submission deadline. Though there is not a functional equivalent to supply of key in the manual system, a bidder not providing the key could be equated with a bidder withdrawing its bid after expiry of the bid submission deadline. In the manual system, bids received before expiry of bid submission deadline are fully under control of the buyer and buyer shall ensure safe custody of these bids, open and take them into consideration during bid evaluation. Bidders are not entitled to withdraw their proposal after expiry of bid submission timeline. Whereas when bids are encrypted using the bidder’s key, buyers are dependent on the bidders to provide the key required to open bids after bid expiry timeline. In a certain e-Procurement installations where encryption of bid proposal using bidder’s key is implemented, the system allows bidders to view other participants in a tender during the Tender Opening Event (i.e.) before the bidders are required to provide the key for decrypting proposals. After knowing about the competition, one or more bidders could decide not to provide the key with mala-fide intent. In which case, the e-Procurement system will have facilitated cartel formation amongst the bidders which have responded to a tender. b) Disallow Mandatory Submission of Originals before Bid Submission Deadline: In certain e- Procurement systems, bidders are asked to submit the original copy of EMD / Bid Security in the office of procurement entity before the bid submission deadline. When this requirement is implemented, procurement entity will know about the bidders participating in a tender just as it is in the manual system. The potential e-Procurement has to bring about transparency in bid submission (i.e. number of bidders participating in a tender will not be known until the tender opening event) cannot be realized when bidders are required to submit original documents before bid submission deadline. The Bank recommends provision of two working days gap between the “Bid submission deadline� and “Deadline for tender opening�. Bidders can submit physical copy of Bid Security and other original documents to procurement entity on or before the deadline for tender opening. c) Bidders Shall Submit Originals Before Tender Opening: There are instances where bidders are allowed to submit the original documents with “x� number of days after tender opening. A bidder may choose to collude with other bidders participating in the tender and choose not to submit the Page | 22 original documents within the prescribed deadline. In which case – just as it is with point “a� in this section, the e-Procurement system will have facilitated cartel formation amongst the bidders which have responded to a tender. The Bank recommends submission of original documents by bidders before start of the tender opening event. The bids for which original documents were not received shall be rejected during tender opening. Such clarity in policy will remove discretion in decision making amongst tender inviting authorities. 3.2.3 Functionality a) Application shall always Allow Issuance of Corrigendum before Bid Submission Timeline: In a certain attachment based e-Procurement application software, government user is not allowed to issue corrigendum (i.e. replace excel sheet) to commercial bid before tender closing time (i.e.) after one or more bidders submitted their bids in response to that tender. Ideally, the application software should allow authorized government user to issue a corrigendum anytime till tender closing time, regardless of whether bids are received or not for that tender. b) Upload of Bid Documents in Chunks of 1 MB File: A certain e-Procurement application software restricts bidders from uploading file attachments larger than 1 MB file size. Bidders are required to address this constraint by splitting their proposals into multiple documents of 1MB each using file splitting software and upload the split files individually one after another. Government officials will then have to download the split files and then merge them together to re-create the technical proposal. This restriction causes operational inconvenience to users of that e-Procurement software. c) Application shall Disallow Issuance of Corrigendum after Expiry of Bid Submission Deadline: In a certain e-Procurement application, the provision to issue corrigendum continues to be available even after expiry of bid submission deadline. This feature can be used to extend bid submission deadline or upload file attachments. d) Bid Submission Shall Not be done in Bits and Pieces: In a certain e-Procurement application, the action wherein bid gets transferred from supplier to buyer is not well defined. Instead, the application issues an acknowledgement for each mandatory bid part uploaded by supplier. A bid is considered for tender opening when all mandatory forms / documents are submitted on or before bid submission timeline. Suppliers can continue to upload additional documents as Annexures even after submission of the mandatory forms / document (i.e.) until the due date and time for bid submission. The Bank has recommended that the action wherein bid gets transferred from supplier to buyer has to be well defined. Further, the application should issue a single acknowledgement and modify the bid status to “submitted�. Post submission, any changes made to the bid will be marked as “modification� or “substitution� to the submitted bid. 3.2.4 Data Governance a) Recovery Point Objective (RPO) and Recovery Time Objective (RTO) Undefined: Government agencies in general could not commit on the RPO and RTO during system demonstration. The Bank is particularly concerned about RPO because it defines the maximum possible data loss. During assessment of a certain e-Procurement installation, the ASP and Government representatives Page | 23 discussed whose responsibility was it to take the back-up (i.e. if it is that of the ASP or the Government). Such a discussion illustrates lackadaisical approach adopted towards data governance. This issue is particularly concerning because government agencies nowadays seek 100% e-Bid submission and tenders worth thousands of crores of Rupees are handled through such systems. The Bank has recommended that a data back-up policy document (database and file contents) should be prepared and it should clearly specify the maximum possible data loss and such loss should be kept as minimal as possible (e.g. 5 minutes) b) Lack of Commitment on Record Keeping: Refer guideline 9.4 of MBD’s e-Tendering guideline: “EGP systems and information security shall ensure that secure records are kept of every process, procedure, transmission, receipt, transaction in terms of the content, executing individual and authorizations, time and date. These records shall be kept for at least two years after the closing date of the Loan Agreement and be made available for audit on request�. Many projects funded by the World Bank close after 5+ years. In which case, all procurement data including the audit trails as defined in guideline 9.4 shall be kept stored carefully and made available for audit upon request anytime in the next 7+ years. Presumably, Government of India will have similar norms requiring safe keep of procurement data for certain number of years at the minimum. The concern here is that the contract that government agencies have with e-Procurement ASP’s in the private sector is valid typically for a period of 5 years. After expiry of the contract, it is not known whether the e-Procurement application will be made available to read the data and extract information about “…process, procedure, transmission…�. Or else, special care should be taken to extract and transition not just the content but also other process related information as part of Exit Management of the ASP. Further, software has to be developed specifically to read the extracted content including the process related information. 3.2.5 Policy a) System Malfunction Procedure Undefined: e-Procurement system which facilitates a time bound activity will face unexpected down-time due to heavy work load or other unforeseen issues related to software or hardware (e.g. break-down of switch, network failure and power failure). If such break-down happens just 10 minutes before the standard tender closing time (e.g. 1700 hours), a significant number of bidders would not be able to submit their bids for no fault of their own. When such instances happen in practice, the recourse to be taken is being decided by government agencies as they see it fit. The Bank has recommended that a policy laid down by the Government should guide the procedure to be adopted to recover from such unexpected shut down. Else, decision making in such circumstances tends to be arbitrary causing conflict between procurement entities and bidder community. Though the Bank has had access to a System Malfunction procedure prepared by an ASP, none of the government agencies have enacted the procedure till date that the Bank is aware of. b) Discontinue Sale of Tender Documents: Refer 11.1 of MDB’s e-Tendering guidelines: “Prospective bidders shall have open and free access to all Specific Procurement Notices (SPNs) and bidding documents published in electronic format. No payment shall be required.�. In certain e-Procurement installations, bidders are required to pay for purchase of official copy of tender documents (i.e. the link to download tender documents including the commercial bid format will get enabled after the payment is made). Alternatively, in other systems only bidders participating in a tender are required Page | 24 to pay for tender document. In certain other systems, bidders are required to pay tender document fees and tender processing fees. Wherein, the former is paid to the concerned procurement entity that has published the tender and the latter is paid to the Project Management Unit (PMU) managing implementation of e-Procurement system. There is a clear need to standardize the procedure to be adopted across e-GP installations for publication / sale of tender documents. In a fully transparent system, bidders should download documents for free of charge and make payment if any required for bid submission through e-Payment. c) e-Payment to be Implemented: The Banking infrastructure in India has advanced significantly in the last 4 -5 years. There is at least one e-Procurement installation in India, where EMD / Bid security, tender document / processing fees and supplier registration fees are handled using e-Payment only. The vendor community can make e-Payment using multiple payment modes viz.: NEFT / RTGS, Over the Counter, Credit Card and Direct Debit. EMD is collected as electronic cash (instead of Demand Draft as in the manual system) and deposited in a single designated bank account of the State Government and also refunded online to the supplier’s bank account based on change in bid status in e-Procurement software. The maturity levels in implementation of e-Payment in other e- Procurement installations vary. For example, e-Procurement system is integrated with payment gateway only for collection of tender processing fees. In a different system, certain procurement entities require bidders to deposit EMD as electronic cash using NEFT payment mode in their respective bank accounts. The use of e-Payment to handle all e-Tendering related payments will make the system user friendly and transparent. Hence, an effort should be made to implement all modes of e-Payment consistently across e-Procurement installations in India. The Ministry of Finance, Government of India has been actively promoting the use of electronic Bank Guarantees in the recent months using the Structured Financial Messaging System (SFMS) developed by Institute for Development and Research in Banking Technology (IDRBT). It is understood that the Ministry of Finance is promoting the use of SFMS for online issuance of BG in an effort to prevent fraudulent / forged BG. The BG’s in the manual system are provided by an Issuing Bank directly to its customer, who in turn will submit a manual copy of the BG to Government. Under SFMS, specific formats for issuance of BG (MT760) electronically has already been developed, wherein BG / LC will be issued in electronic format from one Bank to another Bank (i.e.) issuing Bank to the Advising Bank. The Advising Bank will receive electronic copy of the BG/LC and hand over the same to their customer (i.e.) final recipient of the financial instrument, which is Government in this case. EMD amount in certain high value tenders can be in excess of 1 Crore Rupees (i.e. USD 250,000). In such cases, prospective bidders tend to seek submission of EMD in BG format. The various e- Payment modes now integrated with e-Procurement system allow transfer of electronic cash but not BG. Hence, government agencies now require bidders to upload scanned copy of BG in their electronic bid and submit the original BG on or before the bid opening timeline. The technical infrastructure required for issuance of BG in electronic format is already there. With a bit of effort, at least a pilot project could be initiated to receive electronic copy of BG from the Advising Bank to e-Procurement system online. The e-Procurement system can then provide assurance on authenticity of the BG submitted. Page | 25 d) Procurement of Digital Signature Certificate by Foreign Bidders: The World Bank actively encourages both National and International bidders to participate in Bank funded projects. Certain high value tenders are tagged as International Competitive Bidding (ICB) tenders specifically to attract participation from bidders across the World. The e-Procurement systems in India are Digital Signature Certificate (DSC) enabled, wherein a company representative seeking to participate in tenders advertised in the platform will require a DSC issued by a Certification Authority (CA) authorized by the Controller of Certification Authority (CCA). The CA’s require certain documentary evidence about identity of the applicant and for issuance of Class – III certificate they seek to personally verify the applicant’s identity. The certificates thus issued will be valid for a maximum period of 2 years. Refer 10.4 in MDB’s e-Tendering guidelines: “The certification process shall accept an electronic signature or a digital certification/signature issued by certifying authorities within the country of the bidder, or the process shall accept submission of online or offline documentation for certifying the authenticity of the bidder representative, accepting such documentation that can be obtained under commonly used procedures in the country of the bidder (for example, no notarization in consulate or embassy shall be required).� The Bank is informed that bidders can obtain DSC’s issued by Indian CA’s by establishing their identity with the Indian Embassy in their country of residence. However, the Bank is circumspect about practicality of the process for a foreign bidder to obtain DSC from his/her country of residence within a short period of time (e.g. 5 working days). The European Union initiated a project named Pan-European Public Procurement Online (PEPPOL) in 2008 with objective of “…developing and implementing the technology standards to align business processes for electronic procurement across all governments within Europe, aiming to expand market connectivity and interoperability between eProcurement communities.�. A key component in PEPPOL is development of the set-up required to validate eSignatures issued in any of the European Union (EU) member countries. It will be good if Government of India can take the initiative to inter- link the certification establishment in India with that of the PEPPOL infrastructure. 3.3 Important Issues from the View Point of Government of India 3.3.1 Functionality a) Adoption of Form Based System: Refer to section 3.1, wherein e-Procurement application software is broadly classified under: (i) Fully attachment based and (ii) Form based with provision to include file attachments Since bidders submit both technical and financial proposals as file attachments, the e-Procurement application cannot acquire intelligence about the items procured, procurement quantity, estimated cost, bid quotes received and the awarded price. To that extent, the attachment based system’s capability to generate analytical reports is limited. Also, approval workflows cannot be configured based on: (i) Estimated quote (ii) Percentage of deviation between estimated quote and award price (iii) Quantum of variation between the approved contract and work execution Page | 26 Many e-Procurement solutions in India are fully attachment based, which limits their potential as detailed above. The limitations of attachment based application will certainly show when e- Procurement application is integrated with external systems. For example, Treasury system will require information about approved bill in a readable format so the data can be taken for automated processing. This integration will not be effective if approved bill is provided as a file attachment. Also, the attachment design is not amenable to extend e-Procurement application by adding new functionality such as Schedule of Rate (SoR) management and Rate analysis. The fully attachment based application is however easy to use since government users and bidders can upload tender / submit bid just by uploading few documents. All key information are captured in online forms and stored in database in Form based systems. This data can be extracted and presented in many different formats as required. A full-fledged e- Procurement application to be used as a shared infrastructure by multiple government agencies should ideally be form based. Further, the use of online forms would facilitate standardization of forms across procurement entities. As a result of which, a standard set of forms will get developed over a period of time. It is worth investing in Research and Development (R&D) of a standard set of online forms for procurement of different types of works, goods and services. At least a couple of the key e-Procurement solutions widely used by Government agencies in India are fully attachment based. The ASPs owning these solutions would not be keen to introduce online forms, since such modification will require design level changes. It is for Government of India to decide whether to mandate online form based design for e-Procurement platforms. Such a mandate should ideally specify the key data fields to be necessarily captured in online form. The e- Procurement assessment undertaken should verify whether the data fields specified in the mandate are recorded in online forms. b) Serious Usability Issues: Many of the e-Procurement solutions in India are very user un-friendly. This unfriendliness is introduced primarily under the guise of security. In one system for example, government official has to decrypt commercial bids item-by-item for each bidder. If there are 10 items and 5 bidders, government user has to follow the decryption procedure 50 times. Users are required to act upon so many functions which they neither require nor understand and the outputs – some record created in the name of security in database the effectiveness of which no one has really verified – of which they neither sought nor understood. Consequently, usage of the system is restricted to one or two specialist resources in the procurement entity or even worse the Application Service Provider (ASP) designates a resource as “Hand-holding� support to upload tenders and download bid responses on behalf of the Government. Ideally, users should perform procurement activities (e.g. bid opening and evaluation approval) online by self in e-Procurement application just as they would in the manual system. In which case, use of electronic system will be all pervasive and a decision may be taken to fully replace manual procurement with e-Procurement. As implementation of e-Procurement is typically mandated, users – both government users and suppliers – are left with no option but to use the system regardless of the serious usability issues. In many installations, users are not even provided with a forum to express their concerns. Usability concern raised by users is typically viewed as an act towards subverting the use of e-Procurement, hence ignored. Page | 27 Poor quality systems implemented by enforcement of a mandate will not succeed and sustain in the long run. Hence, it is very important that usability of a system is evaluated and properly graded during assessment of e-Procurement systems. Such grading will act as a guide to government agencies during procurement of e-Procurement system. 3.3.2 Policy a) Sustained Uptake of e-Procurement: The uptake of e-Procurement is not sustained over a period of time in certain installations. The decision to mandate the use of e-Procurement for a category of tenders was subsequently reversed by making the use of e-Procurement optional. In such circumstances, it will take significant effort to bring the initiative back on track. It is suggested that Government of India should at least discourage such reversal viz.: (i) Hold back allocation of grants (if any) towards implementation of e-Procurement to the defaulting procurement agency (ii) Include “sustained uptake of e-Procurement� as a key metric in the e-Procurement assessment framework. The e-Procurement installations with roll back will be graded lesser and consequently ranked lower as against their peers. It is hoped that such grading and relative ranking of the e-Procurement installations will creation competition amongst government agencies in implementation of best practices. b) Electronic Systems should Replace Manual Systems and Not Co-Exist: Many government agencies in India process a significant percentage of procurement using e-Procurement, wherein bidders submit technical and financial proposals only in electronic format. Certain procurement related workflows such as those given below are handled online in few of the e-Procurement installations: (i) Administrative sanction (ii) Technical sanction (iii) Tender document approval (iv) Addendum / corrigendum approval (v) Pre-qualification / technical / commercial bid evaluation approval Users are typically required to digitally sign to authenticate key activities during online bid submission and approval workflows. The bids submitted online and approvals accorded online have due legal effect given the systems apparently put in place. Despite which, government agencies have the tendency to redundantly build a physical file with: (i) Printed copies of bid documents (ii) Write note sheets and (iii) Take the approvals yet again in the file Such duplication increases workload of government officials, causes printing piles of papers and the physical files occupy valuable real estate space. This duplication is done by government officials due to: (i) Fear of potential loss of electronic records and (ii) Possible non acceptance of electronic records by Accountant General (AG) The implementation of e-Procurement should make cause reduction in workload for Government officials and not be on the contrary. Hence, it is suggested that an effort is taken to: (i) Establish a robust data back-up and disaster recovery set-up Page | 28 (ii) Define the activities to be audit logged and ship a copy of the audit logs to government control on a period basis. Further, verify during e-Procurement assessment whether activities are correctly logged and that logs are shipped to government control as defined (iii) Get the necessary buy-in from AG about validity of electronic records and also create a provision in e-Procurement system for the AG to audit electronic records directly Ideally, a Government Order (GO) should be issued to clarify to all concerned that necessary precautions are taken to protect against potential loss of electronic records and that AG will audit procurement related activities solely based on electronic records. Hence, Government officials need not maintain physical copy of the documents in duplicate. The issuance of such a GO will imply that the system has reached a certain level of maturity. 3.3.3 Security a) e-Procurement Servers should be under Government Control: Few e-Procurement installations in India are operated on the “Cloud� model, wherein the application is hosted in servers located in a 3rd party Data Center. The e-Procurement requirement of a government agency is addressed by creating an instance of the software in the Cloud environment, which is accessible over the Internet. The cost of setting up e-Procurement will be significantly less when the application is hosted in a shared environment. Especially, the small and medium sized government agencies will benefit from the cloud initiative, since e-Procurement as a Service is available at an affordable cost. However, it is not advisable that the servers and database used for processing hundreds of crores of rupees worth procurement are completely outside Government’s control. Even the audit logs are not shipped at regular intervals to an infrastructure under Government control. The e-Procurement ASP’s are attempting to address security concerns fully at the application level instead of distributing security implementation across both application and infrastructure. Such restrictive focus on security has made the e-Procurement software quite cumbersome and user unfriendly. When the infrastructure is located in a Data Center under government control, the audit logs could be backed up and e-Procurement traffic will also get monitored. Such system administration work could be assigned to Data Center resources directly employed by the Government and not to e- Procurement ASP’s resources. The assessment of e-Procurement installation should verify: (i) Whether e-Procurement servers, database and audit logs are under Government control (ii) How well critical system administration tasks are distributed between ASP and Government resources 3.3.4 Project Management a) Weak Project Management Unit (PMU): The Project Management Unit (PMU) in place to manage implementation of an e-Procurement platform shall be manned with suitably qualified resources. In many of the e-Procurement systems assessed by the Bank, the PMU was found to be quite weak. This weakness has adversely impacted the quality of e-Procurement implementation. Where the PMU is weak, ASP has direct contact with the end user departments and configures the system as per department specific needs without due application of thought on whether the configurations Page | 29 sought by the departments adversely impact security and transparency. For example, ASP configured the system for symmetric key (system generated) encryption – instead of the standard asymmetric key encryption – since the user department did not procure encryption certificates. The e-Procurement installation tends to be quite robust in terms of policy, data governance, security, functionality and transparency where the PMU is strong. It will be good if e-Procurement assessment by Government of India evaluates the quality of PMU establishment set-up to monitor and manage implementation of e-Procurement system. b) Lack of Service Orientation: An e-Procurement system is a mission critical infrastructure and it should be maintained as per well-defined service level standards. This service level orientation could not be found in most of the e-Procurement installations. Service level requirements were undefined or ill-defined and or defined but not monitored. Due to which, there is lack of focus on maintenance of e-Procurement system as per certain prescribed standards. The project management staff observes in general that the system has had no major downtime and issues in the platform are fixed at the earliest. Such practice has to change and it is important that the installation is governed by well laid service levels, adherence of which shall be monitored by PMU on a periodic basis. Hence, it is suggested that service level orientation system is specified as a criterion for assessment of e- Procurement installations. Benchmark for certain Service Level Agreement (SLA) criteria such as “System uptime� (e.g. 99.5% availability) and “Resolution of critical faults� (< 6 hours) could be set in the assessment framework. The extent of adherence to the various SLA criteria for the prescribed time period (e.g. month-on-month for the last 6 months) could be verified during assessment of e- Procurement installation. 3.3.5 Standardization a) Lack of e-Procurement Specific Standard Bid Document Clauses: The conditions for bid process management have got standardized based on decades of experiential knowledge. Many of these clauses will now have to be revised to fit the e-Procurement process. For example, the bid submission process in the manual system as defined below will be inapplicable for electronic bid submission: “THE BIDDER SHALL SUBMIT A SEALED COVER CONSISTING OF TWO (2) COPIES OF ALL THE BID DOCUMENTS. SEALED PROPOSALS WILL BE RECEIVED AT THE FRONT DESK OF THE <> BY 17: OO HRS ON 20th MARCH, YYYY, TO THE ATTENTION OF << DESIGNATED REPRESENTATIVE>>.� A clause corresponding to this requirement is “Bidders shall submit their response to this tender only electronically vide the e-Procurement system of <>: http://www.eproc.state.gov.in Procurement entity shall not accept manual submission of bids from the bidders. Bidders are informed to get acquainted with the bid submission process in e- Procurement system of the State Government of <> by contacting <>. <> conducts training sessions for prospective bidders at regular intervals. Refer to http://www.eproc.state.gov.in for further details.� e-Procurement specific standard bid clauses covering different aspects of e-Procurement have to be drafted and the same included in bid documents. The process of including e-Procurement installation specific Standard Bid Document (SBD) clauses in tender documents was not institutionalized in many of the e-Procurement installations assessed by the Bank. When there is an issue with e-Procurement system in relation to a tender, the tender document will not provide guidance on the process to be adopted to resolve the issue. Instead, a Page | 30 solution will have to be decided on a case to case basis often at the displeasure of one or more of the stakeholders. The assessment could seek to verify whether a standard set of SBD clauses are inserted in a sample of 50 tenders selected randomly from the e-Procurement installation. b) De-duplicated Vendor Database: In most of the e-Procurement systems, the process for registering a vendor in e-Procurement platform does not seek to authenticate, validate and confirm identity of the vendor. Due to which, one vendor can create multiple identities in a single e-Procurement platform. Implementation of e-Procurement system provides a unique opportunity to develop a single master database of vendors serving various procurement entities in the Government. If a standardized approach could be adopted for vendor registration and for unique identification of vendors across multiple e-Procurement systems, the basis for development of a Master database of suppliers serving the Government will be established. By this, Government agencies can generate intuitive analytical reports about supplier performance not just in one single e-Procurement installation but across multiple e-Procurement installations. The Department of Income Tax issues PAN number to uniquely identify an Individual or a Company. Government has the option to query the Income Tax database to know about identity details pertaining to a PAN number. The e-Procurement assessment guidelines could be modified such that: (i) Submission of PAN information is made mandatory for vendor registration (ii) e-Procurement Project Management Unit shall adopt the following process to authenticate the identity of an applicant seeking to register in e-Procurement platform: o Query the Income Tax database to find out if the Company / Individual name provided for vendor registration matches with the name corresponding the PAN in Income Tax database o Obtain a Power of Attorney / Affidavit from the entity seeking registration declaring that he/she is an authorized representative of the legal entity seeking registration in the e-Procurement platform o Verify whether the person name provided in the Power of Attorney / Affidavit matches with the Common Name specified in the Digital Signature Certificate (DSC) used by the application A separate process has to be developed to authenticate and register vendors from foreign countries. e-Procurement systems will add the Contract Management functionality in the years to come, wherein Award of Contract and supplier performance information will be captured online. A protocol could be established for online sharing of experience certificates across multiple e- Procurement systems. The development of a Master database of suppliers is an essential pre- requisite for implementing this vision. A pictorial view about the National Database on Supplier’s Performance taken from an ADB case study on e-Procurement in India is given below: Page | 31 National Database on Supplier’s Performance Evaluates Bidder Performance Downloads performance Government Officer Details of supplier from National Database E-GP System 2 National Database of Supplier’s Performance Refers to National Database on Participates in tender in Contractor Performance to Different e-GP platform Cite Past Experiences Performance of Supplier Updated Supplier Executes E-GP System 1 work Verifies Work Updates e-GP System Government Officer Figure 5: National Database on Supplier's Performance The process of registering vendors ought to be standardized but implemented in a de-centralized manner (i.e. specific to e-Procurement installations). The assessment undertaken should verify whether the standardized process of supplier registration is correctly adopted in various e- Procurement installations. c) Lack of Adoption of Item Codification Standard: Although a significant percentage of tenders are published online in various e-Procurement installations, a mechanism is not put in place so far to codify the procurement using a standard item code classification scheme. The European Union (EU) has adopted a standard titled “Common Procurement Vocabulary� (CPV) to codify all procurement advertised in the Tenders Electronic Daily (TED). By this, EU-wide reports can be generated on the quantum of money spent by Government in procurement of a certain category of works, goods or services. For example, Government can know about the quantum of work awarded for construction of Bridges across India in the last fiscal year. Also, a query can be executed about the extent of competition (i.e. average number of bidders) and about the margins (whether higher or lower) in a certain area such as “Bridge Construction and Repair Service� (item code: 72141107) or “Drainage System Construction Service� (item code: 72141205). Further, vendors can specify Segment / Family / Class / Commodity of procurement for which they seek to know about the opportunities advertised across India. As on date, a standard mechanism for assignment of item code does not exist. Item code is assigned by the user publishing a tender or as per department specific norms or by the e-Procurement Page | 32 system. Most certainly, there is a need for adoption of item code classification standard across India. UNSPSC (United Nations Standard Products and Services Code) is a reputed global item code classification master which Government of India may choose to adopt as a standard. Trying to create a standard afresh is tantamount to re-invention of the wheel. d) Central Public Procurement Portal (CPPP) as One Stop Shop: The Government of India has already developed the Central Public Procurement Portal to facilitate publication of “…Tender Enquiries, Corrigendum and Award of Contract details� by all Central Government Organizations. This portal is designed “…to provide a single point access to the information on procurements made across various central government organizations.�. It is suggested that this initiative is extended to cover all government organizations in India and not just the Central Government Organizations. The portal already has a provision to publish web-link of other e-Procurement web sites. The tenders published in CPPP can be searched by “Organization name� and / or “Product Category�. A published tender is tagged with a product category selected from a set of about 75 pre-defined product categories. Bid award details published in CPPP identify the selected bidder by “Name� in a non-standardized text format (e.g. “Sudha associates�). The same “Sudha Associates� could have been the selected bidder for a different tender and the award details published in the CPPP, but still the CPPP portal cannot firmly identify that the same “Sudha Associates� got selected for the two different tenders. Instead of publishing only the web links, the following information may be gotten from all e- Procurement web sites (including Non Central Government agencies) in a readable format and published in the CPPP: (i) Tender enquiries (ii) Corrigendum and (iii) Award of Contract Such implementation will propel CPPP as a single source of information for all Government tenders in India. As non-Central government organizations cannot be mandated to publish tender information in CPPP due to the Federal structure, it is suggested that an effort is made to induce them to do so. Specifically, e-Procurement installations could be assessed to verify and evaluate the extent to which certain key tender related information are uploaded in CPPP. The installations which do not publish information in CPPP could be graded lesser on that account. Further, it is recommended that tenders published in CPPP are tagged with item code from a global and well established hierarchical item code set such as the UNSPSC. Such tagging will enable generation of rich analytical reports on procurement spend at the National level. The European Union has already established the Portal “Tenders Electronic Daily� (http://ted.europa.eu/), similar to the CPPP, wherein it is mandatory to publish procurement notices for all tenders exceeding certain threshold values (e.g. 5 Million EUR for Public works and 200,000 EUR for service contracts). Each advertised procurement notice shall be tagged with an appropriate item code from a detailed hierarchical (4 layers) codification standard “Common Procurement Vocabulary� (CPV). Interested suppliers can register in TED portal and subscribe for e-mail alerts for tenders published under a certain category or sub-category as illustrated below:  03000000 - Agricultural, farming, fishing, forestry and related products o 03100000 - Agricultural and horticultural products Page | 33  03110000 - Crops, products of market gardening and horticulture  03111000 – Seeds o 03111100 - Soya beans As all tenders published in TED are codified using CPV, it is possible to generate detailed analytical reports (i.e. slice and dice, drill down and roll up) on procurement spend at the European Union (EU) level. A system needs to be put in place for assignment of a unique National Identity for suppliers. Wherein, each supplier will be assigned a unique ID specific to an e-Procurement (e-GP ID) system and this e-GP ID of the supplier will be mapped with the National ID assigned to the Supplier. A supplier can thus have multiple e-GP ID’s, each of which will be mapped to the same National ID (NID). The request for generation of NID for a supplier has to necessarily come from an authorized e- Procurement system with the following key fields: (i) PAN reference of the supplier (ii) e-GP system seeking the NID reference (iii) Identity reference issued by the e-GP system seeking the NID reference The NID system will verify whether the given PAN is already assigned NID (i.e.) possibly in response to a request from a different e-GP system. As per the verification, the NID system will take a decision on whether to: (i) Respond with the already generated NID reference of the supplier or (ii) Generate a new NID reference and respond to the NID request submitted by the concerned e-Procurement system The NID thus received in response will be mapped with the e-GP ID in both the concerned e- Procurement system and NID system. The bid award details submitted online by e-Procurement systems shall specify NID of the selected supplier(s). Thereby, rich insights can be obtained on: (i) Extent of competition in a certain item code (ii) The average premium / discount rates quoted by a certain supplier (iii) Number of unique suppliers servicing government (or a State Government or a department) in a certain item code (iv) Ratio of unique selected suppliers to tenders within a certain item code. A ratio of 1.0 would mean that as many suppliers were selected as the number of tenders. For example, selection of 100 different suppliers in 100 tenders implies that competition for tenders in this item code is the highest. Whereas a ratio of 0.47 implies the competition is not as good and there is room for improvement. The sample procurement analytics report given in this section is prepared based on analysis of procurement spend data obtained from an e-Procurement installation in India. The Government may decide to publish such reports to enhance completion for tenders under a certain item code in a certain geographical area. For example, “Bridge Construction and Repair Service� (item code 72141107) in State X may attract lesser number of bidders (i.e. 1.64 per tender) and demand an average premium of 29.06%. With publication of this statistics: (i) Suppliers in a related area may choose to participate in bridge construction tenders (ii) Government may choose to dilute some of the evaluation criteria to attract suppliers from related work areas and thereby enhance competition in bridge construction Page | 34 (iii) Bridge construction suppliers operating in other States may choose to bid for tenders in State X, as they stand better chance of winning. A diagrammatic view of the approach suggested for establishing a National database of registered suppliers is provided in this section. This picture is taken from a report by the Asian Development on e-Procurement in India. Figure 6: Snippet from a Procurement Analytics Report In this regard, the assessment of e-Procurement system should seek to verify whether: (i) Tender publication notice and award details are provided in a standardized machine readable format to CPPP o Since award of contract is typically done manually, procurement officials typically do not update bid award details in the e-Procurement application in India. An approach has to be developed for enforcing the publication of bid award details (ii) Identity of a supplier is verified before the supplier is provided with a e-GP ID (i.e.) as defined in the section “vendor registration� Page | 35 National Database of Registered Suppliers Supplier’s National id E-GP id E-GP id National id E-GP id National id National id E-GP System 1 E-GP System 2 E-GP System n Figure 7: National Database of Registered Suppliers e) Need for Mechanism to Address Vendor Lock-in and Exit Management: Many e-Procurement systems are implemented in Private Public Partnership (PPP) mode, wherein the ASP is from the Private Sector. The contract with Private Partner typically lasts for a period of 5 years on Built-Own- Operate (BOO) model with Transfer as an option. Government can choose to renew the contract after its expiry as per provisions laid down in the agreement. In 5 years, the technology tends to get outdated and invariably Government needs to a take a decision on whether to continue with the same solution or opt for a different solution instead. Where the Transfer option is available, Government could decide to obtain perpetual license for use of the software and even get access to Source code of the software. Government will then be responsible to manage the source case and ensure secrecy of the source code and may even have to indemnify the ASP for leakage of source code shared by the ASP to Government in confidence. Alternatively, Government could decide to go for a different solution, wherein the data pertaining to “…every process, procedure, transmission, receipt, transaction in terms of the content, executing individual and authorizations, time and date� will have to be backed up as accurately as possible. Then, a system has to be put in place to read and access the transferred data as accurately as possible. It is recommended that Government of India develops a well-defined standard (i.e. what all needs to be backed up and how such backed up information have to be stored) for backup of data during Exit Management and for also reading the backed up data. Many e-Procurement installations are now in operation in BOO model, wherein the contract term will expire at regular intervals in the years to come. In which case, data from the existing e- Procurement system has to be backed up correctly and the newly set-up solution has to read and display the backed up data quite accurately. If the process of back-up and restoration is not standardized and left open to the government establishments managing the various e-Procurement Page | 36 installations, there will be reinvention of the wheel and learning from one instance of exit management will not get transferred to another. Hence, it is recommended that Government of India develops a standard for data back-up and restoration during Exit Management. The assessment of e-Procurement systems should ideally verify whether the data back up and restoration has happened as per the standards laid down. The various government agencies would certainly appreciate and welcome the standardization effort as they can rely on an established standard instead of inventing it afresh. The development of this standard will allow Government agencies to transition from one e-Procurement ASP to another with less inhibition. Thus, the fear of lock-in will get addressed as well. f) Standardize the Encryption Methodology: The encryption requirement of MDB is defined in section 5.3 of its guidelines as under “… At no time shall bids/proposals be in unencrypted format. Copies taken and decrypted for bid evaluation purposes shall not affect the integrity of the original record.�. From the MDB’s viewpoint, it is adequate if bids are kept in encrypted format and a copy of the original encrypted bid is available after decryption of the same has happened. Few of the e-Procurement ASPs operating in the Indian market are actively trying to differentiate themselves by the encryption technique implemented in their solution. Consequently, many different encryption approaches exist, developed primarily through combination of: (i) Symmetric / Asymmetric key and (ii) Key provided by Government / Supplier / Both The encryption techniques invented by the ASP’s tends to be technology focused and in some cases do not comply with the bid submission norms evolved over several decades. Refer to the sub- section “Transparency� and the heading “Encryption of Bid Proposals Using Bidder’s Key� for a detailed analysis about functional non-compliance of one encryption technique and how such non- compliance can effectively facilitate cartel formation amongst the sub-set of bidders participating in a tender; thereby causing transparency issues. A key assumption underlying the development of the different encryption techniques is that bidders, government officials and ASPs would connive with one another. The encryption techniques proposed are designed to prevent / disallow this connivance amongst the key stakeholders. If all the stakeholders decided to connive with one another, the strongest possible security implementation cannot prevent cartelization and sub-optimal outcome from the tendering process. An e- Procurement system or any system for that matter cannot be completely secure. It is a well-established fact that adding layers and layers of security will make the system unfriendly causing inconvenience to the end users. Few of the e-Procurement systems in India are already quite unfriendly. For example, in certain systems bidders shall necessarily3 be present during bid opening for providing the key required to unlock their respective bids. There is bound to be confusion when one or more of the bidders are not present during bid opening or when they have misplaced the key. In another case, online bid submission is a two stage activity wherein bidder has to necessarily perform two different actions in a time bound manner (i.e. two deadlines) to submit their bids4. 3 In the manual system, presence of a bidder during bid opening is not mandatory 4 In the manual system, bid submission is a one-time activity and it should be done before a certain deadline. Page | 37 The STQC guidelines do not recommend a specific encryption technique. Instead, it provides high level guidance: “If the e�procurement system uses PKI for bid�encryption, it has to satisfactorily address the above issues and consequent concerns … through suitable functionality built into the e� procurement application.…it is the responsibility of the individual vendors to design and develop their applications in a manner that addresses the outlined concerns…�. The status quo as explained above is not a healthy situation. It is recommended that Government of India defines a standard approach for encryption of bids and prescribes adoption of that approach across all e-Procurement installations. The e-Procurement assessment could evaluate whether the prescribed encryption technique is implemented as per the norms laid down by the Government of India. 3.3.6 System Architecture a) Well Define Functional Scope of e-Procurement System: e-Tendering is only one of the modules of e-Procurement system. A full-fledged e-Procurement system is more comprehensive and has the following additional modules: (i) Indent Management (ii) Work estimate Management (iii) e-Auction (iv) Contract Management (v) Catalogue Management A few government agencies use e-Auction for sale of scrap items, forest produce, licenses and land parcels. The indent management / work estimate management modules are used for approvals preceding tender publication in few of the installations. There has not been much success reported in implementation of an integrated e-Procurement system including the use of post tendering components (i.e.) contract & catalogue management modules. Most e-Procurement ASPs claim to have already developed full-fledged e-Procurement solution. But still, the modules other than e-Tendering have not been used. Technically, in installations where e- Tendering is widely adopted, it should be relatively easier to implement the remaining modules due to the following: (i) The requisite IT infrastructure and network connectivity to use e-Procurement will have been established (ii) Both suppliers and government users will have Digital Signature Certificate and would have got accustomed to the e-Procurement user interface (iii) Project Management establishment (i.e. including nodal officers) will be in place It could be that lethargy has set in after pushing the uptake of e-Tendering and a fresh initiative is required to encourage adoption of the remaining modules. The e-Procurement assessment should evaluate the extent of usage of all the modules and suggest a target timeline for implementation of end to end e-Procurement system. A matrix could be developed to rank and position e-Procurement installations across India. Thereby an element of completion would get introduced and government agencies will tend to compete and seek to move up the ladder. An example of such a matrix taken from an ADB report is provided in this section as an illustration: Page | 38 e-Governance initiatives are being taken up in domains related to Procurement such as Treasury and budgeting. The functionality in those initiatives could cover certain procurement related aspects such as work estimate management and contract management. For example, an integrated treasury system could include the following functionality typically implemented in the Contract and Catalogue management modules: (i) Bill preparation (ii) Payment approval workflows (iii) Maintenance of bank account information of vendors required for e-Payment Defining the functional coverage (boundary) of e-Procurement in relation to other systems is a key pre-requisite for standardized definition of e-Procurement Enterprise Architecture. Such definition by design will assign accountability for development and implementation of procurement related functional components to a certain designated agency. Also, clarity in functional boundary will prevent Government agencies from making redundant investments in development of the same functionality in two or more different e-Governance projects. Malaysia 2000 1,1 Philippines 2000 8,2 High Thailand 7,3 2007 Geographical Coverage 2010 India 5,4 Indonesia 4,5 Georgia 2008 2009 1,6 Vietnam 2010 9,7 Nepal Low 6,8 Bangladesh 2010 2011 Afghanistan 3,9 2011 11,10 Uzbekistan 10,11 2011 Low High Functional Coverage Figure 8: e-GP Position Map Taken from a Survey Conducted by the ADB b) Need to Define e-Procurement Enterprise Architecture: There in an obvious lack of clarity in the Enterprise Architecture (i.e. how many installations there should be in India) for e-Procurement, which has translated into co-existence of multiple systems within a geographical area (i.e. a State or even a procurement entity). Development of a full-fledged e-Procurement system requires significant planning and investments and it takes many years of sustained effort for the system to Page | 39 reach a certain level of maturity. More importantly, significant efforts would have been made towards ‘change management’ viz. training, workshops and grievance redressal. Hence, it is important that Vision for e-Procurement is laid down upfront with good clarity and all the efforts are taken for achieving this Vision. The fact that multiple systems operate in a State or even within a procurement entity illustrates the lack of Vision for e-Procurement. There are instances where the progress made in implementation of e-Procurement system is rolled back due to poor planning. Given that setting up a strong PMU and conduct of third party audit is expensive, the limited amount of available resources is distributed to manage multiple systems within an Enterprise (i.e.) instead of investing in development of a single robust platform. Consequently, the systems implemented typically are not of good quality and vendor community will have no option but to work with multiple systems to pursue their business objectives. In the long run, Government will be forced to invest effort in managing the cumbersome Exit Management and Transition Management for each of the many e-Procurement systems. Further, Government will have difficulties in converging the procurement spend data across multiple e-Procurement systems. To that extent, generating enterprise level (e.g. State Government) analytical reports on procurement spend will be difficult. This lack of Vision and a Policy establishment is hurting the development of e-Procurement in India. Just as it is in software development, it is easier and less expensive to address architectural problems during the design phase instead of the implementation phase. The European Union (EU) for example has established a robust e-Procurement policy establishment and is going about development of e-Procurement architecture in a very systematic manner. It is important that at least now an initiative is taken to define an Enterprise and the number of e- Procurement installations required to handle Government Procurement in India. Then, a steadfast effort should be taken focused on realizing this Vision. Further, a strong e-Procurement Policy establishment has to be set-up to catalyze the uptake of e-Procurement and actively provide directions for implementation of e-Procurement in India. The cost of establishing and operating an e-Procurement policy institution will be insignificant when compared to the tens of thousands of crores (Billions of USD) worth procurement processed year-on-year using e-Procurement platforms. Assessment of e-Procurement taken should not be restricted to specific installations, instead it should also evaluate whether e-Procurement systems are developed in line with the envisioned Enterprise Architecture. c) Integration of e-Procurement with External Systems: Both government and private sector agencies are continuously investing in implementation of Information Technology to improve their operations. The Banking system for example has developed significantly in recent years with implementation of Core Banking Systems (CBS) and wide-spread adoption of Nation-wide payment systems such as National Electronic Fund Transfer (NEFT) / Real Time Gross Settlement (RTGS). Banks now have new service offerings such as “Virtual account number5� by leveraging on NEFT 5 Few of the Banks in India have devised an innovative solution in the last couple of years to handle reconciliation problems in e-Payment. In this solution, a virtual account number is created for each transaction. A bidder seeking to remit EMD / bid security using NEFT/RTGS will print a challan in the e-Procurement system, wherein a (virtual) unique bank account number will be printed along with the IFSC code, amount and other information required to execute the NEFT/RTGS transaction. The sender bank will send money to the virtual account number specified in Page | 40 related developments in the Banking sector. Similarly, Income Tax department now provides “Bulk PAN Query� service to verify identity of PAN holders. Integration of e-Procurement system with external systems such as the Income Tax, Banking, Budgeting and Human Resource Management System (HRMS) is recommended specifically to: (i) Improve end user experience. For example: o EMD / Bid security is both received and refunded online using the NEFT infrastructure and wrapper services provided by Banks. Bidder need not visit government office to the collect the EMD paid by way of demand draft (ii) Better achieve the e-Procurement project objectives. For example: o Development of a robust supplier database wherein identity of each supplier is verified and confirmed from Income Tax database using Bulk PAN query during supplier registration o Provide authenticated financial statements (e.g. annual turnover) pertaining to a bidder downloaded directly from the Ministry of Corporate Affairs (MCA) o Effect transfer of government officers in e-Procurement system as per execution of transfer registered in Human Resource Management System (HRMS) o Enhanced efficiency in payment processing achieved by online exchange of approved bill details from e-Procurement system to the Treasury system o Authentic Bank Guarantees received online through integration with the Banking system Just as it is with development of standards for data back-up during Exit Management, efforts should be made to define standards for integration between e-Procurement system and external systems (e.g. Income Tax, MCA and Banks). The development of such standards will help simplify the interlinking of e-Procurement with external systems. The extent to which external systems are integrated should be evaluated during assessment of e- Procurement installations. Further, a plan should be developed at the National level to at least pilot the envisaged external integrations within a certain prescribed timelines. 3.3.7 Legal Framework a) Need for e-Procurement Rules: Many e-Procurement installations in India have been in operation in excess of 5 years. Despite being in operation for such long time, the installations operate without being backed by well-defined set of e-Procurement rules. The rules should: (i) Provide clarity on roles and responsibilities of key stakeholders involved (ii) Detail the course of action to be taken during exceptional circumstances (e.g. unexpected shut down of the system) (iii) Explain the manner in which key procurement related activities (e.g. tender publication, EMD payment, tender opening and appeal) will be performed in e-Procurement system the challan. A middleware developed by the recipient bank will parse the virtual account number to identify the real account to which the money belongs and also identify the purpose for which the money is received basis the virtual account number. Page | 41 Ideally, the rules should be approved by an authorized authority and published online for view by all parties involved. This drafting and publication of rules is not a one-time exercise, instead the rules should be revisited continuously and evolved based on real life experiences. Over a period of time, the maturity of an e-Procurement installation can be gauged by the level of clarity in the rules governing the installation. In a situation where there is lack of rules, end users are at liberty to define and interpret e- Procurement related processes any which way they think is appropriate. For example: (i) Procurement entity may seek submission of original documents before bid submission deadline or seek a week after the bid opening. Either of the two processes has drawbacks as explained earlier in this section (ii) A government agency may require receipt of e-Payment of tender processing fees in government account before the bid submission deadline for a bid to be considered for evaluation. Alternatively, certain systems will allow a bidder to submit a bid only after online reconciliation (i.e. confirmed receipt) of tender processing fee. (iii) A bidder may get unfairly rejected because the bidder has not loaded certain documentary evidence in the specified slot provided in e-Procurement system. But the document however is submitted as part of the bid in a different slot. Those defending the status quo will tend to argue that the lack of rules has not adversely affected the uptake of e-Procurement and that thousands of tenders are processed on a regular basis without any issues. Given which, they ask why there is a need for rules at all. A key point to note in this regard is that the issues mostly do not get reported primarily because the aggrieved bidders will want to maintain good relationship with the procurement entity. Hence, the lack of reported issues does not imply that there are not issues at all. Given the sample size (i.e. large number of tenders) anything which could go wrong will have gone wrong. Few adversely affected bidders may choose to submit an appeal as per the tender guidelines at first and then seek intervention from the Court of Law. In which case, the Judiciary will require the Government to submit the rules governing implementation of e-Procurement. At that instance, the Court will tend to chide the government for processing thousands of crores worth procurement by e-Procurement without the rules required to administer the platform. Hence, it is recommended that e-Procurement rules are drafted to govern each of the e-Procurement installations at the very earliest. A draft of the rules could be prepared at the National level and shared with the various procurement agencies for them to enact and adopt. The e-Procurement assessment could seek to verify whether an e-Procurement installation is governed by e-Procurement rules and also evaluate completeness of rules governing the installation. b) Reference to e-Procurement in the Draft Public Procurement Bill 2012: e-Procurement is referred to as a “Method of Procurement� in Section 29 (4) of the Bill wherein it is stated that “The Central Government may make rules relating to electronic procurement and may, by notification, declare adoption of electronic procurement as compulsory for different stages and types of procurement, and on such declaration, every requirement for written communication under this Act shall be deemed to have been satisfied if it were done by electronic means.� Page | 42 The Central Public Procurement Portal (CPPP) is envisaged as a medium “…accessible to the public for posting and exhibiting matters relating to public procurement� in the Draft Public Procurement Bill 2012. Specifically, publication of the following information in CPPP is envisaged in the Bill: (i) Procurement plan – section 7 (4) (ii) Information about selection of registered bidders – section 14 (5) (iii) Pre-bid clarifications – section 18 (4) (iv) Exclusion of bids – section 22 (4) (v) Award of contract – section 25 (3) (vi) Invitation for Open Competitive Bidding and Limited Competitive Bidding – Sections 30 (5) and 31 (b) Section 56 (2) (x) of the bill empowers Central Government to make rules for electronic Procurement. Refer to the Annexure for details about e-Procurement related sections in the Draft Public Procurement Bill 2012. The following two subjects would come into focus if direct reference could be provided in the bill: (i) Adoption of a unified item code classification standard by all government agencies under purview of this bill (ii) Assignment of National Identity Reference (NID) to Bidders If not direct reference in the Bill, these two requirements should find mention in the rules for e- Procurement to be drafted subsequently. Page | 43 4 Review of STQC’s e-Procurement Guidelines The Government of India (GoI) set-up “Standardization, Quality and Testing Certification� (STQC) as an attached office to the Department of Electronics and Information Technology (DeitY) to provide quality assurance services in the area of Information Technology (IT) and Electronics. STQC developed guidelines to verify “…compliance to quality requirements of eProcurement systems� and specifically to assess the essential quality characteristics of e-Procurement system covering Security, Transparency & Functionality. The latest version of this guideline is dated 05th of August 2011. Preparation of this guideline is a good initiative by STQC, which demonstrates that there is a certain level of understanding in Government about the need to assess and verify “…Quality and Security of an e�Procurement system�. The introduction of this guideline has certainly created awareness amongst the key stakeholders that e-Procurement application / system (read installation) has to meet certain standards. Government agencies regard a STQC certified e-Procurement system as secure and transparent. The Bank’s primary objective for preparation of this report is to synchronize the e-Procurement assessment procedure adopted by the Bank with that of the STQC, so as to achieve the following outcome: “An e-Procurement system certified by STQC or a designated Government of India institution need not be assessed by the World Bank separately. A STQC or GoI certified e-Procurement system can be used as such to process tenders in World Bank funded projects.� The e-Procurement applications deployed in many of the e-Procurement installations assessed by the Bank were STQC certified, yet there were several issues with the e-Procurement installations as explained in the previous section. A plausible reason for this situation is that the e-Procurement application software used in the installations was audited by STQC but not the installations per se (i.e.) “…the complete eprocurement system, viz the application along with the server in a specific hosting environment�. A comparative study of MDB’s e-Tendering guideline with the STQC’s e-Procurement guideline shows that certain requirements of MDB is not already covered in the STQC’s guideline. The specific points which the Bank seeks to include in STQC’s guideline are listed later in this section. Further, a few suggestions are made to improve the guidelines such that it is more specific, actionable, decisive and trend-setting. Though certain requirements – such as 3rd party audit at regular intervals and the need to adopt certain standardized procedures to recover from disasters6 – are defined in STQC guidelines, e-Procurement 6 The requirement to audit e-Procurement systems at regular intervals is specified in page 22 of the STQC guidelines as: “It is always recommended that system should be audited at least once in a year and as and when the infrastructure (i.e hardware and software) is augmented by additions of new hardware and software.� Despite rd the references in STQC guidelines, many e-Procurement installations have not been subject to 3 party audit. rd Refer to the issue titled “Audit by 3 Party Agency� in Section 3 of this document for more details. Similarly, the procedure to be followed in case of System Malfunction is not defined in most of the installations (refer issue titled: “System Malfunction Procedure Undefined� ) though this requirement finds a reference in page 44 of STQC guidelines: “Procedures to follow in the event of an unrecoverable disaster e.g. retrieval of off�site back�ups or relocating to a “warm recovery� server which contains all historical data.� . Page | 44 installations have not adopted the same in practice. Hence, development of the guidelines in itself will not serve any purpose unless and otherwise it is backed up by: (i) Well laid down e-Procurement policy and (ii) A robust institutional framework to govern / ensure compliance of e-Procurement installations to the specified guidelines An overview of the institutional framework required to verify and confirm compliance of e-Procurement systems to the laid down e-Procurement guidelines is explained later in this report. 4.1 An Overview of STQC’s e-Procurement Guidelines The guideline by STQC dated 05th of August appears as a compendium of requirements developed from risks defined as per ISO 27001 norms, e-Procurement guidelines issued by the Central Vigilance Commission (CVC), General Financial Rules (GFR), IT Act of 2000 and certain other security guidelines. Detailed section-wise overview is provided below: (i) Risks associated with implementation of e-Procurement, corresponding control reference to the risk in ISO 27001 Information Security Management System (ISMS) standard and recommended remedial (the guideline) measures (Annexure I). The following risks are identified: a. Concerns related with electronic vs. manual procurement b. Concerns related to implementation of e-Procurement using PKI based encryption c. Concerns relating to situations where bids before being transmitted from the bidder’s computer are protected with only SSL Encryption and Database level Encryption is done before the bid is stored in the Database Server d. Concern about Symmetric key based Bid�Encryption done at the Bidder’s computer e. Concerns/ clarifications based on s42(1) of the IT Act 2000 relating to Digital Signatures, a User Organization’s Administrative Hierarchy, and some related aspects f. Some other functionality/ Security/ Transparency related requirements of a Manual Tendering System and Conformance its Availability in the offered e�tendering system g. Concerns/clarifications relating to preventing other Bidders from Bidding in the e� Tendering Scenario, and Miscellaneous Concerns/ Clarifications h. Concerns relating to Bidders making false assertions based on non�existing functionality in their e�tendering software (Important Eligibility/ Qualifying Criteria) (ii) Guidelines issued by the Central Vigilance Commission (CVC); Annexure II a. General security issues b. Infrastructure security issues c. Application security issues at design level d. Application security issues during deployment & use e. Application security issues during data storage & communication (iii) Government of India procurement procedures as defined in Chapter 6 of General Financial Rules (GFR) 2005; Annexure III (iv) IT Act of 2000; Annexure IV (v) Top 10 vulnerabilities defined by OWASP (Open Web Application Security Project); Reference document – 3 and (vi) Penetration testing and vulnerability testing as per NIST 800 – 115 standards Besides the above, the guideline provides directions on the procedure to be adopted by Government agencies on the process to be adopted for selection of e-Procurement ASP. Page | 45 4.2 Fitment of MDB’s e-Tendering Guideline with that of the STQC The MDB have defined a set of about 50 requirements under 12 sub-sections to assess e-Tendering systems in India and elsewhere in the World. A point-by-point comparison is made to verify whether MDB’s e-Tendering requirements are addressed in the STQC’s guidelines. Only those points of MDB’s e- Tendering guidelines which do not find proper reference in STQC guidelines are listed in this section. The Bank seeks inclusion of these missing MDB’s e-Tendering requirements in the STQC’s guideline. Page | 46 S.no. MDB’s e-Tendering Requirement Corresponding Reference in STQC Guidelines 1 Section 1.1: System access shall be open, equal and Refer page 78: Access shall be provided to the general public for unrestricted to all prospective bidders / consultants and accessing any other `Public Information’ sections of the e�tendering members of the public. Those who want to submit portal, such as –Information pertaining to forthcoming Tendering information or receive online alerts or notifications of Opportunities, Information pertaining to `Award of Contracts i.e. amendments or clarifications shall be offered an online Purchase Orders’.� enrollment facility7. Enrollment shall be free. Refer page 86: “…Where required, registration by an authorized user for particular Department/ Buyer for a particular classification of trade, region and vendor class for a particular duration� Refer page 58: “Cost of priced Tender Documents should be payable online at the time of downloading tender documents, or payable offline parallel to the online bid�submission before the bid� submission deadline.� Remarks: This requirement is largely addressed except that there is not a suitable reference to the requirement “Enrolment shall be free� 2 Section 1.2: The principle of single sign-on shall apply. Single Remarks: STQC guideline is silent on the e-Procurement System enrollment shall allow Bidders/ Consultants the multiple use Architecture (i.e.) whether or not multiple user departments of the same electronic system for different projects should share one single e-GP system as a shared infrastructure. from different parts of the government. Refer to the section titled “Need to Define e-Procurement Enterprise Architecture� in Section 3 of this document for further details. Section 2.3: The bidding period shall be measured from the Remarks: No reference corresponding to this requirement could date of publication on the required sites / media, and where be found in STQC guidelines. It is proposed that all bid these dates vary the date of publication will be whichever is opportunities are advertised in the CPPP as explained under the 3 later. A secure log of these entries shall be available for heading “Central Public Procurement Portal (CPPP) as One Stop audit as required. Shop� (i.e. in addition to the e-Procurement portal where the advertisement is originally published) 4 Section 3.3: In case of any amendments or substitutions to Remarks: The STQC guidelines do not state anything averse to this the Bidding Document/RFP by the Contracting Authority, the requirement. However, a corresponding reference to this Contracting Authority shall not replace the Bidding requirement is not explicitly stated 7 Users must be notified of successful registration by means of email, web message or equivalent and also must be able to update their profile at any time. Page | 47 S.no. MDB’s e-Tendering Requirement Corresponding Reference in STQC Guidelines Document/RFP by a new one, but provide such changes by means of an additional document in line with the same distribution mechanism as for the Bidding Document/RFP. 5 Section 3.5: Where Contracting Authorities stage online pre- Remarks: No reference corresponding to online pre-bid bid conferences and clarifications including, for example, conferences could be found in STQC guidelines online conferencing and chat facilities, such facilities shall not function after the bid submission deadline. 6 Section 3.6: Correspondence during bid evaluation for the Remarks: Specific reference to correspondence during bid purpose of clarification may also be done electronically with evaluation between buyer representatives and participating the normal restrictions against modification of the substance bidder is not found in STQC guidelines and price of the bid. Any correspondence of this type shall be directed through the Chairperson of the evaluation committee. Confidentiality of the bid evaluation process shall be maintained. 7 Section 4.3: Contracting Authorities shall ensure the integrity Refer page 31 in STQC guidelines: “For authenticity and for of Bidding Documents in electronic format8, and their online assurance that it has not been tampered, the electronic publication. Amendments shall be similarly secure and Tender Notice (which is an electronic record), should have an stored with the Bidding Document. Contracting Authorities audit�trail within the application of its creation/ approval/ posting. shall inform bidders / consultants where the legally binding Also, the tender notice should be digitally signed by an authorized Bidding Documents can be accessed. Also, There shall be no officer of the Purchase/ Buyer organization.� difference between electronic and print versions of the Bidding Documents/RFPs. Splitting documents into Remarks: Reference to “Legally binding bidding documents�, combinations of electronic and printed portions should be “difference between electronic and print versions of the bidding avoided. documents� and “splitting of documents into combinations of electronic and printed portions…� are not explicitly addressed. 8 Section 5.2: Bids/proposals submitted online shall be virus Remarks: Screening of documents for virus real-time during file scanned by the Contracting Authority before being uploaded upload is not explicitly stated in STQC guidelines. However, the and accepted into the online bid box, and where this causes need for installation of Anti-virus systems to protect the server a bid to be rejected the bidder/consultant shall be notified environment is acknowledged in multiple places immediately. 9 Section 5.5: Bidders/consultants shall be advised that their Remarks: No reference corresponding to this requirement could 8 The ADB also requires a certified paper copy Page | 48 S.no. MDB’s e-Tendering Requirement Corresponding Reference in STQC Guidelines bids/proposals must be readable through open standards be found in STQC guidelines interfaces. 10 Section 5.6: Bidders/consultants shall be allowed to submit Remarks: This requirement is partially addressed, except for the modifications to bids/proposals or withdraw previously following: “Receipt of modification or notice of withdrawal submitted bids/proposals electronically up to, but not after, including the date and time must be acknowledged, and shall also the time of the bid submission deadline. Receipt of be done electronically.� modification or notice of withdrawal including the date and time must be acknowledged, and shall also be done electronically. 11 Section 7.3: Bids/proposals in electronic format shall be Remarks: Refer page 38: “Reading out, ie allowing bidders to protected against access by unauthorized persons until the download the electronic version of the salient points of each publication of the contract award. opened bid (opened in the simultaneous online presence of the bidders)� The meaning of salient points is unclear. Should the system allow download of technical documents uploaded by bidders competing for a tender? Refer the issue titled “Confidentiality of Bid Documents� in Section 3 of this document for further details 12 Section 10.3: The certification process shall allow bidders to Remarks: No other reference except for the following statements take all actions required for their certification within their could be found with regard to procurement of a valid Digital own countries, without the need to travel abroad. Signature Certificate (DSC) by foreign bidders: Section 10.4: The certification process shall accept an “Additionally, the foreign bidders should be able to quote.� electronic signature or a digital certification/signature issued “Whether e�procurement solution has also been customized to by certifying authorities within the country of the bidder, or process all type of tenders viz Limited/Open/Global Tenders?� the process shall accept submission of online or offline documentation for certifying the authenticity of the bidder representative, accepting such documentation that can be obtained under commonly used procedures in the country of the bidder (for example, no notarization in consulate or embassy shall be required). Section 10.5: The certification process shall not require bidders to submit mandatory information with origin outside the bidders own country. Figure 9: Fitment of MDB's e-Tendering Guideline with that of the STQC Page | 49 4.3 Improvements Suggested to STQC’s e-Procurement Guideline 4.3.1 Assess e-Procurement Installation and Not e-Procurement Application Software An e-Procurement application being actively used by tens of thousands of users in an installation will undergo modifications on a regular basis. New versions of software will be released at least once in a month (i.e.) after the installation is somewhat stabilized, else new releases will be more frequent. The ASP’s tend to productize the application software and make it parameterized such that the application in configured to address department specific requirements. Though a feature is available in the software, the ASP could decide to enable / disable the feature in a specific installation as such or even for a department within an installation. Given which, the assessment of e-Procurement system should seek to verify whether e-Procurement software in a certain installation complies with the laid down guidelines and not whether the application per se has complied. It could well be that the version of application subjected to STQC audit had the requisite functionality but the same version of software is not installed at all or that it was configured differently in an installation. For example, in page 32 of STQC guideline it is stated that “…At a higher level, there should be clearance (which is audit�trailed within the application and digitally signed) before a Corrigendum is issued.�. The corrigendum approval workflow is available in most of the e-Procurement applications, but they are rarely used in practice. The e-Procurement application is configured such that the corrigendum can be published by a designated official without taking approvals at a higher level. In such cases, it is said that the requisite approvals are taken offline. When manual and electronic systems are adopted as a mix, it is not possible to enforce implementation of laid down procedures. For example, the requisite manual approval might not have been taken before online publication of the corrigendum. Further, Government agencies do not seem to recognize the difference between audit of e-Procurement application and the audit of “…the complete eprocurement system, viz the application along with the server in a specific hosting environment�. This lack of understanding has created a misperception that security and quality requirement of an e-Procurement installation is assured if the application is STQC certified. Further version number of application software is just a number. There is not a definite mechanism to verify whether the version of software subjected to STQC audit is the exact same as the version deployed in a certain installation. It could well be the case that the version number shown in the application was not modified, but certain changes to the application were made. Hence, a decision has to be taken on whether at all to continue assessment of only e-Procurement application. The suggestion here is that STQC focuses its effort on assessment of complete e-Procurement systems instead. 4.3.2 Supply Driven and Not Demand Driven Assessment The assessment of e-Procurement systems is now demand driven, wherein: “The applicant shall submit the request to Testing and auditing agency (like STQC) to get eprocurement System assessed and certified. During the application the applicant shall clearly mention the scope of certification (i.e. the application system along with the associated infrastructure).The audit team nominated by STQC will conduct formal audit as per the defined criteria and submit a report to the applicant highlighting the Page | 50 non�compliances. The applicant shall submit his closure report after taking necessary corrective measures to STQC. The team of auditors will verify the closure and submit its final report along with its recommendations to the STQC Certification body. A certificate of compliance will be issued by STQC if it is satisfied with the compliance status of the eProcurement System.� Such certification will be done at base cost of Rs. 1 Lakh. Vulnerability Testing, Penetration Testing, SLA monitoring and application security will be assessed at an additional cost. Given that procurement is a de-centralized function and since there is not a designed public procurement agency in India, a database of e-Procurement installations is not there as yet. A dedicated effort has to be made at the very earliest to build a database of e-Procurement installations in India and also assess and verify their compliance to laid down guidelines. Such assessment should be supply driven and evolved as a de facto practice. It is suggested that the subjects to be covered under an assessment (both and initial and periodic) are well defined and that the cost of assessment is borne centrally. The assesse need not pay to have their systems assessed. A communication could be sent out to all known e-Procurement installations about the assessment service provided and those interested to avail the service should revert back with their acceptance. Subsequent to their acceptance, a team will be deployed to assess the e-Procurement installations. At end of year or on a periodic basis, a report on assessment status of various e-Procurement installations could be published. Government agencies will most certainly appreciate this assessment effort because there is widespread recognition about the need for securing e-Procurement installations and as of now there is lack of expertise amongst government agencies to undertake / manage such assessment. 4.3.3 Consolidate and Classify the Requirements in a Certain Logical Sequence The present version of STQC guidelines is presented as a compendium of requirements taken from multiple sources such as the GFR, IT Act of 2000, ISO 27001 norms and CVC guidelines. Due to this, subjects such as Tender opening are addressed in many different places as given below: (i) Refer page 28, risk analysis as per ISO 27001: “E�Procurement system should have functionality such that the physical presence of bidders should not be mandatory during Online Public TOE.� (ii) Refer page 37, risk analysis as per ISO 27001: “Some relevant processes of a fair and transparent online public TOE should include: i. Opening of the bids in the simultaneous online presence of the bidders with proper online attendance record of the authorized representatives of the bidders… iii. One�by�one opening of the sealed bids in the simultaneous online presence of the bidders� (iii) Refer page 45, as defined by CVC: “Whether online Public Tender opening events feature are available in the application?� (iv) Refer page 54, related to GFR: “The e�procurement system must have a very transparent and comprehensive Online Public Tender Opening Event.� It is suggested that an effort is made to consolidate the assessment requirements and classify them under multiple sub-headings. Any duplicate references to the same subject should be removed during the consolidation exercise. Such an exercise will make the guidelines quite crisp and the document will reduce in size significantly from the present size of 91 pages. Page | 51 4.3.4 Make it as a Check-list The current version of STQC guideline is not amenable for assessment because the document has argumentation and reasoning along-side the requirements. In the maze of this argumentation, the key assessment requirement tends to get lost. Take for example the following recommendation about tender opening: Page 19 of STQC guidelines: “A system in which Public Key of a bidder’s representative is used for bid� encryption at bidder’s office, and where decryption will be done by the bidder’s representative himself using his private key during the Online Public TOE…. Note: Private key cannot be transmitted by the bidder over the internet. Furthermore, during the Online Public TOE, bids cannot be allowed to be downloaded from the server to the bidder’s computer. This would tantamount to the bids being taken away from the tender�box back to the bidder’s office for opening. This cannot be allowed. Therefore the bidder will have to be physically present during the Public TOE, and such a system will never be able to have a proper Online Public TOE.� It is not clear whether e-Procurement systems which use Public Key of bidder’s representatives to encrypt the bidder’s commercial quote will be regarded as complying with this statement. The issue with encrypting using bidder’s key is explained in detail under the title “Bid Proposals shall not be Encrypted Using Bidder’s Key� in Section 3. If the system is not complying with this requirement, will a decision be taken to certify the system? In other words, is this a critical requirement? To remove such lack of clarity, it is suggested that the assessment requirements are compiled in the form a check-list under multiple sub-headings. The assesse could then be asked to explain their compliance to all the requirements specified in the check-list. Subsequently, the responses would be evaluated and non-compliance if any will be identified with reference to one or more of the requirements specified in the check-list. The check-list will help in verifying all the requirements in sequence without missing one or more requirements by mistake. 4.3.5 Be as Specific as Possible Few of the requirements are left open ended in the current version of STQC guideline such as: (i) Refer page 19: “If the e�procurement system uses PKI for bid�encryption, it has to satisfactorily address the above issues and consequent concerns … through suitable functionality built into the e�procurement application.…it is the responsibility of the individual vendors to design and develop their applications in a manner that addresses the outlined concerns…�. (ii) Refer page 22: “It is important that even if a clandestine copy is made and stolen as above, the bid encryption methodology should be such that it should not be possible to decrypt the bids in connivance with any officer of the Buyer organization or the Service Provider organization….in case PKI�based bid encryption is done, the software functionality has to be suitably augmented to mitigate this security threat.� (iii) Refer page 50: “Suitable management procedure shall be deployed for regular back�up of application and data. The regularity of data backup shall be in commensurate with the nature of transaction/ business translated into the e�procurement system� Page | 52 The adverse impact open-ended guideline on encryption methodology has had in the emergence of multiple user-unfriendly e-Procurement systems is explained in detail under the issue titled “Standardize the Encryption Methodology� in Section 3 of this document. Ideally, the guidelines should provide specific guidance on the manner in which a particular functionality has to be implemented. Provision of such clarity is all the more important for critical functionality such as bid encryption. A concept note evaluating the various bid encryption techniques could be prepared as a separate document and a group of experts could be consulted to arrive at a consensus etc. After all that, the guidelines should precisely explain the encryption method (i.e. guideline) to be implemented. 4.3.6 Visualize the Larger Picture The requirements laid down in the STQC guidelines are focused on translating and determining functional equivalence of various procurement related requirements. Such focus is required, but the potential of e-Procurement to transform public procurement in India has to be acknowledged. For example, it is now possible to generate analytical reports on the amount of money spent by the Government in procurement of individual items such “Notebook computers�: (i) All government agencies identify Notebook computer vide a standardized item code reference (UNSPSC item code: 43211503) (ii) All government agencies advertise their tender details in a centralized web portal such as the CPPP The issues titled “Lack of Adoption of Item Codification Standard� and “Central Public Procurement Portal (CPPP) as One Stop Shop in Section 3 have more details in this regard. The assessment guidelines should ideally imbibe certain key elements – such as adoption of UNSPSC code and publication of tender advertisement in CPPP – of the larger picture and thereby catalyze adoption of such standards within the Government establishment. STQC should strive to establish / identify e-Procurement specific standards where required and verify compliance of the various e-Procurement installations to the laid down standards. Such standards should be established with the intent to make the different e-Procurement systems work together in synergy. The Nation should be treated as the unit of analysis. Those involved should recognize that implementing these standards and making them work will happen only after many years of sustained effort. A set of issues to be addressed in this regard is explained in Section 3: (i) De-duplicated Vendor Database a. If a vendor could be uniquely identified across multiple e-Procurement systems, a National database on the vendor’s performance could be developed. With such a set-up in place, vendors can cite their project experiences in authentic electronic format (i.e. refer to a link in National supplier performance database) instead of submitting scanned copies as it is now practiced. Multiple standards have to be developed and implemented across e-Procurement installations to realize this vision. (ii) Need for Mechanism to Address Vendor Lock-in and Exit Management a. When contracts with e-Procurement ASP’s invariably expire or when a decision is taken to develop e-Procurement afresh, the data – i.e. procurement data, audit logs etc. – has Page | 53 to be transitioned as accurately as possible from the legacy system to the newly established e-Procurement system. It is proposed that such transition is done as per certain laid down standards. The assessment will seek to certify whether such transition has happened as per the standards. (iii) Need to Define e-Procurement Enterprise Architecture a. How many e-Procurement installations are required and how large should be an e- Procurement installation for it to sustain (i.e. worth enough to make the necessary investments)? The guidelines should in some form discourage smaller size installations (or in other words pressurize them to grow in size) because in the long run good quality e-Procurement systems can be sustained only through investments in development of policies, PMU establishment, 3rd party audit and requisite server infrastructure. 4.3.7 Establish Benchmarks STQC’s guidelines should strive to set the pace for evolution of e-Procurement in India by setting the benchmark metrics for key requirements such as: (i) Recovery Point Objective; RPO (i.e. what’s could be the maximum possible data loss) – e.g. 5 minutes (ii) Recovery Time Objective; RTO (i.e. when the system will be operational in case of unexpected downtime) – e.g. 6 hours (iii) Load handling capabilities of the system – For example, system handled 5000 concurrent connects to the server without performance issues (iv) Duration of periodic audit (e.g. done once in 6 months) (v) Number of government agencies using e-Procurement as a shared infrastructure (e.g. 250 government agencies) (vi) Number of suppliers registered in a single instance of e-Procurement platform (e.g. 10,000 suppliers) (vii) Percentage of tender documents with e-Procurement specific SBD clauses; 100% of the sample is ideal. The assessment would then seek to verify whether an e-Procurement installation complies with the benchmarks and associated metrics laid down. The Benchmarks will have to be gradually made more stringent with the intent to enhance quality, thus the guidelines will catalyze enhancement of quality in e-Procurement installations. For example, as the systems mature: (i) The Recovery Point Objective and Recovery Time Objective could be brought down to 1 minute and 4 hours respectively (ii) Number of government agencies using a single instance of e-Procurement platform and number of suppliers registered can be increased to 300 and 25,000 respectively 4.3.8 Introduce Competition by Grading and Ranking e-Procurement Installations Setting the benchmarks by itself may not spur government agencies to achieve better by improving the quality of their e-Procurement installation. Hence, it is suggested that each installation is graded during the assessment as per weightage allocated to various sections of the e-Procurement assessment framework. The assessment will seek to evaluate not only the application but also other aspects of the Page | 54 installation such as adoption of the standards and the policies, data governance, project management, adoption (functional and geographical coverage) and legal framework. Certain requirements – such as implementation of the prescribed encryption methodology and RPO of 1 hour – will be termed as mandatory, non-adherence of which will result in non-compliance of the system to the laid down guidelines. Beyond that, grades will be assigned to the extent the installation satisfied the requirements or the metrics. For example, an installation with RPO of 5 minutes will be graded better than an installation with RPO of 1 hour. All the grades thus assigned will be consolidated to arrive at a single mark as per which e-Procurement installations will be ranked. An annual report on assessment of e-Procurement systems in India will be published ranking the various e-Procurement installations. A key assumption herein is that a sense of competition will get introduced amongst the various e-Procurement installations and Government agencies will compete and seek to obtain better rank by upgrading their installations vis-à-vis the laid down e-Procurement guidelines. A new version of e-Procurement guidelines has to be released on a periodic basis to set new benchmarks and upgraded metrics. Page | 55 5 Concept Model of the Proposed e-Procurement Assessment Guideline A new assessment guideline is proposed herein to address the various issues identified in Section 3 and the improvements to STQC’s existing e-Procurement guidelines suggested in Section 4. The focus in this Section is on explaining Concept model of the proposed assessment guideline. The various sections in the concept model are identified and certain distribution of weightage across the sections is suggested. It is emphasized that the model per se is only a draft and weightage could get allocated differently. The guideline proposed herein is for assessment of e-Procurement installations (i.e. “…the complete eprocurement system, viz the application along with the server in a specific hosting environment�) and not just the e-Procurement application software. Application software is a key component within the overall assessment guideline. Requirements under various sections of the Model have not been detailed in this document. Extensive discussions with key stakeholders may have to be held before requirements in subjects such as bid encryption methodology can be finalized. Also, certain key requirements included in the guideline may have to be backed by a concept note wherein the logic adopted to arrive at a certain guideline or set of guidelines have to be spelled out. In an effort to illustrate the concept more clearly, an effort is made to draft the e-Procurement assessment guideline in its full form and specific to the Indian context. The base content required for drafting this detailed requirement is drawn from the latest version of STQC’s e-Procurement guidelines and other key resource documents such as CVC guidelines, IT Act of 2000 and GFR. A full draft of the proposed assessment guideline will be prepared as a separate document and it is not included as part of this report. 5.1 Tabular Format Adopted to Define Requirements The proposed assessment guideline is prepared a la the technical evaluation criteria in Quality and Cost Based Selection (QCBS) procurement. Herein, the guideline is conceptually sub-divided into layers and then subjects. The subjects to be covered in the assessment framework are classified under the 4 layers of classification defined in STQC’s e-Procurement guideline: Data, Application, Infrastructure and Process. Each identified subject can have one or more requirements. An e-Procurement installation will be assessed for a total of 1000 marks. The marks are distributed at first amongst the 4 layers. The mark assigned to a layer is further sub-divided across multiple subjects underneath the layer and then across the many requirements specified in each subject. The break-down structure of the proposed framework will allow comparison of e-Procurement installations as a whole, layer-by-layer and at the subject levels. An e-Procurement installation ranked less can find out the layers / subjects wherein they were outscored and identify the measures to be undertaken to move up the rank. A provision is made to mark a requirement as mandatory and requirements marked as mandatory shall necessarily be adhered to. Else, the e-Procurement installation will not be certified. Requirements which have metrics tagged to them will be assigned marks as per the slab-wise grading defined in the guidelines. A pictorial view of the proposed tabular framework is given below: Page | 56 Mand Marks S.no. Requirement atory Allocated Assigned 1. Application 325 Tender publication 25 The advertisement shall be posted in the publicly accessible web portal of e-Procurement platform that is 1.1.1 Yes 5 well known nationally, well maintained, functional, and affords free and unrestricted access. Where bidding is restricted or subject to pre-qualification, this shall be clearly disclosed in the bid advertising. The bid advertisements and results disclosures shall not be 1.1.2 Yes 5 restricted. Also, there is to be no material difference 1.1 between the paper documents (i.e. newspaper advertisements) and those advertised online. All clarifications and amendments or substitutions of the bidding documents, as well as any pre-bid conference minutes, shall be posted simultaneously onto a bid 1.1.3 Yes 5 advertising website that is freely accessible to all. Bidders who have already expressed an interest should be directly informed electronically of any such postings. 1.1.n … … 1.2 Bid submission 30 1.2.1 … … 1.n … … 2. Data 150 2.1 Data Governance 50 Back-up policy should be put in place such that Recovery Point Objective (RPO) of at the minimum 5 minutes is achieved. Award of marks will be as under: RPO of 1 2.1.1 50 minute = 50 marks RPO of 3 minutes = 25 marks RPO of 5 minutes = 10 marks 3. Process 375 3.1 Adoption of Standards 50 The works / goods / services procured shall be identified 3.1.1 25 by an item code from UNSPSC code set. An .xml file of advertisements shall be uploaded into the Central Public Procurement Portal (CPPP) within 24 hours 3.1.2 25 from the time of publication in the web portal of e- Procurement platform 4. Infrastructure 150 Note 1. Standard functional requirements in section 1.1 are assigned 30 marks, but made mandatory. An installation will not be certified unless the requirements in 1.1 are complied with. 2. A total of 50 marks is assigned under 3.1 in an effort to encourage government agencies to adopt the specified standards Figure 10: Tabular Framework Adopted to Define Requirements Page | 57 5.2 Classification of Assessment Requirements The e-Tendering guideline of MDB is divided into 2 sections, wherein the first section has a set of requirements under 12 headings: 1. System access 2. Advertising 3. Correspondence, Amendments and Classification 4. Bidding documents 5. Submission of bids 6. Bid securities 7. Price bid opening 8. Bid evaluation and contract award 9. Information security management 10. Authentication 11. Payment 12. Other considerations The second section has a set of 25 technical requirements such as the adoption of SSL and storage of password in encrypted format. The Quality and Security evaluation model of STQC consists of four layers viz.: 1. Data 2. Application 3. Infrastructure and 4. Process The proposed assessment requirements are explained within the 4-layer framework of STQC, wherein MDB’s e-Tendering requirements are categorized under the Application layer. 5.3 Concept Model of the Proposed Assessment Guideline The skeletal structure of the proposed assessment guideline is referred to as “Concept Model�. The Concept model identifies the subjects to be covered in e-Procurement assessment and suggests weightage allocation to each of the identified subjects. A total of 33 subject areas are identified under 4 layers viz. Data, Application, Infrastructure and Process. Also, the issues identified in Section 3 of this report are mapped with the identified subjects. Refer to the Figure below for details. The subjects to be covered in the proposed assessment framework are identified herein based on the following sources of information: (i) STQC’s e-Procurement guidelines (ii) MDB’s e-Tendering guidelines & (iii) Issues identified during assessment of e-Procurement systems, as explained in Section 3 A sample of the Tabular framework proposed for defining the requirements is illustrated earlier in this section. Detailed assessment requirements under each of the identified subjects will be prepared in the Tabular Format and submitted as a separate document. Page | 58 Concept Model of the Proposed Assessment Guideline S.no. Subject Weightage Issues Addressed Data 150  Recovery Point Objective (RPO) and Recovery Time Objective (RTO) 1.1 Data governance 50 1 Undefined 1.2 Exit Management 50  Lack of Commitment on Record Keeping 1.3 Data storage 50 Application 325 2.1 System Design 40  Adoption of Form Based System 2.2 Supplier registration 25  De-duplicated Vendor Database  System shall always Allow Issuance of Corrigendum before Bid 2.3 Tender publication 25 Submission Timeline  Upload of Bid Documents in Chunks of 1 MB File  System Shall Disallow Issuance of Corrigendum After Expiry of Bid 2.4 Bid submission 30 Submission Deadline  Bid Submission Shall Not be done in Bits and Pieces 2.5 Bid evaluation and contract award 25 2  Confidentiality of Bid Documents  Original Encrypted Bid Document shall not be replaced by 2.6 Information security management 50 Decrypted Bid Document  Bid Proposals shall not be Encrypted Using Bidder’s Key  Standardize the Encryption Methodology 2.7 Authentication 10  Procurement of Digital Signature Certificate by Foreign Bidders 2.8 Payment 30  e-Payment to be Implemented 2.9 Usability 30  Serious Usability Issues  Well Define Functional Scope of e-Procurement System 2.10 System Architecture 30  Need to Define e-Procurement Enterprise Architecture 2.11 External integration 30  Integration of e-Procurement with External Systems Infrastructure 150 3.1 Anti-virus scan 10  Anti-virus Scan of Uploaded Documents 3 3.2 Disaster Recovery Set-up 25 3.3 Server side infrastructure 20  e-Procurement Servers should be under Government Control 3.4 Hosting environment 20 Page | 59 Concept Model of the Proposed Assessment Guideline S.no. Subject Weightage Issues Addressed 3.5 End user infrastructure 15 3.6 Deployment architecture 15 3.7 Load handling capabilities of the system 20 3.8 Infrastructure security 25 Process 375 Final Acceptance Testing (FAT)  Message Definition in Digitally Signed and Submitted Bids 4.1 25 Compliance 4.2 Third party audit 25  Audit by 3rd Party Agency  Disallow Mandatory Submission of Originals before Bid Submission Enforcement of Operational Guidelines Deadline 4.3 25 in using e-Procurement system  Bidders Shall Submit Originals Before Tender Opening  Discontinue Sale of Tender Documents  System Malfunction Procedure Undefined 4.4 e-Procurement Policy 40  Electronic Systems should Replace Manual Systems and Not Co- Exist  Sustained Uptake of e-Procurement 4 4.5 Usage of e-Procurement system 75  Usage of functional components 4.6 Project Management Unit (PMU) 35  Weak Project Management Unit (PMU) 4.7 e-Procurement specific Bid Documents 10  Lack of e-Procurement Specific Standard Bid Document Clauses 4.8 Vendor registration 20  De-duplicated Vendor Database  Lack of Adoption of Item Codification Standard  Central Public Procurement Portal (CPPP) as One Stop Shop 4.9 Adoption of standards 50  Need for Mechanism to Address Vendor Lock-in and Exit Management  Need for e-Procurement Rules 4.10 Legal framework 45  Reference to e-Procurement in the Draft Public Procurement Bill 2012 4.11 SLA monitoring 25 Figure 11: Concept Model of the Proposed Assessment Guideline Page | 60 6 Institutional Establishment Required for the Assessment of e- Procurement Installations The assessment of an e-Procurement installation requires extensive effort because a broad list of requirements under 35 subjects has to be studied. The agency conducting the assessment has to necessarily do a thorough job through activities such as: (i) Check for compliance to information security requirements in the database level (ii) Restore and confirm that the data is backed up correctly and RPO objectives are actually met (iii) Verify whether the e-Procurement application is configured in line with the norms laid down in the assessment guideline (iv) Assess load handling capabilities of the system in the production set-up and (v) Evaluate the capabilities and know-how of the Project Management Unit (PMU) established to manage implementation of the e-Procurement system and (vi) Subject the e-Procurement set-up to penetration testing and vulnerability testing When a supply driven model of assessment is adopted – as explained in Section 4 – the onus is on the designated assessment agency to: (i) Develop institutional capabilities required to assess multiple e-Procurement systems. As on date, there are about 15 adequately large e-Procurement installations. This number would eventually increase to about 509 installations in the years to come. (ii) Standardize assessment methodology through development of metrics such that all e- Procurement installations are evaluated in a standardized manner (iii) Complete periodic (e.g. year on year) assessment of e-Procurement in a timely manner such that an annual report on grading and ranking of e-Procurement systems can be published within a set time period (e.g. by end of June of every year) Besides the above, the assessment agency needs to: (i) Continuously evolve the methodology (i.e. development of evaluation approaches and metrics) required to assess e-Procurement installations (ii) Keep up to date on e-Procurement related developments in India and elsewhere in the World (iii) Update the assessment guidelines on a periodic basis by modification of assessment requirements and re-distribution of weightage allocations (iv) Contribute towards development of e-Procurement related standards and policy making by “Visualizing the Larger Picture� (refer Section 4.3 for details) The World Bank representatives did not interact with STQC for preparation of this report, hence in- house capabilities of STQC in e-Procurement is not known. However, STQC presumably has in-house expertise in e-Procurement since it prepared the e-Procurement guidelines. The Government of India may choose to leverage upon the set-up already existing in STQC to build the institution required for assessment of e-Procurement systems. An adequately large sized team is required to assess the many e-Procurement installations year on year. Such a team would need to have strong expert resources from technology and public procurement domains. The Government of India has to decide on whether to develop the full team of resources in- 9 Maintaining and monitoring quality of large number of small sized e-Procurement installations will be expensive. Hence, there is a need to define Enterprise Architecture for e-Procurement at the National level. In other words, how many e-Procurement systems does India need and how large is adequately large? Page | 61 house or to outsource the assessment work to 3rd party agencies. The outsourcing of work while it is adopted shall be restricted to routine and well defined activities such as verification of compliances to laid-down process and technology standards. Key activities such as visualizing the larger picture and preparation of the Annual e-Procurement Assessment Report shall necessarily be prepared by a core establishment set-up within the Government. The Government may choose to segregate the routine e-Procurement assessment work and policy development work and assign them to two different organizations. In which case, an e-Procurement policy making authority has to be constituted. It is recommended that the cost of setting up and managing the institutional establishment for assessment of e-Procurement systems is borne by the Central Government as expenditure incurred towards National Policy Making. Page | 62 7 Six Step Procedure to Operationalize the Assessment of e- Procurement Installations A six-step iterative procedure proposed to operationalize the assessment of e-Procurement installations is explained in this Chapter. This six-step procedure and the institutional establishment required to undertake this assessment is jointly referred to as the Assessment Framework. 7.1 Step One: Develop Assessment Guidelines A team of experts have to be set-up to operationalize the concept model. Each of the 35 subject categories in the model framework needs to be expanded to prepare the detailed Assessment guidelines: (i) Detailed evaluation criteria to be defined just as technical bid evaluation criteria in Quality and Cost Based Systems (QCBS) tenders are defined (ii) Methodology for assessment has to be defined individually for the assessment criteria. For example, develop the test case to ascertain whether the bid is encrypted as proclaimed by the bidder (iii) Metrics for assessment criteria – where applicable – has to be defined (e.g. RPO of 10 minutes) (iv) Grades to be allocated for each of the assessment criteria An Expert Committee needs to be constituted to review and approve the detailed Assessment guidelines Preparation of detailed Assessment guidelines will require research work. For example, the team has to evaluate the item code classification options and decide on the option most suitable for India. An Expert team has to be set-up for preparation of the guidelines, to initiate and manage assessment of e-Procurement installations. Key roles and responsibilities of the Expert team are: (i) Prepare the assessment guidelines (ii) Contact government agencies and obtain their content for assessment of their e-Procurement installations (iii) Conduct the initial set of assessments in person and thus obtain first-hand experience about the status quo (iv) Obtain necessary support from the market by procurement of consultancy services as required or liaison with relevant government agencies such as Cert IN (v) Prepare the Annual e-Procurement Assessment Report (vi) Market findings of the report to the larger audience (vii) Revise the assessment guidelines for the next assessment cycle The Expert team needs to have resources dedicatedly deployed. A 4-member team as given below should be adequate to operationalize the assessment procedure: (i) Team Lead (1 resource) (ii) Domain expert (1 resource) (iii) Technical experts (2 resources) This team will report to the Expert Committee and work as per its guidance. This team shall have the expertise to: (i) Develop e-Procurement related standards (ii) Prepare the detailed assessment guideline (iii) Conduct the assessment and (iv) Prepare the Annual assessment report. Page | 63 The Expert team can hire resources from the market on a need basis either for specialized skill set or to expedite the assessment work. It is important that the Domain and Technical experts engaged for this work are well informed about their respective areas of expertise. The Technical experts for example should have had hands-on expertise in development, deployment and administration of web based transactional systems. With such knowledge, they shall have the expertise to evaluate and clarify technical issues raised by the Expert Committee and e-Procurement Application Service Providers. Selection of capable and well informed domain and technical experts is a key requirement for successful implementation of the Assessment framework. The establishment of Standards will elicit stiff resistance from the Application Service Providers (ASP), as the ASP’s would have to customize their software to comply with the Standards. All discussions pertaining to the Standards will be quite technical, most of which will be security related. Hence, it is important that this Expert team is technically quite strong and capable. 7.2 Step Two: Prepare Assessment List for the Year The Nodal agency designated for conducting the assessment has to obtain consent for the assessment from owner of e-Procurement installations. Ideally, the assessment guidelines should be shared with the Government agencies upfront while seeking the consent. The agencies which have agreed to the assessment will be included in the assessment list. It is proposed that a smaller sub-set of e- Procurement installations are taken up in the first year on a pilot basis and this list is gradually expanded to cover all sizeable e-Procurement installations in India. Figure 12: Six Step Procedure to Operationalize the Assessment of e-Procurement Installations Page | 64 7.3 Step Three: Conduct Assessment It is proposed that the Expert team set-up for preparation of the guidelines visits the field and does the assessment first-hand. The team will learn about the effectiveness of the metrics and completeness of the assessment guidelines per se by doing the assessment first hand. Further, the team needs to collect the evidences required for assignment of marks and substantiate their claims of default. An effort has to be made to standardize the field study. After the team has acquired adequate expertise in conducting the assessment and when the field study is adequately standardized, a decision may be taken to gradually outsource the assessment work or a section of the assessment work to external agencies. 7.4 Step Four: Grade and Rank Systems The installations assessed will be graded as per the assessment methodology (i.e.) just the way technical bids are evaluated in a Quality and Cost Based Selection (QCBS) tender. Then, all the assessed systems will be compared and ranked just the way bidder’s technical proposals are ranked during bid evaluation. 7.5 Step Five: Publish Annual Assessment Report A detailed assessment report will be prepared every year explaining: (i) Assessment guidelines (ii) Methodology adopted to conduct the assessment (iii) Key findings from the assessment (iv) Grades and ranking of the assessed systems (v) Areas of improvement (i.e. compared to the previous year) (vi) Areas of concern (e.g. data is not getting backed-up at regular intervals and exit management readiness is not there in most of the installations) (vii) Revisions to the assessment criteria for the subsequent year Adequate publicity has to be given to the Annual Assessment Report, wherein all key stakeholders have to be invited for a detailed discussion of the Report. A ceremony may be conducted to award the best ranked system and the most improved system and therein key stakeholders will be consulted on the areas of concern and revisions proposed to the assessment guidelines. Thus, this assessment exercise will act as a vehicle for policy implementation and encourage de facto adoption of laid down policies. 7.6 Step Six: Feedback The Expert team will propose revisions to the assessment guideline for the subsequent year. Such revisions could refer to: (i) Stringent service levels (e.g. RPO of 5 minutes as against 1 hour in the previous year) (ii) Inclusion of new criteria to expand the horizon (e.g. adherence of application software to export and import data in accordance with an Exit Management standard newly developed (or evolved) by the Government) (iii) Improved methodology for assessment (e.g. revised test case to verify whether commercial bids are encrypted at the client side) Page | 65 8 Workability of the Proposed Solution The World Bank conducted assessment of e-Procurement installations with full approval from the Government agencies. The assessment team was welcomed well and information sought by the team was provided with earnestness. At end of the field visit, Government agencies typically enquired how their system fared vis-à-vis e-Procurement installations elsewhere in the country. Thus, there is a genuine requirement to rank and grade e-Procurement installations and State Governments and Public Sector Undertakings (PSU) are likely to welcome the initiative to rank and grade e-Procurement installations. Government agencies implementing e-Procurement systems in general lack the requisite capabilities to objectively assess their systems vis-à-vis well established standards. Herein, it is suggested that Government of India builds strong capabilities at the National level at first to assess and guide development of e-Procurement systems. Subsequently, a concerted effort can be taken to build capabilities to monitor and evaluate e-Procurement system and e-Governance systems in general in a decentralized manner. Figure 13: Workability of the Proposed Solution There is a genuine need to standardize the various e-Procurement systems such that they seamlessly inter-operate and it is better if such standardization happens sooner than later. Given the Federal establishment in India, imposing standards de jure on the State Governments will not work. Hence, it is suggested herein that government agencies are encouraged to adopt certain specified standards (e.g. UNSPSC code set, PAN verification during supplier registration and publication in CPPC) by allocation of marks. The States would fall in line and take efforts to adopt the laid down standards if the central team could: (i) Develop a good detailed assessment framework (ii) Do the assessment objectively (iii) Gain repute by publishing the National e-Procurement assessment report year-on-year and (iv) Provide adequate publicity to the assessment report Standards in the proposed model will thus evolve as de facto instead of the de jure. Page | 66 The setting up and maintenance of the institutional establishment – required to develop e-Procurement assessment guidelines and to assess the many e-Procurement installations – can be done with minimal (i.e. in relation to the overall public procurement spend) investments. Most of this investment will be spent on human resources and operational expenditure. It is expected that this initiative will catalyze the development of robust e-Procurement systems and enable inter-networking of the many systems through adoption of standards. The Value for Money (VfM) from investments made in this initiative will be quite high if this expectation is realized. The Assessment framework explained in this report could be used as a vehicle to implement policy in other federated e-Governance systems such as e-District and State Data Center (SDC). In e-District Mission Mode Project (MMP) taken as an example, Government needs to define e-District specific guidelines (i.e. e-District Policy), assess e-District installations year on year to verify for compliance vis-à- vis the laid down guidelines and prepare e-District Annual Assessment Report. Thus, the Assessment Framework concept explained herein has wider application in implementation of e-Governance policy. Page | 67 9 Annexure 9.1 e-Procurement Related Sections in Draft Public Procurement Bill 2012 9.1.1 Section 7: Determination of Need for Procurement “(4): The procuring entity may publish information regarding planned procurement activities for the forthcoming year or years on the Central Public Procurement Portal. Explanation.—For the removal of doubts, it is hereby declared that the publication of information under this sub-section shall not be construed as initiation of a procurement process and cast any obligation on the procuring entity to issue bidding document or confer any right on prospective bidders.� 9.1.2 Section 14: Registration of Bidders “(5) The results of the registration process shall be intimated to the bidders and the list of registered bidders for the subject matter of procurement shall be exhibited on the Central Public Procurement Portal.� 9.1.3 Section 18: Pre-bid Clarifications “All requests for clarification and responses thereto shall be intimated to all bidders and where applicable, shall be exhibited on the Central Public Procurement Portal.� “(4) A procuring entity may hold a pre-bid conference to clarify doubts of potential bidders in respect of a particular procurement and the records of such conference shall be intimated to all bidders and where applicable, shall be exhibited on the Central Public Procurement Portal.� 9.1.4 Section 22: Exclusion of Bids “(4) Every decision of the procuring entity under sub-section (3) shall be—(a) communicated to the concerned bidder in writing; (b) exhibited on the Central Public Procurement Portal.� Clause (3) refers to reason for exclusion 9.1.5 Section 25: Award of Contract “(3) As soon as the procuring entity, with the approval of the competent authority, decides to accept a bid, he shall communicate that fact to all participating bidders and also exhibit the decision on the Central Public Procurement Portal;� 9.1.6 Section 29, Method of Procurement “(4) The Central Government may make rules relating to electronic procurement and may, by notification, declare adoption of electronic procurement as compulsory for different stages and types of procurement, and on such declaration, every requirement for written communication under this Act shall be deemed to have been satisfied if it were done by electronic means.� 9.1.7 Section 30: Open Competitive Bidding “(5) In case of an open competitive bidding, the procuring entity shall invite bids by exhibiting an invitation on the Central Public Procurement Portal, its own website and by giving wide publicity in the manner as may be prescribed.� Page | 68 9.1.8 Section 31: Limited Competitive Bidding “(b) the procuring entity shall also exhibit the invitation to bid on the Central Public Procurement Portal.� 9.1.9 Section 38: Central Public Procurement Portal “(1) The Central Government shall set up and maintain a Central Public Procurement Portal accessible to the public for posting and exhibiting matters relating to public procurement. (2) Subject to the provisions of section 28, each procuring entity shall cause the procurement related information to be exhibited as required under this Act or the rules made thereunder on the Portal referred to in sub-section (1). (3) Without prejudice to the generality of the provisions contained in sub-section (2), the Central Public Procurement Portal shall provide access to the following information in relation to procurement governed by the provisions of this Act, namely:— (a) pre-qualification document, bidder registration document, bidding document and any modification or clarification including those pursuant to pre-bid conference, and corrigenda thereto; (b) list of bidders that presented bids including for pre-qualification or bidder registration, and of those bidders which were pre-qualified and registered, as the case may be; (c) list of bidders excluded under section 22, with reasons thereof; (d) decisions taken during the process of grievance redressal under the provisions of Chapter III; (e) details of successful bids, their prices and bidders; (f) names and the particulars of bidders who have been debarred by the Central Government or a procuring entity together with the name of the procuring entity, cause for the debarment action and the period of debarment; (g) any other information as may be prescribed. (4) The information exhibited in terms of this section shall be available on the portal for such period as may be prescribed.� 9.1.10 Section 56: Power of Central Government to make Rules “(2) In particular, and without prejudice to the generality of the foregoing powers, such rules may provide for all or any of the following matters, namely:— (x) electronic procurement under sub-section (4) of section 29;� Page | 69