Policy Research Working Paper 10390 Introducing E-Procurement in Bangladesh The Promise of Efficiency and Openness Jürgen René Blum Arkopal Datta Mihály Fazekas Sushmita Samaddar Ishtiak Siddique Governance Global Practice April 2023 Policy Research Working Paper 10390 Abstract Governments around the world spend about one-third of probability of a single bidder decreasing by 7.8–13.5 per- their budgets through public procurement systems where centage points. Economy also improves as discounts firms electronic administration of public tenders promises great offer increase by 7.4–8.0 percentage points. Administrative benefits. However, surprisingly little is known about how, efficiency greatly improves too: the total time of process- under which circumstances, and through which features ing a tender—starting from the public call for tenders to electronic systems work. To address these questions, this contract signature—drops by 15.6–19.2 days. However, paper looks at the introduction of an electronic procure- it is possible that low performance and rent-seeking were ment system compared to a fully paper-based system in displaced to the contract implementation phase, which Bangladesh in 2011–16. The impact of the electronic pro- remained principally paper-administered. These results curement system on access to public tenders, their economy, indicate that the government directly saved US$460 mil- and administrative efficiency is estimated. Contracts were lion to US$513 million in the analyzed electronic tenders, matched both within procuring entities and years, and fixed largely due to increased winning rebates and lower adver- effects regressions were run to address biases emanating tising costs. Considering the indirect macro effects, the from nonrandom assignment to treatment. The findings introduction of electronic procurement increased Bangla- show an overwhelmingly positive impact. Access improves, desh’s gross domestic product by 0.48 to 0.54 percent, or with the number of bidders increasing by 1.6–2.2 and the US$1.4 billion to US$1.6 billion in 2019. This paper is a product of the Governance Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at jblum@worldbank.org and fazekasm@ceu.edu. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Introducing E-Procurement in Bangladesh: The Promise of Efficiency and Openness Jürgen René Blum1, Arkopal Datta2, Mihály Fazekas3, Sushmita Samaddar4, Ishtiak Siddique5 JEL Codes : H57, O12, O53 Keywords: E-Procurement; Public Procurement; Impact Evaluation; Bangladesh 1 Senior Public Sector Specialist and the Task Team Leader for the Bangladesh procurement impact evaluation 2 Field coordinator for this Bangladesh procurement impact evaluation 3 Assistant professor at Central European University and the academic counterpart for the impact evaluation 4 Field coordinator for this Bangladesh procurement impact evaluation 5 Senior Procurement Specialist and the co-Task Team Leader for the Bangladesh procurement impact evaluation Acknowledgments: The team is gratefully for the support and cooperation from the Government of Bangladesh, and particularly, the Central Procurement Technical Unit (CPTU), the Local Government Engineering Department (LGED), the Roads and Highways Department (RHD), and the Bangladesh Water Development Board (BWDB). The team is particularly thankful to Mr. Amulya Kumar Debnath, Mr. Md. Faruque Hossain, Mr. Md. Ali Noor, and Mr. Mohammed Shoheler Rahman Chowdhury, for their help as the Director General, CPTU during the activities of the project. The authors are thankful for the financial support received from the Global Procurement Partnership: Multi-Donor Trust Fund at the World Bank. The report greatly benefited from useful inputs and feedback from the World Bank colleagues, including Mr. Vinay Sharma, Ms. Alma Kanani, Ms. Elmas Arisoy, Mr. Zafrul Islam, Mr. A.N.M. Mustafizur Rahman, Mr. Md. Kamruzzaman, and Mr. Arafat Istiaque. The team is grateful to the multiple reviewers for their helpful comments. The team also recognizes the efforts of Ms. Mahmuda Nusrat Hussain, Mr. Shourov Kumar Sharma and Ms. Shahnun Nima Taznia who helped with the logistics of the project. 1. INTRODUCTION 1. Government contracting accounts for roughly 30 percent of general government spending across OECD countries (OECD, 2015) and for an even higher share in developing economies (World Bank, 2016). Governments depend on public procurement for delivering virtually all public services—be it defense, roads or health care. Yet, the risk of rent seeking in procurement is high, and it has often been marred by corruption and collusion among bidders, undermining service delivery. On the flipside, efforts to reduce the discretion of procurement agents, a widely used reform avenue, have frequently led to inefficiency and red tape (Collier et al., 2015; OECD, 2010; World Bank, 2009). 2. Electronic procurement (e-procurement henceforth) reforms promise helping tackle the dual challenge of rent seeking and inefficiency (Fazekas & Blum, 2021; OECD, 2016). As a widely advocated tool (OECD, 2016), online publication and submission of tenders can increase transparency and make it harder for corrupt agents to seek rents. Electronic processing can cut transaction costs both for businesses and the government, making procurement more efficient. Both factors have the potential to boost value for money from public tenders. 3. In practice, however, these benefits are far from guaranteed. If e-procurement makes it harder for corrupt officials to hinder unwanted competitors from bidding, they are likely to resort to alternative—and potentially socially more costly— rent-seeking strategies (Fisman & Golden, 2017), such as allowing connected bidders to lowball bids and later to still make nice profits by cutting corners when implementing public works projects. In addition, Small and Medium Enterprises (SMEs) may lack the Information and Communication Technology (ICT) skills for submitting bids online, and hence lose out under e- procurement. 4. Furthermore, the evidence on the impacts of e-procurement poses a puzzle. The most reliable study on the impact of e-procurement (Lewis-Faupel et al., 2014) finds evidence that e-procurement broadened the bidder pool (i.e. increased the share of non-local winners) in India and Indonesia. Yet, interestingly, it finds no evidence that e-procurement increased the number of bidders or lowered prices, contrary to theoretical predictions. Why? Part of the answer may be context. To fully yield its benefits, e-procurement may require enabling conditions such as an ample number of potential market entrants (Straub, 2014) and sufficient capacity in the public and private sectors to use e-procurement (Thai, 2009 section III). Furthermore, e-procurement systems vary in the scope of procurement processes they digitize and in their quality, impacting results (Croom & Brandon-Jones, 2005; Mendes & Fazekas, 2017). 5. As the effects of e-procurement are ambiguous and context-contingent and the evidence so far is limited and inconclusive, further research on its effects and the underlying mechanisms is needed. Seeking 2 to contribute to this literature, this paper report findings from an impact evaluation of Bangladesh’s e- procurement system. It poses the research question: Does electronic procurement, in particular electronic tendering and contract award, improve value for money (i.e. economy), access to public tenders and administrative efficiency? If yes, under which conditions? 6. When the Government of Bangladesh (GOB) introduced e-procurement in 2011, the country provided both a promising and challenging context. Procurement-related violence frequently made the headlines of newspapers, as local “strongmen” connected to insider firms systematically used armed gangs 6 to physically hinder competing bidders from submitting paper-based bids. Part of e-procurement’s promise was that it could cut through such practices and drastically increase access to public tenders. At the same time, limited administrative capacity and high staff turnover made introducing e-procurement challenging (World Bank, 2002). 7. Crucially to our claim of novel insights, the scope of e-procurement in Bangladesh goes beyond most if not all such systems in low and middle-income countries. It digitizes the entire contracting process, from advertisement to contract award, including buyer-bidder communication, bidder registration, bid submission, and bid evaluation, with no parallel “paper trail”. Hence, it is a fully functional transactional system. By contrast, most other e-procurement systems only provide for online advertisement and submission of tenders, but the “back-end” of tender evaluation and award remains largely paper-based (OECD, 2016). 8. This impact evaluation focuses on the adoption of e-procurement by the three government agencies that are jointly responsible for most7 of Bangladesh’s public works procurement: the Local Government Engineering Department (LGED), the Roads and Highways Department (RHD), and the Bangladesh Water Development Board (BWDB). These three agencies were also the first to introduce e-procurement in Bangladesh, starting in FY2012-13. 9. Our primary data source is administrative data on public procurement. We obtained detailed electronic procurement data for all electronic tenders of BWDB, LGED and RHD from Bangladesh’s Central Procurement Technical Unit between FY2011-12 and FY2017-18, a total of 69,240 tenders for 185 district-level procuring entities.8 To obtain equivalent data for paper-based tenders, we surveyed paper- 6 In 2014, Bangladesh ranked 145th out of 175 on TI’s Corruption Perception Index. Numerous high -profile, high-value corruption scandals in public procurement further support the perception that corruption is a widespread problem. 7 Based on the number and value of contracts. 8 Bangladesh has 64 districts, but each agency does not have an office in each of these districts. For example, BWDB does not have offices in districts without any significant rivers or canals. 3 based procurement records for a sample of 10,319 tenders from the same district-level procuring entities between FY2011-12 and FY2016-17. FY2011-12 serves as the baseline and FY2017-18 as the endline year, by which all three agencies exclusively procured through e-procurement. A survey of 600 procurement officials in the three agencies served to capture their perceptions of the new e-procurement system and to measure management practices. We also surveyed 600 firms that participated in public works contracts of the three agencies across 16 districts, to understand their transaction costs and perceptions about the e- procurement system. 10. Identifying the causal effects of e-procurement is challenging because e-procurement has been introduced and phased in at the discretion of implementing authorities and their choices may be strategic, for example trying to minimize the exposure of corrupt practices. The central identification challenge in our non-experimental setting is that the district-level procuring entities (PE) in our sample had the discretion to choose when and for which tenders to adopt e-procurement. This poses the risk of bias due to i) self- selection of PEs into e-procurement, ii) strategic sorting of tenders into e-procurement within each PE, iii) potential spillovers between electronic and paper-based tenders (for example because officials focus their attention on electronic tenders), and iv) potential learning effects to take place for procuring entities adopting the new system at different speeds. We combine Coarsened Exact Matching with fixed effects regressions in order to approximate causal effects. We run two parallel specifications. First, we match tenders within procuring entities but across years. Second, we match tenders within the same years, but across procuring entities that have similar baseline dependent variable values. In each of these cases, we look at the differences in the dependent variables of tenders least likely to be influenced by spillover effects. Specifically, we compare manual tenders administered by procuring entities that run mainly manual tenders with electronic tenders administered by procuring entities that run mainly electronic tenders. 11. We find overwhelmingly positive impacts of e-procurement on a wide range of outcomes tracked. First, e-procurement improves open and fair access to public tenders as bidder numbers go up, single bidding decreases and non-local firms gain market share. These positive impacts are largely due to effects on the “lower tail”. That is, e-procurement led to large improvements in access for PEs with low access at baseline. Second, e-procurement also improves value for money (i.e. economy) as winning rebates, that is discounts firms offer, increase. This impact results both from increasing the number of bidders and from enabling more intense competition among a given number of bidders. By contrast, we find no robust impact on contract implementation performance, namely on cost and time overruns. Third, e-procurement enhances administrative efficiency, as the total time for processing a tender—from the public call for tenders to contract signature—drops substantially. The time required for every major stage of this process shortens, 4 9 except for the decision-making period. This is in part because the electronic system effectively eliminates outlier processes, ensuring that virtually all tenders are processed within the legally mandated timelines. 12. These predominantly positive effects translate into large and continuous cost savings for the government—as well as for bidding firms, albeit to a lesser degree. We estimate that, between 2012 and 2016, the three government agencies saved USD 459 million to USD 513 million, mainly in purchase prices, which far surpass the cost of setting up the e-procurement system (about USD 70 million). If e-procurement were to cover 100 percent of public procurement spending, the GOB could save up to USD 1.76 billion per year, assuming the same savings rates across the whole budget. 13. Using the above direct impacts of the introduction of e-procurement, it is also possible to estimate economy-wide effects, based on a macroeconomic model that takes into account indirect and multiplier effects. For 2019, the model suggests that e-procurement increased Bangladesh’s gross domestic product (GDP) between 0.48 and 0.54 percent. It also suggests that e-procurement increased employment by 0.15 to 0.17 percent, adding around 100,000 extra jobs. These jobs are widely dispersed across different sectors. The largest macroeconomic impacts of e-procurement come from government savings, which are assumed to translate into increased government spending across sectors. Direct e-procurement impacts on private companies are small, overall. One noteworthy negative impact is that e-procurement reduces the need for advertising and hence lessens revenues for print media/newspaper publishers. However, this effect is far outweighed by the indirect positive impacts of higher government spending (Pollitt et al, forthcoming). 14. Our results contribute to the literature on e-procurement and anti-corruption. First, they confirm the beneficial effects of e-procurement on a range of outcomes, delivering further evidence for further investing in e-procurement systems. Second, our findings also underpin arguments in the literature that greater transparency and tighter monitoring of some procurement stages may displace rent-seeking to less transparent and less monitored stages, such as contract implementation. Third, contrary to empirical evidence from India and Indonesia, but in line with theoretical predictions, we find that e-procurement has a positive effect on the number of bidders and prices. Fourth, our analysis suggests that e-procurement delivers the most value where prior performance and regulatory compliance was weak, in line with cross- country evidence (Bosio, Djankov, Glaeser, & Shleifer, 2022). 15. The paper proceeds as follows. First, we summarize Bangladesh’s institutional context and the e- procurement reform design. Second, we identify the relevant theory and causal mechanisms for understanding how e-procurement can affect the outcomes of interest. Third, we outline the data and 9 Decision Period is the time taken by the agency officials to evaluate the bids and declare the winner. It is measured as the time elapsed in days from the tender opening date to the notification of award date. 5 indicators used. Fourth, we discuss the challenges of causal identification and our proposed solutions. Fifth, we present and discuss our results, including microeconometric estimates of direct impacts and macrosimulation estimates of indirect impacts. We conclude by highlighting major policy implications and the next steps for further research. 2. INSTITUTIONAL CONTEXT AND REFORM DESIGN 16. The GOB has steadily improved the procurement environment since 2002, with support from two consecutive World Bank–funded projects (Public Procurement Reform Project I and II). In 2002, the government established the Central Procurement Technical Unit (CPTU), a national procurement policy unit. The government enacted a Public Procurement Law in 2006 and associated Procurement Rules in 2008, simplifying procurement procedures and making them more transparent. 17. As part of this broader procurement reform agenda, in 2009, the government commissioned the design of a custom-made e-procurement system, intended to improve access to public tenders and to speed up contract awards. Bangladesh’s e-procurement system is one of the world’s most advanced and comprehensive. It allows for a comprehensive, completely paperless management of the public procurement process starting from annual procurement plans to the contract award. It requires procuring entities to advertise tenders electronically, and firms to submit bids online. All administrative decisions about competing bids, from the opening of the bids to the awarding of contracts, are also recorded online. The system hence generates uniquely rich electronic records on the procurement process, which permits the tracing of decision-making processes in detail, such as the composition of bid-evaluation committees and the choices made by individual committee members. 18. The initial e-procurement roll-out between FY2011-12 and FY2015-1610 focused primarily on the four largest government agencies with the largest number and value of contracts. The aforementioned three—BWDB, LGED and RHD—and the Bangladesh Rural Electrification Board (BREB). BWDB, LGED and RHD share a similar organizational structure, with procuring entities at multiple hierarchical levels. Figure 1 illustrates this structure for the example of LGED. Each of the three agencies has 64 district- level procuring entities, one for each district. These district PEs are the focus of this study, because they are responsible for the largest share of tenders, both in number and value. They also shape comparable clusters, with similar tasks and levels of authority. BREB was excluded from the study, as it only has a few central PEs. 10 Please note that our dataset used for the analysis also includes 2 additional years after the initial roll-out (FY16-17 & FY 17-18) in order to observe PEs once they fully adopted e-procurement. 6 FIGURE 1: STYLIZED ORGANIZATIONAL STRUCTURE OF LOCAL GOVERNMENT ENGINEERING DEPARTMENT11 Agency Head Office Chief Engineer Divisional Offices Additional Additional Chief Engineer Chief Engineer (8 Divisions) (Division 1) (Division 8) Regional Offices Superintending Superintending Superintending Engineer Engineer Engineer (20 Regions) (Region 1) (Region 2) (Region 20) Units of District Offices Executive Executive Executive Engineer Engineer Engineer primary (64 Districts) (District 1) (District 2) (District 64) interest Upazilla Offices Upazila Upazila Upazila Engineer Engineer Engineer (492 Upazillas) (Upazilla 1) (Upazilla 2) (Upazilla 492) Source: Authors 19. Uptake of e-procurement has skyrocketed since 2012 in the three agencies under investigation. At the end of our observation period, virtually all the tenders in LGED, BWDB and RHD were processed through e-procurement (Figure 2). Similarly, on the bidder side, the number of firms registered in the e- procurement system has grown exponentially. As of the beginning of 2022, about 95,000 firms had registered in the e-procurement system. 11 A complete organogram of Local Government Engineering Department (LGED) can be found here: Local Government Engineering Department, Organogram, Approved in Year 2018, available at https://www.lged.gov.bd/site/page/e66db6f5-aca3- 488c-92d7-1c833ff4448e/ 7 FIGURE 2: SPEED AND SCALE OF E-PROCUREMENT ADOPTION12 100% 100% 100% 100% 100% 100% 100% 100% 100% 94% 96% 90% 87% 90% 80% 70% 60% 55% 49% 50% 37% 40% 30% 20% 14% 10% 5% 0% 0% 0% 1% 0% 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 BWDB LGED RHD Source: Data collected from procuring entities and e-procurement system 3. THEORETICAL FRAMEWORK 20. Well-functioning public procurement systems are meant to achieve value for money, ensure open and fair access for bidders and run efficiently. We consider these three distinct sets of outcomes—(i) economy; (ii) access and (iii) efficiency—separately, while recognizing their interdependence. In this section, we define these concepts and identify the main mechanisms through which e-procurement can affect them. In the following section, we will map these concepts to specific indicators. 3.1 OUTCOME DEFINITIONS 21. “Economy” or “value for money” for a given quantity of products refers to the quality of goods or works obtained for a given procurement price. Accordingly, economy can increase because a given quality is achieved at a lower price or because a higher quality is achieved at a given price or both. 22. “Access” in the context of public procurement captures the extent to which the procurement process de facto ensures open and fair competition among bidders. “Limited access” refers to a situation in which public procurement contracts are allocated and managed in ways that benefit a closed network of firms and government officials, while intentionally restricting access to others. This definition applies North, Wallis, & Weingast's (2009) concept of “limited access orders” to the context of public procurement. Limiting access typically involves bending explicit rules and principles of good public procurement. Such a conceptualization is very close to political science concepts that are recently gaining prominence such as quality of government as impartiality (Rothstein & Teorell, 2008), and particularistic governance regimes 12 A granular version of this graph showing the transition of each procuring entity has been included in Appendix 9.8. 8 (Mungiu-Pippidi, 2015). In practical terms, limited access can be achieved if public officials steer contracts to a favored bidder by, for example, unjustified sole sourcing or direct contract awards; favoring a certain bidder by tailoring specifications or sharing inside information; or paying connected suppliers on time while delaying due payments to others (World Bank, 2009). Moreover, access might also be limited by colluding firms barring non-cartel member firms from access (e.g. through physically preventing them from submitting paper-based bids). 23. “Administrative efficiency” refers to the total transaction costs for completing a procurement process to society. It is approximated by the total administrative cost incurred by the government for achieving the predetermined outcome of public procurement, that is, the successful completion of the contract. We largely refrain from the full analysis of administrative costs for bidders due to a lack of sufficiently detailed time-series data (i.e. not considered in the micro-estimates but taken into account in the macro modeling). 24. The total direct savings achieved through an e-procurement intervention could hence be approximated by adding monetized values of the three outcome dimensions. FORMULA 1 = ∆ + ∆ + ∆ (. +) 3.2 MECHANISMS 25. The main conceptual challenge of linking e-procurement to the above three outcomes is that e- procurement is a bundled intervention. Bangladesh’s e-procurement system, like most e-procurement interventions, bundles together a variety of changes which have a range of impact channels (Blum et al., 2018). We consider three main impact mechanisms as the most important for explaining the three outcomes of interest. While there is no simple mapping between mechanisms and outcomes, their relationship can be summarized as follows: 26. E-procurement can (i) make it more costly for public officials to erect intentional barriers to entry, by reducing their de facto discretion and by increasing the risk of detection, among others. It can (ii) reduce nonintentional barriers to entry by lowering the transaction costs for bidding firms. Both mechanisms are expected to directly affect access—and indirectly economy, as a more open and fair competition among bidders is expected to enhance value-for-money. E-procurement can (iii) increase administrative efficiency by automating the procurement process and removing information transfer costs for the government (e.g. postal costs). We discuss these mechanisms in greater depth below and set out hypotheses while also 9 considering how some mechanisms are intertwined with multiple outcome groups and why some of our expectations are a priori ambiguous. 27. First, e-procurement is expected to lower intentional barriers to entry, by making it harder for public officials to create some types of barriers for unwanted competitors. E-procurement can automatically ensure compliance with procurement rules and hence reduce discretion. For example, automatically publishing notices about new tenders on a national website makes it difficult to hinder undesired competitors from accessing information about tender opportunities (Coviello et al., 2018). As e-procurement enables firms to download tender documents and submit bids online, it also makes intentional barriers obsolete that rely on enhancing the cost of bid submission––such as through physical violence, complex administrative procedures, or by denying receipt of tenders from unwanted bidders. This is particularly salient in Bangladesh, as extortion and the use of physical violence to limit competition were widespread under paper- based procurement. 28. E-procurement is also expected to lower intentional barriers to entry because it makes rent-seeking riskier. It produces an easily accessible electronic audit trail13 about the decisions leading to contract award (bid evaluation), facilitating managerial oversight and audit and hence increasing the risk of getting caught (Di Tella & Schargrodsky, 2003; Olken, 2007). This risk is salient in Bangladesh’s context because the CPTU uses e-procurement data for proactively monitoring procurement risks14 and has full authority to act on findings. This may motivate rent-seeking officials to go clean or to adopt strategies that are not easily 15 detectable in electronic records. Consequently, if substitute strategies are unavailable, we expect e- procurement to unambiguously reduce intentional entry barriers. 29. However, e-procurement need not lower intentional barriers to entry if collusive agents16 find alternative strategies for raising these barriers.17 They may respond to reduced discretion or tighter monitoring in one part of the procurement process by substituting rent-seeking strategies away to other procurement phases (Dávid-Barrett & Fazekas, 2020). For example, once e-procurement makes it impossible to prevent competitors from bidding, connected or corrupt firms could switch to a “lowballing” 13 The audit trail comprises detailed records of public procurement procedures and personalized records of actions (administrators log in with unique IDs and their clicks are recorded to the second). Compared to paper-based records (which are stored in deconcentrated PEs), these electronic records are relatively hard to purge without being noticed and easily accessible to auditors. 14 See for example the PROMIS reporting tool: https://cptu.gov.bd/promis/promis-content.html 15 However, the deterrent effect of audits depends heavily on the commitment of political and administrative principals to use audits to fight corruption. In Bangladesh, we expect this commitment to be relatively low on average and heterogeneous across PEs interacting with local politics. 16 “Collusive agents” is here used as an umbrella term to jointly refer to firms, public officials, and politicians engaging in corruption in the procurement process. 17 Such strategies could, for example, comprise (i) tailoring the tender criteria for specific bidders, (ii) excluding bids based on superficial administrative grounds as ineligible, or (iii) using subjective assessment criteria unfairly for scoring bidders (Fazekas & Kocsis, 2015). 10 strategy. They could outcompete other firms with unprofitably low prices at the bidding stage, knowing that they would be weakly supervised during contract execution, enabling them to cut corners on quality and to make a profit. Conversely, non-connected and clean firms may anticipate that even if they win the contract, they will be bullied by local armed strongmen or paid late. This may drive up their risk and price and possibly discourage clean firms from bidding. 30. More generally, no electronic system can eliminate all possibilities for intentionally limiting entry and hence access; at best it can make widely exercised tactics more expensive and hence less likely. While substitute strategies are a priori more costly to collusive agents (otherwise collusive agents would have adopted them in the first place), it is also unclear if they will socially be more or less costly—and hence how they will affect economy. For example, strategic bargaining during the contract implementation stage influences bidder behavior and can inflict considerable costs on public budgets (Bajari et al., 2014; Decarolis & Palumbo, 2015). More broadly, the adaptability and flexibility of corrupt agents in response to anticorruption reform is increasingly being studied (Fisman & Golden, 2017). Given the wide availability of substitute corruption techniques in complex public procurement systems (Fazekas et al. 2016), which concrete sets of strategies dishonest agents will choose under each intervention is only tentatively predictable based on the expected cost-benefit ratio. 31. Second, e-procurement can reduce nonintentional barriers to entry, that is firms’ transaction costs associated with submitting a bid, for several reasons. It can reduce the cost firms face in obtaining information on tendering opportunities. Under a paper-based system, informational costs can represent a substantial entry barrier. In extreme cases, if tender opportunities are only advertised in local newspapers with limited circulation or only on physical notice boards, nonlocal bidders practically cannot obtain information about them. Advertising tenders on a national public procurement website in a standardized format provides firms with free and timely access to information, regardless of their location (Coviello & Mariniello, 2014). Furthermore, e-procurement can reduce the cost of submitting bids over a distance. 32. E-procurement can also reduce the costs of submitting bids. Under a paper-based system, it can be financially costly, time-consuming, and uncertain for firms (for example, because postal packages may not arrive on time) to physically submit bids, which often contain sensitive commercial information. Under an electronic submission system, by contrast, submitted bids are received instantaneously. Receipt acknowledgement eliminates the uncertainty of bid delivery. Moreover, paper-based systems typically entail numerous costly document transfers. For example, firms may be required to resubmit basic qualifying documents repeatedly for different bids. Or they must transfer documents internally between the company headquarters and the local offices submitting the bids. Errors in records, such as a missing stamp on a certificate, can lead to the firm’s exclusion from the tender. An electronic system can sharply decrease these 11 costs because it fills in forms automatically based on past data, makes information transfer instantaneous and practically free, and prevents a range of errors automatically. Furthermore, if e-procurement speeds up the procurement process, it can reduce the opportunity costs for firms from lost interest earnings on bid security deposits. 33. However, e-procurement may also increase transaction costs and hence nonintentional barriers to entry. All firms have to initially register in the system and familiarize themselves with it, which requires an initial investment. 18 Hence, it can affect the composition of bidders (Lewis-Faupel et al., 2016). While e- procurement is expected to reduce nonintentional barriers to entry for most firms, it can raise them for some firms that lack ICT capabilities for using a complex e-procurement platform, especially SMEs. The net effect thus depends on the share of firms in either group, on their adaptiveness over time and on the design features of the e-procurement system itself. 34. As the effects of e-procurement on both intentional and unintentional barriers to entry are ambiguous, so are the net effects of e-procurement on access—and, indirectly economy. Given the overwhelmingly positive impacts documented in countries similar to Bangladesh we formulate the following two hypotheses: H1: E-procurement improves access to public tenders; H2: E-procurement improves economy in public tenders. 35. Third, e-procurement is expected to decrease the cost of administering tenders for civil servants (Buyse et al., 2015; Strand et al., 2011) and hence increase efficiency. This is the least ambiguous mechanism. E-procurement can dramatically decrease staff time of government officials, paper and printing costs, and administrative errors. It fills forms automatically based on past data, whereas, under a paper- based system, the same information has to be recorded repeatedly in different documents. It makes the transfer of documents among the evaluators and the approving authority of the bidding process instantaneous and practically free, which can be time-consuming and costly for paper-based tenders. E- procurement can also automatically prevent a number of costly errors which could sink the whole tendering process (for example, following the wrong procedure type given tender value). 36. However, these transaction cost savings depend on whether public officials using the e- procurement system develop the necessary computer literacy and system-specific knowledge (Blum et al., 2018). If they fail to make full use of the portal’s functionalities (for example, by manually re-entering information which could be automatically filled in) or enter incomplete or inconsistent data the expected savings might not materialize. E-procurement may also increase administrative costs by introducing new 18 E-procurement may also increase firms’ administrative costs by introducing new types of costs due to the rigidity of system design (that is, the system might not be able to accommodate certain atypical cases) or IT system breakdowns. 12 types of costs such as system design rigidity (that is, not being able to accommodate certain atypical cases) and IT system breakdowns. Once again, given the overall positive effects documented by past studies, we propose the following hypothesis: H3: E-procurement improves efficiency of public tendering administration. 4. DATA AND INDICATORS 4.1 DATA SOURCES 37. The sample for this study comprises tenders in the respective district-level PEs of BWDB, LGED and RHD, that is for a total of 185 PE clusters. To ensure homogeneity, our sample only includes public works projects, which represent the bulk of procurement spending in these organizations. In BWDB, most works projects consist of the construction of embankments, canals, riverbank protection work, dredging and flood control work. In LGED, most works projects comprise of construction and maintenance of small roads, schools, buildings etc. In RHD, most works projects consist of the construction of big district and subdistrict roads and highways. 38. Administrative data on e-procurement tenders come from the e-procurement database, exported by the government for the research team. It captures the full population of e-procurement tenders for FY 2011- 12 to FY2017-18, totaling 191,785 tenders. A survey firm collected data from procurement records for a representative sample of paper-based tenders processed between 2011-12 to 2016-17. The sampling frame consisted of the full population of about 31,500 manual tenders. A stratified sampling method was used where the strata were defined by district and method of procurement (open tendering method and limited tendering method), and financial year. Strata with few tenders were oversampled, with the sample size of the stratum being equal to the population of that stratum if the population was not greater than 3 tenders. The physical files for about 15% of the tenders that were sampled could not be located (e.g. because flooding destroyed the records); hence, they had to be replaced from the same stratum. Some of the replacements had to be made from different strata, especially in the cases where the population of the stratum was low. These tenders were collected from the same Procuring Entity but from a different financial year. We sampled about 33% of the population resulting in a sample of 10,319 paper-based tenders. Figure 3 shows the number of observations collected for manual and e-procurement tenders by FY, and the number of firms registered in the e-GP system, reflecting the gradual uptake of e-procurement. 13 FIGURE 3: DATA COLLECTED FROM PROCURING ENTITIES AND E-PROCUREMENT SYSTEM19 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 2017-18 Number of Manual 3953 Tenders in Dataset 3737 1952 424 248 5 Number of e- 87,912 procurement Tenders in Dataset 45,488 32,024 17,810 8,045 15 491 Number of firms 56058 registered in the e- GP system 40472 25996 17271 9698 285 1078 Source: Data collected from e-procurement system and procuring entities 39. The survey teams for manual tenders also conducted in-person interviews with about 600 procuring entity officials from February to June 2018. This additional survey served to capture district officials’ perceptions of the e-procurement system and to obtain measures of the costs associated with administering the procurement process manually vis-à-vis electronically. Each procuring entity office at the district level is headed by an executive engineer (XEN). In addition to the executive engineer, the procuring entity offices also have Sub-divisional Engineers (SDE), Assistant Engineers (AE) and Sub-Assistant Engineers (SAE), who work in conjunction with the XEN to dispose of the duties of the procuring entity office. 40. Furthermore, a survey of about 600 firms participating in public works contracts of the 3 agencies was done from June to August 2019, to capture firm transaction costs. To sample firms for the survey, the population of firms were stratified into 4 strata: (i) eGP winners (firms with revenue from eGP tenders at least 3 times more as compared to manual) – 175 such firms were selected; (ii) Manual winners (firms with revenue from manual tenders at least 3 times more as compared to eGP) – 175 such firms were selected; (iii) Always winners (firms that are not in the eGP or manual winners and had revenue greater than median 19 We did not apply any weighting to the survey data in the subsequent calculations. 14 revenue from both manual and eGP tenders) – 150 such firms were selected; (iv) Always Losers (the remaining firms that were not a part of eGP, manual or always winners). 4.2 INDICATORS 41. This section sets out how the three major outcomes of interest— access, economy and efficiency— will be measured. Table 1 provides an overview of the indicators used for each concept. 42. Measuring limited access in public procurement directly follows from the idea of an unjustified restriction of access. Consistent with a set of objective proxies used in the economics and political science literature (Charron et al., 2017; Chong et al., 2015; Coviello & Gagliarducci, 2017; Fazekas & Kocsis, 2020; Fazekas & Tóth, 2016; Klasnja, 2016), we proxy access through (i) the number of bidders participating in each tender, through (ii) the share of tenders with only one bidder (single bid), and by (iii) identifying if a tender is won by a local or non-local firm. Non-local firms are firms that are not from the district where the tender was invited. 43. Economy measures crucially depend on an adequate conceptualization of quality which is market- specific and contestable in the case of many services. As we lack reliable quality measures, we will focus predominantly on prices which are more readily observable. Whereas for highly homogenous goods, such as office equipment, absolute price comparisons are useful (Bandiera et al., 2009), this is impossible for public works. We hence rely on the winning rebate as a relative measure, which compares the original engineer’s estimate with the awarded contract price and refers to the percent age of costs saved relative to the former (Coviello et al., 2018; Decarolis, 2014). 44. We further use delays and cost overruns in contract execution as proxy indicators of economy. Delays are captured by time overruns, that is by how many days the contract was extended beyond the original deadline. Such delays are typically associated with economic costs—for example, because citizens cannot use new infrastructure. They also often signal quality problems requiring corrections. Cost overruns are defined as the ratio of the final amount paid to a firm and the original contract price. Given that both measures capture performance during the contract implementation stage, which was not digitized by the e- procurement system during the period of study, they are indicative of potential displacement effects. 45. Administrative efficiency can be measured by monetizing and summing the government’s total costs of administering tenders and monitoring the implementation of contracts. These costs primarily include staff time, but also advertising costs, costs for printing and mailing tender documents and costs incurred by misprocurement due to administrative errors. The survey of procurement officials served to gather proxy measures of these costs which are not easily observable. The only efficiency-variable available at the tender-level is the time spent on the procurement process. We consider both the overall time required 15 from tender invitation to contract signing and the break-down of time used for each part of the procurement process (submission period, decision period, and signing period).20 46. Prior to the analysis, a rigorous data cleaning exercise was done. Duplicate tenders, packages that were retendered due to no bids, data entry errors, etc. were removed from the dataset used for analysis. Inconsistent values such negative time periods, and very low contract values were removed. Efficiency indicators with values above the 99.75th percentile were set to missing, similarly to winning rebate values below 0.1 and above 99.9 percentile. These outlier values most likely represent measurement error or highly unusual tender outcomes potentially biasing the analysis. 47. The study controls for two tender-level covariates when estimating the impact on outcomes: procurement method and contract value of the tender. The main procurement methods considered in the study are the Open Tendering Method (OTM) and the Limited Tendering Method (LTM). The population of paper-based tenders was roughly distributed in a 3:2 ratio between OTM and LTM, respectively, as was the random sample stratified by procurement method. OTM tenders are usually nationally competitive tenders. Any firm that fulfills the qualifying criteria of the specific tender can submit their bids for OTM tenders and compete. LTM tenders are advertised only to a limited list of firms that are registered with the procuring entity and fulfill the qualifying criteria sought. Typically, LTM tenders are advertised for smaller value contracts for firms based locally (i.e. in the district of the procuring entity office). The maximum value of an LTM contract can be BDT 20 million (US$ 240k). 20 Please refer to table 1 for definitions of these periods. 16 TABLE 1: DEFINITIONS AND DESCRIPTIVE STATISTICS OF THE VARIABLES USED IN THE ANALYSIS 21 Coefficient/ Mean/Observations p-value Definition Outcomes Variables Treatment Manual e-proc. Total vs Control Ln of 1.523 1.362 1.412 -0.161*** (0.007) (0.004) (0.004) (0.008) Measured as the natural logarithm of the number Number of of tenders submitted as on tender opening day Bidders 30626 69240 99866 0.171 0.194 0.187 0.022*** Single (0.002) (0.002) (0.001) (0.003) =1 if the number of bidders on a tender is equal Access Bidders to 1 30626 69240 99866 0.126 0.216 0.186 0.09*** Non-Local (0.002) (0.002) (0.001) (0.003) =1 if district of the winning firm is not equal to Winners the district from where the tender was invited 30147 60472 90619 0.724 6.968 5.073 6.243 Winning Rebate was calculated as; Winning (0.052) (0.035) (0.03) (0.062) − ( ) Rebate 29922 68705 98627 × 100 0.305 0.191 0.299 -0.114*** Ratio of Cost Overrun is calculated as; Economy Time (0.003) (0.011) (0.003) (0.013) − Overrun ( ) 29251 1349 30600 -0.016 -0.017 -0.016 -0.002 Cost Overrun (0.001) (0.004) (0.001) (0.003) =1 if the contract was extended 28318 1307 29625 25.55 19.48 22 -6.07*** Submission (0.04) (0.02) (0.03) (0.05) Measured as the time elapsed from tender notice 22 Period date to tender submission date 30661 43137 73798 25.4 21.07 22.66 -4.33*** Decision (0.14) (0.07) (0.07) (0.14) Measured as the time elapsed from tender Period opening date to notification of award date 24950 43108 68058 Efficiency 17.85 16.05 16.65 -1.8*** Signing (0.08) (0.04) (0.04) (0.08) Measured as the time elapsed notification of Period award date to contract signing date 29851 60068 89919 83.58 65.91 71.82 -17.68*** (0.24) (0.09) (0.1) (0.21) Measured as the time elapsed from tender notice Lead Time date to contract signing date 30198 60114 90312 1.479 1.828 1.72 0.349*** =1 if procurement methods used is Limited Procurement (0.003) (0.001) (0.001) (0.003) Tendering Method and =2 if method used is Method Independent 31088 69240 100328 Open Tendering Method Variables Ln of 14.805 14.902 14.872 0.097*** (0.008) (0.005) (0.004) (0.009) The natural log of the contract price stated in the Contract original contract signed with the winner Value 31088 69240 100328 Note: weighted figures (for more on weights see section 6). 21 Additional Descriptive Statistics have been included in Appendix 9.7. 22 In the Bangladesh public procurement system, the tender submission date and the tender opening date are only one day apart by design. There is no variation in this under any normal circumstance. 17 5. CAUSAL IDENTIFICATION 48. Identifying the causal impact of e-procurement on access, economy and efficiency is challenging as e-procurement adoption is clearly non-random either at the procuring entity or tender levels (see section 3). In sum, we have to grapple with five main sources of bias: i) omitted variable bias; ii) self-selection of procuring entities into the e-procurement system; iii) strategic sorting of tenders within procuring entities; iv) spillovers from treated to non-treated tenders within procuring entities; and v) learning effects and increasing firm registrations entail heterogeneous effects over time. 49. We employ a range of estimation strategies to address each of the above biases. None of these strategies can address all of these potential sources of bias on their own. Taken together, however, they give a reasonable estimation of causal effects (Table 2). This section first describes each of these challenges in greater detail and then outlines how we address them. 50. First, omitted variable bias stems from our limited ability to observe all relevant characteristics both of individual tenders and of procuring entities, the public bodies administering tenders through manual or e-procurement systems. For example, even if tenders are of similar size and have been procured using the same methods, they might differ on harder-to-observe characteristics, such as the nature of works (such as roads vs. school buildings) that may be associated with outcomes (e.g. the likelihood of time overruns) and treatment status. Similarly, procuring entities have unobserved and hard to precisely measure qualities, like management style, staff motivation, work ethics, or e-government readiness that affect procurement outcomes and may be associated with e-procurement take-up. 51. The second threat to causal identification comes from the potential self-selection of procuring entities into adopting e-procurement. For example, better performing procuring entities or those with a lower propensity for corruption might more rapidly adopt e-procurement—wishing to reap its benefits as early as possible, or in view of signaling their readiness to reform to superiors and the market. 52. Third, procuring entities may sort tenders into e-procurement for strategic reasons (strategic sorting), reflecting their organizational goals and institutional quality. For example, they may initially decide to only use the new e-procurement platform for smaller tenders,23 in view of lowering the risks of making mistakes, due to their lack of experience with the new system. Corrupt procuring entities could try 23 In the survey of procuring entity (PE) officials conducted, 108 procurement officials stated that they assigned tenders to e- procurement according to estimated tender value. 18 to keep processing those tenders that offer the “juiciest” rent-seeking opportunities outside the e- procurement system, and process only less attractive tenders through e-procurement, as long as possible. 53. Fourth, if a procuring entity adopts the e-procurement system only partially, spillover effects could occur between electronic and manual tenders. Public officials could for example direct extra attention and time to electronic tenders and divert it from manual tenders, say because the timely processing of electronic tenders is subject to higher managerial scrutiny. This would entail negative spillover effects from electronic to paper-based tenders. 54. Fifth, we also expect impacts to vary over time. Learning effects are one reason, especially in the first few years of e-procurement adoption. We expect that the positive effects of e-procurement might not or only partially materialize in the initial period after its adoption by a procuring entity. One major reason is that it will take time for procurement staff to acquire experience with the e-procurement platform and operate it effectively. Similar learning effects may also arise on the side of suppliers, after they register on the e-procurement platform. Another reason for time-varying e-procurement impacts is the above-described exponential growth of the number of firms registered in the e-procurement system. The more suppliers sign up and start to use e-procurement nationwide, the higher the expected impact of e-procurement adoption (by a particular procuring entity) on access and economy. 55. We employ two main complementary identification strategies that rely on coarsened exact matching combined with OLS fixed effects regressions, i.e. doubly robust regressions (Funk et al., 2011). Both strategies have three steps in common. First, we restrict the sample of treated and control tenders to procuring entities with high and low e-procurement saturation levels, respectively, to reduce potential bias from strategic sorting and negative spillovers. Second, for both strategies, we carry out coarsened exact matching of treated and control tenders. Third, we estimate OLS regressions with a range of fixed effects (FEs) on the matched samples (Funk et al., 2011). The main difference between the two identification strategies is how matched samples are constructed in step 2. In the first strategy, we match within PEs across years, in view of addressing bias from PE self-selection, accepting the possibility of bias from learning effects. In the second strategy, we match within year, across PEs with similar baseline DV average and experience with the e-procurement process. Here, the focus is on addressing bias from learning effects, accepting the possibility of bias from unobservable PE self-selection. Table 2 summarizes how well the two identification strategies (strategies 1 and 2) and their subsequent robustness address the different potential sources of bias. 19 TABLE 2: SUMMARY OF MAIN IDENTIFICATION STRATEGIES AND HOW THEY ADDRESS POTENTIAL SOURCES OF BIAS self-selection strategic sorting spillovers from treated Learning Identification unobserved of PEs into e- of tenders within to non-treated tenders effects strategy/bias variable bias procurement Pes within PEs Strategy 1: FE Yes regression with Partially Partially Partially (comparisons Yes matching within PE (30 vs 80% sat.) (30 vs 80% sat.) (year FEs) within PEs) across years Strategy 2: FE regression with Partially Partially Partially Partially Yes matching within years (PE FEs) (PE FEs) (30 vs 80% sat.) (30 vs 80% sat.) across similar PEs Robustness test: FE Partially Partially Partially Partially Partially regression without (PE FEs) (PE FEs) (30 vs 80% sat.) (30 vs 80% sat.) (year FEs) matching 56. In both identification strategies, we first address potential bias from strategic sorting and negative spillovers, by considering treated and control tenders in PEs which have high and low e-procurement saturation levels, respectively. Specifically, we restrict the sample of electronic tenders considered as “treated” and of manual tenders considered as “controls” based on the respective PE-year e-procurement saturation level, which is based on the percentage of all awarded contracts processed through e-procurement in the respective year. We restrict “control” tenders to manual tenders in low e-procurement saturation PE- years (0-30 percent) and “treated” tenders to electronic tenders in high saturation PE-years (80-100 24 percent). Both identification strategies then compare these restricted samples of control and treated tenders. 57. This set-up is expected to bound potential bias from strategic sorting and negative spillovers, for the following argument: If PEs have only adopted e-procurement for none or only for a minority of their tenders (0-30 percent), the sorting and spillover effects on the remaining 70-100 percent of manual tenders are expected to remain small, on average. Conversely, once PEs have adopted e-procurement for most of their tenders (80-100 percent), sorting and spillover effects on these electronic tenders from the remaining few manual tenders are expected to remain small—simply due to the lack of manual tenders bureaucrats could sort into or divert attention from. 58. Second, we carry out Coarsened Exact Matching (Iacus et al., 2012) between the restricted sets of treated and control tenders before running our main regressions. This matching takes two different forms: i) matching within PEs across years, and ii) matching within year, across PEs with similar baseline DV 24 The main results were also calculated based on symmetric e-procurement definitions, i.e., low e-procurement saturation PE- years corresponding to 0-25 percent of tenders being electronically processed and that for high saturation PE-years being 75-100 years. Results remain unchanged, for details see appendix 9.6. The rationale for selecting a slightly asymmetric saturation threshold was to maximize the number of manual tenders in the analysis. As there were fewer manual tenders that were administered to begin with and a sample of these tenders were collected, including more manual tenders make the estimates more accurate. 20 averages and length of experience with e-procurement. Besides PE-level covariates, both coarsened exact matching exercises include contract-level covariates in order to control for potential confounders at the contract level widely used in the literature. These comprise contract value and procedure method. 59. The strength of the first approach—matching within PEs across years—is that it accounts for most omitted variable bias at the PE level and bias from self-selection of PEs into e-procurement. Making comparisons within PEs over time hence is powerful in blocking off a range of confounders at the PE level. However, this identification strategy is prone to bias from learning effects and any time-varying external shocks. To partially account for these potential biases, the regression analysis conducted on the matched samples includes PE Fixed Effects (FEs) as well as year FEs (more on the regression set-up below). 60. The second approach—matching within year, across PEs—is designed to reduce bias from learning effects, increasing firm registration in the e-procurement system and other time-varying shocks, by limiting comparisons to within two-year periods. We chose to use two-year, rather than single-year matching windows, as single-year matching returned too small a sample of adequate matches, given the sample restrictions in step one above. While there is significant variance in the speed of e-procurement adoption across PEs, this variance is not large enough to find enough low (0-30 percent) saturation PEs and high (80- 100 percent) saturation PEs within a single year, to draw control and treatment observations from, respectively. This approach partially compromises, however, on the bias from PE-level omitted variables and PE self-selection into the e-procurement system. To reduce bias from these sources, we restrict cross- PE comparisons to similar PEs, by including pre-treatment average dependent variable values, the number of years the PE started to use e-procurement, and the respective government organization (LGED, BWDB, RHD) among the matching covariates.25 61. Third, in order to further mitigate potential biases remaining after matching, we run regressions including covariates already used for matching (i.e. doubly robust regression): FEs for PE and year and contract level controls for contract value and procurement method. We estimate a simple ordinary least squares (OLS) set-up for continuous dependent variables and binary logit for categorical dependent variables at the contract level. The main parameter of interest is the coefficient of the manual versus e- procurement dummy variable while controlling for observed and unobserved characteristics of both the contract and the institutional setting. The regression equation estimated is as follows: = 0 + 1 + + + ℎ + log( ) + 25 Descriptive Statistics of the Coarsened exact matching has been included in Appendix 9.1. 21 62. where represents any of the dependent variables for the ith contract awarded26; Ti stands for the treatment status (i.e. manual or electronic tender administration) of ith contract awarded; while PEp is a dummy for pth PE, and is the corresponding FE for that PE. is a dummy for the t-th financial year and is the FE for that financial year. ℎ is the dummy for the procurement method used for administering the i-th contract and is the FE corresponding to that method. is the dollar value of the ith contract and is included in the regression specification to control for the size of the contract. stands for the error term of the regression model. We use the same regression specification without prior matching as a simplified test of our hypotheses (see Appendix 9.4). 63. In addition to our main identification strategies described thus far, we also offer robustness tests that account for potential interaction effects with e-procurement saturation levels. These robustness tests are run on a similarly matched sample, albeit including medium saturation level PE-years too. They also use the same regression specification, except that they include an interaction term between the contract- level manual-electronic dummy variable and measures of the e-procurement saturation level in the respective PE-year. Saturation is either measured as a continuous variable or in three broad categories: low e-procurement saturation (0-30 percent); medium e-procurement saturation (30-80 percent); and high e- procurement saturation (80-100 percent). In the categorical set-up, the coefficient of the dummy for manual in low saturation PE-year versus e-procurement in the high e-procurement saturation PE-year category is the closest to the main effects estimated previously. The other interaction terms reveal gaming behaviors. These robustness tests offer additional insights based on a broader sample and more explicit modeling of spillover and sorting effects.27 64. The interacted regression equation estimated used for the robustness check is as follows: = 0 + 1 + + + ℎ + log( ) + Where each element is the same as above with the addition of si which stands for treatment saturation level by each PE-year combination for the ith contract awarded. 65. All descriptive statistics and regression models are estimated using frequency weights in order to balance the sample of manual tenders with the full population data on e-procurement tenders. Recall, the full population data of the tenders administered through the e-procurement system was directly acquired from the backend database of the government. Hence, e-procurement tenders received population weights equal to 1 and the sampled manual tenders were given population weights such that the sample size and 26 In the binary logistic regressions DVs are entered after logit transformation. 27 These robustness checks with saturation interactions have been included in Appendix 9.3. 22 proportions were equal to that of the population. The population weights for a tender in a particular stratum equals the ratio of the population of that stratum over the number of tenders collected from that stratum. 66. The standard errors of coefficients are calculated using bootstrapping (500 samples with replacement). The matching strategy and frequency weights have been incorporated into the bootstrapping design through the selection of the bootstrapped samples, where the probability of selection into the bootstrap sample is equal to the product of the matching and the frequency weights.28 Bootstrapping is preferable to traditional standard error calculations given the complex nature of our sample with e-GP data representing the full population and our sample of manual samples oversampling smaller PEs. (For a methodological description of bootstrapping methods, see for example Good, 2006 or Carpentel & Bithell, 2000.) 67. Data collection also included contract implementation information, however following a somewhat different procedure. While for all manual tenders sampled, we also collected contract implementation information, electronic tenders had to be sampled for additional data collection because electronic records did not contain information on contract implementation. A sample of about 1,500 tenders was selected from a population of about 6,500 electronically administered tenders from FY13-14 for data related to contract management indicators. A similar approach was followed for weighting the electronic tenders for which contract management information was collected as described above, that is the weighted tenders were representative of the population. 6. RESULTS 6.1 MAIN EFFECTS 68. First, we examine hypothesis one (H1) which posits that e-procurement improves access to public tenders (Table 3). Across the different models, we find consistent support for H1, using three different 2 proxies for open access. The models have generally high albeit varying explanatory power (R falls between 15 percent and 38 percent, with logit models having generally lower explanatory power). In Models 1a (within PE matching) and 1b (within year matching), we find that there is a statistically significant, large 29 impact of e-procurement on the number of bidders. On average, 1.6-2.2 more firms bid for e-procurement tenders than for manual ones (5.0 vs 2.8 bidders in model 1a and 6.3 vs 4.6 in model 1b). In Models 2a and 2b, we test the impact of e-procurement on the probability that a tender receives a single bid. The predicted 28 The standard errors of the coefficients were also estimated using the traditional method and the results have been included in Appendix 9.2. 29 Note that the natural logarithm of bidder numbers was used as the dependent variable to correct for the highly skewed distribution of bidder numbers. 23 probability of receiving only one bid on a tender is 7.8-13.5 percentage points lower for e-procurement than for manual tenders (30.1 percent in manual vs 16.6 percent in e-procurement in model 2a and 20.7 percent in manual vs 12.9 percent in e-procurement in model 2b). In models 3a and 3b, we look at the impact of e- procurement on the probability that the winning firm is headquartered outside the district of the awarding procuring entity. The predicted probability of a non-local firm winning is 3.7-6.3 percent points higher for e-procurement than for manual tenders (15.2 percent vs 18.9 percent in model 3a and 13.3 percent vs. 19.6 percent in model 3b). This result was further corroborated as 27% of bidders surveyed mentioned becoming more confident in bidding outside their own districts. TABLE 3: MAIN EFFECTS: ACCESS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model (1b): Model (2b): Model (3b): Model (1a): Model (2a): Model (3a): CEM CEM CEM CEM CEM CEM Model within within within within PEs within PEs within PEs years across years across years across across years across years across years PEs PEs PEs Regression type OLS Binary Logit Binary Logit Control: Manual Processing Treatment: E- 0.592*** 0.302*** -0.960*** -0.688*** 0.304* 0.596*** procurement (0.041) (0.042) (0.122) (0.139) (0.153) (0.138) 0.019*** -0.075*** -0.091*** 0.095*** 0.424*** 0.464*** Ln of Contract Value (0.003) (0.005) (0.009) (0.015) (0.010) (0.018) Procurement Method: -1.260*** -1.203*** 0.662*** 0.528*** 0.464*** 0.187*** OTM (LTM – base) (0.010) (0.015) (0.026) (0.040) (0.032) (0.048) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.379 0.376 0.149 0.178 0.174 0.225 N 85372 34429 84135 31230 76783 33118 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 69. Second, we examine H2 which proposes that e-procurement improves economy of public tenders (Table 4). Overall, we find some albeit not unequivocal support for H2, in models of varying explanatory power (R2 ranging from 6 to 34 percent). While the impact on winning rebates is as expected, the effect on time and cost overruns is inconsistent across models. The lack of a clear positive impact on contract implementation-related outcomes may signal that positive effects are at least partially offset by strategically displacing rent-seeking from the bidding phase to contract implementation (Dávid-Barrett & Fazekas, 2020). 24 70. Models 1a and 1b show that there is a statistically significant and large impact of e-procurement on winning rebates, defined as the percent discount achieved compared to the initial estimate. Winning rebates are 7.4-8.0 percentage points higher in e-procurement than in manual tenders (7.0 percent in e- procurement versus negative 0.4 percent in manual as per Model 1a; and 8.9 percent versus 0.9 percent, as per Model 1b). These results are robust to removing different ranges of outlier winning rebate values (see Appendix 9.11). 71. In Model 2a, e-procurement has a small, negative and statistically significant impact on cost 30 overruns, while the effect in Model 2b is small, positive and statistically insignificant. In Model 3a, e- procurement has a negative, and statistically significant impact on time overrun, that is e-procurement decreases the probability of contract extension. However, the effect is positive and significant in model 3b. Taken together, Models 2 and 3 point at no clear pattern of the impact of a bidding and award stage-focused e-procurement system on contract implementation. To further explore such patterns, appendix 9.12 systematically tests whether abnormally low bids (i.e. high winning rebates) lead to higher time overruns (used as a proxy for cost overruns as the cost overrun sample was unreliably small). If e-procurement makes this relationship steeper due to its heavy focus on effective competition at the bidding stage, it may offset some of the positive effects we see across the board. Reassuringly, while the most excessively low bids (i.e. very high winning rebates) are weakly associated with more contract modifications, e-procurement did not increase the size of this effect. 30 Please note that sample size drops for models with within bi-annual period matching (models 2b and 3b) because contract implementation data was only collected for 1 financial year for the e-procurement sample. 25 TABLE 4: MAIN EFFECTS: ECONOMY Dependent variable Winning Rebate Cost Overrun Time Overrun Model Model Model Model Model Model (1a): CEM (1b): CEM (2a): CEM (2b): CEM (3a): CEM (3b): CEM Model within PEs within within PEs within within PEs within across years across years across years years across PEs years across PEs years across PEs Regression type OLS OLS Binary Logit Control: Manual Processing Treatment: E- 7.345*** 7.984*** -0.027** -0.251** -1.035*** 1.866* procurement (0.454) (0.452) (0.008) (0.087) (0.170) (0.782) -1.255*** -1.672*** -0.004*** -0.026*** 0.699*** 0.797*** Ln of Contract Value (0.026) (0.047) (0.001) (0.004) (0.018) (0.151) Procurement Method: 0.877*** -0.636*** 0.013*** 0.116*** 0.187*** 0.588 OTM (LTM – base) (0.069) (0.138) (0.002) (0.032) (0.044) (0.369) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.332 0.341 0.065 0.095 0.181 0.092 N 84633 33790 24513 1048 22798 834 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 72. Third, we examine H3, that is whether e-procurement improves efficiency of public tender administration (Table 5). We find strong support for H3 for the whole tender administration period (lead time). Overall, e-procurement reduces lead time— that is the time elapsed from tender advertisement to 31 contract signing — by 15.6-19.2 days compared to manual tenders (84.1 versus 65.0 days, respectively in Model 4a; and 80.9 versus 65.3 days, respectively in Model 4b). We also find support for most of the different parts of the process, except for the decision period, in models of generally high explanatory power 2 (R falling between 15 percent and 35 percent). 73. E-procurement has a statistically significant impact on the submission period of a tender, that is on the time elapsed from the tender advertisement date to the tender submission date, as shown in Models 1a and 1b. Processing tenders through e-procurement decreases the submission period by 6.1-6.4 days, on average, in comparison to tenders processed manually (25.8 vs. 19.4 days as per Model 1a and 26.1 and 19.9 days as per Model 1b). While this is clearly an improvement from an administrative efficiency perspective, it could hamper healthy competition by cutting the advertisement process too short. A closer look at the distributional impact of e-procurement on submission periods suggests that the reduced average 31 Please note that the lead time effect is not a simple sum of the 3 other effects due to different sample compositions for the three models, that is we do not have data on submission period and decision period for tenders without a published call for tenders, but we do on signing period. 26 submission period does not adversely impact companies’ capacity to prepare bids adequately (for more details see section 6.2 below). Instead, e-procurement makes very short advertisement periods (less than 15 days) virtually disappear which should improve advertisement quality leaving more time for bidders to prepare bids. Moreover, it also significantly reduces very long periods (more than 30 days) which is the primary reason behind the declining average advertisement length. 74. In Models 2a and 2b, we observe that, contrary to our expectations, e-procurement increases decision period length compared to manual tender evaluation. While overall small, our models predict that e-procurement increases the decision period by 2.0-2.2 days (from 20.5 to 22.7 days in Model 2a and from 22.8 to 24.9 days in Model 2b). The surprising increase in decision period length can be attributed to two potential reasons. First, as shown, the number of bidders per tender increased and more bidders naturally mean more work for bid evaluators. When normalizing the decision period length by the number of bidders, we find a decrease in decision making time as expected (see normalized regressions in Appendix 9.5). Second, the increasing participation of non-local bidders implies more validating work for procuring entities because they have to obtain documents to assess the validity of the experience quoted by bidders. This takes comparatively longer compared to a procuring entity contacting a bidder it already contracted with. 75. Regarding the signing period, that is the number of days needed for signing the contract once the contract is awarded, we find a substantial decrease in processing time. E-procurement reduces the average signing period by 4.1-6.9 days compared to manual tenders (21.9 versus 14.9 days in Model 3a; and 18.2 versus 14.1 days in Model 3b). 76. The observations from the administrative data are further corroborated by the perceptions of the times as reported by surveyed procuring entity officials. PE officials reported similar decreases in processing times (a detailed analysis has been given in Appendix 9.10). Moreover, 22% of PE officials mentioned the lower turn-around time required for processing tenders as one of the two main advantages of the e-procurement system. Similarly, 59% of bidders reported preparation times of tenders as one of the main drawbacks of the paper-based system that is alleviated by the transition to the electronic system. 27 TABLE 5: MAIN EFFECTS: EFFICIENCY Dependent variable Submission Period Decision Period Signing Period Lead Time Model (1a): Model (1b): Model (2a): Model (2b): Model (3a): Model (3b): Model (4a): Model (4b): CEM CEM CEM CEM CEM CEM CEM CEM Model within PEs within within PEs within within PEs within within PEs within across years across years across years across years years across PEs years across PEs years across PEs years across PEs Regression Type OLS OLS OLS OLS Control: Manual Processing Treatment: E- -6.417*** -6.176*** 2.210* 2.067+ -6.995*** -4.160*** -19.344*** -15.615*** procurement (0.340) (0.377) (1.003) (1.107) (0.556) (0.532) (1.655) (1.519) 1.082*** 1.163*** 2.151*** 3.239*** 0.572*** 1.032*** 5.755*** 8.066*** Ln of Contract Value (0.022) (0.039) (0.067) (0.118) (0.034) (0.052) (0.096) (0.153) Procurement Method: 0.330*** 0.841*** 4.090*** 3.284*** 0.304** 0.044 7.567*** 9.003*** OTM (LTM – base) (0.057) (0.105) (0.175) (0.308) (0.104) (0.186) (0.264) (0.447) PE Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Constant Yes Yes Yes Yes Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 500 500 500 500 2 R 0.387 0.338 0.201 0.284 0.155 0.189 0.246 0.296 N 62506 22378 58013 21905 76367 33965 76589 33482 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, 2b, 3b and 4b use bi-annual time windows. 6.2 MECHANISMS 77. After estimating the main effects and concluding that they are largely in line with our theoretical expectations, we further probe some of the key underlying impact mechanisms for each hypothesis. 78. First, we explore one impact mechanism underlying increased access due to e-procurement (H1). One of our main theoretical arguments was that e-procurement lowers intentional barriers to entry by making it harder to erect barriers for unwanted competitors and making rent-seeking riskier. If this is the dominant mechanism underpinning H1, we would expect e-procurement to have a larger impact on those PEs which had limited access to their tenders prior to the intervention. Hence, PEs in different quantiles of the access distribution at baseline are expected to react differently to e-procurement. We analyze this formally by interacting PE-level measures of the degree of open access at baseline with the e-procurement treatment variable. We add this interaction term to the same regression setup as above. 79. In each of the regressions containing the interaction between e-procurement and PEs’ baseline average dependent variable, the coefficient of the interaction is negative while the coefficient of the PE’s pre-e-procurement average dependent variable is large and positive (Table 6). This suggests that the higher the baseline level of access, the lower the impact of e-procurement on access. In other words, e-procurement is particularly suited to improving access in PEs where access was limited to start with, while its impact is much weaker for PEs which were more open to competition prior to the intervention. 28 TABLE 6: MECHANISMS: ACCESS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model (1a): Model (1b): Model (2a): Model (2b): Model (3a): Model (3b): CEM CEM CEM CEM CEM CEM Model within PEs within within PEs within within PEs within across years across years across years years across PEs years across PEs years across PEs Regression type OLS Binary Logit Binary Logit Control: Manual Processing 1.413*** 1.059*** -0.082 0.748*** 0.878*** 1.186*** Treatment: E-procurement (0.046) (0.048) (0.121) (0.150) (0.171) (0.145) Pre-e-procurement ln bidder number of 0.488** 1.530*** PE (0.166) (0.192) Pre-e-procurement single bidding rate 7.201*** 8.535*** of PE (1.020) (1.116) Pre-e-procurement non-local winning 6.929 6.891 rate of PE (5.572) (13.200) Baseline: Pre-e-procurement ln bidder number of PE*manual Pre-e-procurement ln bidder number of -0.501*** -0.429*** PE*electronic (0.011) (0.016) Baseline: Pre-e-proc. single bidding rate of PE*manual Pre-e-procurement single bidding rate -4.976*** -5.828*** of PE*electronic (0.129) (0.258) Baseline Pre-e-proc. non-local winning rate of PE*manual Pre-e-procurement non-local winning -2.531*** -2.767*** rate of PE*electronic (0.155) (0.193) Ln of Contract Value Yes Yes Yes Yes Yes Yes Procurement Method Yes Yes Yes Yes Yes Yes PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.400 0.385 0.172 0.205 0.176 0.228 N 85358 34428 84423 31625 76823 33204 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 80. Second, we explore to which extent e-procurement affects economy (H2) through increased access, that is by reducing intentional and non-intentional barriers to entry and hence intensifying competition. We would expect e-procurement to affect economy, and in particular winning rebates through two parallel channels: First, it increases competition, simply by increasing access—that is by enabling more bidders to participate. This in turn leads to lower prices and larger winning rebates. We call this the “extensive” impact mechanism. The positive impact of e-procurement on access was partially shown above (Table 3). We further explore this channel by testing the impact of increased access (e.g. number of bidders) on economy (i.e. winning rebates) without interactions. 29 81. Second, e-procurement can also intensify competition among bidders, at a given level of access. In other words, a higher level of access is expected to have a stronger impact on winning rebates in the e- procurement system than in paper-administered tenders. We call this the “intensive” impact mechanism. We explore the intensive impact channel by interacting measures of access with the e-procurement treatment. By implication, the regressions test whether the interaction term between measures of open access and e-procurement are positive, that is the interaction further increases the effect size of open access (please note that the direction of impact for single bidding is opposite to the direction of bidder number and non-local bidders). In simple terms, open competition is expected to have a stronger impact on prices (winning rebates) in the e-procurement system rather than in paper administered tenders. We look at these effects in the same specifications as above for winning rebates (Table 4, Models 1a and 1b), but adding in variables for open access and interacting them with the e-procurement dummy. 82. Our findings confirm both impact channels. The number of bidders and non-local bidders increases winning rebates while single bidding lowers them. This result combined with the above finding on e- procurement increasing bidder numbers and the prevalence of non-local bidders while lowering the incidence of single bidding provides support for the extensive impact channel. Looking at the coefficients of the interaction terms, we find that the effects of bidder number, non-local bidders, and single bidding are markedly stronger for electronic tenders than for manual tenders which underpins the intensive impact channel (Table 7). 30 TABLE 7: MECHANISMS: ECONOMY Dependent variable Winning Rebate Model Model Model Model Model Model (1a): (1b): (2a): (2b): (3a): (3b): CEM CEM CEM CEM CEM CEM Model within within within within within within PEs years PEs years PEs years across across across across across across years PEs years PEs years PEs Regression type OLS Control: Manual Processing 5.062*** 6.469*** 7.224*** 8.965*** 7.286*** 7.695*** Treatment: E-procurement (0.444) (0.449) (0.443) (0.448) (0.463) (0.453) 2.432*** 2.221*** Ln Number of Bidders (0.029) (0.035) 0.523*** 0.567*** ln bidder number*electronic (0.042) (0.085) Baseline: single bidder contract=No -4.568*** -3.529*** single bidder contract=Yes (0.157) (0.148) single bidder contract=Yes * -2.520*** -6.703*** electronic (0.166) (0.259) Baseline: winner is from outside the district?=No winner is from outside the 0.695*** 1.561*** district?=Yes (0.202) (0.190) winner is from outside the 0.820*** 0.465 district?=Yes * electronic (0.216) (0.320) Ln of Contract Value Yes Yes Yes Yes Yes Yes Procurement Method Yes Yes Yes Yes Yes Yes PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes Constant Yes Yes Yes Yes Yes Yes Bootstrapping Iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.415 0.404 0.401 0.389 0.339 0.347 N 84451 33542 84451 33542 76014 33013 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 83. Third, we explore the claim that e-procurement increases administrative efficiency (H3) by standardizing and speeding up recurrent administrative processes such as tender advertisement. As shown above, on average, e-procurement shortens lead time as well as submission period length. A closer look at the distributional impact of e-procurement on submission periods indeed suggests that the reduced average submission period identified in Table 5 results from a considerable standardization of this process. Compared to manual tenders, e-procurement leads to shorter and more homogenous submission periods (Figure 4). E-procurement substantially reduces the kurtosis and variance of the submission period (27.2 vs 57.6 days, a difference significant at the 0.001 level) around a lower mean. It makes very short advertisement periods (less than 14 days) virtually disappear; it significantly reduces the frequency of very 31 long periods (more than 30 days). In other words, e-procurement has virtually eliminated non-compliance with the legally mandated minimal advertisement period of 14 days.32 84. Similarly, E-procurement significantly reduces the number of contracts that had a signing period of greater than 28 days. It also reduces the variance of the singing periods (9.3 days as compared to 14.4 days, a difference significant at the 0.001 level) indicating the standardization of and speeding up recurrent administrative processes. E-procurement virtually eliminates non-compliance with the legally mandated maximum singing period of 28 days from the date of the issuance of the NOA (Notification of Award). 33 Both the reduction of non-compliance with the submission period and signing period indicate that the e- procurement system is helpful for firms as they are able to more easily follow the procurement rules. 85. This e-procurement impact mechanism on submission and signing periods is likely indicative of a major channel through which e-procurement improves administrative efficiency more generally: e- procurement standardizes internal administrative processes by effectively eliminating outlier procedures and by moving tenders closer to the range of expected processing times. 32 Schedule II: Public Procurement Rules 2008, CPTU, IMED, Ministry of Planning, Government of Bangladesh. Rule 64(1): Time for preparation and submission of Tenders for National Procurement of Goods, Works and Physical Services under the Open Tendering Method from the date of advertisement: (1) Not less than fourteen (14) days for Procurement up to Tk. 3 million (2) Not less than twenty-one (21) days for contacts above Tk 3 million and up to Tk. 50 (fifty) million, (3) Not less than twenty-eight (28) days for contacts above Tk. 50 (fifty) million, (4) Not less than fourteen (14) days for emergency Procurement following a catastrophe, (5) Not less than fourteen (14) days for re-Tendering Rule 64(5): Time for preparation and submission of Tenders from the date of publication of advertisement in the newspaper under Limited Tendering Method (1) Not less than fourteen (14) days 33 Schedule II: Public Procurement Rules 2008, CPTU, IMED, Ministry of Planning, Government of Bangladesh. Rule 102(11): Time for signing of Contract by the successful Tenderer: (1) Within twenty-eight (28) days of the issuance of NOA. (2) Within twenty-eight (28) days of the issuance of NOA for international Procurement. 32 FIGURE 4. HISTOGRAMS OF SUBMISSION PERIOD LENGTH (DAYS) BY TREATMENT STATUS Manual e-GP 20% 20% Mean = 19.5 Mean = 25.6 Submission Period 15% 15% 10% 10% 5% 5% 0% 0% 0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50 8% 8% Mean = 17.8 Mean = 16.1 6% 6% Signing Period 4% 4% 2% 2% 0% 0% 0 5 10 15 20 25 30 35 40 45 50 0 5 10 15 20 25 30 35 40 45 50 Note 1: Number of tenders: Manual = 30661; electronic = 43137 Note 2: The red regions of the graph correspond to the stipulated submission and signing periods as per the procurement rules 7. CONCLUSIONS AND POLICY IMPLICATIONS 86. In this impact evaluation, we estimate the effects of Bangladesh’s e-procurement system on metrics of access, economy and efficiency of public tenders. Bangladesh’s e-procurement system is advanced compared to most of its comparators around the world in that it not only provides an online advertisement portal, but rather requires that all notable administrative actions are administered digitally, from tender preparation to contract signature. To identify the impact of e-procurement on a wide range of outcomes, we construct a novel dataset, comprising all published electronic public procurement tenders and data extracted from a large sample of paper-based procurement records from there major GOB agencies, which are responsible for the bulk of public works projects. 87. We rely on a range of fixed effect regression models combined with coarsened exact matching estimators, to address potential threats to causal identification. We find overwhelming positive impacts of e-procurement on a wide range of outcomes. 33 88. First, e-procurement improves open and fair access to public tenders. It increases bidder numbers by 1.6-2.2, it decreases the probability of single bidding by 7.8-13.5 percentage points and increases the probability of non-local companies winning by 3.7-6.3 percentage points. These positive impacts are down, to a large degree, to e-procurement improving access for PEs with low levels of access at baseline. 89. Second, e-procurement also improves economy, most notably winning rebates: discounts firms offer increase by 7.4-8.0 percentage points. This impact results both from increasing the number of bidders and from enabling more intense competition among a given number of bidders. The impact on contract implementation performance, namely on cost and time overruns, is unclear and mostly insignificant. This suggests that e-procurement may displace rent seeking to the contract implementation phase which remains principally paper-administered. 90. Third, e-procurement greatly improves administrative efficiency. The average time of processing a tender from the public call for tenders to contract signature drops by 15.6-19.2 days with most major stages of this process improving. A detailed look at the distributional impacts of e-procurement underlines that the electronic system is effective in standardizing processing times and in enhancing compliance with the legally mandated minimal advertisement period of 15 days, by sharply thinning the tails of excessively short or long processing times. 91. Our results contribute to the literature on e-procurement and anti-corruption. First, they confirm the beneficial effects of e-procurement on a range of desirable outcomes, lending strong support to global efforts for rolling out comprehensive and well-designed e-procurement platforms. Second, our findings also deliver a warning, however, that the transparency and tighter monitoring of the bidding and decision- making stages of the procurement process may displace rent seeking to later stages, most notably contract implementation. Third, contrary to prior findings from India and Indonesia, but in line with theoretical predictions, we find sizable positive effects on the number of bidders as well as prices. 34 93. 94. Table 8 contrasts our findings for Bangladesh with Lewis-Faupel et al.’s earlier results for India and Indonesia. It remains to be seen why our findings differ, while they already deliver significant refinements to the existing evidence base. Fourth, our analysis suggests that e-procurement delivers the most value where prior performance and regulatory compliance was weak, in line with existing evidence (Bosio, Djankov, Glaeser, & Shleifer, 2022). 35 TABLE 8: SUMMARY OF IDENTIFIED E-PROCUREMENT IMPACTS ACROSS INDIA, INDONESIA, ANF BANGLADESH Outcome Indicator India Indonesia Bangladesh Number of bids submitted 1.6-2.2 bids Access Non-local winners 11% 23% 3.7-6.3% Road Quality on Audit 9.7% higher quality N/A N/A Winning Rebate 7.4-8.0% Economy Time Overruns Cost Overruns N/A Lead Time 54.13 days 15.6-19.2 days Submission Period N/A N/A 4.1-6.9 days Efficiency Decision Period 2.0-2.2 days Signing Period N/A N/A 4.1-6.9 days Source: Lewis-Faupel et al. for India and Indonesia and this study for Bangladesh 95. Based on the different estimation strategies, delivering a range of savings impacts, it is possible to broadly estimate the total public money saved by the e-procurement system (Pollitt et all, forthcoming). While our estimation does not encompass all expected monetizable impacts of e-procurement, it does provide a useful lower bound estimate of the total savings achieved. Using the total savings formula in section 4, the predominantly positive impact of e-procurement on access, economy and efficiency can be translated into savings estimates for the government and for bidding firms. We estimate that, between 2012 and 2016, the three government agencies saved USD 424 million to USD 471 million in purchase prices, and USD 35 million to USD 42 million in administrative costs. These figures dwarf the cost of setting up the e-procurement system, estimated at USD 70 million. Bidders’ transaction cost savings are also positive albeit rather low: USD 4 million to USD 7 million. If e-procurement were to cover 100 percent of public procurement spending, the GOB could save up to USD 1.76 billion per year, assuming that the same average savings due to increasing winning rebates would accrue to the whole budget. The full calculations can be found in Appendix 9.9. In addition to the cost savings of the GOB due to lower purchase prices, there would also be macroeconomic effects. These effects have been estimated by assuming that the government increases the provision of public services using the savings from the lower purchase prices. Including multiplier effects, e-procurement is estimated to increase Bangladesh’s GDP by between 0.48% and 0.54% in 2019 or, in absolute terms, between USD 0.94 million and USD 1.07 billion (in 2011 prices, between 36 USD 1.4 billion and USD 1.6 billion in 2019 prices) based on macro-economic modeling. Bangladesh’s e- procurement is also estimated to have led to employment increases in Bangladesh, in the region of 0.15% to 0.17%, or around 93,000 to 105,000 extra jobs in total. These additional jobs are widely dispersed among different sectors of the economy. The largest economic impacts of e-procurement come from cost savings accruing to the Bangladesh government as also observed in the cost savings estimate. The productivity increase in government tendering would allow the government to increase output in other areas, and this higher level of spending would spread throughout the economy because of multiplier effects (Pollitt et all, forthcoming). 96. It is estimated that bidding firms that compete for government tenders have also seen efficiency improvements from e-procurement, but these would have had negligible impact on the wider economy. While these bidding firms saw reductions in the per-tender cost of procurement, the reduced cost per tender led to an increase in the overall number of bids, which increased tender preparation costs overall. However, these higher tender preparation costs were in turn outweighed by greater savings from security costs, as firms no longer needed to protect their staff from intimidation tactics when submitting paper bid documents in person. E-procurement reduced the need for governments to advertise tenders in print, which has led to a fall in revenues for newspapers, print publishers and advertisers. This is the only industry to be negatively impacted by e-procurement. Overall, the direct impacts on private firms are far outweighed by the indirect impacts of higher government spending. 97. Nevertheless, the present findings could only identify and carefully assess the main effects while highlighting a few notable impact mechanisms without being exhaustive. Among others, future research could look at the impact channels in detail in order to deliver a more nuanced understanding of why and under which conditions e-procurement delivers the hoped for impacts. On the one hand, a more detailed understanding of administrative preconditions for successful e-procurement reform can be explored by combining data from the procuring entity survey with procurement records. On the other hand, a better understanding of the constraints imposed and opportunities presented by different bidding markets could be investigated by additionally drawing on bidder registration data and a tailored bidder survey. 98. While more research is needed in this area, our findings already lend themselves to a range of policy recommendations in and beyond Bangladesh: i) Introduce and expand the scope of e-procurement systems to encompass all public procurement tenders; and ii) Capitalize on the available data in the e-procurement system by developing a contract performance dashboard which enables continuous monitoring and optimization of processes in real-time. 37 8. BIBLIOGRAPHY Bajari, P., Houghton, S., & Tadelis, S. (2014). Bidding for Incomplete Contracts: An Empirical Analysis of Adaptation Costs. American Economic Review, 104(4), 1288–1319. Bandiera, O., Prat, A., & Valletti, T. (2009). Active and Passive Waste in Government Spending: Evidence from a Policy Experiment. American Economic Review, 99(4), 1278–1308. https://doi.org/10.1257/aer.99.4.1278 Politt, H., McGovern, M., Smith, A., Blum, J. R., & Fazekas, M. (Forthcoming). Macroeconomic Effects of Introduction of Electronic Procurement in Bangladesh. Governance Notes. Blum, J. R., Siddique, I., Fazekas, M., & Samaddar, S. (2018). Evaluating the Impact of Introducing E- procurement in Bangladesh. Policy Note. Bosio, E., Djankov, S., Glaeser, E. L., & Shleifer, A. (2022). Public Procurement in Law and Practice. American Economic Review, 112(4), 1091-1117. Buyse, A., Dewyngaert, N., Loozen, N., Lopez Potes, M., Simons, G., & Ziemyte, A. (2015). e- Procurement Uptake. PricewaterhouseCoopers. Carpenter, J. and Bithell, J. (2000), Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Statist. Med., 19: 1141-1164. Charron, N., Dahlström, C., Fazekas, M., & Lapuente, V. (2017). Careers, Connections, and Corruption Risks: Investigating the impact of bureaucratic meritocracy on public procurement processes. Journal of Politics, 79(1), 89–103. https://doi.org/10.1086/687209 Chong, E., Klien, M., & Saussier, S. (2015). The Quality of Governance and the Use of Negotiated Procurement Procedures: Evidence from the European Union (No. 2015–3; EPPP DP). Collier, P., Kirchberger, M., & Söderbrom, M. (2015). The cost of road infrastructure in low and middle income countries. Coviello, D., & Gagliarducci, S. (2017). Tenure in Office and Public Procurement. American Economic Journal: Economic Policy, 9(3), 59–105. Coviello, D., Guglielmo, A., & Spagnolo, G. (2018). The Effect of Discretion on Procurement Performance. Management Science, 64(2), 715–738. Coviello, D., & Mariniello, M. (2014). Publicity requirements in public procurement: Evidence from a regression discontinuity design. Journal of Public Economics, 109, 76–100. Croom, S. R., & Brandon-Jones, A. (2005). Key Issues in E-Procurement: Procurement Implementation and Operation in the Public Sector. Journal of Public Procurement, 5(3), 367–387. Dávid-Barrett, E., & Fazekas, M. (2020). Anti-corruption in aid-funded procurement: Is corruption reduced or merely displaced? World Development, 132, 105000. http://www.sciencedirect.com/science/article/pii/S0305750X20301261 38 Decarolis, F. (2014). Awarding Price, Contract Performance, and Bids Screening: Evidence from Procurement Auctions. American Economic Journal: Applied Economics, 6(1), 108–132. http://www.aeaweb.org/articles.php?doi=10.1257/app.6.1.108 Decarolis, F., & Palumbo, G. (2015). Renegotiation of public contracts: An empirical analysis. Economics Letters, 132, 77–81. Di Tella, R., & Schargrodsky, E. (2003). The Role of Wages and Auditing during a Crackdown on Corruption in the City of Buenos Aires. Journal of Law and Economics, 46(1), 269–292. http://ideas.repec.org/a/ucp/jlawec/y2003v46i1p269-92.html Fazekas, M., & Blum, J. R. (2021). Improving public procurement outcomes: review of tools and the state of evidence base (No. 9690; Policy Research Working Papers). https://openknowledge.worldbank.org/handle/10986/35727 Fazekas, M., & Kocsis, G. (2020). Uncovering High-Level Corruption: Cross-National Corruption Proxies Using Government Contracting Data. British Journal of Political Science, 50(1), 155–164. https://doi.org/10.1017/S0007123417000461 Fazekas, M., & Tóth, B. (2016). Assessing the potential for detecting collusion in Swedish public procurement. http://www.konkurrensverket.se/globalassets/publikationer/uppdragsforskning/forsk_rapport_2016- 3.pdf Fazekas, M., Tóth, I. J., & King, L. P. (2016). An Objective Corruption Risk Index Using Public Procurement Data. European Journal of Criminal Policy and Research, 22(3), 369–397. https://doi.org/10.1007/s10610-016-9308-z Fisman, R., & Golden, M. (2017). How to fight corruption. Science, 356(6340), 803 LP – 804. http://science.sciencemag.org/content/356/6340/803.abstract Funk, M. J., Westreich, D., Wiesen, C., Stürmer, T., Brookhart, M. A., & Davidian, M. (2011). Doubly Robust Estimation of Causal Effects. American Journal of Epidemiology, 173(7), 761–767. https://doi.org/10.1093/aje/kwq439 Good, P.I. (2006) Resampling Methods. A Practical Guide to Data Analysis. 3rd edition, Birkhauser, Boston. Iacus, S. M., King, G., & Porro, G. (2012). Causal Inference without Balance Checking : Coarsened Exact Matching. Political Analysis, 20(1), 1–24. https://doi.org/10.1093/pan/mpr013 Klasnja, M. (2016). Corruption and the Incumbency Disadvantage: Theory and Evidence. Journal of Politics, 77(4), 928–942. Lewis-Faupel, S., Neggers, Y., Olken, B. A., & Pande, R. (2014). Can Electronic Procurement Improve Infrastructure Provision? Evidence From Public Works in India and Indonesia (No. 20344; NBER 39 Working Paper Series). http://www.nber.org/papers/w20344 Lewis-Faupel, S., Neggers, Y., Olken, B. A., & Pande, R. (2016). Can Electronic Procurement Improve Infrastructure Provision? Evidence from Public Works in India and Indonesia. American Economic Journal: Economic Policy, 8(3), 258–283. http://www.aeaweb.org/articles?id=10.1257/pol.20140258 Mendes, M., & Fazekas, M. (2017). Towards More Transparent and Efficient Contracting - Public Procurement in the European Union. https://opentender.eu/blog/assets/downloads/digiwhist_recommendations.pdf Mungiu-Pippidi, A. (2015). The Quest for Good Governance. How Societies Develop Control of Corruption. Cambridge University Press. http://www.cambridge.org/us/academic/subjects/politics- international-relations/comparative-politics/quest-good-governance-how-societies-develop-control- corruption North, D. C., Wallis, J. J., & Weingast, B. R. (2009). Violence and Social Orders. A Conceptual Framework for Interpreting Recorded Human History. Cambridge University Press. OECD. (2010). Collusion and Corruption in Public Procurement. OECD. OECD. (2015). Government at a glance. 2015. OECD. OECD. (2016). Survey on Public Procurement. https://qdd.oecd.org/subject.aspx?Subject=GOV_PUBPRO_2016 Olken, B. A. (2007). Monitoring Corruption: Evidence from a Field Experiment in Indonesia. Journal of Political Economy, 115(2), 200–249. http://www.nber.org/papers/w11753 Rothstein, B., & Teorell, J. (2008). What Is Quality of Government? A Theory of Impartial Government Institutions. Governance, 21(2), 165–190. Strand, I., Ramada, P., & Canton, E. (2011). Public procurement in Europe. Cost and effectiveness. PricewaterhouseCoopers, London Economics and Ecorys. Straub, S. (2014). Political Firms, Public Procurement, and the Democratization Process (TSE‐461; Working Papers). Thai, K. V (Ed.). (2009). International Handbook of Public Procurement. CRC Press. World Bank. (2002). Bangladesh Country Procurement Assessment Report. World Bank. World Bank. (2009). Fraud and Corruption. Awareness Handbook. World Bank. World Bank. (2016). Benchmarking Public Procurement 2016. World Bank. 40 9. APPENDICES 9.1 DESCRIPTIVE STATISTICS OF COARSENED EXACT MATCHING 99. The following are descriptive tables of coarsened exact matching. The tables show the number of strata present and the number of strata that were matched and unmatched for each of the dependent variable treatment effect estimations. TABLE 9. COARSENED EXACT MATCHING: MATCHING DESCRIPTION STATISTICS FOR MAIN EFFECT (MATCHING WITHIN PES ACROSS YEARS) Number of Matched Observations Unmatched Observations Number Multivariate Outcome Variable Matched of Strata Control Treatment Control Treatment L1 distance Strata Number of Bidders 372 321 8110 60583 35 2096 0.6443 Single Bidders 372 321 8110 60583 35 2096 0.6443 Non-Local Winners 372 321 8110 60583 35 2096 0.6443 Cost Overrun 372 321 8110 60583 35 2096 0.6443 Time Overrun 372 321 8110 60583 35 2096 0.6443 Winning Rebate 372 321 8110 60583 35 2096 0.6443 Lead Time 372 321 8110 60583 35 2096 0.6443 Submission Period 372 321 8110 60583 35 2096 0.6443 Decision Period 372 321 8110 60583 35 2096 0.6443 Signing Period 372 321 8110 60583 35 2096 0.6443 TABLE 10: COARSENED EXACT MATCHING: MATCHING DESCRIPTION STATISTICS FOR MAIN EFFECT (MATCHING WITHIN YEARS ACROSS PES) Number of Matched Observations Unmatched Number Observations Multivariate L1 Outcome Variable Matched of Strata distance Strata Control Treatment Control Treatment Number of Bidders 216 59 7938 10488 207 52191 0.9735 Single Bidders 156 42 7934 10488 211 52191 0.9566 Non-Local Winners 186 51 8066 10525 79 52154 0.9670 Winning Rebate 168 50 7922 10543 223 52136 0.9712 Cost Overrun 77 24 7147 10537 998 52142 0.9071 Time Overrun 97 33 7136 10537 1009 52142 0.9311 Submission Period 190 50 8084 10288 61 52391 0.9792 Decision Period 200 54 8076 10532 69 52147 0.9729 Signing Period 251 74 8048 10542 97 52137 0.9690 Lead Time 273 79 8050 10542 95 52137 0.9757 9.2 REGRESSIONS WITH CONVENTIONAL ESTIMATES OF STANDARD ERROR 41 100. The standard errors in this section have been estimated using the conventional method of estimations (using a variance-covariance matrix as opposed to bootstrapping) after matching. We observe that the point and the standard error estimates were very similar to those found using bootstrapping. We also observe that the average treatment effect obtained by both methods was equal to the second significant digit. TABLE 11: MAIN EFFECTS WITH NO BOOTSTRAPPING: ACCESS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model (1b): Model (2b): Model (3b): Model (1a): Model (2a): Model (3a): CEM CEM CEM CEM CEM CEM Model within within within within PEs within PEs within PEs years across years across years across across years across years across years PEs PEs PEs Regression type OLS Binary Logit Binary Logit Control: Manual Processing Treatment: E- 0.589*** 0.302*** -0.949*** -0.685*** 0.321* 0.600*** procurement (0.04) (0.04) (0.13) (0.14) (0.16) (0.14) 0.019*** -0.075*** -0.091*** 0.095*** 0.423*** 0.461*** Ln of Contract Value (0.00) (0.00) (0.01) (0.02) (0.01) (0.02) Procurement Method: -1.260*** -1.203*** 0.662*** 0.528*** 0.465*** 0.190*** OTM (LTM – base) (0.01) (0.02) (0.03) (0.04) (0.03) (0.05) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes R2/Pseudo-R2 0.378 0.373 0.148 0.177 0.172 0.223 N 85371 34429 84436 31625 76823 33260 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 42 TABLE 12: MAIN EFFECTS WITH NO BOOTSTRAPPING: ECONOMY Dependent variable Winning Rebate Cost Overrun Time Overrun Model Model Model Model Model Model (1a): CEM (1b): CEM (2a): CEM (2b): CEM (3a): CEM (3b): CEM Model within PEs within within PEs within within PEs within across years across years across years years across PEs years across PEs years across PEs Regression type OLS OLS Binary Logit Control: Manual Processing Treatment: E- 7.347*** 7.948*** -0.027** -0.250 -1.013*** 1.873*** procurement (0.45) (0.40) (0.01) (0.21) (0.17) (0.57) -1.253*** -1.669*** -0.004*** -0.026*** 0.693*** 0.780*** Ln of Contract Value (0.03) (0.04) (0.00) (0.01) (0.02) (0.11) Procurement Method: 0.881*** -0.618*** 0.013*** 0.116*** 0.186*** 0.569 OTM (LTM – base) (0.07) (0.12) (0.00) (0.02) (0.04) (0.39) PE Fixed Effects Yes Yes Yes Yes Yes No Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes 2 2 R /Pseudo-R 0.331 0.336 0.059 0.079 0.181 0.081 N 84634 33790 24514 1048 22996 836 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 43 TABLE 13: MAIN EFFECTS WITH NO BOOTSTRAPPING: EFFICIENCY Dependent variable Submission Period Decision Period Signing Period Lead Time Model Model Model Model Model Model Model Model Model (1a): CEM (1b): CEM (2a): CEM (2b): CEM (3a): CEM (3b): CEM (4a): CEM (4b): CEM within PEs within within PEs within within PEs within within PEs within across years across years across years across years years across PEs years across PEs years across PEs years across PEs Regression Type OLS OLS OLS OLS Control: Manual Processing Treatment: E- -6.414*** -6.142*** 2.172+ 2.037* -6.939*** -4.132*** -19.160*** -15.595*** procurement (0.33) (0.37) (1.13) (1.01) (0.58) (0.59) (1.67) (1.40) Ln of Contract Value 1.081*** 1.166*** 2.154*** 3.244*** 0.574*** 1.033*** 5.760*** 8.064*** (0.02) (0.04) (0.07) (0.10) (0.03) (0.06) (0.10) (0.14) Procurement Method: 0.331*** 0.845*** 4.084*** 3.280*** 0.302** 0.043 7.565*** 8.993*** OTM (LTM – base) (0.06) (0.11) (0.17) (0.30) (0.11) (0.18) (0.26) (0.43) PE Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Constant Yes Yes Yes Yes Yes Yes Yes Yes 2 R 0.385 0.334 0.197 0.276 0.153 0.185 0.244 0.292 N 62516 22379 58020 21903 76363 33965 76584 33483 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, 2b, 3b, and 4b use bi-annual time windows. 44 9.3 REGRESSIONS I NCLUDING INTERACTIONS WITH SATURATION LEVELS 101. The following tables display the results from regression where the main intervention variable (e- procurement) is interacted with categorical as well as continuous saturation variable at the PE level. In line with the main results in Section 6, we look at the magnitude and direction of change in outcomes with changes in the level of saturation. 102. We observe that the treatment effect of e-procurement on electronic tenders for PEs with over 80% saturation is comparable in magnitude and showed the same direction of change to the treatment effect obtained in the regressions in the main results. The results obtained from the regressions with interactions are consistent for all the dependent variables tested in the main results. 103. These regressions with interactions also provide us with an estimate of spillover effects in the form of the average treatment effect on the untreated. We observe that the average treatment effect on the untreated (i.e., the treatment effect on manual tenders for PEs with over 80% e-procurement saturation and 30%-80% e-procurement saturation) is in the same direction as the average treatment effect on the treated obtained in the main results for access and economy indicators, albeit to a smaller magnitude. The direction of the change in the indicators however is in the opposite direction of the efficiency indicators. This adds confidence to our estimates as it shows there was little strategic sorting of the tenders into e-GP which might have confounded our results. If strategic sorting had indeed taken place, there would have been a negative treatment effect on the untreated. 104. There can be multiple reasons behind the positive spillovers of the e-procurement on manual tenders: a. Signalling effect – E-procurement might have had a signalling effect for the procuring entity offices. B. Demonstration effect – E-procurement might have demonstrated transparency to firms that would otherwise not have participated in a manual tender given the barriers to participation. Firms that participated in electronic tenders could have therefore also participated in manual tenders from the same procuring entity. 45 TABLE 14: INTERACTED EFFECTS: ACCESS WITH CATEGORICAL SATURATION INTERACTIONS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model Model (1a): Model (1b): Model (2a): Model (2b): Model (3a): Model (3b): CEM within CEM within CEM within CEM within CEM within CEM within PEs across years across PEs across years across PEs across years across years PEs years PEs years PEs Regression type OLS Binary Logit Binary Logit Baseline: Manual # Treatment Saturation: <0.30 E-procurement # Treatment -0.002 0.053 0.410*** 0.612*** 0.958*** 1.115*** Saturation: <0.30 (0.032) (0.034) (0.114) (0.140) (0.122) (0.133) Manual # Treatment Saturation: 0.231*** 0.199*** -0.491*** -1.322*** -0.127 -0.966*** 0.30-0.80 (0.027) (0.037) (0.084) (0.151) (0.114) (0.177) E-procurement # Treatment 0.435*** 0.436*** -0.402*** -0.579*** 0.723*** 1.154*** Saturation: 0.30-0.80 (0.025) (0.032) (0.076) (0.131) (0.104) (0.139) Manual # Treatment Saturation: 0.342*** 0.231*** -0.619*** -0.577*** 0.835*** 1.440*** >0.80 (0.041) (0.038) (0.123) (0.145) (0.141) (0.156) E-procurement # Treatment 0.528*** 0.446*** -0.680*** -0.379** 0.718*** 1.313*** Saturation: >0.80 (0.029) (0.037) (0.089) (0.145) (0.116) (0.152) Ln of Contract Value 0.015*** 0.001 -0.072*** -0.018+ 0.425*** 0.514*** (0.003) (0.003) (0.009) (0.011) (0.009) (0.012) Procurement Method: OTM -1.270*** -1.565*** 0.633*** 0.897*** 0.482*** 0.326*** (LTM-base) (0.009) (0.013) (0.024) (0.043) (0.030) (0.043) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrapping iteration 500 500 500 500 500 500 2 2 R /Pseudo-R 0.385 0.515 0.149 0.193 0.174 0.201 N 97726 70953 96554 66885 88676 65482 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows 46 TABLE 15: INTERACTED EFFECTS: ECONOMY WITH CATEGORICAL SATURATION INTERACTIONS Dependent variable Winning Rebate Cost Overrun Time Overrun Model Model Model Model Model Model Model (1b): (2b): (3b): (1a): CEM (2a): CEM (3a): CEM CEM CEM CEM within within within within within within PEs across PEs across PEs across years years years years years years across PEs across PEs across PEs Regression Type OLS OLS Binary Logit Baseline: Manual # Treatment Saturation: <0.30 E-procurement # Treatment 6.665*** 5.383*** -0.041*** 0.027** -0.753*** 14.684*** Saturation: <0.30 (0.543) (0.579) (0.012) (0.008) (0.198) (0.802) Manual # Treatment Saturation: 0.522* 0.156 -0.004 -0.060 0.531*** 0.30-0.80 (0.264) (0.421) (0.005) (0.077) (0.116) E-procurement # Treatment 7.618*** 6.414*** -0.006 -0.016 -0.465*** 15.294*** Saturation: 0.30-0.80 (0.285) (0.416) (0.005) (0.074) (0.117) (1.054) Manual # Treatment Saturation: 1.033** -0.029 -0.001 0.093 0.603*** 14.078*** >0.80 (0.395) (0.444) (0.006) (0.077) (0.160) (0.967) E-procurement # Treatment 7.869*** 6.943*** -0.018* -0.009*** -0.993*** 0.558*** Saturation: >0.80 (0.298) (0.437) (0.009) (0.002) (0.166) (0.050) Ln of Contract Value -1.362*** -1.373*** -0.007*** 0.647*** (0.025) (0.033) (0.001) (0.015) Procurement Method: OTM 1.069*** 2.744*** 0.012*** 0.172*** (0.066) (0.091) (0.001) (0.037) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrapping iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.327 0.368 0.102 0.171 0.169 0.138 N 96478 69818 33555 5216 31117 3708 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows 47 TABLE 16: INTERACTED EFFECTS: EFFICIENCY WITH CATEGORICAL SATURATION INTERACTIONS Dependent variable Submission Period Decision Period Signing Period Lead Time Model Model Model Model Model Model Model Model Model (4a): (4b): (1b): (2b): (3b): (1a): CEM (2a): CEM (3a): CEM CEM CEM CEM CEM CEM within within within within within within within within PEs across PEs across PEs across PEs years years years years years years years across across across PEs across PEs across PEs years PEs Baseline:Manual#Treatment Saturation: <0.30 E-procurement # Treatment -2.470*** -2.505*** -1.695 -5.567*** -1.046* -0.656 -14.826*** -13.024*** Saturation: <0.30 (0.345) (0.458) (1.147) (1.213) (0.488) (0.472) (1.385) (1.480) Manual # Treatment Saturation: -0.967*** -2.253*** 5.227*** -2.757* -2.630*** -2.010*** -0.407 -2.041 0.30-0.80 (0.209) (0.353) (0.765) (1.189) (0.419) (0.532) (1.234) (1.694) E-procurement#Treatment -4.445*** -5.290*** -2.161** -5.201*** -6.338*** -5.441*** -22.716*** -22.693*** Saturation:0.30-0.80 (0.224) (0.339) (0.705) (1.092) (0.398) (0.494) (1.161) (1.618) Manual # Treatment Saturation: 3.129*** 2.183*** 23.695*** 11.111*** -4.305*** -2.718*** 18.369*** -0.247 >0.80 (0.356) (0.382) (1.436) (1.272) (0.550) (0.555) (1.882) (1.687) E-procurement # Treatment -4.964*** -5.962*** -0.236 -4.052*** -4.433*** -4.085*** -17.859*** -23.849*** Saturation: >0.80 (0.243) (0.361) (0.813) (1.221) (0.424) (0.528) (1.229) (1.657) Ln of Contract Value 1.154*** 1.497*** 2.266*** 3.111*** 0.615*** 0.652*** 6.150*** 7.986*** (0.021) (0.032) (0.065) (0.078) (0.030) (0.032) (0.091) (0.108) Procurement Method: OTM 0.399*** 1.398*** 4.278*** 4.194*** 0.240* 1.825*** 7.544*** 7.603*** (Base: LTM) (0.057) (0.083) (0.171) (0.252) (0.098) (0.134) (0.251) (0.334) PE Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 500 500 500 500 2 R 0.371 0.402 0.207 0.312 0.144 0.144 0.248 0.369 N 72184 47867 66883 45035 88278 66142 88575 63284 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, 2b, 3b and 4b use bi-annual time windows 48 TABLE 17. INTERACTED EFFECTS: ACCESS WITH CONTINUOUS SATURATION INTERACTIONS Dependent Variable Ln Number of Non-Local Single Bidders Bidders Winners Model Model (2): Model (3): Binary Model (1): OLS Binary Logit Logit Control: Manual Treatment : e-procurement 0.034 0.420*** 1.281*** (0.026) (0.095) (0.096) Treatment Saturation (Continuous) 0.106* -0.152 1.003*** (0.043) (0.132) (0.147) Control: Manual # Treatment Saturation (Continuous) Treatment: e-procurement # Treatment 0.282*** -0.703*** -1.171*** Saturation (Continuous) (0.043) (0.145) (0.152) Ln of Contract Value 0.018*** -0.076*** 0.431*** (0.002) (0.008) (0.009) Procurement Method: OTM (LTM – -1.270*** 0.697*** 0.515*** base) (0.010) (0.026) (0.030) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap Iterations 500 500 500 2 2 R /Pseudo-R 0.396 0.159 0.186 N 99865 98687 90457 Note 1: +p<0.10 *p<0.05 **p<0.01 ***p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients 49 TABLE 18. INTERACTED EFFECTS: ECONOMY WITH CONTINUOUS SATURATION INTERACTIONS Dependent Variable Winning Rebate Cost Overrun Time Overrun Model (3): Model Model (1): OLS Model (2): OLS Binary Logit Control: Manual Treatment : e-procurement 7.610*** -0.033*** -0.253 (0.366) (0.009) (0.161) Treatment Saturation (Continuous) 1.160** 0.003 1.320*** (0.429) (0.005) (0.178) Control: Manual # Treatment Saturation (Continuous) Treatment: e-procurement # Treatment -0.996+ 0.039** -1.326*** Saturation (Continuous) (0.543) (0.015) (0.272) -1.307*** -0.006*** 0.649*** Ln of Contract Value (0.024) (0.001) (0.015) 1.239*** 0.009*** 0.086* Procurement Method: OTM (0.068) (0.002) (0.038) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap iterations 500 500 500 R2/Pseudo-R2 0.328 0.075 0.193 N 98627 33465 30250 Note 1: +p<0.10 *p<0.05 **p<0.01 ***p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients 50 TABLE 19. INTERACTED EFFECTS: EFFICIENCY WITH CONTINUOUS SATURATION INTERACTIONS Submission Decision Signing Dependent Variable Lead Time Period Period Period Model (1): Model (2): Model (3): Model (4): Model OLS OLS OLS OLS Control: Manual -1.341*** 1.246 -3.046*** -11.968*** Treatment: e-procurement (0.275) (0.810) (0.387) (1.127) 2.097*** 18.674*** -1.685** 19.313*** Treatment Saturation (Continuous) (0.355) (1.383) (0.559) (1.834) Control: Manual # Treatment Saturation (Continuous) Treatment: e-procurement # Treatment -4.741*** -18.156*** 0.843 -20.279*** Saturation (Continuous) (0.426) (1.511) (0.629) (1.994) 1.218*** 2.516*** 0.652*** 6.485*** Ln of Contract Value (0.021) (0.068) (0.031) (0.098) 0.600*** 4.372*** 0.303** 8.211*** Procurement Method: OTM (0.056) (0.157) (0.098) (0.260) PE Fixed Effects Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes constant Yes Yes Yes Yes Bootstrap iterations 500 500 500 500 R2 0.371 0.220 0.154 0.257 N 73789 68049 89914 90308 Note: +p<0.10 *p<0.05 **p<0.01 ***p<0.001 51 9.4 OLS REGRESSIONS WITHOUT MATCHING 105. The following tables show the fixed effect OLS regressions without Coarsened Exact Matching in order to show the robustness of our findings to the use of simpler analytical methods. We observe that the point estimates for the treatment effect on all dependent variables are comparable in magnitude and have the same direction as the main results. TABLE 20: MAIN EFFECTS: ACCESS WITHOUT SATURATION INTERACTIONS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model (2): Binary Model (3): Binary Model Model (1): OLS Logit Logit Control: Manual Processing Treatment: E- 0.419*** -0.718*** 0.583*** procurement (0.034) (0.122) (0.122) 0.020*** -0.084*** 0.434*** Ln of Contract Value (0.003) (0.009) (0.010) Procurement Method: -1.268*** 0.732*** 0.544*** OTM (LTM – Base) (0.010) (0.028) (0.032) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap Iterations 500 500 500 R2/Pseudo-R2 0.397 0.161 0.191 N 87436 85817 78300 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients 52 TABLE 21: MAIN EFFECTS: ECONOMY WITHOUT SATURATION INTERACTIONS Dependent variable Winning Rebate Cost Overrun Time Overrun Model Model (1): OLS Model (2): OLS Model (3): Binary Logit Control: Manual Processing 7.617*** -0.018* -0.854*** Treatment: E-procurement (0.409) (0.007) (0.154) Ln of Contract Value -1.182*** -0.005*** 0.729*** (0.025) (0.001) (0.020) Procurement Method: OTM 1.233*** 0.010*** 0.133** (LTM – Base) (0.069) (0.002) (0.047) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap Iterations 500 500 500 R2/Pseudo-R2 0.335 0.068 0.214 N 86570 23990 21132 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients TABLE 22: MAIN EFFECTS: EFFICIENCY WITHOUT SATURATION INTERACTIONS Dependent variable Submission Period Decision Period Signing Period Lead Time Model Model (1): OLS Model (2): OLS Model (3): OLS Model (4): OLS Control: Manual Processing Treatment: E- -5.466*** 0.724 -5.255*** -14.601*** procurement (0.267) (0.914) (0.430) (1.300) 1.179*** 2.489*** 0.647*** 6.230*** Ln of Contract Value (0.023) (0.071) (0.032) (0.104) Procurement Method: 0.507*** 4.307*** 0.399*** 8.180*** OTM (LTM – Base) (0.060) (0.168) (0.101) (0.274) PE Fixed Effects Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Constant Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 R2 0.389 0.228 0.172 0.252 N 63880 59065 77854 78232 Note: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 53 TABLE 23: INTERACTED EFFECTS: ACCESS WITH SATURATION INTERACTIONS Ln Number of Non-Local Dependent variable Single Bidders Bidders Winners Model (2): Binary Model (3): Binary Model Model (1): OLS Logit Logit Baseline: Manual # Treatment Saturation: <0.30 E-procurement # Treatment -0.066* 0.566*** 1.068*** Saturation: <0.30 (0.028) (0.104) (0.115) Manual # Treatment Saturation: 0.114*** -0.242** 0.073 0.30-0.80 (0.025) (0.082) (0.088) E-procurement # Treatment 0.352*** -0.290*** 0.891*** Saturation: 0.30-0.80 (0.022) (0.074) (0.085) Manual # Treatment Saturation: 0.202*** -0.398** 0.966*** >0.80 (0.039) (0.124) (0.127) E-procurement # Treatment 0.436*** -0.496*** 0.854*** Saturation: >0.80 (0.026) (0.088) (0.093) 0.018*** -0.076*** 0.432*** Ln of Contract Value (0.002) (0.008) (0.009) Procurement Method: OTM (LTM -1.272*** 0.702*** 0.516*** – Base) (0.009) (0.026) (0.030) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap Iterations 500 500 500 R2/Pseudo-R2 0.397 0.159 0.186 N 99865 98687 90457 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients 54 TABLE 24: INTERACTED EFFECTS: ECONOMY WITH SATURATION INTERACTIONS Dependent variable Winning Rebate Cost Overrun Time Overrun Model (3): Binary Model Model (1): OLS Model (1): OLS Logit Baseline: Manual # Treatment Saturation: <0.30 E-procurement # Treatment 6.490*** -0.045*** -0.642*** Saturation: <0.30 (0.502) (0.011) (0.178) Manual # Treatment Saturation: 0.328 -0.003 0.509*** 0.30-0.80 (0.222) (0.003) (0.096) E-procurement # Treatment 7.572*** -0.007+ -0.400*** Saturation: 0.30-0.80 (0.241) (0.004) (0.100) Manual # Treatment Saturation: 0.706+ 0.004 0.581*** >0.80 (0.408) (0.005) (0.146) E-procurement # Treatment 7.563*** -0.019** -0.894*** Saturation: >0.80 (0.266) (0.007) (0.146) -1.305*** -0.006*** 0.651*** Ln of Contract Value (0.024) (0.001) (0.015) Procurement Method: OTM (LTM 1.249*** 0.008*** 0.113** – base) (0.068) (0.001) (0.039) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes Bootstrap Iterations 500 500 500 R2/Pseudo-R2 0.328 0.076 0.193 N 98627 33465 30250 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients 55 TABLE 25: INTERACTED EFFECTS: EFFICIENCY WITH SATURATION INTERACTIONS Submission Decision Signing Dependent variable Lead Time Period Period Period Model (1): Model (2): Model (3): Model (4): Model OLS OLS OLS OLS Baseline:Manual#Treatment Saturation: <0.30 E-procurement # Treatment -2.460*** -1.552 -0.185 -12.340*** Saturation: <0.30 (0.347) (1.129) (0.449) (1.368) Manual # Treatment Saturation: -0.939*** 4.632*** -1.108*** 3.341*** 0.30-0.80 (0.180) (0.662) (0.311) (0.993) E-procurement#Treatment -4.199*** -1.900** -4.927*** -18.561*** Saturation:0.30-0.80 (0.187) (0.591) (0.297) (0.899) Manual # Treatment Saturation: 3.067*** 21.915*** -3.499*** 19.186*** >0.80 (0.349) (1.508) (0.544) (1.934) E-procurement # Treatment -4.637*** -0.368 -3.731*** -15.858*** Saturation: >0.80 (0.220) (0.702) (0.334) (1.002) 1.223*** 2.528*** 0.648*** 6.499*** Ln of Contract Value (0.021) (0.068) (0.031) (0.098) Procurement Method: OTM (LTM 0.580*** 4.281*** 0.333*** 8.209*** – base) (0.056) (0.155) (0.098) (0.262) PE Fixed Effects Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes constant Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 2 R 0.373 0.223 0.156 0.257 N 73789 68049 89914 90308 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 56 9.5 NORMALIZED DECISION PERIOD LENGTH RESULTS 106. These regression tables replicate regression tables for decision period length but replacing the dependent variable with normalized decision period length, that is total decision period length (days) divided by the number of bids received. For comparison, we also reproduce the standard tables using non- normalized decision period length regressions. TABLE 26: MAIN EFFECT ON NORMALISED DECISION PERIOD Dependent variable Decision Period Decision Period/#bidders Model (1a): Model (1b): Model (1c): Model (2a): Model (2b): Model (2c): CEM within CEM within OLS with CEM within CEM within OLS with Model PEs across years across FE without PEs across years across FE without years PEs matching years PEs matching Control: Manual Processing 2.210* 2.067+ 0.724 -2.846*** -1.304+ -2.724*** Treatment: E-procurement (1.003) (1.107) (0.914) (0.678) (0.678) (0.486) 2.151*** 3.239*** 2.489*** 0.646*** 1.657*** 0.795*** Ln of Contract Value (0.067) (0.118) (0.071) (0.045) (0.074) (0.044) Procurement Method: 4.090*** 3.284*** 4.307*** 5.448*** 4.875*** 5.615*** OTM (LTM – Base) (0.175) (0.308) (0.168) (0.114) (0.214) (0.109) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 500 500 2 R 0.201 0.284 0.228 0.195 0.251 0.200 N 58013 21905 59065 57912 21801 58921 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, and 2b use bi-annual time windows 57 TABLE 27: INTERACTED EFFECT ON NORMALISED DECISION PERIOD Dependent variable Decision Period Decision Period/#bidders Model Model (1a): Model (1b): Model (1c): Model (2a): Model (2b): Model (2c): CEM within CEM within OLS with CEM within CEM within OLS with PEs across years across FE without PEs across years across FE without years PEs matching years PEs matching Baseline:Manual#Treatment Saturation:<0.30 E-procurement # Treatment Saturation: -1.695 -5.567*** -1.552 0.094 -0.916 0.421 <0.30 (1.147) (1.213) (1.129) (0.734) (0.692) (0.675) Manual # Treatment Saturation: 0.30- 5.227*** -2.757* 4.632*** 0.454 -1.779* 0.893* 0.80 (0.765) (1.189) (0.662) (0.496) (0.755) (0.395) E-procurement#Treatment -2.161** -5.201*** -1.900** -4.090*** -3.610*** -3.825*** Saturation:0.30-0.80 (0.705) (1.092) (0.591) (0.450) (0.714) (0.357) Manual # Treatment Saturation: >0.80 23.695*** 11.111*** 21.915*** 6.744*** 5.830*** 6.927*** (1.436) (1.272) (1.508) (0.809) (0.798) (0.921) E-procurement # Treatment Saturation: -0.236 -4.052*** -0.368 -3.752*** -2.572*** -3.212*** >0.80 (0.813) (1.221) (0.702) (0.515) (0.761) (0.444) Ln of Contract Value 2.266*** 3.111*** 2.528*** 0.725*** 1.525*** 0.812*** (0.065) (0.078) (0.068) (0.040) (0.051) (0.040) Procurement Method: OTM (LTM – 4.278*** 4.194*** 4.281*** 5.262*** 5.306*** 5.377*** base) (0.171) (0.252) (0.155) (0.112) (0.151) (0.108) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 500 500 2 R 0.207 0.312 0.223 0.196 0.291 0.197 N 66883 45035 68049 66744 44988 67879 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, and 2b use bi-annual time windows 58 9.6 REGRESSIONS WITH DIFFERENT SATURATION THRESHOLDS 107. The main treatment effects were estimated by comparing manual tenders from procuring entities with low saturation of e-procurement tenders with electronic tenders from procuring entities with high saturation of electronic tenders. The threshold for defining low and high saturation procuring entities has been made symmetric and the main results have been duplicated. TABLE 28: MAIN EFFECTS WITH DIFFERENT SATURATION THRESHOLD FOR TREATMENT VARIABLE: ACCESS Dependent variable Ln Number of Bidders Single Bidders Non-Local Winners Model Model (1a): Model (1b): Model (2a): Model (2b): Model (3a): Model (3b): CEM CEM CEM CEM CEM CEM within PEs within years within PEs within years within PEs within years across years across PEs across years across PEs across years across PEs Regression type OLS Binary Logit Binary Logit Control: Manual Processing Treatment: E- 0.783*** 0.377*** -1.346*** -0.840*** 0.771*** 0.970*** procurement (0.04) (0.04) (0.13) (0.14) (0.19) (0.16) Ln of Contract Value 0.020*** -0.076*** -0.091*** 0.091*** 0.420*** 0.450*** (0.00) (0.00) (0.01) (0.02) (0.01) (0.02) Procurement Method: -1.266*** -1.226*** 0.666*** 0.550*** 0.440*** 0.135** OTM (LTM – base) (0.01) (0.02) (0.03) (0.04) (0.03) (0.05) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 500 500 2 2 R /Pseudo-R 0.378 0.373 0.148 0.177 0.172 0.223 N 85751 34405 84842 32255 77187 33504 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 59 TABLE 29: MAIN EFFECTS WITH DIFFERENT SATURATION THRESHOLD FOR TREATMENT VARIABLE: ECONOMY Dependent variable Winning Rebate Cost Overrun Time Overrun Model Model (1a): Model (1b): Model (2a): Model (2b): Model (3a): Model (3b): CEM within CEM within CEM within CEM within CEM within CEM within PEs across years across PEs across years across PEs across years across years PEs years PEs years PEs Regression type OLS OLS Binary Logit Control: Manual Processing Treatment: E- 6.282*** 6.862*** -0.001 -0.253** -0.663*** 1.914** procurement (0.447) (0.470) (0.009) (0.087) (0.174) (0.719) Ln of Contract Value -1.265*** -1.734*** -0.004*** -0.021*** 0.695*** 0.805*** (0.027) (0.046) (0.001) (0.005) (0.017) (0.106) Procurement Method: 0.923*** -0.459*** 0.014*** 0.083*** 0.110* -0.450+ OTM (LTM – base) (0.069) (0.134) (0.002) (0.024) (0.043) (0.251) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 500 500 2 R 0.329 0.335 0.060 0.065 0.181 0.081 N 85002 33698 24666 1371 23234 1184 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients Note 3: Models 1b, 2b, and 3b use bi-annual time windows. 60 TABLE 30: MAIN EFFECTS WITH DIFFERENT SATURATION THRESHOLD FOR TREATMENT VARIABLE Dependent variable Submission Period Decision Period Signing Period Lead Time Model Model Model Model Model Model Model Model (4a): (4b): (1a): CEM (1b): CEM (2a): CEM (2b): CEM (3a): CEM (3b): CEM CEM CEM Model within PEs within within PEs within within PEs within within within across years across years across years PEs years years across PEs years across PEs years across PEs across across years PEs Regression Type OLS OLS OLS OLS Control: Manual Processing Treatment: E- -5.759*** -6.203*** 3.678** 3.762** -8.443*** -4.765*** -21.104*** -16.150*** procurement (0.331) (0.409) (1.149) (1.191) (0.656) (0.556) (1.866) (1.770) 1.072*** 1.125*** 2.138*** 3.121*** 0.557*** 1.003*** 5.706*** 7.756*** Ln of Contract Value (0.023) (0.041) (0.072) (0.114) (0.035) (0.054) (0.098) (0.158) Procurement Method: 0.386*** 0.807*** 3.995*** 3.006*** 0.316** -0.086 7.431*** 8.252*** OTM (LTM – base) (0.056) (0.107) (0.170) (0.288) (0.108) (0.181) (0.267) (0.438) PE Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes Yes Yes Constant Yes Yes Yes Yes Yes Yes Yes Yes Bootstrap Iterations 500 500 500 500 500 500 500 500 2 R 0.387 0.341 0.198 0.281 0.154 0.187 0.244 0.287 N 62660 22268 58282 21975 76736 33984 76964 33775 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: Models 1b, 2b, 3b, and 4b use bi-annual time windows. 61 9.7 ADDITIONAL DESCRIPTIVE STATISTICS 108. This section includes tabulation of the mean and standard error of dependent and independent variables for tenders administered manually and electronically. The tabulations have also been done by procurement method and by financial year. TABLE 31: DESCRIPTIVE STATISTICS OF OUTCOME INDICATORS (UNWEIGHTED) Mean/Observations Coefficient/ p-value Outcomes Variables Manual E-proc. Total Treatment vs Control 1.537 1.362 1.384 -0.175*** Ln of Number of (0.013) (0.004) (0.004) (0.012) Bidders 10009 69240 79249 0.169 0.194 0.191 0.025*** Access Single Bidders (0.004) (0.002) (0.001) (0.004) 10009 69240 79249 0.129 0.216 0.204 0.087*** Non-Local Winners (0.003) (0.002) (0.002) (0.004) 9850 60472 70322 0.947 6.968 6.219 6.021*** Winning Rebate (0.092) (0.035) (0.033) (0.098) 9753 68705 78458 0.305 0.191 0.291 -0.115*** Economy Time Overrun (0.005) (0.011) (0.004) (0.013) 9551 1349 10900 -0.015 -0.017 -0.016 -0.002 Cost Overrun (0.001) (0.004) (0.001) (0.003) 9256 1307 10563 25.7 19.48 20.65 -6.21*** Submission Period (0.08) (0.02) (0.03) (0.06) 10022 43137 53159 25.78 21.07 21.83 -4.71*** Decision Period (0.25) (0.07) (0.07) (0.2) 8203 43108 51311 Efficiency 17.6 16.05 16.27 -1.54*** Signing Period (0.14) (0.04) (0.04) (0.11) 9763 60068 69831 83.24 65.91 68.35 -17.34*** Lead Time (0.41) (0.09) (0.1) (0.28) 9879 60114 69993 1.481 1.828 1.783 0.346*** Procurement Method (0.005) (0.001) (0.001) (0.004) Independent 10163 69240 79403 Variables 14.82 14.902 14.892 0.082*** Ln of Contract Value (0.014) (0.005) (0.004) (0.013) 10163 69240 79403 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 62 TABLE 32: DESCRIPTIVE STATISTICS OF INDICATORS BY PROCUREMENT METHOD (WEIGHTED) LTM OTM Coefficient/ Coefficient/ Mean/Observations Mean/Observations Outcome Variables p-value p-value Treatment Treatment Manual E-proc Total Manual E-proc Total vs Control vs Control Ln of 1.92 2.533 2.184 0.613*** 1.1 1.118 1.114 0.020** Number of (0.012) (0.013) (0.009) (0.019) (0.006) (0.003) (0.003) (0.007) Bidders 15835 11932 27767 14791 57308 72099 0.17 0.107 0.14 -0.058*** 0.18 0.212 0.205 0.034*** Single Access (0.003) (0.003) (0.002) (0.004) (0.003) (0.002) (0.002) (0.004) Bidders 15835 11932 27767 14791 57308 72099 0.07 0.115 0.09 0.042*** 0.18 0.237 0.225 0.053*** Non-Local (0.002) (0.003) (0.002) (0.004) (0.003) (0.002) (0.002) (0.004) Winners 15758 10135 25893 14389 50337 64726 1.37 3.723 2.378 2.353*** -0.01 7.649 6.134 7.66*** Winning (0.038) (0.029) (0.026) (0.05) (0.101) (0.041) (0.04) (0.096) Rebate 15929 11923 27852 13993 56782 70775 0.34 0.175 0.334 -0.16*** 0.27 0.192 0.262 -0.076*** Time Economy (0.004) (0.031) (0.004) (0.038) (0.004) (0.011) (0.004) (0.013) Overrun 15786 154 15940 13465 1195 14660 -0.01 -0.05 -0.014 -0.036*** -0.02 -0.013 -0.017 0.005 Cost (0.001) (0.014) (0.001) (0.006) (0.001) (0.005) (0.001) (0.003) Overrun 15601 154 15755 12717 1153 13870 25.59 18.525 23.373 -7.065*** 25.51 19.677 21.367 -5.829*** Submission (0.056) (0.049) (0.046) (0.088) (0.067) (0.028) (0.03) (0.061) Period 16036 7331 23367 14625 35806 50431 20.17 17.961 19.353 -2.209*** 30.64 21.71 24.016 -8.934*** Decision (0.145) (0.119) (0.102) (0.211) (0.233) (0.081) (0.087) (0.195) Period 12496 7327 19823 12454 35781 48235 19.44 16.286 18.218 -3.158*** 16.04 16.005 16.012 -0.034 Signing Efficiency (0.116) (0.087) (0.079) (0.161) (0.119) (0.042) (0.042) (0.102) Period 15850 10060 25910 14001 50008 64009 76.78 61.165 70.736 -15.611*** 91.2 66.859 72.253 -24.346*** Lead Time (0.259) (0.19) (0.181) (0.359) (0.398) (0.104) (0.126) (0.289) 15956 10069 26025 14242 50045 64287 Ln of 14.67 14.541 14.614 -0.125*** 14.96 14.977 14.973 0.022+ Contract (0.008) (0.011) (0.007) (0.014) (0.014) (0.005) (0.005) (0.012) Value 16202 11932 28134 14886 57308 72194 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 63 TABLE 33: DESCRIPTIVE STATISTICS OF INDICATORS BY PROCUREMENT METHOD (UNWEIGHTED) LTM OTM Coefficient/ Coefficient/ Mean/Observations Mean/Observations Outcomes Variables p-value p-value Treatment Treatment vs Manual E-proc. Total Manual E-proc. Total vs Control Control Ln of 1.949 2.533 2.357 0.585*** 1.103 1.118 1.117 0.015 Number of(0.022) (0.013) (0.012) (0.025) (0.011) (0.003) (0.003) (0.012) Bidders 5139 11932 17071 4870 57308 62178 0.159 0.107 0.123 -0.052*** 0.18 0.212 0.209 0.032*** Single Access (0.005) (0.003) (0.003) (0.005) (0.006) (0.002) (0.002) (0.006) Bidders 5139 11932 17071 4870 57308 62178 0.074 0.115 0.101 0.042*** 0.189 0.237 0.233 0.047*** Non-Local (0.004) (0.003) (0.002) (0.005) (0.006) (0.002) (0.002) (0.006) Winners 5116 10135 15251 4734 50337 55071 1.408 3.723 3.022 2.316*** 0.425 7.649 7.11 7.223*** Winning (0.067) (0.029) (0.03) (0.062) (0.18) (0.041) (0.041) (0.152) Rebate 5176 11923 17099 4577 56782 61359 0.335 0.175 0.331 -0.16 0.271 0.192 0.254 -0.078*** Time Economy (0.007) (0.031) (0.006) (0.038) (0.007) (0.011) (0.006) (0.014) Overrun 5130 154 5284 4421 1195 5616 -0.014 -0.05 -0.015 -0.036*** -0.018 -0.013 -0.017 0.005 Cost (0.001) (0.014) (0.001) (0.007) (0.001) (0.005) (0.002) (0.004) Overrun 5071 154 5225 4185 1153 5338 25.636 18.525 21.481 -7.111*** 25.761 19.677 20.397 -6.084*** Submission (0.097) (0.049) (0.058) (0.1) (0.119) (0.028) (0.03) (0.087) Period 5216 7331 12547 4806 35806 40612 20.088 17.961 18.723 -2.127*** 31.459 21.71 22.714 -9.75*** Decision (0.25) (0.119) (0.118) (0.246) (0.421) (0.081) (0.086) (0.279) Period 4094 7327 11421 4109 35781 39890 19.274 16.286 17.299 -2.987*** 15.719 16.005 15.981 0.286+ Signing Efficiency (0.201) (0.087) (0.09) (0.189) (0.201) (0.042) (0.042) (0.152) Period 5156 10060 15216 4607 50008 54615 76.484 61.165 66.379 -15.319*** 90.742 66.859 68.903 -23.883*** Lead Time (0.449) (0.19) (0.206) (0.417) (0.687) (0.104) (0.116) (0.401) 5196 10069 15265 4683 50045 54728 Ln of 14.68 14.541 14.584 -0.139*** 14.971 14.977 14.977 0.006 Contract (0.014) (0.011) (0.009) (0.02) (0.025) (0.005) (0.005) (0.019) Value 5270 11932 17202 4893 57308 62201 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 64 TABLE 34: DESCRIPTIVE STATISTICS OF INDICATORS BY FINANCIAL YEAR 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 Outcomes Variables Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-pro 1.693 0.922 1.531 1.078 1.295 1.298 1.194 1.641 1.171 1.429 0.726 1.290 1.242 Ln of Number (0.013) (0.193) (0.013) (0.040) (0.015) (0.013) (0.022) (0.012) (0.032) (0.009) (0.146) (0.008) (0.007) of Bidders 11,205 9 11,683 345 5,735 5,785 1,274 10,402 713 17,041 16 15,145 0 20,513 0.137 0.111 0.191 0.200 0.214 0.183 0.141 0.155 0.102 0.176 0.313 0.191 0.233 Access Single Bidders (0.003) (0.118) (0.004) (0.022) (0.005) (0.005) (0.010) (0.004) (0.011) (0.003) (0.124) (0.003) (0.003) 11,205 9 11,683 345 5,735 5,785 1,274 10,402 713 17,041 16 15,145 0 20,513 0.132 0.000 0.110 0.433 0.141 0.182 0.140 0.208 0.134 0.180 0.375 0.217 0.263 Non-Local (0.003) (0.000) (0.003) (0.027) (0.005) (0.005) (0.010) (0.004) (0.013) (0.003) (0.129) (0.004) (0.003) Winners 10,976 8 11,541 337 5,652 5,702 1,267 10,079 695 15,331 16 12,931 0 16,084 1.001 -0.116 0.821 12.124 -0.118 9.118 0.929 7.577 1.125 8.752 1.204 7.310 4.283 Winning (0.082) (4.612) (0.083) (0.864) (0.126) (0.168) (0.276) (0.106) (0.319) (0.073) (1.785) (0.069) (0.039) Rebate 10,946 9 11,423 311 5,555 5,490 1,268 10,304 714 16,957 16 15,124 0 20,510 0.312 0.284 0.333 0.191 0.336 0.251 0.300 Economy Time Overrun (0.004) (0.004) (0.006) (0.011) (0.014) (0.017) (0.161) 10,714 0 11,284 0 5,375 1,349 1,188 0 680 0 10 0 0 0 -0.018 -0.011 -0.016 -0.017 -0.026 -0.030 -0.274 Cost Overrun (0.001) (0.001) (0.001) (0.004) (0.002) (0.005) (0.140) 10,469 0 11,081 0 5,146 1,307 1,059 0 556 0 7 0 0 0 25.85 28.00 25.24 22.30 25.19 21.72 26.76 20.71 26.54 19.43 29.88 19.45 18.25 Submission (0.07) (2.45) (0.07) (0.51) (0.10) (0.10) (0.22) (0.07) (0.34) (0.05) (2.74) (0.05) (0.04) Period 11,198 4 11,732 242 5,761 3,595 1,246 6,571 708 10,362 16 9,431 0 12,932 24.40 19.20 24.38 25.11 26.96 19.30 35.80 23.26 25.08 20.36 63.75 21.80 20.42 Decision (0.23) (5.45) (0.22) (1.43) (0.34) (0.27) (0.90) (0.18) (1.09) (0.14) (3.76) (0.16) (0.12) Period 8,785 5 9,799 245 4,704 3,588 1,087 6,575 567 10,348 8 9,421 0 12,926 Efficiency 18.53 28.38 17.25 14.67 18.19 14.24 16.48 14.82 16.31 16.38 42.71 16.65 16.71 Signing Period (0.15) (10.86) (0.12) (0.46) (0.20) (0.12) (0.39) (0.10) (0.30) (0.07) (10.94) (0.08) (0.07) 10,805 8 11,498 336 5,578 5,673 1,249 10,031 707 15,229 14 12,836 0 15,955 81.75 73.13 81.04 63.76 86.67 65.52 102.73 68.81 94.27 66.06 100.06 66.39 63.72 Lead Time (0.38) (14.22) (0.36) (1.69) (0.58) (0.33) (1.39) (0.23) (1.64) (0.18) (8.22) (0.20) (0.18) 10,921 8 11,619 336 5,671 5,676 1,261 10,035 710 15,231 16 12,853 0 15,975 1.426 1.778 1.393 1.933 1.607 1.888 1.899 1.797 1.936 1.878 1.750 1.941 1.698 Procurement (0.005) (0.156) (0.004) (0.013) (0.006) (0.004) (0.008) (0.004) (0.009) (0.003) (0.115) (0.002) (0.003) Method Independent 11,363 9 11,881 345 5,836 5,785 1,278 10,402 714 17,041 16 15,145 0 20,513 Variables 14.61 16.51 14.79 14.44 14.90 14.54 15.50 14.81 15.98 14.99 16.39 15.05 14.88 Ln of Contract (0.01) (0.13) (0.01) (0.08) (0.02) (0.02) (0.03) (0.01) (0.05) (0.01) (0.27) (0.01) (0.01) Value 11,363 9 11,881 345 5,836 5,785 1,278 10,402 714 17,041 16 15,145 0 20,513 Note: There were no Manual tenders in FY17-18 and contract management information for e-procurement tenders was only collected for FY13-14 65 TABLE 35: DESCRIPTIVE STATISTICS OF INDICATORS BY FINANCIAL YEAR FOR OTM TENDERS 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 Outcomes Variables Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Ln of 1.173 0.988 1.042 1.013 1.075 1.092 1.133 1.244 1.031 1.212 0.737 1.130 0.948 Number of (0.011) (0.250) (0.011) (0.037) (0.012) (0.010) (0.020) (0.009) (0.024) (0.007) (0.199) (0.007) (0.006) Bidders 4,846 7 4,631 322 3,490 5,137 1,145 8,295 667 14,969 12 14,254 0 14,324 0.157 0.143 0.216 0.202 0.179 0.200 0.144 0.185 0.109 0.196 0.417 0.202 0.258 Single Access (0.005) (0.154) (0.006) (0.022) (0.006) (0.006) (0.010) (0.004) (0.012) (0.003) (0.155) (0.003) (0.004) Bidders 4,846 7 4,631 322 3,490 5,137 1,145 8,295 667 14,969 12 14,254 0 14,324 0.194 0.000 0.179 0.452 0.196 0.185 0.145 0.220 0.143 0.187 0.500 0.222 0.339 Non-Local (0.006) (0.000) (0.006) (0.028) (0.007) (0.005) (0.010) (0.005) (0.014) (0.003) (0.157) (0.004) (0.004) Winners 4,673 6 4,504 314 3,410 5,060 1,141 8,013 649 13,349 12 12,145 0 11,450 0.057 1.255 -0.035 12.824 -0.551 9.723 0.851 8.327 0.849 9.468 1.301 7.473 4.742 Winning (0.178) (6.014) (0.199) (0.919) (0.199) (0.189) (0.304) (0.131) (0.335) (0.080) (2.433) (0.073) (0.052) Rebate 4,546 7 4,337 288 3,291 4,842 1,139 8,201 668 14,888 12 14,233 0 14,323 0.259 0.283 0.243 0.192 0.326 0.260 0.500 Time Economy (0.007) (0.007) (0.008) (0.011) (0.014) (0.017) (0.245) Overrun 4,380 0 4,240 0 3,142 1,195 1,059 0 638 0 6 0 0 0 -0.017 -0.015 -0.015 -0.013 -0.027 -0.034 -0.639 Cost (0.001) (0.002) (0.002) (0.005) (0.003) (0.005) (0.000) Overrun 4,222 0 4,109 0 2,930 1,153 936 0 517 0 3 0 0 0 25.27 28.00 25.42 22.64 24.96 21.80 27.58 20.91 27.06 19.53 31.83 19.41 18.55 Submission (0.11) (2.45) (0.13) (0.53) (0.13) (0.10) (0.23) (0.08) (0.36) (0.05) (3.53) (0.05) (0.05) Period 4,765 4 4,589 224 3,480 3,243 1,117 5,216 662 9,170 12 8,838 0 9,111 29.84 19.20 30.62 26.44 30.50 19.56 36.92 23.56 25.62 20.30 63.75 21.82 22.61 Decision (0.40) (5.45) (0.40) (1.50) (0.47) (0.29) (0.98) (0.21) (1.12) (0.15) (3.76) (0.17) (0.16) Period 3,972 5 4,082 227 2,857 3,240 988 5,220 547 9,156 8 8,828 0 9,105 Efficiency 17.24 30.17 14.96 14.69 15.27 14.64 17.05 14.91 16.75 16.27 48.60 16.78 16.28 Signing (0.25) (15.10) (0.19) (0.49) (0.21) (0.13) (0.43) (0.11) (0.30) (0.08) (15.32) (0.09) (0.09) Period 4,466 6 4,405 313 3,336 5,039 1,123 7,975 661 13,257 10 12,051 0 11,367 90.14 81.17 89.75 65.13 87.97 65.94 107.22 69.85 97.42 66.18 105.42 66.78 66.10 Lead Time (0.70) (18.28) (0.71) (1.77) (0.79) (0.36) (1.48) (0.27) (1.67) (0.20) (10.69) (0.20) (0.22) 4,548 6 4,464 313 3,422 5,042 1,132 7,978 664 13,261 12 12,068 0 11,377 Ln of 14.65 16.58 14.99 14.50 14.92 14.57 15.55 14.87 16.06 15.03 16.70 15.08 15.03 Independent Contract (0.03) (0.15) (0.03) (0.08) (0.03) (0.02) (0.04) (0.01) (0.05) (0.01) (0.31) (0.01) (0.01) Variables Value 4,846 7 4,666 322 3,545 5,137 1,149 8,295 668 14,969 12 14,254 0 14,324 Note: There were no Manual tenders in FY17-18 and contract management information for e-procurement tenders was only collected for FY13-14 TABLE 36: DESCRIPTIVE STATISTICS OF INDICATORS BY FINANCIAL YEAR FOR LTM TENDERS Outcomes Variables 2011-2012 2012-2013 2013-2014 2014-2015 2015-2016 2016-2017 2017-2018 66 Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Manual E-proc Ln of 2.090 0.693 1.853 1.984 1.636 2.927 1.732 3.204 3.199 2.997 0.693 3.843 1.923 Number of (0.020) (0.000) (0.019) (0.257) (0.031) (0.043) (0.119) (0.026) (0.185) (0.030) (0.000) (0.039) (0.018) Bidders 6,359 2 7,052 23 2,245 648 129 2,107 46 2,072 4 891 0 6,189 0.122 0.000 0.174 0.174 0.268 0.048 0.109 0.038 0.000 0.034 0.000 0.013 0.175 Single Access (0.004) (0.000) (0.005) (0.083) (0.009) (0.008) (0.028) (0.004) (0.000) (0.004) (0.000) (0.004) (0.005) Bidders 6,359 2 7,052 23 2,245 648 129 2,107 46 2,072 4 891 0 6,189 0.087 0.000 0.065 0.174 0.059 0.164 0.095 0.161 0.000 0.134 0.000 0.139 0.076 Non-Local (0.004) (0.000) (0.003) (0.083) (0.005) (0.015) (0.026) (0.008) (0.000) (0.008) (0.000) (0.012) (0.004) Winners 6,303 2 7,037 23 2,242 642 126 2,066 46 1,982 4 786 0 4,634 1.671 -4.915 1.344 3.348 0.512 4.591 1.613 4.654 5.132 3.605 0.912 4.706 3.219 Winning (0.061) (0.021) (0.055) (0.673) (0.104) (0.085) (0.365) (0.038) (0.662) (0.074) (0.000) (0.052) (0.045) Rebate 6,400 2 7,086 23 2,264 648 129 2,103 46 2,069 4 891 0 6,187 0.348 0.284 0.459 0.175 0.419 0.119 0.000 Time Economy (0.006) (0.005) (0.011) (0.031) (0.044) (0.051) (0.000) Overrun 6,334 0 7,044 0 2,233 154 129 0 42 0 4 0 0 0 -0.018 -0.009 -0.018 -0.050 -0.015 0.018 0.000 Cost (0.001) (0.001) (0.002) (0.014) (0.004) (0.012) (0.000) Overrun 6,247 0 6,972 0 2,216 154 123 0 39 0 4 0 0 0 26.28 0.00 25.13 18.00 25.54 21.00 19.64 19.94 19.07 18.70 24.00 19.95 17.52 Submission (0.09) (0.00) (0.08) (1.41) (0.14) (0.39) (0.38) (0.13) (0.66) (0.12) (0.00) (0.20) (0.05) Period 6,433 0 7,143 18 2,281 352 129 1,355 46 1,192 4 593 0 3,821 19.91 0.00 19.92 8.33 21.49 16.81 24.66 22.11 10.20 20.84 0.00 21.58 15.18 Decision (0.22) (0.00) (0.21) (1.95) (0.42) (0.60) (1.31) (0.29) (1.85) (0.30) (0.00) (0.39) (0.15) Period 4,813 0 5,717 18 1,847 348 99 1,355 20 1,192 0 593 0 3,821 Efficiency 19.43 23.00 18.68 14.35 22.53 10.98 11.40 14.46 9.91 17.16 28.00 14.55 17.77 Signing (0.19) (0.00) (0.16) (1.50) (0.35) (0.27) (0.60) (0.18) (0.91) (0.20) (0.00) (0.27) (0.13) Period 6,339 2 7,093 23 2,242 634 126 2,056 46 1,972 4 785 0 4,588 75.76 49.00 75.61 45.09 84.69 62.17 63.37 64.80 48.87 65.27 84.00 60.46 57.85 Lead Time (0.41) (0.00) (0.36) (3.52) (0.81) (0.91) (2.15) (0.43) (2.52) (0.42) (0.00) (0.61) (0.27) 6,373 2 7,155 23 2,249 634 129 2,057 46 1,970 4 785 0 4,598 Ln of 14.59 16.27 14.67 13.61 14.87 14.34 15.00 14.57 14.79 14.70 15.47 14.44 14.52 Independent Contract (0.01) (0.39) (0.01) (0.23) (0.02) (0.05) (0.09) (0.03) (0.16) (0.03) (0.00) (0.04) (0.02) Variables Value 6,517 2 7,215 23 2,291 648 129 2,107 46 2,072 4 891 0 6,189 Note: There were no Manual tenders in FY17-18 and contract management information for e-procurement tenders was only collected for FY13-14 67 9.8 ADOPTION OF E-PROCUREMENT 109. The following graph shows the transition of the procurement system to electronically administered tenders. It is also observed that the BWDB started to transition earlier than LGED and RHD. The transition for RHD was the quickest and the district-level procuring entities office of the RHD had completely shifted to the electronic procurement system by the 2014-2015 financial year. The transitions for LGED and BWDB took longer. FIGURE 5: ADOPTION OF E-PROCUMENT BY PROCURING ENITIES OVER TIME (EACH LINE CORRESPONDS TO THE SATURATION LEVELS OF ELECTRONIC PROCURMENT FOR A PARTICULAR PROCURING ENTITY) 68 9.9 COST SAVINGS CALCULATION DETAILS This appendix sets out the details of the savings model and calculations underpinning the high-level figures in the main text. Savings considered are direct savings resulting from the adoption of the e-procurement system in Bangladesh and fall into three broad categories: economic savings for the public budget, government administrative cost savings, and bidding firm transaction costs. While the savings estimates can be summed by sector (i.e. public and private sectors), we refrain from combining these two high-level savings estimates as it is unclear whether some parts of the accrued savings result from transfers between the two sectors. All savings calculations take the main results models from the text as a baseline (section 7.1). The savings calculations are based on three scenarios or samples to which savings are extrapolated: • Narrow sample: savings in the analysis sample of main results which delivers the most rigorous savings estimation but on an arguably narrow sample in comparison to the total procurement spending in Bangladesh. • Extended sample: savings in sample of all electronic tenders conducted up until the end of FY18. This sample delivers the most reliable estimate of the total savings achieved through e-procurement while relying on some assumptions underpinning extrapolation from the analysis sample to further electronic tenders. • Full-government annual sample: sample of all public procurement spending in Bangladesh for one financial year. Savings estimates corresponding to this broad sample represent a hypothetical scenario in which all government tenders would be processed through e-procurement. While such a broad extrapolation arguably has to rely on a number of assumptions, it nevertheless sets the boundaries to the maximum annual savings to be expected from the system at full implementation. 9.9.1 NARROW SAMPLE ESTIMATE The estimated savings categories and their precise definitions for the narrow sample estimate are outlined in Table 37 below. 69 TABLE 37: SAVINGS CATEGORIES AND DEFINITIONS Component Description Calculation Strategy 1. Government Economic Cost Savings a. Higher rebate Savings due to lower = ∑ ( Savings barriers for entry, higher − ) competition reduced × . ) collusive, coercive and corrupt practices 2. Government Administrative Cost Savings a. Advertisement Savings arising due to = ∑ # × ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ Cost Savings lower number of − # × ̅̅̅̅̅̅̅̅̅̅̅̅̅̅ newspaper advertisements required for eGP tenders b. Tender Savings due to not = ∑ ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ Preparation requiring printing, paper, Cost Savings mail, transportation c. Security Cost Savings due to not = ∑ ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ Savings requiring security support × % while accepting bids d. Staff-time Staff-time savings due to = ∑ ( − ) savings increased efficiency of Procuring Entity officials ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ × 3. Bidder Transaction Cost Savings a. Staff-time Staff-time savings due to = ∑ × # ( savings increased efficiency of × # − ) bidders ̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅̅ × b. Tender Savings due to lower = ∑ × # ( Preparation tender preparation costs × # − ) Cost Savings c. Security Cost Savings due to not = ∑ × # Savings requiring security support while submitting bids d. Bribe Savings Savings due to reduced = ∑ (+ℎ bribe payments × # − +ℎ × # ) 1. Government Economic Cost Savings: The economic cost savings are a result of multiple mechanisms as discussed in the paper. The savings were estimated using the models obtained after two matching methods, as done for the Winning Rebate models in the results. The savings for the 69240 tenders was 70 estimated to be Tk. 3576 crore (US$ 421 million – model 1a – Table 4) when matching by procuring entity, and Tk. 3163 crore (US$ 372 million - model 2a – Table 4) when matching by financial year. Bangladesh has had an inflation rate of about 6% since 2012, when the e-procurement system was introduced. The inflation rates are given in the table below. TABLE 38: BANGLADESH INFLATION RATES BY YEAR (CPI & GDP DEFLATOR) Inflation, GDP Inflation, consumer Year 34 deflator (annual prices (annual %) 35 %) 2011 11.40% 7.86% 2012 6.22% 8.16% 2013 7.53% 7.17% 2014 6.99% 5.67% 2015 6.19% 5.87% 2016 5.51% 6.73% 2017 5.70% 6.28% 2018 5.54% 5.60% The savings are adjusted for inflation using the inflation deflator rates as it is indicative of the rate of change of prices of the economy. Adjusting for inflation the savings was estimated to be Tk. 4,004 crore (US$ 471 million – model 1a – Table 4) when matching by procuring entity, and Tk. 3,606 crore (US$ 424 million - model 2a – Table 4) when matching by financial year of tender at the current taka value for the 69,240 tenders. 2. Government Administrative Cost Savings a. Advertisement Cost Savings: Among the major savings that has resulted from the shift to the electronic procurement system was the decrease in the need for newspaper advertisements. All electronically administered tenders are published on the e-procurement portal and hence to need to advertise it decreases significantly. The advertisement cost savings is estimated by the difference of the product of the average manual advertisement cost and the predicted number of advertisements had the electronic tender been not done electronically and the product of the average e-procurement advertisement cost and the number of advertisements of that electronic tender. The number of advertisements of an 34 Inflation as measured by the consumer price index reflects the annual percentage change in the cost to the average consumer of acquiring a basket of goods and services that may be fixed or changed at specified intervals, such as yearly. The Laspeyres formula is generally used. (International Monetary Fund, International Financial Statistics and data files.) 35 Inflation as measured by the annual growth rate of the GDP implicit deflator shows the rate of price change in the economy as a whole. The GDP implicit deflator is the ratio of GDP in current local currency to GDP in constant local currency. (World Bank national accounts data, and OECD National Accounts data files.) 71 electronic tender had it not been advertised electronically, has been predicted using the same two models (using matching methods) used for the main data indicators. The models are shown below along with a fixed effects regression for comparison. TABLE 39: TREATMENT EFFECT ON NEWSPAPER ADVERTISEMENTS Dependent variable Number of Newspaper Advertisements Model Model (1a) – Model (1b) – Model (1c) – Matching within Matching within year OLS with agency across year across PEs Fixed Effects Control: Manual Processing Treatment: E-procurement -1.484*** -1.463*** -1.598*** (0.04) (0.04) (0.03) Ln of Contract Value 0.016*** -0.007+ 0.009*** (0.00) (0.00) (0.00) Procurement Method: OTM 0.335*** -0.049*** 0.357*** (LTM – Base) (0.01) (0.01) (0.01) PE Fixed Effects Yes Yes Yes Year Fixed Effects Yes Yes Yes constant Yes Yes Yes R2 0.458 0.557 0.482 N 85674 34590 87824 Note: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 We observe that the number of advertisements has gone down in all three models, corroborating the expected decrease. The rates for advertisement are regulated and revised by the Department of Films and Publication, Government of Bangladesh depending on the circulation of respective newspaper. Based on the survey of Procuring Entity Officials, the average cost of an advertisement for a tender was approximately Tk. 25,200 (approximately US$ 295). It has been assumed that the advertisement costs have increased commensurate with inflation. Based on these figures the cost savings were between Tk. 266 crore (US$ 31 million, as per model 1a) and Tk. 210 crore (US$ 24 million as per model 1b) for about 69,000 tenders. b. Tender Preparation Costs – The shift to electronic tendering has meant a number of costs pertaining to tender preparation are no longer incurred. There were significant printing costs, transportation costs, postal costs that are being saved due to the shift. The savings per tender is estimated on the basis of responses of the PE official’s survey. The averages are calculated by organization as the number of tenders vary significantly by organization (with LGED the majority of tenders in consideration), and the tender preparation cost vary in a statistically significant way across organizations. As the PE officials were asked the average tender preparation cost over a time period, the tender preparation costs have not been adjusted for inflation. The total tender preparation cost savings was estimated to be Tk. 53.5 crore (US$ 6.3 million). 72 TABLE 40: TENDER PREPARATION COST SAVINGS BY ORGANIZATION Savings per Savings per tender Number of Total Savings Total Savings Organization tender (taka) (US dollar) tenders (taka) (US dollar) BWDB 7,280 86 6,793 5.0 crore 0.6 million LGED 7,279 86 54,124 39.4 crore 4.6 million RHD 10,964 129 8,323 9.1 crore 1.1 million c. Security Cost Savings: Due to widespread coercive practices employed by bidders preventing other bidders from physically submitting bids that frequently led to violent incidents, procuring entity officials often needed to request the help of security forces, primarily the police, to deal with such situations. Soliciting the help of the police implied certain costs that were often informal in nature. Procuring Entity Officials were asked for what percentage of tenders did they need to engage security personnel and what was the amount the organization paid or spent for it. The product of the two was calculated and averaged for the organizations. As for tender preparation costs, the security cost savings have not been adjusted for inflation. The total savings for not requiring security was estimated to be Tk. 11.5 crore (US$ 1.4 million). TABLE 41: SUMMARY OF SECURITY COST SAVINGS BY ORGANISATION Savings per Savings per tender Number of Total Savings Total Savings Organization tender (taka) (US dollar) tenders (taka) (US dollar) BWDB 1271 15 6,793 0.9 crore 0.1 million LGED 1777 21 54,124 9.6 crore 1.1 million RHD 1276 15 8,323 1.1 crore 0.1 million d. Staff-time savings: We observe that eGP reduced the times for the different constituent processes of procurement. However, these reduced in times do not translate to staff-time savings for the Procuring Entity Official. PE officials were asked about the time they spent on each constituent activity of the procurement process using the manual and the electronic systems. These times were averaged by the method of procurement used and the difference of the times between the manual and electronic systems was multiplied by the remuneration per unit time. The table given below shows the time saved per activity. TABLE 42: TIME TAKEN FOR EACH ADMINISTRATIVE ACTIVITY OTM LTM Activity Description Time Time Time Time Time Time taken - taken - taken - taken - Saved Saved Manual eGP Manual eGP Creating the tender in the e- Tender 4 hours 3 hours 2 hours procurement system, defining 1 hour 30 mins 2 hours Preparation 15 mins 15 mins 30 mins the technical specifications. 73 Evaluating the winner of the Tender tendering process, 8 hours 3 hours 5 hours 2 hours 1 hour 45 45 mins Evaluation establishing responsiveness 30 mins 15 mins 15 mins 30 min mins of the bids Preparing for sending the Tender decision of evaluation for 1 hour 15 mins 45 mins 1 hour 15 mins 45 mins Approval approval. 13 hours 4 hours 9 hours 1 hour 30 4 hours Total 6 hours 45 mins 30 mins 15 mins mins 30 mins The times mentioned in the table above correspond to the approximate times that procuring entity official inviting the tender takes to complete main aspect of that activity, that has been affected by the e- procurement process. However, this does not account for all the activities done at each stage of the tendering activity. For instance, verification of work completion letters of the winning bidder during the evaluation stage delays the process by multiple days, however, the PE official does not need to dedicate any significant portion of his time while the verification is being acquired. Similarly, the PE official only needs to prepare a document to send it for approval which takes less than an hour. The majority of the time is spent is receiving the approval from the Head of the Procuring Entity or other authority depending on the size of the contract. The average monthly remuneration of PE officials was assumed to be Tk. 70,000 (US$ 830) based on National Pay Scale, 2015. This corresponds to an hourly remuneration of Tk. 500 (US$ 6). The savings for each OTM tender was thus estimated to be Tk. 4,700 (US$ 55), while it was Tk. 2,300 (US$ 27). The total staff savings was estimated to be Tk. 30 crore (US$ 3.5 million) for about 69,000 tenders. TABLE 43: SUMMARY OF ADMINISTRATIVE STAFF TIME SAVINGS Savings per Savings per tender Number of Total Savings Total Savings Procedure tender (taka) (US dollar) tenders (taka) (US dollar) OTM 4,700 55 57,308 26.9 crore 3.2 million LTM 2,300 27 11,932 2.7 crore 0.3 million 3. Bidding Firm Transaction Cost savings a. Staff-time savings: The transition to e-procurement has enabled participating bidders to prepare and submit the bid in a shorter time. The staff-time savings for the bidders was estimated by multiplying the mean salary by the difference of the product of the number of bids if the tender had been paper based by the average tender preparation time for paper-based tenders and the product of the actual number of bids and the average tender preparation time for electronic tenders. The number of bids had the tender been manually administered is predicted using the same two models as used for the main results and have been given in the Table 3, Models 1a and 1b. Based on the survey of bidders, we find that the average staff-time required to prepare a bid is found to be dependent on the contract value only for OTM tenders, as shown in the table. This is expected as 74 LTM tenders are of similar complexity given the lower range of contract sizes, but OTM tenders can be of different levels of complexity thus time are dependent on the contract size. We also observe that the times increases with contract size are larger for paper vis-à-vis e-procurement OTM tenders. The models given in the table are used to estimate the average bid preparation times for OTM tenders whereas a constant average bid preparation time is assumed for LTM tenders. The average bid preparation time for manual LTM tender is found to be 2 hours and 15 minutes, while an e-procurement LTM tender is 1 hour and 15 minutes. TABLE 44: EFFECT OF CONTRACT VALUE ON TENDER PREPARATION TIMES FOR BIDDERS Tender Preparation Times Dependent Variable OTM LTM Paper eGP Paper eGP Model OLS OLS OLS OLS 0.472*** 0.235** -0.063 -0.027 Ln of Contract Value (0.13) (0.09) (0.09) (0.07) Constant Yes Yes Yes Yes R2 0.030 0.014 0.001 0.000 N 458 488 501 524 Note: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 The bidder staff time savings for about 69,000 tenders was estimated to be Tk. 1.93 crore (US$ 0.23 million – using model 1a in Table 7) when matching by procuring entity, and Tk. 0.82 crore (US$ 0.10 million – using model 2a in Table 7) when matching by financial year of tender. The staff time savings for bidders overall are low because of the opposite effects of lower time required to prepare the bids and higher number of bids. b. Tender Preparation Cost Savings: The transition to an electronic system has resulted in significant lowering of tender preparation costs per bid as a number of expenses such as paper, printing transportation, are no longer required. It is worthwhile to note that a there are new costs associated with preparing the bid for electronic submission such as internet costs and computer costs, but these are lower than the manual costs. The tender preparation cost savings for the bidders was estimated as the difference of the product of the number of bids if that tender had been paper based by estimated tender preparation cost for that paper-based tenders and the product of the actual number of bids and the estimated tender preparation cost for that electronic tender. The number of bids had the tender been manually administered is predicted using the same two models as used for the main results and have been given in the Table 7. The tender preparation costs are estimated using the model given in Table 37. 75 TABLE 45: EFFECT OF CONTRACT VALUE ON TENDER PREPARATION COSTS FOR BIDDERS Logarithm of tender preparation cost Dependent Variable OTM LTM Paper eGP Paper eGP Model OLS OLS OLS OLS 0.295*** 0.239*** 0.219*** 0.159*** Ln of Contract Value (0.03) (0.03) (0.03) (0.03) Constant 3.475*** 4.272*** 4.294*** 5.256*** (0.44) (0.46) (0.50) (0.48) R2 0.182 0.125 0.081 0.048 N 540 533 519 537 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 The total bidder tender preparation cost savings was estimated to be negative using both models indicating the overall cost increased due to the transition to electronic bids. The increased overall cost for about 69,000 tenders was estimated to be Tk. 51.1 crore (US$ 6.01 million – using model 1a in Table 7) when matching by procuring entity, and Tk. 60.0 crore (US$ 7.06 million - model 2a in Table 7) when matching by financial year of tender. The increased cost is obtained because the effect of higher number of bids overshadows the lower bid preparation costs. c. Security Cost Savings: One of the primary reasons for the introduction of the electronic government procurement system was frequent coercive incidents while physically submitting the bid at the procuring entity office. This was especially true when the bid was being submitted by a non-local firm. These firms would therefore spend to ensure their safety while submitting these bids. The security cost savings is estimated by multiplying the number of bids had the tender been a manual tender with the average security cost incurred for submitting a bid, as obtained from the responses to the bidders’ survey. The average security cost for submitting a bid was Tk. 2,950 (US$ 35). The bidder security cost savings for about 69,000 tenders was estimated to be Tk. 105.9 crore (US$ 12.46 million – using model 1a in Table 7) when matching by procuring entity, and Tk. 97.2 crore (US$ 11.4 million – using model 2a in Table 7) when matching by financial year of tender. We observe that the security costs are on average higher for firms that submit or win a larger proportion of their bids, which indicates that security expenses are higher for non-local firms. 76 TABLE 46: EFFECT OF OUT OF DISTRICT BIDDERS ON SECURITY COST Dependent Variable Security Cost Model 1a – Model 1b – Model OLS +FE OLS +FE Control: 0-40% Percentage of out of 476.303 district paper-based 40-80% bid submissions (870.45) 1719.289 80-100% (1947.27) Control: 0-40% Percentage of out of 313.856 district paper-based 40-80% bid wins (1038.07) 1165.378 80-100% (2247.02) Ln of contract value Yes Yes District Fixed Effects Yes Yes Constant Yes Yes R2 0.066 0.077 N 470 269 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 d. Reduction in Bribes: One of the reasons behind the transition to the electronic system was to reduce corrupt practices. There were two kinds of bribes that were given by firms, one to other firms that would otherwise hinder their participation and, the other to government officials. As expected, the bribes given to other firms reduced substantially due to the transition to electronic procurement, as firms are now able to participate in any tender across the country without physically being present there. However, the bribes given to government officials remain largely unaffected. The bribes paid to government officials are not paid to secure the contracts, but after the contract has been awarded, and is only paid by the winning firm. Two different estimates for the reduction in bribes have been given. This is because the respondents in a survey will often underreport their participation in corrupt activities, and data from firms that responded with a non-zero figure has been used to obtain an upper limit for the estimate. The table below reports the average bribes as a percentage of contract value. 77 TABLE 47: TREATMENT EFFECT ON BRIBES Coefficient/p- Reduced Reduced Bribes as a %age of contract value value Bribes Bribes Observations Outcomes overall (in overall (in used Treatment vs BDT US$ Manual Electronic Total Control crore) million) All 0.657 0.066 0.357 0.590*** 445.17 55.65 Bribe to Responses (0.077) (0.024) (0.041) (0.079) other bidders Non-zero 1.840 0.841 1.654 0.998** 753.38 94.17 Responses (0.186) (0.277) (0.162) (0.411) All 0.754 0.668 0.710 0.087 65.46 8.18 Bribes to Responses (0.077) (0.080) (0.056) (0.112) govt. officials Non-zero 1.734 2.003 1.853 -0.268 -202.26 -25.28 Responses (0.155) (0.206) (0.125) (0.253) Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Bribes have not been categorized as savings as a part of this reduction is passed on as greater rebate rates. As we observe that the transition has very little impact on the bribes to government officials, it may be assumed that these costs are accounted for in the quoted price of a bid. 78 TABLE 48: SUMMARY OF ESTIMATED SAVINGS FOR LGED, RHD, AND BWDB WORKS TENDERS ADMINISTERED USING ELECTRONIC PROCUREMENT FROM FY12 TO FY18 (69,240 TENDERS) Savings (in BDT Savings (in US$ crore) million) Components Lower Upper Lower Upper bound bound bound bound 1. Government Economic Cost Savings a. Winning Rebates (inflation adjusted) 3606.8 4004.0 424.3 471.1 2. Government Administrative Cost Savings a. Advertisement Cost Savings 210.7 266.4 24.8 31.3 b. Tender Preparation Cost Savings 53.5 53.5 6.3 6.3 c. Security Cost Savings 11.5 11.5 1.4 1.4 d. Staff-time savings 29.7 29.7 3.5 3.5 Total Government Savings 3912.2 4365.1 460.3 513.6 3. Firm Transaction Cost Savings a. Staff-time savings 0.8 1.9 0.1 0.2 b. Tender Preparation Cost Savings -60.0 -51.1 -7.1 -6.0 c. Security Cost Savings 97.2 105.9 11.4 12.5 Total Private Sector Savings 38.0 56.7 4.4 6.7 Note: lower bound estimates are produced by models employing CEM matching within years, while upper bound estimates are based on models using CEM matching within PEs. 9.9.2 EXTENDED SAMPLE ESTIMATE Based on the estimates of the narrow sample, savings estimates were made for all tenders administered using the e-procurement system. The average savings by component has been mentioned in the table below. 79 TABLE 49: SUMMARY OF AVERAGE ESTIMATED SAVINGS PER TENDER ADMINISTERED ELECTRONICALLY FROM FY12 TO FY18 (BASED ON THE NARROW SAMPLE) Savings per tender (in Savings per tender (in BDT) US$) Components Lower Upper Lower Upper bound bound bound bound Government Administrative Cost Savings a. Advertisement Cost Savings 30,424 38,480 362 458 b. Tender Preparation Cost Savings 7,727 7,727 91 91 c. Security Cost Savings 1,661 1,661 20 20 d. Staff-time savings 4,289 4,289 51 51 Administrative Savings per tender 44,101 52,157 524 620 Firm Transaction Cost Savings a. Staff-time savings 116 274 1 3 b. Tender Preparation Cost Savings -8,666 -7,380 -103 -87 c. Security Cost Savings 14,038 15,295 165 181 Total Private Sector Savings 5,488 8,189 64 97 Note: lower bound estimates are produced by models employing CEM matching within years, while upper bound estimates are based on models using CEM matching within PEs. Based on the average savings given in Table 49, the savings components for all tenders administered till FY19 electronically is extrapolated,36 For estimating the savings due to higher winning rebates, the average percentage savings on the estimated cost was used. The savings was estimated to be between 6.5% and 7.4% of the estimated cost of a tender. The components of the savings have been given in Table 50. TABLE 50: SUMMARY OF ESTIMATED SAVINGS FOR E-PROCUREMENT WORKS TENDERS FROM FY12 TO FY19 (275,201 TENDERS, BDT 2.84 LAKH CRORE OR US$ 33.8 BILLION) Savings (in BDT Savings (in US$ crore) million) Components Lower Upper Lower Upper bound bound bound bound 1. Economic Cost Savings 36 This means extrapolating in three dimensions: - From electronic tenders in district level Procuring Entities of three government agencies to electronic tenders in all Procuring Entities of all government organizations; - From e-procurement tenders for FY11-12 to FY17-18 to e-procurement in FY18-19; - From electronic tenders for public works only to all electronic tenders, including for goods and services. The assumptions made for the extrapolation are: - For savings due to increased winning rebates, that the average reduction in procurement costs per tender as a percentage of the official cost estimate due to e-procurement is the same across (a) all PEs from all organizations, (b) works, goods and service contracts, and (c) FYs; - For government administrative savings and private sector savings, the average reduction in procurement costs per tender in absolute terms due to eProcurement is the same across (a) all PEs from all organizations, (b) works, goods and service contracts, and (c) FYs. 80 a. Winning Rebates (inflation adjusted) 18466.3 21023.1 2198.4 2502.8 2. Government Administrative Cost Savings a. Advertisement Cost Savings 779.5 985.9 92.8 117.4 b. Tender Preparation Cost Savings 198.0 198.0 23.6 23.6 c. Security Cost Savings 42.6 42.6 5.1 5.1 d. Staff-time savings 109.9 109.9 13.1 13.1 Total Government Savings 19596.2 22359.4 2332.9 2661.8 3. Firm Transaction Cost Savings a. Staff-time savings 3.0 7.0 0.4 0.8 b. Tender Preparation Cost Savings -222.0 -189.1 -26.4 -22.5 c. Security Cost Savings 359.7 391.9 42.8 46.7 Total Private Sector Savings 140.6 209.8 16.7 25.0 Note: lower bound estimates are produced by models employing CEM matching within years, while upper bound estimates are based on models using CEM matching within PEs. 9.9.3 FULL GOVERNMENT SAMPLE ESTIMATE The annual expenditure in public procurement was estimated to US$24 billion in FY19, representing 45.2% of the annual budget and 8% of GDP.37 Based on the savings due increased winning rebate with an upper bound of 7.4%, the annual estimated savings if all procurement in Bangladesh is administered electronically could be as high as US$ 1.76 billion per annum, based on the same average higher rebate rate.38 9.9.4 POLICY RECOMMENDATIONS Based on the cost savings analysis, the following policy recommendations have been put forward:- 1. We observe that the majority of the savings was obtained from the transition to the e-procurement system was from the increased rebate rates and the initial investment in the system is recuperated multiple times over in a single year. Investing in the procurement functions, IT skills, capacity building of personnel, are tiny compared to the saving that can be obtained from it. Most of the training of procurement officials is legal in nature. Any training focusing on professional development of government officials involved in procurement would also be justified. Hiring of experienced procurement professionals can also be justified based on the savings that can be obtained. 37 Assessment of Bangladesh Public Procurement System Report 2020. (World Bank). 38 The other cost components have not been considered; the economic cost savings comprise more than 89% of the total government savings. Additionally, over 90% of tenders by number are currently administered electronically but only constitute about 65% of the total public procurement by value. As the economic cost savings is based on the value of the tender, whereas all the other cost components are primarily dependent on the number of tenders, the proportion of economic cost savings as a part of the whole will be even higher than 89%. 81 2. Investment for private sector development are also be justified based on the savings that may be obtained. We observe that tenders with higher participation saw higher rebate rates. Similarly tenders with single bidders saw substantially lower rebate rates. 9.10 VALIDATION WITH PERCEPTION INDICATORS This appendix shows how the impact of e-procurement was perceived by different stakeholders of the procurement process. We reported in the main text the impact of electronic procurement on access, economy and efficiency based on administrative data. These observations are further corroborated by the perception of change in these indicators. The following tables summarize the advantages and disadvantages of the 2 modes as reported by PE officials and bidders. 82 TABLE 51: ADVANTAGES AND DISADVANTAGES OF PAPER BASED AND ELECTRONIC PROCUREMENT AS REPORTED BY PE OFFICIALS Manual Electronic Advantage/Disadvantage for PE Advantage Disadvantage Advantage Disadvantage Bidder capacity development 13% 1% 6% 6% Cost of processing a tender 2% 20% 12% 3% Inappropriate bidding practices (e.g. 1% 34% 21% 7% collusion) No. of tenderers applying for a tender 27% 1% 6% 12% Processing different lots at the same time 9% 1% 3% 6% Quality of project implementation 10% 1% 9% 1% Technical complexity (internet connectivity, 6% 2% 3% 76% software usability) Tenderer quality and skills 17% 11% 6% 7% The stress related to processing a tender 4% 37% 6% 3% Time taken to process a tender 9% 1% 22% 3% Training needed to learn the processing 4% 11% 5% 29% method Translations for standard documents 6% 1% 4% 4% Transparency 0% 0% 58% 1% Trust or reliability of the system 4% 4% 21% 8% TABLE 52: ADVANTAGES AND DISADVANTAGES OF PAPER BASED AND ELECTRONIC PROCUREMENT AS REPORTED BY BIDDERS Manual Electronic Advantage/Disadvantage for Bidders Advantage Disadvantage Advantage Disadvantage Time taken to prepare a tender. 7% 56% 29% 5% Cost of preparing a tender 5% 22% 10% 13% Technical knowledge requirements 45% 0% 1% 25% Level of competition/ number of competing 17% 13% 3% 24% firms Inappropriate bidding practices (e.g. 23% 0% 10% 23% collusion) Translations for standard documents 3% 5% 0% 4% Labour requirements 31% 10% 2% 26% Technical complexities (internet 48% 0% 0% 60% connectivity, software glitches) Transparency in the procurement process 0% 0% 9% 0% Ease of submission of tenders 0% 6% 72% 1% Coercive practices/use of muscle power 0% 84% 64% 0% The perceptions of each indicator have been given below: 1. Efficiency Indicators 83 Procuring Entity Officials were asked about the number of days taken for completing different administrative processes as per their estimation. Their estimates/perception of these times have been compared with the actuals times and tabulated in the table below. TABLE 53: COMPARISON OF TENDER PROCESSING TIMES BASED ON ADMINISTRATIVE DATA AND ESTIMATION OF PROCURING ENTITY OFFICIALS Efficiency Model (4a): CEM within PEs Model (4b): CEM within Student t-tests Indicator across years years across PEs Treatment Treatment Treatment Manual e-proc Manual e-proc Manual e-proc vs Control vs Control vs Control Lead Time 84.1 65.0 -19.2*** 80.9 65.3 -15.6*** 64.9 50.0 -14.9*** Evaluation Period 9.0 19.2 10.2*** 9.3 21.3 12.0*** 15.3 10.5 -4.8*** Approval Period 11.0 3.5 -7.5*** 13.4 3.4 -10.0*** 19.8 14.9 -4.9*** Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 We observe that PE officials have an overwhelming positive perception of the impact of the e-procurement system. On average they reported a positive effect of the transition on evaluation periods whereas administrative data shows a negative impact. Moreover, 22% of PE officials mentioned the lower turn- around time required for processing tenders as one of the two main advantages of the e-procurement system. Similarly, 59% of bidders reported preparation times of tenders as one of the main drawbacks of the paper- based system that is alleviated by the transition to the electronic system. 2. Access Indicators The analysis using administrative data of access indicators showed a positive impact of the introduction of e-procurement. We observed a significant increase in the number of non-local winners of contracts. This finding was corroborated by bidders in the firm survey. Bidders were asked about their estimation of the percentage of tenders they participated in and won outside their district. On average, bidders reported a 16.6% increase in the proportion of tenders they participated in and a 14.8% increase in the proportion of contracts won outside their district. When bidders were asked about their how confident they were in bidding outside their district, 164 reported they had become more confident in bidding outside their districts while 58 said they had become less confident as result of the transition to the e-procurement system, with 361 bidders reporting no change in how confident they were bidding outside their district. It’s worthwhile to note 100 bidders (out of the 583) reported becoming equally confident in bidding in-and-out of their respective districts as opposed to being more confident in their own district previously. The table given below tabulates the frequency of the responses. 84 TABLE 54: OUT OF DISTRICT BIDDING CONFIDENCE Electronic Much more confident Much less confident More confident Less confident Similar Total Much less confident 7 3 2 3 4 19 Less confident 0 8 5 4 3 20 Manual Similar 4 7 71 7 16 105 More confident 0 8 47 95 11 161 Much more confident 5 6 66 21 180 278 Total 16 32 191 130 214 583 Additionally, 24% of the surveyed bidders mentioned the increased competition as one of the main disadvantages of e-procurement. This further corroborates our findings on how e-procurement achieved greater competition. 9.11 ROBUSTNESS TEST FOR WINNING REBATE REGRESSIONS : FURTHER OUTLIERS REMOVED There were a wide range of values for winning rebates, with certain contracts being signed at a fraction of the estimated cost while others at a multiple of the estimated cost. These outlier values can affect the results of the regressions depending on the threshold used to remove these outlier values. coefficients show the regression results for winning rebates, with no outliers removed, 0.1% of the tails removed (used in the main analysis) and values which were more 2 standard deviations away from the mean removed. 85 TABLE 55: EFFECT OF OUTLIERS ON THE WINNING REBATE REGRESSION COEFFICIENTS IN TABLE 4 2σ from μ 0.1 -99.9 %ile No Outliers 2σ from μ 0.1 -99.9 %ile th th th th Outlier threshold No Outliers Model Model (1a): Model (1b): Model (1c): Model (2a): Model (2b): Model (2c): CEM within CEM within CEM within CEM within CEM within CEM within PEs across years across PEs across PEs across years across PEs across years PEs years years PEs years Regression type OLS OLS OLS OLS OLS OLS Control: Manual Treatment: E- 6.411*** 7.347*** 8.729*** 5.870*** 7.948*** 9.539*** procurement (0.38) (0.45) (0.56) (0.34) (0.40) (0.48) Ln of Contract Value -0.899*** -1.253*** -1.548*** -1.178*** -1.669*** -2.207*** (0.02) (0.03) (0.03) (0.04) (0.04) (0.05) Procurement Method: 0.572*** 0.881*** 1.599*** -1.147*** -0.618*** 0.750*** OTM (LTM – Base) (0.06) (0.07) (0.08) (0.10) (0.12) (0.15) PE Fixed Effects Yes Yes Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes Yes Yes constant Yes Yes Yes Yes Yes Yes 2 R 0.324 0.331 0.303 0.314 0.336 0.378 N 82157 84634 85428 31806 33790 33939 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 9.12 EFFECT OF HIGH WINNING REBATES ON TIME AND COST OVERRUNS It was hypothesized previously in the paper that the transparency and tighter monitoring of the bidding and decision-making stages of the procurement process (as a result of the transition to electronic procurement) may displace rent-seeking to later stages, most notably in contract implementation. Additionally, it was feared that the gains made from the higher rebate rates that were observed for electronic procurement were eroded in the contract implementation phase through contract amendments (mainly in the form of cost and time overruns). If this is the case, then it might suggest that bidders were aggressively bidding for the contracts at abnormally low rates, even at loss making rates, to secure the contracts and using contract amendments to recuperate the losses. The contractors might also employ more sophisticated strategies such as compromising on the quality of the infrastructure work. This analysis has been restricted by the limited amount and quality of data available on contract modifications, especially for the electronic tenders. The analysis of the effect of high winning rebates was only done for time overruns as the data for the cost overruns was not reliable. 86 TABLE 56: EFFECT OF HIGH WINNING REBATES ON CONTRACT EXTENTIONS Dependent Variable Contract Extension Model (1a): Model (1b): Model (1c): Model (1d): CEM within CEM within CEM within CEM within Model PEs across PEs across PEs across PEs across years years years years Regression Type Logit Control: Manual -1.044*** -0.944*** -1.190*** -1.101*** Treatment: e-procurement (0.17) (0.18) (0.17) (0.18) -0.006* -0.005+ Winning Rebate (Continuous) (0.00) (0.00) Control: Manual # Winning Rebate (Continuous) Treatment: e-procurement # Winning -0.019 Rebate (Continuous) (0.01) Control: Winning Rebate – Normal (<17%) Winning Rebate – High (17-27%) 0.211 0.317 (0.17) (0.19) 0.699** 1.124*** Winning Rebate – Very High (>27%) (0.24) (0.32) Control: e-procurement # Winning Rebate – Normal (<17%) e-procurement # Winning Rebate – High -0.541 (17-27%) (0.44) e-procurement # Winning Rebate – Very -1.098* High (>27%) (0.55) 0.706*** 0.706*** 0.719*** 0.720*** Ln of Contract Value (0.02) (0.02) (0.02) (0.02) Procurement Method: OTM (LTM – 0.152** 0.159*** 0.175*** 0.174*** base) (0.05) (0.05) (0.05) (0.05) PE Fixed Effects Yes Yes Yes Yes Year Fixed Effects Yes Yes Yes Yes constant Yes Yes Yes Yes Pseudo R2 0.171 0.172 0.172 0.172 N 22270 22270 22270 22270 Note 1: + p<0.10 * p<0.05 ** p<0.01 *** p<0.001 Note 2: All Binary Logit Models in the table display logit coefficients We observe that e-procurement in general had lower contract extensions as compared to manual which corroborates the previous finding that e-procurement leads to better competition. We find weak mixed evidence suggesting that firms bidding at very competitive rates were recuperating their low prices in the contract implementation phase through contract extensions. Please note that the main channel of recuperating losses happens through cost overruns rather than time extensions with our models tracking only the latter. We find that across manual and electronic systems, winning rebates either have a significant 87 negative (continuous version) or insignificant positive (categorical version) impact on contract extensions. So, we find a weak signal that lower winning rebates are associated with less likely delays to the contract which would be consistent with better quality suppliers winning these tenders. When we look at the impact of the e-procurement system on this relationship we find no effect. That is, e-procurement does not change the impact of winning rebates on contract extensions in our sample. 88