Taking Stock of the Impact of Power Utility Reform in Developing Countries: A Literature Review

This paper reviews recent literature on the impacts of various components of power sector reform on the performance of electric utilities in developing countries. Recent literature is heavily focused on statistical testing of the significance of the links between four components of sector reform (unbundling, private sector participation, regulation, and competition) and various performance indicators (relating to utility performance, user outcomes, and broader economic development). Some studies exhibit methodological shortcomings in failing to account for interactions between reforms or degrees of reform; however, others appear to be robust. The strongest result is that the introduction of private sector participation is linked to a significant improvement in labor productivity and distribution losses. Moreover, unbundling reforms in isolation is found to have hardly any significant impact on utility performance. The introduction of an independent regulator or competition is not generally significant across studies. A notable feature of all of the studies is very limited testing of the effects of policy introduction on performance indicators, such as bill collection and the duration and frequency of outages. Poor performance on these indicators of state-owned power companies is well documented and bill collection has been identified as a major hidden cost of unreformed power sectors. The materiality of the impact of private sector participation, on the various performance indictors found to be significant, is calculated for studies that provide sufficient information to do so. The size of the impact of private sector participation on utility performance is substantial in a couple of studies, although much more modest in others.

This paper reviews recent literature on the impacts of various components of power sector reform on the performance of electric utilities in developing countries. Recent literature is heavily focused on statistical testing of the significance of the links between four components of sector reform (unbundling, private sector participation, regulation, and competition) and various performance indicators (relating to utility performance, user outcomes, and broader economic development). Some studies exhibit methodological shortcomings in failing to account for interactions between reforms or degrees of reform; however, others appear to be robust. The strongest result is that the introduction of private sector participation is linked to a significant improvement in labor productivity and distribution losses. Moreover, unbundling reforms in isolation is found to have hardly any significant impact on utility performance. The introduction of an independent regulator or competition is not generally significant across studies. A notable feature of all of the studies is very limited testing of the effects of policy introduction on performance indicators, such as bill collection and the duration and frequency of outages. Poor performance on these indicators of state-owned power companies is well documented and bill collection has been identified as a major hidden cost of unreformed power sectors. The materiality of the impact of private sector participation, on the various performance indictors found to be significant, is calculated for studies that provide sufficient information to do so. The size of the impact of private sector participation on utility performance is substantial in a couple of studies, although much more modest in others.

Introduction
The dis-satisfaction with the performance of state-owned vertically integrated electricity utilities that emerged in the 1970s from several different contexts led to a policy agenda that rapidly became highly stylized. The overall approach adopted was similar when applied to highly industrialized countries (the U.K.), to transition countries of Eastern Europe and the former Soviet Union, and to developing countries (large and small).
At the center of the reform agenda was the policy of unbundling (vertically and horizontally) the state-owned monopoly. Unbundling by itself was not expected to achieve large improvements in performance, and there are some arguments to suggest that it could worsen performance. However, an unbundled system could make further policy steps possible. Private entry could become feasible, and this would require regulation. Privatization of existing assets could also be encouraged. These steps could facilitate competition, while regulation could ensure that some of the benefits of competition accrued not only to the owners of the private capital, but ultimately also to consumers.
However, almost from the initial adoption of such an approach, there were doubts that this emphasis on private ownership, regulation, and competition would be able to improve the very poor power sector performance observed in many developing countries (Brown, 2002;Hogan, 2002;Pineau, 2002;Byrne and Mun, 2003;Wamukonya, 2003;Banks, 2006;Gratwick and Eberhard, 2008). A major review of the experience of the previous two decades ( Besant-Jones, 2006) concluded that a "one-size-fits-all" approach to power sector reform had not been successful, and that policies had to be adapted to the specifics of the country in which improved performance of the power sector was required. At the same time, there was rather limited statistical, as opposed to anecdotal, evidence on the actual impacts of policies that had been implemented, and so the actual benefits of power sector reform were uncertain.
After 2006 some changes have occurred that make it important to revisit the design of power sector policies. First, there has been an increased weight placed on the fiscal costs to governments of subsidizing state electricity companies. The very large increase of oil prices between 2005 and 2015 resulted in governments in oil-importing countries that relied on thermal generation having to shoulder the sharply increased cost of the subsidies resulting from decisions to limit power tariff increases relative to costs. In this situation, there has been renewed interest in improving the efficiency of state owned utilities as a way of reducing the total fiscal cost. A second, and related, issue has been the increased concern with improving access to electricity of lower income households in developing countries. Although increased access has been one of the objectives of power sector reform, it has more recently been of greater concern in certain countries where power sector reform has been discussed for decades but has not brought about a substantial increase in access. A third change has been the very great increase in emphasis on the environmental damage of CO2 emissions from thermal generation in both developed and developing economies. Some developing countries are starting to have large emissions and are adjusting policies to reduce their growth, in part through the adoption of renewable forms of energy generation. At the same time, the global drive for lower cost renewable technologies has provided increasingly attractive 3 alternatives even for countries with relatively low emissions. All three of these changes have implications for the design of power systems and the adjustment of reform strategies.
A final factor pointing towards the need for a re-evaluation has been the accumulation of studies estimating the benefits of sector reform and subjecting these to statistical testing. The consensus of such studies on the possible benefits of sector reform must be an important element in a possible rethink of policies towards the sector in those countries that have not yet changed their mode of operation. If it can be shown that unbundling and privatization have generally led to sector improvements in performance then the case for refining the policy approach, rather than looking for a different model, will be strengthened.
This literature review of the role of unbundling and ownership in power sector reform concentrates on material from the last decade because Besant-Jones (2006) had provided an extensive bibliography and had incorporated material from many of these sources. A search through the published literature of the period from 2006 to the present has yielded a number of studies with some link to the topics of unbundling and ownership in the power sector. A very noticeable feature of this literature is the heavy concentration on the statistical evaluation of the impacts of sector reform on the various targets it was designed to improve. This paper reviews the problems of power sector performance that led to the adoption of unbundling and private sector involvement as principal tools to help meet the goals for power sectors in developing countries. It links the list of problems to be solved to the policies put forward for their solution, discussing the mechanisms by which they were expected to work. The review of the evidence on the impact of using various reform strategies provides some guidance as to future directions for both policy design and further analysis.
The paper is organized as follows. Section 2 discusses the various failures of the power sector that reform has been designed to ameliorate. It notes how different failures existed in different types of economies. These failures are an important background to all policy discussions, but they are particularly relevant to the discussion of unbundling and private sector ownership, since these policies are essential perquisites to the implementation of all other policies, such as regulation and competition. Some studies that have attempted quantification of some of these inefficiencies are described.
Section 3 discusses the various policies that together made up the power sector reform synthesis. Links between the policies and their expected impact are discussed, because these provide the basis for statistical analysis of the impact of separate reform instruments. The section then describes the three types of indicator that have been used to assess whether the performance of the sector has improved, and whether this can be attributed to the adoption of reform policies.
Section 4 provides a detailed analysis of the various attempts to quantify the impact of unbundling and privatization on the various targets they were designed to improve. A distinction is made between the "output" of a policy and the "outcome" of that output. For example, legislating and implementing the creation of regulatory authority is an output, while holding prices below where they would have been in an unregulated market is an outcome. Success is measured by the improvement in outcomes, but if the outputs have not been created there can be no improvement

The drivers of reform
The drivers of energy sector reform have been described by many authors (Bacon, 1995;Bacon and Besant-Jones, 2001;Wamukonya, 2003;Joskow, 2008), and derive from forces internal and external to the sector (Jamasb, Nepal, Timilisina, and Toman, 2014). Different countries had very different drivers leading to their adoption of a reform strategy, but there appeared to be a common core of problems with a common approach to solving them: the response to this was the emergence of a "neo-liberal" power sector reform strategy.
The internal drivers of reform of state-owned electricity sector companies included: (a) Poor performance in terms of high costs, unreliable supply, inadequate expansion of access, and low quality of service. (b) The inability of the state sector to finance needed expenditure on maintenance and expansion. (c) The burden on the government of providing subsidies to consumers through holding tariffs below costs to avoid unpopular price increases. (d) The use of costly generation technologies.
The external drivers included: (e) Technological innovation such as development of CCGTs. (f) National economic reform programs adopted resulting from an economic crisis. (g) Desire to introduce private sector ownership in the transition economies of Eastern Europe and the former Soviet Union. (h) The desire of a government to raise capital by asset sale through privatization. (i) OECD energy deregulation leading to the creation of new energy multinationals looking for investment opportunities. 6 (j) Political and economic ideology favoring the use of market forces through competition, made possible by unbundling and privatization. (k) In economies suffering from high unemployment, governments using state enterprises as a source of employment for political reasons, even though the productivity of labor was very low.
The principal driver in developing countries was the poor performance of the state-owned enterprise. This was compounded and even encouraged by governments' tariff policy of using subsidies to soften the impact of cost increases brought about by external shocks. The increasing financial burden this created made it harder for utilities to finance maintenance, repair and expansion, leading to further deterioration in performance. Blackouts and brownouts increased in frequency and severity, and access remained almost static. At the same time the willingness of governments to allow a worsening financial performance provided no incentive for the management of the utility to cut costs and improve efficiency.
Transition economies faced an entirely different scenario. They largely had excess generation capacity, and high rates of access, but charged prices that were far below replacement costs. Over and above these sector issues was the general reform strategy of creating a market economy based on private ownership and competition. A further incentive to adopt these strategies has been provided by the desire of certain countries to join the European Union, which would require them to bring the power sector into line with EU directives.
The principal drivers in developed countries were not linked to the short-run quality of service (blackouts were few and access was universal), but rather related to consumers wanting to see lower prices and governments seeing the opportunity to improve their short-run financial position through asset sales. The advent of new technologies, particularly combined cycle gas turbines, that could lower generation costs and did not require very large units to reap the benefits of economies of scale, reduced the strength of the case for allowing a vertically integrated monopoly. Without economies of scale, plants and firms could be smaller, and this opened the possibilities for multiple players and competition, once private capital could enter the industry.

How large were the inefficiencies of state-owned enterprises?
Despite the large volume of literature describing the inadequacies of state-owned vertically integrated utilities, few studies have attempted to quantify these shortcomings. Recent emphasis on quantifying the benefits due to sector reform mainly concentrates on the actual difference in performance achieved, rather than on the possible benefits from attaining optimal performance.
A companion literature review on cost recovery (World Bank 2017a) identified several studies that had quantified the extent of cost recovery in developing countries, but many of these did not separate the magnitude of inefficiencies from that of subsidies. The International Monetary Fund and the World Bank developed a method of quantifying some losses from poor performance of the utility sectors in transition countries, following the breakup of the former Soviet Union. Ebinger (2006) showed how to split the quasi-fiscal deficit of state enterprises, in Eastern Europe and Central Asia, into components that corresponded to inefficiencies with respect to certain aspects of managing the utility.
Recent studies of the composition of the quasi-fiscal deficit (QFD) of state utilities in Sub-Saharan Africa (Briceño-Garmendia et al., 2008;Alleyne et al. 2013;Trimble et al., 2016) and the Middle East and North Africa (Camos et al., 2017) provide an important insight into the magnitude of certain inefficiencies. The QFD of state-owned public utilities can be defined (Saavalainen and ten Berge, 2006) as: [the] value of the implicit subsidy computed as the difference between the average revenue charged and collected at regulated prices and the revenue required to fully cover the operating costs of production and capital depreciation.
The QFD can be broken into components (the so called "hidden costs") to highlight the major sources of underperformance of a utility: (a) excess transmission and distribution losses; (b) overstaffing costs; (c) bill collection failure; and (d) underpricing. The first three items are defined relative to an "ideal" performance target for that country or region. Trimble et al. defined T&D losses of 10% and bill collection of 100% as achievable for a well-managed utility. The target number of staff was calculated from studies on Latin American distribution utilities. The underpricing element is then determined as a residual. Camos et al. used a similar approach. Other possible sources of inefficiency, for example, generation plant run in a sub-optimal fashion, are not shown explicitly and are therefore included in the underpricing element. Other shortcomings, such as the rate of access, are not measured by such a calculation.
Most of the countries included in these studies had not undertaken deep reforms of the electricity sector, so that the costs identified give an insight into the magnitude of certain shortcomings of unreformed power sectors. The few countries where there has been long-standing private engagement (beyond management contracts) are shown with an asterisk in the following tables. Table 1 shows the hidden costs as a percentage of GDP for Sub-Saharan countries for a reference year (between 2013 and 2015). The median QFD was around 1% of GDP-with a maximum country value of 6%. Underpricing was the most important item, accounting for about 40% of the total QFD. Bill collection was the smallest item, but it may be the easiest to rectify. However, for 13 countries the underpricing component was negative. This can be interpreted as showing that had the other costs (incomplete bill collection, excess T&D losses, and overstaffing) been removed by some policy action, the tariffs being charged would have more than covered costs.  Table 2 presents the parallel results for Middle East and North African countries as of 2013. In this region the median QFD is much larger than in Sub-Saharan Africa, but this is almost entirely due to the much greater degree of underpricing. The largest QFD of almost 9% of GDP in Lebanon indicates that problems are not confined to very poor economies that find it difficult to manage the utility effectively, or to high-income economies that can afford a large fiscal deficit. Large subsidies may exist for reasons of political economy where consumer unrest could be reflected in popular opposition to price increases. These tables also indicate that excess T&D losses, partial bill collection, and overstaffing amounted to a substantial fraction of GDP in many developing countries so that, as well as the tariff setting issue, there were substantial inefficiencies in operation.

3: Power sector reform policies and their expected impacts
In attempting to assess whether power sector reform has been effective, it is necessary to start with a view of what reform steps are entailed. Different lists have been supplied by different authors, with the simplest list corresponding to the steps identified in most econometric studies assessing the impact of reform packages adopted by different countries. These various approaches to understanding what is meant by sector reform are discussed in section 3.1.
Based on such lists it is necessary to understand the role of each step in improving the performance of a country's power sector. An important aspect of such analysis is the separation of the benefit of a step taken by itself (e.g. unbundling) with the benefit when combined with another step (e.g. privatization). If one policy must be put in place before gains in performance are to be expected from the implementation of a second policy, then the partial reform would be ineffective. Section 3.2 outlines the roles of unbundling and privatization in combining to achieve improvements in sector performance. Section 3.3 discusses the types of indicators of performance that have been utilized in the various statistical studies, noting that some indicators relate directly to user welfare (prices, quality of supply), others refer to the internal performance of the firm (T&D losses, thermal efficiency) and yet others measure benefits external to the sector (GDP, HDI). The latter two types of indicator do not indicate whether users were better-off because of the performance improvement.

The components of energy sector reform policies
The components of a power sector reform strategy have been identified in various combinations by different studies. Those which are more analytically based (Joskow, 2008;Gratwick and Eberhard, 2008) tend to be more detailed and identify more components than statistical studies. Statistical studies that use an overall index of reform measured as the sum of components (ESMAP, 1999;Erdogdu, 2010;2011b;Urpelainen, 2017) tend to use more components than studies estimating the impacts of individual reform steps (Ba and Gasmi, 2017). Table 3 indicates the reform components identified in various studies. 3.2 The expected role of the main energy sector reform policies

11
Each of the separate policy actions that make up the reform strategy for a given country is expected to have a role in achieving an overall improvement in sector performance. Some of these policies have a direct effect, but other policies act as necessary facilitators. Without their enactment certain direct policies cannot be effective. This distinction is important for statistical testing where it is desired to make an evaluation of the impact of each policy component.
The starting point for policy design is the state-owned enterprise (SOE) with its monopoly on all aspects of electricity production and supply. Privatizing the SOE can lead to improved performance as the profit motive provides an incentive to lower costs but private monopolies, in the absence of regulation, tend to use their monopoly power to maximize profits by restricting output via excess prices, and hence absorb much of the gain from efficiency improvement, resulting in little benefit to consumers even though production costs have been reduced. Regulation attempts to redistribute the gains between consumers and the producer.
Unbundling the sector, both vertically and horizontally, allows the possibility of introducing competition in the market, both between generators and between distributors. This would provide a further push to improve efficiency, since lower costs can be used to increase market share as prices are lowered relative to rivals. By this process, consumers can benefit from the change in market structure. Larger consumer gains are expected to accrue from the presence of several competitors with no dominant firms. Where there is still market power on the supply side, regulation can again be used to improve the welfare of consumers.
This simple account of the underlying arguments for sector reform including the four key components allows qualitative evaluation of the impact of each policy component singly and in combination with other policies.
(a) Unbundling on its own (U). Unbundling the SOE vertically and horizontally into separate SOEs would create smaller but state-owned companies. The potential costs and benefits from such a move have been analyzed by Pollitt (2008b) and Vagliasindi (2012). Both authors warn that there are costs as well as benefits to unbundling, and that by itself this policy cannot be counted on to improve performance of the system overall. Some authors have emphasized that there are different degrees of unbundling with different consequences for performance (Vagliasindi and Besant-Jones, 2013), 2 and there are arguments suggesting that unbundling improves performance through improved transparency, accountability, and management focus. (b) Private sector participation (PSP) on its own (P). Without unbundling, privatization could entail the creation of a private monopoly electricity company. In many developing countries the performance of the SOE is so poor that it would be hard to attract bids for the entire utility. Although there are efficient, fully integrated, private sector electricity companies, there is limited experience with this model in developing countries. 3 Privatization would be expected to increase efficiency as the firm seeks to reduce costs and increase profits (as well as to increase tariffs when these have previously been held below the profit maximizing value). In the absence of regulation, tariffs could be set too high and consumers would lose out.
The introduction of Independent Power Producers (IPPs), selling power to the vertically integrated SOE through power purchase agreements (PPA), introduces a limited form of private sector participation without unbundling the existing SOE. In this case the SOE remains unchanged and purchases power from this new form of supply. Without direct pressure on the management of the SOE the incentive to improve performance will be low. Most statistical studies do not separate the different forms of unbundling and treat the introduction of IPPs as being the same as the vertical or horizontal separation of the existing SOE.
(c) Regulation on its own (R). In the absence of other policy changes, the introduction of regulation, as used in countries where there is private sector participation and competition, would be expected to have a negligible effect on the performance of an SOE. The regulator would essentially be duplicating the control of whichever government department was responsible for the actions of the SOE. Regulation makes sense when there has been unbundling and some private sector entry into the sector. The greater the market power of the private participants in the market the greater the need for an independent regulator who can control excess profit making and ensure some efficiency gains are passed on to consumers. (d) Introduction of competition on its own (C). In the presence of a single state-owned utility, it is impossible to introduce competition in the market. Competition in the market requires unbundling and privatization, permitting the presence of several firms, each with an incentive to increase its profits at the expense of the others. Competition for the market would be possible even without unbundling--it corresponds to the privatization of the entire SOE as a single entity from some form of auction. Competition can be between generators (wholesale) and between distributors (retail).
This simple discussion makes it clear that certain policies, if applied on their own, would be expected to have little or no effect on sector performance-instead, policies need to be combined. However, most of the econometric studies reviewed below do not recognize this aspect of reform and test whether each policy component contributes separately to improved performance whether or not other policies are in place. The discussion above suggests that the policy combinations that could be expected to improve performance substantially are (U+P), (U+R+P), and (U+R+P+C). The last corresponds to the full policy reform implementation, while the former two do not attempt to create a competitive environment in the sector. Other combinations are expected to make little or no impact on sector performance, but they may act as stepping stones to the full policy implementation-hence some countries may exhibit the implementation of partial reform policies, with little effect on performance.
The survey of reform steps undertaken by developing countries (World Bank 2017b) provides insights on combinations of policies adopted that can be regarded as incomplete and unlikely to be effective unless further reforms are undertaken. The percentages of the 88 developing countries surveyed exhibiting various different ineffective reform combinations in 2015 are shown in table 4. The percentage of countries that have some private sector participation without having undertaken any unbundling is very high (34%). Quite a few have created an independent regulator in the absence of any PSP (12%) with a similar value allowing competition in the absence of an independent regulator (12%).
14 A separate issue in the analysis of the impacts of reform policies is that of degree. Many of the earlier econometric studies characterized reform steps as being either "on" or "off", implying that the benefits achievable were of a fixed size, and did not depend on the "degree" to which the reform step was implemented. Some more recent studies have noted that each of the reform steps can be characterized as being implemented to a greater or lesser extent, and hence suggest that the largest impacts on sector performance would be attained when each of the reform steps is fully implemented. This refinement is important in econometric modeling of the impacts of reform policies and is also important for the construction of an overall "reform index" used to indicate how far a country has gone towards the full implementation of this reform strategy.
 The degree of unbundling. Most studies have treated unbundling as a two-valued state, defined by whether generation is separated from transmission, thus facilitating entry (Nakayama, 2007;Erdogdu, 2011). Vagliasindi (2012) termed separating generation from transmission as "partial vertical unbundling" and added a second criterion of "full vertical unbundling" when transmission was separated from both generation and distribution. Besant-Jones (2006) described degrees of unbundling, but this has not been reflected in the econometric studies evaluating the various policy options, apart from Vagliasindi (2012).  The degree of privatization. Earlier studies also treated privatization as two-valued. A privatized sector was one where there was some private sector participation, and nonprivatized indicated entirely state-owned (Nakayama, 2007;Zhang et al., 2005;Erdogdu, 2014b). A variation on this approach defined privatization having occurred when there was at least 50% private ownership in any subsector (Balza et al., 2013). These measures of privatization imply the same gains in performance would accrue to privatization however much greater private ownership was than the threshold value (zero or 50% in the two cases). Nakayama (2007) defined privatization assets by two "on/off" variables: the introduction of independent power producers (IPP), and the privatization of existing utility assets, but did not model further increases in performance as impacted by increasing shares of private capital. Gassner et al. (2009) identified four forms of private sector participation:  Full divestiture. Transfer of 100% of assets to private hands for an indefinite period  Partial divestiture. Transfer of 51% (but less than 100%) of assets to private hands for an indefinite period  Concession. Transfer of assets and rights for a limited period 15  Lease contract. State ownership of infrastructure assets, joint ownership of operating assets, and private ownership of operating rights for a limited period; management contract is similar to lease contract except that state owns operating assets.
Each of these can be treated as a different "on/off" variable, and this allows for the various forms of private involvement to have different sized impacts of the performance outcomes. Partial divestiture can change over time and it is possible that the impact on performance would be larger the greater the degree of divestiture.
The alternative approach to treating privatization as an "on/off" driver has been to measure privatization by the amount of private sector investment that entered the sector. Once the initial entry had occurred there would likely be further increases in private ownership and, as the share of the sector controlled by private capital increased, the overall performance of the whole sector could be expected to rise. Zhang et al. (2008) measured privatization by the share of generation capacity owned by private investors, while Balza et al. (2013) measured the degree of private participation by the cumulative private investment in the sector (relative to gross capital formation at the beginning of the period). Vagliasindi (2012) distinguished the generation and distribution sectors, measuring privatization by the share of privately owned generation capacity, or the share of privately owned retail sales.
 The degree of regulation. Most econometric studies of the impacts of reform policies have treated regulation as being either "on" or "off". If there is a regulator, independent of the government, then regulation can be considered "on", while in countries with no regulator, or where the regulator is not independent of the government, regulation is "off". The definition of independence is important in this context and indicates the existence of an agency not directly under the control of a ministry (Zhang et al. 2005). This approach implies that the effectiveness of regulation is the same for all "independent" regulators and is ineffectual for all "non-independent" (or non-existent) regulators.
In an attempt to test the hypothesis that different degrees of regulatory independence could have a more varied impact, Cubbin and Stern (2006), building on earlier work by Domah, Pollitt and Stern (2002), proposed a set of four indicators (each of which took a value of zero or one, depending on its absence or presence). The sum of the four indictor scores was defined as the regulatory index (ranging between 0 and 4). The indicators selected were:  An electricity regulatory law (1) or no law (0);  An autonomous (1) or ministry (0) regulator  License fee (1) or government budget (0) regulatory funding  Free (1) or mandatory (0) civil service pay scales for regulatory staff.
The regulatory index was utilized instead of a simple dummy variable in an equation measuring the impacts of policies on sector performance. Balza, Jimenz and Mercado (2013) followed this approach, while Zhang et al. (2008) used the same approach except that the fourth component was replaced by the alternative:  There is a fixed term appointment for the head of the regulatory body (1) or no fixed term (0).
These studies differentiated between regulatory situations that were more or less independent of the government and that may therefore have been more or less effective in controlling excess pricing. Also, when the index increased over time, regulatory effectiveness performance could be expected to increase. In practice, the quality of regulation could be expected to be important but, as noted by Cubbin and Stern (2006), there are no data available to measure quality on a comparative basis.
 The degree of competition. Existence of competition has usually been linked to the introduction of wholesale competition in the generation market (Zhang et al., 2005;Nakayama, 2007;Erdogdu, 2011) and retail competition in the distribution segment (Nakayama, 2007). Some studies have noted that the degree of competition within these markets, and the likely effect on efficiency, will depend on the size distribution of the constituent firms. Zhang et al. (2008) used the square root of (100 minus the percentage market share) of the three largest generators as the index of competition-the larger the share of these three firms the lower the degree of competition. Vagliasindi (2012) used the inverse of the Hirschman-Herfindal index (HHI) as a measure of disaggregation 4 : for generation the HHI was based on the capacities of firms, and the retail sales of firms were used for the distribution sub-sector. Both approaches predict that the higher the value of the measure, the greater will be the increase in efficiency.

Indicators of sector performance
The review of problems with the traditional SOE model of the power sector indicated that there were problems facing users, such as excessive blackouts and brownouts, lack of access, and high prices, that could result in pressure for change. There were also problems for governments, such as excessive costs of supporting subsidies, and the inefficiencies of the sector, leading to higher costs of production and higher tariffs or larger subsidies.
The application of reform policies was expected to reduce these impacts, and various ways of measuring the improvements due to sector reform have been proposed. Earlier studies were very limited in the range of data on suitable variables that was publicly available and, because some studies were carried out at country level while others were at the utility level, there are substantial differences between studies in terms of the impact variables used. The indicators fall into three categories:  Type I indicators measure some aspect of internal performance of the utilities in the sector. Indicators such as transmission and distribution losses, customers per employee, generation capacity, or thermal efficiency can show the extent to which the utility has improved its overall efficiency and lowered its costs relative to where they would have been in the absence of reform. The reduction in costs does not guarantee that consumers are better-off, and therefore this group of indicators cannot be used by themselves to support the argument that sector reform will improve the general welfare of the population. The higher level of profits, made possible by cost reductions, may primarily benefit the new investors in the sector, whose entry has been made possible by unbundling and privatization, together with no or weak regulation.  Type II indicators measure some aspect of performance of utilities felt by users. The number of connections, the average duration of interruptions, or the tariffs charged have direct consequences for users, and so can be used to support the argument that consumers have benefited from sector reform. The improvement in the performance of these indicators is often due to the improvement in the type I indicators.  Type III indicators measure some aspect of the economy external to the sector. GDP per capita, GDP per unit of energy used, or the Human Development Index (HDI) can be used as evidence that the reformed sector is creating benefits to the whole economy. Because variables of this type are impacted by many factors, not just sector reform, it is particularly important to separate out the contributions of reform from the other factors involved in their determination during the period in which reform was active.
Studies of the impact of sector reform usually focus on the impact on a few indicators, the choice of which is often constrained by data availability.
In summary, this review of the components of sector reform and their anticipated impacts points to three important conclusions: 1. Reform processes include many distinct policies. There has been no testing of the impacts of a reform program carrying out all the steps regarded as defining the complete policy. Econometric studies have always tended to concentrate on a small number of steps.

Section 4: Quantifying the impact of sector reform
Three approaches to the measurement of the benefits of sector reform are found in the literature (Jamasb, Mota, Newbery and Pollitt, 2005). Case studies of individual country or utility experience (Galal, Jones, Tandon and Vogelsang, 1994;Newbery and Pollitt, 1997) base calculations about the size and distribution of costs and benefits of reform on assumptions made about the working of that economy, rather than on statistical estimation based on a sample of experiences. Efficiency and productivity analysis, such as Data Envelopment Analysis (DEA), is well suited for the case where there several firms within the same economy, and the focus of reform is the impacts on efficiency measures of performance. Econometric analysis is attractive because it can include the effect of non-reform factors, as they change over time, or differ between countries.
In the literature search undertaken for this study attention was paid primarily to studies on developing countries. An earlier survey (Jamasb, Mota, Newbery and Pollitt, 2005) had identified several empirical studies on the impacts of sector reform, but few were on developing countries. The last decade has seen a sharp increase in empirical work on the impacts of sector reform in developing countries, with the largest part of this concentrated on econometric studies.
Section 4.1 outlines the hypotheses about the impacts of sector reform linked to unbundling, private sector participation, regulation, and competition that are tested, and issues involved in doing so. Section 4.2 describes the results obtained from the regression and other studies surveyed for each of the hypotheses outlined. Studies are first evaluated by the significance (or not) of the various reform policies on various performance indicators, and then for the studies where private sector participation was found to be significant, the magnitude (materiality) of the effect is calculated.

Hypotheses about the impact of sector reform on sector performance
The hypotheses concerning the impacts of sector reform policies are usually stated in their positive form: e.g. the efficiency of the sector improves after privatization. It should be noted that in formal statistical hypothesis testing the "null" hypothesis (e.g. sector efficiency is unaffected by privatization) is assumed to be correct and is rejected in favor of the alternative hypothesis (sector efficiency is affected by privatization) only if the evidence is sufficiently strong. In this survey the hypotheses will be stated in their positive form to be consistent with the reform literature.
The data used to test these hypotheses come in one of three forms. A time series of data on a single utility (or country) covers the period before and after the implementation of the reform policy. Changes in the performance indicator are linked to the level of the policy variable as well as other variables expected to impact performance. The policy variable may be assumed to have the same impact in each year or may change if the intensity of the policy changes. In a simple cross-section model there are data on a given year for all the countries in the sample. Those countries where the policy is on are expected to have a higher and common value of the performance indicator once allowance has been made for differences among countries on the other determining variables. Panel analysis combines time series and cross section data, and in the simplest case the impact of the policy on the performance indictor (for the years after which it is was implemented in the various countries) is assumed to be the same in all countries and all periods. In regression models the fixed impact of a policy when switched "on" is modeled by a "dummy" variable (taking a value of unity in "policy on" periods or countries, and zero when the policy is off). The coefficient of this dummy variable measures the impact on the performance variable of the policy.
It is not expected that every policy variable would lead to an improvement in all performance variables. For example, regulation itself may focus largely on prices or interruptions of service, so that it would be expected to have only an indirect effect on performance measures such as T&D losses. The hypotheses specifically relating measures of sector performance to unbundling and privatization policies are of the following generic forms, in which the details of the performance variables are not specified:

Sector performance improves when the sector is unbundled.
Unbundling is an essential step if other sector reform steps are to be taken. 5 There are various hypothesis about the effect of unbundling by itself. Some authors have pointed to costs due to the loss of economies of scale and scope, while others have pointed to the benefits of increased transparency, and management accountability and focus. This hypothesis is usually tested by the inclusion of a "dummy" variable in regressions testing for the influence of policy on sector performance. If the coefficient (δ1) on the dummy is not significantly different from zero, then unbundling by itself is estimated to have no impact on the performance measure. This formulation assumes that the benefits from the unbundling policy are the same in all years and all countries for which the policy is "on".
However, the decision to reform is taken before the actual implementation of the reform, and indeed the reform policies may deliberately be implemented in sequence. The anticipation of reform may lead the components targeted for unbundling and private sector entry to start to improve performance before the policy is implemented (Andrés, Foster and Guasch, 2006;Gassner, Popov and Pushak, 2009). Furthermore, unbundling may take place before privatization (Zhang, Parker and Kirkpatrick, 2005) so that improvements in sector performance due to announced privatization may occur before the actual private sector entry at a time when unbundling is occurring. The existence of a policy anticipation effect may provide an explanation of why unbundling on its own sometimes has a significant correlation with performance.
This approach can be extended to the case where the hypothesis relates the improvement in performance to the degree of unbundling. Vagliasindi (2012) introduced separate dummy variables for partial vertical unbundling (generation separated from transmission plus distribution) and full vertical unbundling (transmission also separated from distribution). These steps usually happen in sequence so that the impact of unbundling is expected to increase over time. As with the single measure of unbundling it can be expected that vertical unbundling alone (whether partial or full) would not lead to a substantial improvement in performance.
2. Sector performance improves when there is some private involvement. The simplest approach to testing this hypothesis has been to define privatization as the presence of some private interest in the sector (whether by management contract, concession, partial, or full divestiture). A dummy variable for privatization is added to the regression model, and the coefficient (δ2) on this dummy variable measures the impact of privatization in the absence of unbundling. To capture the additional benefits of privatizing an unbundled system, an interaction dummy variable (taking a value of 1 if both policies are in action and zero if not) with coefficient δ3 should be included (Erdogdu, 2011a). The impact of both unbundling and privatizing is equal to the sum of these coefficients (δ1+δ2+ δ3), while privatizing an unbundled system has impact (δ2+ δ3). 6 The coefficient δ3 measures the incremental benefit of the policies acting together relative to their separate contributions. The coefficient δ1 measures the contribution of unbundling alone.
If the data set covers a number of utilities it is likely that different forms of privatization will be utilized across the data set. Gassner et al. (2009) created separate dummy variables for the use of each of the four types of private sector involvement they analyzed so that comparisons could be made between the size of impacts of the different forms of private intervention.
The extension of this hypothesis to the case where the impact on performance depends on the degree of privatization is achieved by replacing the simple two-value dummy by a variable that takes value zero until the starting date of privatization, and then takes a value measuring the degree of privatization. Zhang et al. (2008) used the share of generation capacity that is privately owned, while Vagliansindi (2012) for distribution used shares in the retail sales (MWh). If it is expected that no extent of privatization would be significant unless there had also been unbundling, then an interaction term is required (dummy for unbundling times the measure of degree of privatization) as well as the unbundling dummy and the degree of privatization variable.
3. Sector performance may start to improve before the actual privatization reform policy is implemented and may not reach its full effect for some years after that date. The "transition effect" hypothesis introduces the possibility that the effect of the introduction of a reform policy can start before the formal initiation of the policy but not reach full impact until some years after the policy introduction date. This is modeled through the use of two dummy variables. The first takes a value of unity from a date before the policy implementation (perhaps the announcement date) until a date after policy implementation (one or two years typically), while the second dummy takes a value of unity from the time of the finish of the first effect. Under the null hypothesis of no policy impact, the coefficients on both dummies would be zero. Under the hypothesis of no transition effect, the coefficients of the two dummy variables would be equal (Andrés et al. 2006). Transition effects can also be combined with models in which the degree of privatization is used as the driving variable.
4. The impact of regulation is to improve certain aspects of sector performance in the presence of unbundling and privatization. A number of studies test the hypothesis that the presence of an independent regulator improves performance and have done so by the introduction of a dummy variable related to the existence or not of an independent regulator. Because independent regulation is likely to be found only where there is privatization, but not in all cases of privatization, it should be treated as a separate effect and a dummy variable included for privatization, as well as one for unbundling. It is possible that the regulatory authority is created before the introduction of private sector involvement, and it might have an impact on the performance of non-privatized firms, although this is likely to be small. As with other models an interaction term between regulation and other policies can be included. Zhang et al. (2008) included interaction terms between regulation and privatization, between regulation and competition and between privatization and competition, as well as simple dummy variables for all three policies. Unbundling was not included in this study, and this would be a specification error if there were some unbundling effect on observations that were not privatized, regulated, or competitive. Nagayama (2007; included interaction variables between regulation and five other reform variables (privatization, IPP introduction, unbundling, retail competition, and wholesale competition) as well as dummies for each of these effects. 7 The omission of an interaction term between unbundling and privatization may represent a specification error in that there may have been observations where there was some private sector activity without unbundling. Sen et al. (2016) included an interaction dummy between regulation and the privatization of distribution, as well as other interaction dummies (corporatization and unbundling, IPPs and open access). Zhang et al. (2008) and Balza et al. (2013) included variables that measured the degree of independence of the regulator.
The generalization of interaction terms introduced by Zhang et al. (2008) points to a difficulty in models where there are 3 (or more) policy variables. A more complete consideration of the interaction terms might conclude that the largest impacts of performance would be obtained when all three policies were implemented and was larger than the sum of the benefits from all three policies singly. This review has not identified any study in which the magnitude of the benefits of three policies carried out together is estimated. This problem is further discussed below in the context of introducing competition.

Increased competition improves performance in the presence of unbundling and privatization.
The entry of competition in the electricity sector can be represented by dummy variables for wholesale competition, and for retail competition. Cubbin and Stern (2006), in their analysis of the generation sector, used the date of the legal right of independent power producer sales (a simple dummy variable), while Zhang et al. (2008) used an index based on the market share of the three largest generators, as well as interaction terms between the competition index and regulation, and between the competition index and privatization. For the distribution sector, Vagliasindi (2012) used the inverse of the HHI for retail sales, without any interaction terms.
No study appears to have estimated the total benefit for performance when all four policies were introduced, allowing for interactions between policies. Because, as was argued in section 3.2, all four policies are likely to have small benefits when used on their own (with the possible exception of privatization) and large benefits from sector reform may arise only when 3 or 4 of the policies are in force, models not allowing for this possibility may underestimate the importance of reform. Leaving out these higher-order interaction terms, where most of the benefit is expected to accrue, is likely to lead to bias in the estimation of the terms that are included.
An important special case of the multi-policy model is where the policy applications follow a "recursive" pattern. Suppose that all privatized observations are unbundled (but not necessarily the reverse); all regulated observations are privatized (but not the reverse); and all competitive observations are regulated (but not the reverse). In this case, as discussed for the combination of privatization and unbundling, there are no observations to allow the separate influence of competition from all other policies, of regulation for other policies, and privatization from other policies. The four combination dummy variables (C+R+P+U, R+P+U, P+U, U) are identical to the four individual dummies (C, R, P, U). In this case the model could be estimated using just the four dummy variables for the individual policies, but the coefficients have to be interpreted as the effects of policy combinations. In this case, attempts to add interaction terms to individual effect terms would result in perfect multicollinearity and a failure of estimation. The fact that studies testing for the policy dummies and interaction effects have not led to perfect multicollinearity indicates that the policies do not follow a strict recursive pattern.
6, The overall extent of sector reform (including unbundling and privatization) impacts sector performance. A few studies have combined the various policies adopted into a single indicator and correlated this with the performance variables. The greater the extent of sector reform the greater the increase in performance expected. Erdogdu (2010;2011b) constructed an index based on eight steps. Each step taken was given a value of one, and those steps not taken a value of zero, so that the reform score index could vary between zero and eight. Various performance indicators were regressed on the reform score and other variables. Erdogdu (2014b) used a similar approach by constructing an "openness" index for the power sector that reflected entry barriers, public ownership, and vertical integration. Performance indicators were regressed on this index and other variables. Nepal and Jamasb (2012) included a power reform index, as well as other reform indices, in an analysis of the impact on performance in transition countries. Urpelainen (2017) also included the number of reforms undertaken as determinant of the performance variables. The average number of reforms taken in neighboring countries was also included as a way of modeling a demonstration effect.

23
The background paper (World Bank 2017b) calculated a reform index over the period 1995 to 2015 for 88 developing countries and 22 OECD countries. The index was based on four subindices (regulation, restructuring, competition, and private sector participation), each given a weight of 25%, whose values were determined by the categorizations shown in table 5. Regulation stayed with an "on/off" approach; restructuring generalized the approach of Vagliasindi (2012) who distinguished full and partial privatization; competition distinguished five categories; while private sector participation was a continuous variable depending on shares of private sector participation. The conclusions to that study found that the rapid diffusion of policies in the decade 1995-2005, with the aggregate reform index for developing countries growing from 12 points to 35 points, was followed by a much slower diffusion in the next decade (35 points to 43 points). The slowdown was especially notable with respect to regulation, and privatization of generation and distribution. By the end of the period, IPPs had been introduced in 96% of developing countries, 72% had a regulatory agency, 44% had undertaken vertical unbundling, but only 7% had a wholesale power market and 12% still were operating with a vertically integrated state-owned monopoly. It is also important to note that even among the OECD countries there was not full uptake of these policies-one-third of these countries had a reform score below 70 in 2015.
These figures do not support an unqualified acceptance of the reform model. Rather they point to important differences between countries in terms of their willingness to adopt certain reform steps. World Bank (2017b) shows how the degree of uptake varies with a number of country characteristics, not just income level. Progress on adopting reform was slowest in Sub-Saharan Africa where the performance as measured by QFDs is known to be weak. These findings point to the need for deeper analysis of why these countries were slow to adopt reformincluding the possibility that the reform model was completely unsuitable and as a result they either failed to adopt it at all, or else they just undertook a small segment of the whole range of policies and then made little further adjustment.

24
The use of aggregate reform indices imposes two restrictions on the model, as compared to models that allow for separate effects for the various policies utilized. First, all policies used are assumed to have equal effect on the performance variable and second, the effectiveness of a policy does not depend on which other policies have been undertaken. Both of these are strong assumptions, perhaps justifiable only when data are lacking for the construction of individual policy variables.

The impact of sector reform on performance
This review of the literature identified 26 studies that used a formal model with which to estimate the impacts of sector reform on some aspect of firm or sector performance in developing countries. These studies are listed in table 6, giving details of: authors, countries involved in estimation and the time period covered; the policy variables tested; the impacted variables measured; other determining variables; the estimation method used; and the principal results. The findings from these studies are grouped into two: first are those based on regression models (mainly panel data covering many countries and several time periods). This group allows comparisons between studies of the effects of certain reform policies on the same impact (performance) variables. The analysis of these studies begins with the significance tests for the impact of the various reform policies on performance of the utilities. Next, focusing on those cases where a significant result was obtained, the magnitude of the impact (materiality) is calculated. The second group of studies are less homogeneous in approach but present particular cases of substantial relevance to the issue of understanding the impact of the sector reform policies.

Results from regression-based models
This group of 16 studies used very similar approaches to estimation and testing, allowing a comparison of results across a number of studies based on common assumptions and method.
Although there are overlaps in country and time period coverage, and in variables utilized, there are enough differences between studies to argue that finding similar results across a number of such studies strengthens the overall conclusions drawn.
In this group of studies, the impact (performance) variables were linked to variables measuring whether policies were "on" or "off" and in some cases to the degree to which policies were "on".
In addition, other main determining (control) variables were added in order to allow for shifts in the performance variables not due to policy changes. Coefficients were estimated by variants of least squares.
All of these studies used a panel framework-that is each country was represented by observations for a number of years, and it was assumed that the impact of adopting a reform policy had the same marginal effect on performance irrespective of the year in which the reform was adopted or the country in which it was adopted. Differences in the overall levels of the performance indicators were accounted for by the level of the control variables. In addition, country specific dummy variables were included. Within a country, the policy on variable represents the change in performance when the policy was adopted, while between countries the variable measures the difference in performance between countries with or without the policy. The inclusion of the control variables allows for the possibility that performance of an SOE improved over time, even though no reform step had been adopted.
The problem of endogeneity was raised by some authors who varied their estimation technique to allow for it. As pointed out by Bacon and Besant-Jones (2001), some countries appeared more willing to adopt sector reform than others, and these countries had some common featuresnotably a higher level of GDP per capita. Extending this to allow for other determinants of the number and extent of reforms adopted, it is possible that the current levels of certain performance variables encouraged some governments to adopt reform, which itself might then impact performance. For example, the existence of a badly-run distribution sector within the SOE may have acted as an inducement to unbundle and privatize in order to improve performance via the pressures of profit maximization and competition. If this were so, the adoption of policy would not be exogenous with respect to performance, and estimation methods should be used that take account of the simultaneity between them. For example, Sen et al. (2016), Cubbin and Stern (2006), Sen and Jamasb (2012), and Urpelainen and Yang (2017) all used instrumental variables estimation or three-stage least squares in order to avoid endogeneity bias. Urpelainen (2017) used two-stage least squares estimation and compared the results to ordinary least squares. For transmission and distribution losses the benefits of undertaking a reform step were underestimated by four-fifths using ordinary least squares, while for generation capacity the ordinary least squares estimator was not significant, while the IV estimator was. These results confirm the importance of allowing for endogeneity in estimating the impacts of sector reform. Table 7 gives the results of significance tests of the effect of policy variables on the impact variables selected by that study. Where the coefficient is of the expected sign and significant this is indicated by the symbol (Y: ↑) for a positive impact, or (Y: ↓) for a negative impact). The symbol N indicates that the coefficient was insignificant (this includes being the incorrect sign). The policy variables (U=unbundling, P=private sector involvement, R=regulation, C=competition) are entered as dummy (on/off) variables, except when the degree of policy is used, indicated by the superscript (º) attached to the significant coefficients. Interaction terms (defined as the product of two of the dummy variables) are denoted UP (unbundling and private sector involvement) etc.  Notes: Y: ↑ indicates coefficient of dummy variable has expected positive sign and is significant at 5% level. Y: ↑ º indicates that policy intensity variable has expected positive sign and is significant at 5% level. Y: ↓ indicates coefficient of dummy variable has expected negative sign and is significant at 5% level. Y: ↓ º indicates that policy intensity variable has expected negative sign and is significant at 5% level. N indicates that policy variable was not expected sign or not significant at 5% level U indicates unbundling P indicates private sector participation R indicates regulation C indicates competition T indicates transition effect UP indicates interaction effect between U and P; etc.

Significance
Type I indicator measuring performance internal to utility Type II indicator measuring performance as felt by user Type III indicator measuring performance external to sector The first group of hypotheses (type I) are concerned with measures of the internal efficiency of operations of the firm. As a result of private sector involvement and competition, once unbundling has taken place, efficiency measures are expected to improve.
Connections per worker increased significantly with respect to private sector involvement, while unbundling was not included as a separate policy. This matches expectations as private ownership is expected to seek ways to reduce costs and reducing the labor relative to the number of connections would reduce costs. Provided that all observations that had private involvement had also been unbundled, the effect measured is the joint effect of unbundling and the private sector. However, if some privatized observations had been unbundled while others had not, then the specification is incorrect, and the effect of the privatization dummy is overestimated because it would include some effect due solely to unbundling (unless there is zero effect to unbundling alone).
Energy sold per worker increased significantly with privatization in three studies, while being insignificant in a fourth. Two of these studies also tested for an effect due to unbundling, while one of these also tested for effects due to regulation and competition. Neither study found a separate effect for unbundling, but regulation increased efficiency. This supports the view that unbundling on its own makes no change in efficiency, but when combined with private sector involvement the labor force was reduced relative to energy sales.
Distribution or T&D losses showed a significant decrease in four studies when there was private sector involvement. No other variable was significant in the other five studies. Two of these included all four policy components and two included only interaction terms. The reasons for the differences between the different studies are worth further investigation because this is known to be a large component of inefficiency in many of the countries covered by the QFD analysis referred to in section 2.2.
Energy generation per worker is expected to improve with private sector involvement and competition, once unbundling has been undertaken. The role of regulation is less clear because it is unlikely to be focused on this aspect of firm behavior. Private sector involvement alone was insignificant in the three studies, but it was significant in increasing this indicator when regulation and privatization were combined.
Plant load factor was significantly increased when there was unbundling, but showed no link to private sector involvement, regulation or competition when these were taken separately. 8 Unbundling and competition together were significantly linked to a higher plant factor. These results together support the argument that relying only on the dummy variables for the four policies without interaction terms may miss important benefits of reform.
The reserve margin was significantly related to interaction between unbundling and competition, but no other study included this performance indicator, and the lack of a private sector involvement variable in the specification cast some doubt on the interpretation of the result.
Bill collection improved significantly with private sector involvement and showed a transition effect. The omission of the other policy variables limited the value of this result, which was based on a single study. There was a surprising lack of other studies focusing on this indicator, given its evident importance in QFD calculations in many developing countries.
Generation capacity is expected to increase after the reform of the sector. The results tend to support this hypothesis. Two studies found that private sector involvement and regulation (allowing for different degrees of policy intervention) significantly were linked to a significant increase in capacity. These studies did not allow for any interaction effect. Two studies had significant interaction effects (UP and UR in one, and PC and RP in the other) and in these studies the simple dummy variables were insignificant. Unfortunately, no study tested for the impact of all four policies using interaction effects. Also, the studies using interaction effects did not allow for the degree of sector reform in the specification of the policy variables. These studies suggest that generation capacity did respond to reform policies but that there was no measurable impact of any of the policies on their own.
The number of employees is an ambiguous performance measure. As demand increases then the sector expands and more employees are needed, but the entry of the profit motive will provide an incentive to reduce the labor force where it had been inefficiently utilized. The two studies measuring this variable found that employment fell once there was private sector involvement. No other variables were tested, but there was a significant transition effect.
This group of results supports the hypothesis that reform can improve the efficiency of the sector, although none of these results could by itself indicate that consumers had benefited as a result. Connections per worker and energy sold per worker both increased once there was private sector involvement, suggesting that the response of such firms had been to reduce the labor force (connections could not be decreased and it is unlikely that sales would fall in response to the policy switch). The result for electricity generation per worker is particularly interesting. Both of the dummy variables for regulation and competition had a significant positive effect, and the interaction term between them also showed a significant effect. It is argued above that this approach is superior to those not specifically allowing for the interaction effects. Two of the indicators, T&D losses and bill collection, have been identified as contributing large shares of the 39 QFDs of various countries. Four studies identified private sector involvement as leading to a significant reduction in losses, and one study found a significant positive relation to bill collection.
One surprising finding is that unbundling on its own was found to be significant in only one case (plant load factor), but only five out of 22 tests included this variable. Four tests included the interaction between unbundling and competition (three were significant) but none of these tests included private sector involvement in any form. This also points to a lack of coherence in the treatment of facilitating variables in these regression models. Several tests included regulation as a policy variable, although it is not normally concerned with the interior working of the firm. Out of 12 tests only three showed a significant effect of regulation on these performance indicators.
A second group of tests (type II) looked at performance indicators that directly link the firm to its customers, and, if policies were identified as successful, consumers would have benefited from their adoption.
Access as measured by the number of connections per household or in total was subjected to some of the most sophisticated modeling. Only one of the five studies tested for the effect of unbundling on its own, but all tested for the effect of private sector involvement. In four cases private sector involvement was significantly linked to an increase in access, and three studies also indicated that regulation contributed significantly to access. Two studies checked for the effects of competition and one of these was also significant, as were transition variables. Rather than using simple dummy variables, three of the studies also used measures of degree for private sector involvement, for the independence of regulation, and for competition. Overall, these results pointed to sector reform being successful in increasing access, particularly when a strong regulator existed and there was a substantial private sector involvement with ample competition.
The quality of service as measured by the frequency and the duration of interruptions is expected to improve after the entry of the private sector and these affects were found to be significant in the one study which included these variables. No other policy factors or the degree of private sector involvement were included.
Some studies also linked reform policies to prices charged, but the role of subsidies was not evaluated at the same time. A reforming government might well have decided to also reduce subsidies, hoping that efficiency gains from reform would help offset the subsidy reduction effect on prices. Without an analysis of the subsidy policy, correlating tariffs with policy variables cannot be relied on to give an unbiased estimate of these policy interventions. For this reason, tests of the relation between prices and policy variables are not reported.
The tests on performance variables that directly impact users of electricity provided some strong evidence that reforms produced significant improvements in performance. Access responded positively to private sector involvement, regulation, and competition; generation capacity 40 responded positively to private sector involvement and regulation; while the quality of service improved with private sector involvement.
The final group of tests (type III) are those in which reform was correlated with economywide objectives. Improvements in sector performance could facilitate improvements in GDP and the HDI, but links would be indirect.
Emissions of CO2 per kWh have not been a key part of sector reform but some studies tested whether reform led to decreased emissions. One study found that reduced emissions were not linked to unbundling or private sector involvement but were linked to regulation and competition.
GDP per capita and the HDI were expected to increase as a result of sector reform but only two studies tested for this effect. One study found that private sector/competition interaction was significant, but the other found that none of the four policy dummy variables was significant. A similar negative result was found with respect to the HDI.
The study by Gassner et al. (2009) disaggregated private sector involvement into sub-categories by type of private sector contract. Nine performance indicators were each regressed on four dummy variables for type of private sector involvement (full divestiture, partial divestiture, concession, lease or management contract), a utility specific fixed effect, and a time trend. No other reform variables were included, nor any non-policy variables. Because this is the only study that provides this level of disaggregation for private sector participation, it is valuable to compare the results for the separate forms of contract with the results obtained with no disaggregation by contract type. The study also tested for the impact of any type of private sector participation. Table  8 reports the significant coefficients.  Gassner et al. (2009) Lease or managements contracts were not associated with performance improvement of any of the indicators, suggesting that the short-term nature of this private sector involvement did not make it worthwhile to spend time or money on improving the performance of the utility. Concessions provide a limited incentive to improve the billing rate because this is likely to be the simplest to achieve. The significant increase in residential connections associated only with concessions reflects the fact that concession contracts often included this as an explicit target. The increasing private sector participation implied by partial and then full divestiture is reflected in the wider range of performance indicators that are significant. None of the forms of private sector involvement is associated with an improvement in supply disruptions, and only full divestiture is associated with increased sales per connection. The improvement of these indicators is likely to require substantial investment and time before a response becomes visible, and this may help to explain why significant results were not obtained. Vagliasindi (2012), and Vagliasindi and Besant-Jones (2013) tested a further hypothesis concerning the effects of reform (in particular vertical and horizontal unbundling): the impacts of reforms are expected to be the most significant for countries with large size power systems and high income per capita and least effective for countries with low system size and income per capita. To test whether the impact of the reform variables varies by system size and by income, it was necessary to create an interaction variable between the privatization dummy (say) and a dummy for those countries that have low system size and income per capita. For access, although all four policy dummy variables improved performance, the interaction terms with size were always significantly negative supporting the hypothesis that reform is least effective in such countries.

Materiality
For a reform policy to be assessed as having made a contribution to improving the performance of the sector, studies have largely focused on statistical significance of the relevant parameter estimates. Where there was concurrence between studies, as indicated above, on the statistical significance of the impact of a particular policy, the case for using that policy is strengthened.
However, for a policy to be useful it must also have a sufficiently large impact on performance to offset the costs of implementing that policy. Hence, information on the size of the impact is needed in order to move to an assessment of the desirability of introducing that policy. The regression coefficients that are reported (together with standard errors) for the purposes of establishing significance can also be used to indicate the magnitude of the impact.
In the present review the largest number of significant results was for the effects of private sector participation. Unbundling, regulation, and competition were tested in fewer studies and for fewer performance indicators, and generally were insignificant (apart from the positive impact of regulation on access). As a result, it was decided to concentrate on the calculation of the impacts of the PSP variable from those studies for which it was significant.
The choice of performance indicators is also important. Some indicators, such as energy sold or energy produced, react to changes in external forces independently of reform. SOEs may have been inefficient, but they were capable of meeting some increases in demand. Post-reform an increase in energy production cannot then be solely attributed to the impact of reform, since some of the increase would have been met even in the absence of reform. Andrés et al. (2006) model this effect but many studies ignore it and tend to claim all the change observed as due to the impact of reform. To avoid this problem the magnitude of policy impacts is not calculated for indicators that are likely to have included a growth or scale effect, while performance indicators, such as the percentage of transmission and distribution losses, that are unlikely to experience trend effects are included.
For the calculations on the effect of PSP on the selected indicators, most studies used a semi-log or double-log functional form, in which the log of the indicator was regressed on the policy dummy and on other variables. If the coefficient on the dummy variable is δ, then the percentage impact of the policy introduction (from 0 to 1) is equal to (exp [δ] −1). To convert a % change into an absolute number measuring the impact of policy it is necessary to multiply it by the mean value of the indicator before the policy was applied. The subset of studies that included data on the mean of the performance indicators did so only for the whole period (i.e. both pre-and post-reform) and some studies did not present any data on the mean. Studies that were carried out using a linear formulation yield the predicted change in performance, and this can be converted into a percentage change if the mean is available.  The materiality of the results can be judged by comparing the overall mean values with the increase (or decrease) implied by the % change figures. For example, connections per employee rose by 56% (relative to pre-reform values) in the transition period, and by 125% relative to the pre-reform value in the post-reform period. Given that the mean connections per employee over the whole period were 559, the post-privatization value increased by 700. In fact, since the pre-reform mean will have been lower than the post-reform mean (because of this increase in performance) and hence lower than the overall mean, the value of 700 is an overestimate. Nevertheless, an increase of 125% of the pre-reform connections per employee is a very substantial change in performance. The transition value, although a much lower percentage figure, is also substantial.
Energy sales per employee (MWh) show a similar picture, and it appears that employment must have been sharply reduced to achieve these results.
Coverage shows a 14% post-reform increase. Based on the overall mean of 75 connections per 100 households, the post-reform value would have been around 85-but given that the 43 pre-reform mean would have been lower, the increment would have been fewer than 10 households per 100, but still substantial.
The value for distribution losses has to be interpreted with care. The estimate of a 19% reduction in losses has to be compared to the overall average value of 15.3% losses. This reduction is equivalent to 2.9 percentage points (taking the total loss to 12.4%), but the prereform mean would be larger than the overall mean, so that the actual reduction would be larger and the post-reform value lower. In this case the transition value showed very little change (down to 15%), but the total reduction is clearly substantial. It should be noted that QFD studies suggested 10% as a target value for efficient firms.
Both the duration of interruptions and the frequency of interruptions fell to about half of the pre-reform values. With a mean duration of around 20 hours per year, the reform resulted in a reduction of around 10 hours a year. This could be valued by using estimates for the cost of shortages (Foster and Steinbuks, 2009) but even without doing so it is clear that privatization leads to a considerable and valuable increase in the quality of supply.
(ii) Gassner et al. (2008) followed a similar approach to the above. Private sector participation was related to performance indicators for distribution utilities using a semi-log function.
The results for significant indicators are shown in table 10. Mean values refer to the whole period rather than the pre-reform period. The results for connections per employee and sales per employee indicate substantial improvements in performance, although not as large as those found by Andrés et al. The reduction in distribution losses is also smaller (equivalent to a fall of 2 percentage points from the mean of 17%). The improvement in bill collection is large, and the overall mean of 78% clearly overstates the pre-reform value. The post-reform value of bill collection is likely to be above 90%.
(iii) Vagliasindi (2012) used a linear model and found significant links between private sector participation and (i) energy sold per worker; and (ii) access.
For the distribution sector privatization was measured by retail sales of the private sector relative to total retail sales. The regression of energy sold per worker on the privatization share gave a value of 9.6, with an overall mean value of energy sold per worker of approximately 25 MWh. A one percentage point increase in the share of private sales is then estimated to increase sales per employee by 0.1 MWh.

44
Access was related to a dummy for privatization in the generation sub-sector, with a coefficient of 3.46, while the overall access rate was of the order of 60%, yielding an increase in access of around 6%. This is lower than the value obtained by Andrés et al. but still is substantial.
(iv) Balsa et al. (2013) used a double-log model. The privatization dummy was "on" when private ownership reached 50%. The only significant coefficient was that for T&D losses with a value of −0.0077. Privatization (as defined in the study) reduced losses by 0.77% which, with a mean loss for the whole period of about 20%, is equivalent to a reduction of 0.15 percentage points in T&D losses. This value is much smaller than that found in other studies and can scarcely be regarded as material.
(v) Zhang et al. (2008) used a semi-log form and included some interaction dummies that involved private sector participation. In the equation for generation per employee (G) the privatization dummy was not significant (but was not zero) while the interaction variable between private sector participation and regulation was significant with a value of 0.001. Private sector participation was measured by the share of privately owned capacity in total (G) capacity, and regulation was measured on a four-point scale. The authors stated that "When the proportion of generation capacity owned by private investors increases one percent[age point] there will be a 0.1% increase in electricity generation per employee." The final equation included both a variable for privatization (P) and a variable for the interaction between privatization and regulation (R) -the equation can be written as: ln (G) = α + δ1 [P*R] + δ2*P For the whole sample the mean R is 0.97, and the mean P is 14.6. The coefficient δ2 was not significant, but rather than re-estimate the equation while omitting the dummy for P, the study keeps the value of δ1 and sets δ2 equal to zero. The equation for the impact of PSP when regulation had been introduced is: Δ ln (G) = δ1 [Δ P*R] = 0.001* [1*0.97] ; ⸫ g = 0.001 Hence the 1 percentage point increase in the share of private participation leads to a 0.1% increase in energy generation per employee. The study does not indicate the units in which generation was measured, so it is not possible to make direct comparisons with other studies.
(vi) Nakayama (2010) used a linear formulation with interaction dummy variables. For T&D losses dummies for unbundling, regulation, private sector participation and competition were all insignificant. The interaction dummy for the entry of IPPs and regulation was significant with a coefficient of −1.43. The equation was not re-estimated omitting the insignificant variables, so that it is concluded that T&D losses fell by 1.4 percentage points when both PSP and regulation were introduced. The mean value of losses was not reported so the importance of this fall cannot be directly compared with other studies.

45
In summary, two of the studies described above -Andres et al. (2006) and Gassner et al. (2008) find quite a substantial impact of private sector participation on the performance of the affected utilities in terms of labor productivity and distribution losses. The other studies that could be analyzed found significant impacts that were nonetheless very small in absolute magnitude.

Geographical coverage of regression studies
An important aspect of the regression studies is their degree of geographical representativeness. The country-based studies, excluding those that were deliberately limited to one region (LAC by Balza et al. 2013;ECA by Erdogdu, 2014b; and South and East Asia combined by Sen et al. 2016), show a fairly even spread in the selection of countries by region, especially when the total number of countries in each region is allowed for-LAC is highly represented in most studies, and Sub-Saharan Africa tends to be under-represented relative to the total number of countries. However, the majority of studies have a reasonable representation of each region, and all studies are implicitly given equal weight in assessing the links between reform and performance, so that it does not appear that the conclusions reached from an overall assessment of these studies are heavily biased by the results from experience of one particular region. This line of argument could be explored using a meta-analysis of the cross-country-based studies.
Several studies pointed out that the number of countries used in their model was constrained by limited data availability on the performance variables. Given that these studies used a panel basis formulation, combining data from several years for each country for each of variables of interest, it was important that each country had data for a minimum number of years. The study by Urpelainen (2017) provided details of countries where indicator data were available for T&D losses, and for generation capacity. For the former, data were available from 96 developing countries, while for generation capacity data were available from 139 developing countries-this confirms that data availability does vary strongly among indicators, and the large number of countries included suggests that some other studies may have not fully explored data sources for the indicators considered.
Fewer studies could be utilized in carrying out materiality calculations, and the overall picture is much more dominated by experience from the LAC region.
The time periods for the data series used in these studies were provided in  Take-aways from regression models 1. The main finding is that private sector participation was often associated with improvements in sector performance. This was most marked for labor productivity and operational efficiency. (Andrés et al., 2006;Ba and Gasmi, 2011;Gassner et al., 2009;Vagliasindi, 2012). It is striking that very few studies analyzed bill collection rates and quality of service despite the well documented and understood magnitude of these dimensions of underperformance. 2. The effects of regulation and competition were more ambiguous. Fewer studies tested for their effects and, with the exception of the impact of regulation on access (Ba andGasmi, 2011, Balza et al., 2013;Vagliasindi, 2012), the majority of studies found no significant beneficial impact of these policies on performance. 3. The interaction between policies has largely been ignored or incompletely treated. Many of the studies used simple (0/1) dummy variables for the four key reform steps, and included no interaction dummies, or not all the interaction dummies corresponding to the set of policy dummies. As explained earlier, this specification error can lead to bias in the estimation of the effects of the various reform steps. 4. The degree of reform is important. Those studies that have introduced a degree of reform variable, rather than an on/off measure, have obtained significant results (Erdogdu, 2011b;Nagayama, 2010;Zhang et al., 2005). Taken across the set of studies examined, all four policies can be expressed in a way that allows for increasing impacts. 5. There is considerable agreement among the various studies that performance indicators internal to the firms are impacted by private sector participation, but not by regulation.
The lack of impact of regulation is not surprising given that regulation tends to focus on performance criteria that impact users rather than on those internal to the firm. 6. The impact of competition is not often significant. Fewer studies test for this variable and most failed to identify a significant impact on any of the performance indicators. The majority of these studies treated competition as an on/off variable rather than one which can vary 48 considerably. One study that measured competition by a concentration index found this to be significant (Vagliasindi, 2012). 7. Bill collection and interruptions are known to be important areas of underperformance but were tested in only two studies. Privatization was associated with significant improvement in these indicators (Andrés et al., 2006;Gassner et al., 2009) but unfortunately other policy variables were not included. 8. The type of private sector participation is important. Leases and management contracts produce no improvement in performance, and concessions appear to improve only those indicators that are specified in the concession contract (Gassner et al., 2009). 9. Transition effects are important. Several studies found significant improvements in performance before the actual implementation of a policy, and in many cases the effects increased further at a later date (Andrés et al., 2006;Gassner et al., 2009). 10. Endogeneity may be important. A few studies explicitly allowed for the endogeneity of reform (i.e. that the decision to take a reform step is influenced by the existing performance of a utility) by using alternative estimation methods utilizing instrumental variables. One study (Urpelainen 2017) found that the use of ordinary least squares substantially underestimated the impact of reform on two performance indicators. 11. The limited analysis that has so far been carried out on differential reform effects by type of country supports the view that the gains will be least in countries with small system size and low income. This finding has important policy implications in terms of where reform efforts should be concentrated. 12. There was substantial agreement between studies that the impact of private sector participation on T&D losses was material as well as being statistically significant (Andrés et al., 2006;Gassner et al., 2009). 13. The few studies that analyzed the impacts of private sector participation on interruptions to supply and bill collection indicated that the improvements in performance were material. 14. The lack of information on pre-reform and post-reform mean values of the performance indicators limited the accuracy of calculations needed to determine the materiality of significant results.

Results from other studies
A number of other studies have made estimates of the impact of sector reform using a variety of approaches. There are too few of each of these to provide general conclusions, as opposed to casespecific results.
Some have tackled the changing performance of utilities post-reform using an explicit production or cost function. Differences among utilities in the same country are used to define an efficient frontier, and differences of actual utilities from this frontier measure firm performance both preand post-reform.
Social cost-benefit analysis, following the original approach by Galal et al. (1990), calculates gains and losses to various groups as a result of the introduction of reform. A key step in such analysis is the construction of the counterfactual: what would have been the outcomes if reform had not happened. Newbery and Pollitt (1997) provide an interesting example of the construction of a counterfactual for Great Britain. In this approach the total impact of all policies has to be considered because it is not simple to allocate the different gains and losses among the policy changes. Pombo and Taborda (2006) analyzed the performance of distribution utilities in Colombia before and after the 1994 regulatory reform, using a Data Envelopment Analysis (DEA) based approach. The reform introduced competition and an independent regulator. The performance analysis showed an overall recovery in profitability, partly due to gains in labor and capital productivity, and partly due to an increase in tariffs. The detailed analysis at a utility level showed that those who had been inefficient pre-reform actually became less efficient post-reform, while the initially efficient distributors stayed on the efficiency frontier and increased performance as the frontier moved. The study concluded that private ownership did not make a significant difference to performance, but that regulation did. The heterogeneity of response to policy changes across utilities in the same country is not explained, but its presence provides a warning that this might be true for other countries. Galán and Pollitt (2014) revisited the issue of inefficiency persistence and heterogeneity among the Colombian electricity utilities. Using a stochastic frontier model and allowing for inefficiency persistence due to fixed costs of adjustment, and for heterogeneity and dynamic adjustment to changes in policy and exogenous factors, they found that small firms (mainly in rural areas and serving smaller customers) face lower adjustment costs and have thus caught up with urban firms and firms serving large users. If this finding were replicated elsewhere it could have implications for incentive regulation and tariff design.
Pérez-Reyes and Tovar (2009) used a DEA model to trace the improvements in efficiency of Peruvian distribution companies in the post-privatization period (1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006). They found that efficiency and productivity increased most in the immediate post-reform period. Generally, the reformed firms, some of which were returned to public ownership later, exhibited the greatest gain in efficiency. The authors attribute an important part of state companies' poorer performance to the fact that the managers were not allowed any autonomy. Estache and Rossi (2004) investigated a sample of distribution companies in Latin America using a stochastic frontier model. Labor requirements were related to a number of firm -specific variables (sales, customers, service area), a dummy variable for private ownership, and dummy variables for the type of regulation (rate of return, price cap, hybrid regimes). Private ownership resulted in significantly lower labor being used, and hybrid regime or price cap regulation also resulted in lower labor use. Rate of return regulation did not significantly affect labor requirements.
Saar (2015) estimated the effect at the country level of the presence of an independent regulatory authority (IRA) on three performance indicators (GDP per unit of energy use, T&D losses, and electricity generation per capita) using propensity score matching as a way of comparing similar firms with and without an IRA (24 had an IRA and 34 did not). Losses were significantly lower, and generation per capita was significantly higher when there was an IRA, while GDP per capita was not significantly different. One of the regression studies also found a significant impact of regulation on generation per capita, while none found that regulation had a significant impact on T&D losses. Malik et al. (2015) used an unusually rich data source to test for the impact on efficiency of unbundling generation units in India. Most studies of the impacts of reform use country-or utilitylevel data, but this study used data at the plant level. It is expected that the thermal efficiency and operating reliability should increase with unbundling because the plant managers have more discretion. The purchase of new equipment and improvement of the pre-treatment of coal could both impact performance positively. In fact, the regressions showed no significant impact on thermal efficiency, but firms that unbundled prior to the 2003 reform showed an improvement in operating reliability. However, a check on capacity utilization showed that this did not improve. This result is in contrast to that of Sen and Jamasb (2012) who found a significant increase in capacity utilization based on statewide data. Further exploration of the data by disaggregating plants into four size groups indicated that the main driving factor was a reduction in forced outages at small (<100 MW and 110-120 MW) plants. Anaya (2010) carried out a social cost-benefit analysis (SCBA) for Peru, following the privatization of the two largest distribution companies. Revenues were compared with costs, and the counterfactual cost fall was based on trends in other public companies. The calculations depend on the discount rate, since there are a stream of costs and benefits to evaluate. At a central discount rate, the government benefitted by $268 million and producers by $335 million, while consumers lost by $390 million, with a net overall benefit to the country of $214 million. The increase in tariffs post-privatization largely explained the loss to consumers. Because privatization resulted in an increase in access above the counterfactual rate, further welfare gains from this were calculated amounting to $328 million, implying a total welfare gain of $541 million. Toba (2007) carried out an SCBA for the Philippines following the introduction of the private sector into generation during the power crisis of 1990-1993. The welfare change was expressed as the sum of changes in benefits to consumers (consumer surplus and avoided cost), the change in private investors' profit, changes to government via income and tax, and the change in externalities. It amounted to $10.4 billion, of which consumers benefited by $10.9 billion, investors by $ 2.8 billion, the government had a net loss of $1.5 billion, and there were external costs of $1.7 billion. Weinmann and Bunn (2004) explored the reasons for delay between general economic reform and the adoption of power sector reform in Latin America. They hypothesized: (i) power sector reform would be delayed if hydropower capacity were strongly concentrated and represented a significant proportion of total generation; (ii) power sector reform will be delayed or not undertaken if the country has high energy autarky; and (iii) reform will be delayed or not undertaken if the sector is small. Delays between general reform and power sector reform were related to these three factors using an event model based on a Weibull distribution appropriate to rare events. All three factors were significant and have relevance for attempts to introduce sector reform in countries that have not yet adopted it. The size and concentration of the hydro sector limits the scope for competition and unbundling; energy autarky (energy exports greater than energy imports) is likely to be accompanied by large consumer subsidies which would be difficult politically to remove as part of the reform process; and small systems are likely to be oligopolistic in nature, and the skilled labor and management required to run several utilities as well as a regulatory authority will be in short supply (Bacon, 1996;Nepal and Jamasb, 2012a).

Take-aways from other studies
15. Unbundling when carried out in the absence of other policies had little or no effect on the performance of generators. The most likely reason for this, apart from the lack of the spur of the profit motive and competition, is that managers of the unbundled companies were not given discretion to make decisions. This result supports the earlier contention that unbundling is purely a facilitating policy, and this should be taken into account when measuring the impacts of sector reform. 16. The detailed structure relating inputs to outputs in DEA and SFA is able to calculate the overall response to reform, but also to identify differences in firm performance.
Patterns of response were noted in Colombia (Pombo and Taborda, 2006;Galán and Pollitt, 2014) and Peru (Pérez-Reyes and Tovar, 2009) and the magnitude of adjustment costs was highlighted as a brake on reform. Small firms (mainly rural) faced lower adjustment costs and were able to catch up productivity to the levels of larger firms. 17. Social cost-benefit analysis measures the benefits or costs, and their allocation, of the implementation of the whole reform package adopted in a country. SCBA is not used to evaluate the contributions of each component of the reform policy. The allocation of benefits among users, firms, and government provides valuable links to the political economy of the reform process in a country. 18. Studies related to the determinants of the output of the reform process may provide valuable insights as to when reform is more likely to be adopted and to its potential impact. Vagliasindi and Besant-Jones (2013) identified country types where the actual gains from reform were smaller (low income, small power system), while Weismann and Bunn (2004) showed that power sector reform tended to be delayed in countries where there was heavy dependence on a highly concentrated hydro generation, energy autarky, and a small power system. Both approaches provide information on where reform should be attempted and where it is likely either to be resisted by the government, or else will bring few gains.
Section 5: A Gap in the literature: Alternatives to "neo-liberal" sector reform Beyond the limitations of the empirical approach to the evaluation of the "neo-liberal" reform agenda is the lack of discussion of alternatives to that approach. Most of the critics of the "neoliberal" approach identified weaknesses in the implementation or outcomes of that policy, but few laid out an alternative route, or provided evidence of the efficacy of such an approach. The study by Gómez-Ibáñez (2007) made a particularly important point relevant to alternative policy approaches. He noted out that, prior to the 1980s adoption of the "neo-liberal" approach to SOE performance improvement, there had been two decades of attempts to improve performance while maintaining public ownership. The failure of these earlier attempts explains in part why there was high initial enthusiasm for the "neo-liberal" approach.
Surprisingly, in the light of the mixed results identified under the 'neo-liberal" approach, there have been few attempts to revisit the earlier policies to see whether they contain valuable lessons for those cases where reform has not been very successful, and there is a dearth of statistical testing that could support the use of certain policies to improve SOE performance.
Gómez-Ibáñez (2007) focused on three areas of policy designed to improve SOE performance that had been widely discussed and used, noting that SOEs often had non-commercial objectives and could be given instructions on how to fulfill these using public service obligations:  Building the internal capacity of the utility through injection of physical capital (plant and equipment), and human capital through training and assistance.  Strengthening managerial incentives to deal with perceived failures of the utility, through the use of performance contracts, and corporatization.  Changing the political-economy of the SOE by strengthening the involvement of those directly affected by the SOE-customers, taxpayers, private investors.
The review of experience with such policies suggested that they had achieved only modest success and concluded that when well-designed they might be useful when privatization could not be introduced, or had proved of limited help. This conclusion appeared to be based more on hope than concrete experience. Irwin and Yamamoto (2004) surveyed the options for governments to improve the performance of SOEs short of privatization, focusing particularly on improving corporate governance, illustrating their arguments by reference to Mexico, New Zealand, the Philippines, and South Africa. They started from the observation that SOEs are often used by politicians to transfer resources to politically influential groups. They note that there are things outside the domain of corporate governance that the government can do to try to improve performance. It can regulate, improve the business environment (enforcing contracts, improving employment law, simplifying tax administration) and strengthen political governance (accountability of government to citizens).
Within the domain of corporate governance, as experience by the private sector, there are a number of policies that may be applicable also to the public sector to some degree:  Make the utility subject to standard private sector company law  Legislate new public-sector governance rules to limit ability of politicians to influence its decisions  Require additional public reporting of performance and policies  Instill a commercial culture  Subject the utility to new pressures from lenders  List a minority of shares  Alleviate the government's conflict of interest as owner and policy maker.
Some relevant evidence was cited, and it was noted that although many of these suggestions had been discussed for decades, they had rarely been acted upon. Vagliasindi (2008a) focused on corporatization and ownership organization and holding structures as tools that could be used to improve the performance of SOEs. The structures considered (drawn from OECD experience) included:  The decentralized or sector model in which ownership is spread among several ministries, of which one may play a coordinating role.  The dual model, in which both sector ministries and a "common" ministry (typically the Ministry of Finance) are owners.  The centralized model where SOEs are put under the responsibility of one ministry or agency.
Some general principles for evaluating these alternatives are derived from theoretical considerations. For example, if policy makers are mainly concerned with allocative efficiency, the model minimizing discretionary costs is the centralized form. Experience with these different structures in a few developing countries was discussed.
As tools to improve performance, the use of organizational and individual performance contracts, and the use of increased transparency are discussed in relation to experience in countries that have used them. Three key conclusions were reached:  Governments should consider increasing the time and effort in monitoring the performance of SOEs.  Given the lack of experience with various structures of ownership in developing countries, it may be best to remain with the centralized model.  General corporate governance guidelines and policy recommendations need to be adapted to the infrastructure sectors, particularly (as with electricity) where there are natural monopoly segments, which have different legal and regulatory frameworks. Vagliasindi (2008b) investigated Boards of Directors of SOEs and ways in which they might be made more effective in improving the performance of the SOE, primarily through an advisory and monitoring role. The study reviews procedural requirements, the composition of the Board, director independence, the selection and evaluation of Board members, and the training of board members. The theoretical literature on Boards of Directors was reviewed and predictions from theory discussed. A review of empirical testing of these propositions is also included, but the majority of this literature is focused on experience in developed countries. The study makes a number of policy recommendations, including:  There should be a sufficient number of independent directors, even though it may prove difficult to empower them to exercise effective monitoring of management.  The sequencing of corporate governance reforms is important-Board formal procedures should be in place before the introduction of independent directors.  More attention should be paid to the process of Board selection and evaluation in order to help insulate the Board from government interference.  The role and responsibility of the Boards of Directors should be clarified.
Again, the limited material on the developing country experience was noted, as well as a lack of testing the various hypotheses using developing country data.
In developed countries some attention has recently been paid to the question of governance of the electricity sector. For example, Holburn (2011) provides a list of policy recommendations to achieve best practice in the governance of regulatory agencies and of government owned utilities in the Canadian electricity sector. This provides a useful check-list, but they would need to be modified and prioritized before they could be applied in a developing country.
A related topic is the benchmarking of utilities. This can provide a valuable tool in assessing the performance of a particular utility and highlighting areas for potential improvement. There is a large literature on benchmarking, with the series of six reports by the Council of European Energy Regulators (2016) providing extensive material on performance by European Utilities, while a report by Tallapragada et al. (2009) focused on monitoring performance in Sub-Saharan Africa. Some lessons on improving performance from experience from SOEs and utilities with private sector participation is given by World Bank (2016) and Antmann (2009). This group of reviews relating to the performance of SOEs and how this might be improved covers a number of topics that could be relevant to those developing countries where reform has not been effective. These studies highlight different aspects of SOE behavior, indicating that in total a large number of policies could be utilized. All the authors note the lack of empirical material on the impacts of such policy changes for developing countries, so that many of the suggestions cannot be evaluated against experience. In particular, the difficulty of keeping the government from intervening in decision making in order to support some political agenda has led to a degree of skepticism on the possibility of permanently improving SOE performance in countries with weak political institutions.
Recommendations for further analysis of policies to improve SOE performance are: 19. Establish a more comprehensive list of aspects of SOE behavior that could be impacted by policies or laws or regulations. 56 20. Identify information on how policies towards improving the performance of SOEs have been applied and the results of this in developing countries.

6: Conclusion
This literature review relating to reform of the power sector in developing countries found that much of the more recent material from the last decade concentrated on statistical analysis of the impacts of reform, where it had been carried out. However, all of these studies concentrated on a subset of the reform steps that have been identified as necessary to obtain the maximum benefits.
These key steps are: unbundling, private sector participation, regulation, and competition. This limited focus was partly because of lack of data, but also because so few developing countries have emulated the more sweeping approach taken by some developed countries.
The strongest evidence that reform can improve performance came from the private sector participation policy. More econometric studies identified this as significant with respect to a greater range of performance indicators, than for regulation or the introduction of competition, but this may be due to the much lower number of countries adopting competition.
The bulk of the statistical studies used regression models to evaluate the impact the key policies on utility and sector performance. In the specification of these models three shortcomings were frequently observed: (i) most studies assumed that a reform step was either "on" or "off" and failed to take into account the extent/quality to which a reform might be implemented; (ii) some studies formulated the model as if any of the key policies might be implemented on its own, failing to recognize, for example, that privatization would be effective only if unbundling had also taken place; and (iii) most studies ignored the possibility that the reform step taken was endogenous (the decision was partly influenced by the performance of the utility). All these shortcomings could lead to mis-estimation of the impacts of reform, and when present reduce the credibility of results obtained.
The overwhelming emphasis in these statistical studies was on the significance (or not) of the various reform steps, and the main finding derived from a comparison of 16 studies was that private sector participation was in many cases associated with an improvement in the performance of the sector, as measured by a number of performance indicators. Unfortunately, the selection of indicators in these studies largely omitted such well-known areas of inefficiency as bill collection and outages. A single study made a systematic distinction between various forms of private sector participation and concluded that leases or management contracts showed no significant performance gains, but that partial or full divestiture produced gains for a number of indicators. The impacts of regulation and competition were not so clear cut. Fewer studies tested for these policies, and those that did so did not show a clear pattern, with the exception that improved access was linked to the presence of an independent regulator.
The studies identified paid little attention to materiality (the actual size of the impact on performance when statistically significant) and many did not present all the information that would facilitate comparisons between studies. There was sufficient information to conclude that transmission and distribution losses had been reduced, bill collection had been increased, and the duration and frequency of interruptions had been reduced by amounts that would have been material.
Overall, the spread of countries across regions was reasonable, although Latin America and the Caribbean was heavily represented, and Sub-Saharan Africa was less well represented in the countries selected partly because few countries in the latter had undertaken much reform at the time of analysis, and partly because of lack of information. The time periods analyzed in this group of 16 studies included only three cases where the terminal date of the information used was post 2010. Even though many of the reform steps taken occurred early on in the process, there is a need to use more up-to-date information, especially for the slow starters.
None of the regression-based studies analyzed in detail the pattern and reasons for failures-which countries were poorly predicted by the estimated models, whether the limited number of countries taking certain reform steps (especially competition) was too small to establish significant relationships, whether the specification of the model was appropriate, and whether the estimation technique was suitable. These findings suggest that there is room for more focused and detailed statistical studies that would include more recent observations and cover the issues mentioned above.
Finally, some studies drew attention to the possibility that there may be other approaches to improving the performance of a state-owned utility without engaging a neo-liberal reform agenda. However, no studies were identified that measured the impact of such alternative policies in practice.
This review of the literature post 2006 on the impacts of reform revealed a number of gaps where new studies might prove valuable. These included: 1. Analysis of the way that the use of new technologies (generation, transmission, and distribution) could be encouraged through a sector reform process, and whether this could be expected to perform better than the status quo of reliance on a state-owned enterprise framework. 2. More extensive econometric testing of the impacts of the various aspects of sector reform on well documented key performance variables, such as bill collection and outages, allowing for possible endogeneity, and ensuring wherever possible the use of the most recent data and a wide selection of countries. The materiality of successful policies should also be highlighted so that as idea of the total benefits available from reform might be obtained. 3. Studies that analyze the simultaneous impact of several reform steps, allowing both for the interaction between the absence or presence of complementary policies and for the "degree" to which a policy had been implemented. 4. Follow-up studies that seek to understand why policies that have been generally successful, as indicated by regression results, have not been successful in particular countries-an analysis of failure could be a valuable complement to existing studies of success.