Citizen Engagement in Rulemaking -- Evidence on Regulatory Practices in 185 Countries

This paper presents a new database of indicators measuring the extent to which rulemaking processes are transparent and participatory across 185 countries. The data look at how citizen engagement happens in practice, including when and how governments open the policy-making process to public input. The data also capture the use of ex ante assessments to determine the possible cost of compliance with a proposed new regulation, the likely administrative burden of enforcing the regulation, and its potential environmental and social impacts. The data show that citizens have more opportunities to participate directly in the rulemaking process in developed economies than in developing ones. Differences are also apparent among regions: rulemaking processes are significantly less transparent and inclusive in Sub-Saharan Africa, the Middle East and North Africa, and South Asia on average than in Organisation for Economic Co-operation and Development high-income countries, Europe and Central Asia, and East Asia and the Pacific. In addition, ex ante impact assessments are much more common among higher-income economies than among lower-income ones. And greater citizen engagement in rulemaking is associated with higher-quality regulation, stronger democratic regimes, and less corrupt institutions.


I. Introduction
Transparency and accountability in government actions are increasingly recognized as central to economic development and political stability (Gisselquist 2012;Fosu, Bates and Hoeffler 2006;Bates and others 2004). Where citizens know the rules that govern their society and have a role in shaping them, they are more likely to comply with those rules. Corruption is lower, and the quality of regulation higher. In addition, citizen access to the government rulemaking process is central to the creation of a business environment in which investors make long-range plans and investments (Diergarten and Krieger 2015;Lindstedt and Naurin 2010;Aidt 2009;Shim and Eom 2008).
Governments have opened their policy-making processes to greater public scrutiny and input over the past few decades. In the early 2000s the governments of such countries as Australia, the United Kingdom and the United States began posting the text of proposed regulations online for citizens to read and comment on, as well as contacting specific stakeholders to discuss their areas of concern. Mexico's government passed a law in 2002 requiring federal ministries and agencies to make all draft regulations publicly available on their websites, and Poland's government launched an online consultation platform in 2013. Online publication and consultation for new regulations have been shown to boost trust in government and to increase compliance by firms and individuals by improving the likelihood that those affected will know about the new rules and by achieving greater buy-in on their scope and application (Vallbé and Casellas 2014;Torriti 2007;Radaelli 2003;Fadairo, Williams and Maggio 2015;Molster and others 2013).
Impact assessments have increasingly formed part of the general public scrutiny of proposed regulations. These assessments open the empirical analysis underlying a proposed regulation to stakeholders, allowing them to review and comment on the assumptions used or the analysis conducted. In this way an impact assessment and public consultation can reinforce each other in improving the quality and effectiveness of regulations (Morrall 2001). In practice, however, studies show that few governments systematically incorporate impact assessments into their consultations but the quality of analysis and consultation varies substantially (OECD 2011(OECD , 2015Kirkpatrick and Parker 2004;Backlund 2009;Torriti 2007;Staronová 2010).
Building on the existing literature and with the aim of extending the analysis to developing countries, we sought to chart the extent to which-and how-citizens, civic organizations and business associations around the world can engage with governments on the content and scope of new regulations. To that end we collected data in 185 countries focusing on three areas. The first area was government transparency around proposed regulations, including whether governments communicate with the public about proposed regulations and, if so, how they do it. Communication could be through a unified website where all (or substantially all) new regulations are posted before their final adoption, websites specific to particular ministries or publicly available registries or journals.
The second area we studied was government practices of consultation on proposed regulations. Here we investigated whether public notices of rulemaking are accompanied by consultation of stakeholders, whether through online solicitation of comments, public meetings or targeted outreach to particular groups. Finally, we explored the extent to which governments assess the possible impact of proposed regulations within their country before finalizing them. These impacts could be economic, social or environmental. We also considered whether that assessment forms part of the consultation process.
Central to this exercise was an effort to capture how rulemaking happens in practice in the different jurisdictions, not just what is required "by laws on the books." For example, while the data collection exercise looked at whether a country's laws or regulations require that the general public be engaged in the design of new regulations, we report what happens in practice. Functionally, we seek to identify where governments offer citizens opportunities to voice their concerns about proposed regulations and where they attempt to measure the possible impact of proposed regulations before their adoption.
This effort was global in scale. In developing the new data set and related analysis, our goal was to help policy makers identify how their government's regulatory practices compare with those of others in the areas of transparency, consultation and impact assessment. Varying practices for citizen engagement and impact assessment can be found in countries at all levels of development. We also lay the foundation for a more comprehensive assessment of the relationship between citizen engagement and different economic outcomes by including both developed and developing countries in our sample.
In summary, this paper presents a new global data set on citizen engagement in rulemaking and provides detailed descriptive statistics for the indicators. The paper then provides preliminary analysis on how the level of citizen engagement correlates with other social and economic outcomes. To support this analysis, we developed a composite citizen engagement in rulemaking score around the publication of proposed regulations, consultation on their content and the use of regulatory impact assessments.
We compare the composite score with indicators from other data sets for two reasons. First, we want to see whether the citizen engagement in rulemaking score aligns with indicators that capture similar concepts. The scores for different countries would be expected to follow the same direction as their scores on good governance and transparency, for example. Second, we explore two hypotheses: 1. In countries that perform well on measures of regulatory quality, the government is more likely to involve the public in developing new regulations. We expect that where civil society, the private sector and other stakeholders have the opportunity to help shape the regulations that affect their day-to-day lives and business operations, the resulting regulations better meet their stated goal and have fewer unintended consequences.
2. In countries with a democratic form of governance, the government is more likely to follow inclusive practices in rulemaking. The essence of a democratic system lies in inclusive and representative governance in which everyone's voice counts. We therefore expect the manner in which national leaders develop the rules and regulations for society to reflect the practices followed in selecting those leaders.
While we conduct initial investigations of these hypotheses, our main purpose in this paper is to present the new data and we do not claim any causal relationships for outcomes.
The rest of the paper is structured as follows: Section II reviews relevant literature on participatory rulemaking and social and economic outcomes. Section III presents the questionnaire design and data collection methodology. Section IV offers descriptive statistics with a primary focus on patterns across regions and income groups. Section V discusses the scoring methodology, and Section VI our main analysis. Section VII concludes.

II. Literature review
A growing body of literature shows a positive association between greater citizen engagement in rulemaking and advanced economic prosperity (Vallbé and Casellas 2014;Torriti 2007;Radaelli 2003;OECD 2009;Denhardt and others 2009). In most economically prosperous countries around the world, the government systematically consults citizens on the scope, content and intended impact of proposed regulations before these regulations are adopted and implemented (Diergarten and Krieger 2015;Gurin 2014).
Policies that encourage citizen participation have also proved to be beneficial for less economically prosperous nations (Adams and Atsu 2005;Denhardt and others 2009;Fishkin 2008). A study examining the effect of regulation on income inequality in 26 Sub-Saharan African countries from 1970 to 2005 shows that government transparency, investment and quality regulation are positively and significantly correlated with economic growth (Adams and Atsu 2005). The study emphasizes that lack of transparency in rulemaking undermines the use of public funds and state resources, jeopardizing the entire system of good governance. The authors advocate for stronger participatory democratic governance and greater transparency, especially in relation to public finance and the budget, as well as for opening laws and procedures to public discussion and scrutiny.
Similarly, a cross-country analysis of developing countries suggests a strong causal link between regulatory quality and economic performance (Jalilian, Kirkpatrick and Parker 2006). This study shows that efficient governance policies and strong, transparent regulatory institutions lead to 5 higher economic growth. These findings are consistent with earlier research showing strong links between productivity growth, income levels and quality of governance (Gisselquist 2012;Fosu, Bates and Hoeffler 2006;Bates and others 2004). Another study, focusing on economic and social development in Asia, echoes these findings (Sen 2014). This study suggests that government systems with high state administrative capacity and state legitimacy-including those with public involvement in rulemaking processes, strong rule of law and programs to combat corruption-lead to strong gains in social development, such as reductions in income poverty and inequality. It also finds that improvements in government effectiveness and regulatory quality are associated with lower headcount poverty.
Efforts to increase transparency and public engagement in rulemaking often have the goal of reducing corruption (United Nations 2012). One study finds that corruption has a negative impact on growth and sustainable development even in economies with already advanced levels of good governance and robust rulemaking institutions (Aidt 2009). And poor governance and weak rulemaking institutions can lead to the misallocation and mismanagement of rich resources, hampering both political and economic development (Bates and others 2004). Corruption in public administration leads to ineffective enactment of public responsibilities, loss of citizen trust, weaker compliance with the rule of law and greater loss of honest human capital (United Nations 2012). Corrupt governments also easily lose the trust of investors (Parker 2002). Moreover, information asymmetry between the regulators and the regulated in countries with little transparency or consultation can lead to "socially sub-optimal outcomes" from new regulations (Jalilian, Kirkpatrick and Parker 2006).
Among relatively recent approaches to combating corruption is the participatory budget movement, now implemented in many cities around the world. This approach allows constituencies to have a say in how policy makers use public funds, either directly or through elected budget representatives. It has been shown to foster greater government transparency and increased levels of civic involvement in rulemaking. This and other types of open data initiatives have managed to attract investors by boosting their confidence in the local rule of law and government accountability (Gurin 2014).
Yet the simple act of making information publicly available may not prevent corruption or enhance accountability. What matters for a government's openness is not how much it has integrated modern communications technology into its day-to-day operations but how responsive it is to citizens (Yu and Robinson 2012). Research shows that achieving the benefits of transparency requires that citizens have the capacity and opportunity to act on the information made publicly available, not just to see it (Lindstedt and Naurin 2010;Schantz 2013). But how much information is made publicly available? According to the 2015 Open Budget Survey, the governments of 78 countries, accounting for about 68 percent of the world's population, provided insufficient budget information to their constituencies (Open Budget Survey 2015).
Other research links inclusive, community-oriented approaches to governance with sustainable and well-functioning democracies. Studies have found the integration of citizens, government and civil society to be part of the formation and maintenance of stable democracies (Denhardt and others 2009;Fishkin 2008). One way to achieve this integration is through modern electronic communications channels-such as online or mobile communications-which can help revolutionize the way citizens engage with rulemakers and participate in important decision making processes. The use of online platforms allows civil society and individuals to challenge government decisions in a way that is relatively inexpensive and requires no extra travel time. It also permits the public to enforce, to at least some extent, government accountability for policy priorities and democratic rule (Farazmand 2012). In addition, online platforms help generate feedback from citizens who would normally not participate in government activities, an outcome that can further improve trust in government (Fadairo, Williams and Maggio 2015).
The pursuit of greater transparency and inclusiveness is not without countervailing challenges, however. Scholars point out that governments may not be inclined to adopt economic measures that promise to be beneficial in the long run but fail to get overwhelming popular support in the short run. This may be especially true in fragile states (Fosu, Bates and Hoeffler 2006). Researchers argue for a balance between "pleasing the masses" in the short run and implementing sound economic policies. Accommodating the interests of powerful minority groups could also pose a challenge to rulemaking, especially when the opinions of these groups diverge from those of larger ones and impede economic reform (Fosu, Bates and Hoeffler 2006).
Interest in tracking and understanding practices related to citizen engagement in rulemaking has grown over time. Numerous studies and data collection efforts on inclusiveness in regulatory decision making have been carried out in the developed world, primarily by the Organisation for Economic Co-operation and Development (OECD). But data on this topic for developing countries have been lacking. Moreover, it is unclear whether adapting the regulatory processes of OECD countries to developing ones is appropriate. This paper offers a comprehensive data set covering multiple forms and stages of citizen engagement globally, attempting to bridge the data gaps for developing countries.

III. Questionnaire design and data collection
To collect data, we developed a questionnaire covering practices of transparency, consultation and impact assessment in the creation of new regulations. These areas were identified by academics and regulatory governance experts as central to good regulatory practices and lacking in global comparative data. Particular care was taken to create questions that would apply to the wide range of countries in the sample and that would allow for cross-economy comparability of the data. Care was also taken to ensure that the questions would be easily understood by the respondents.

7
The questionnaire was organized around five core questions: Do ministries or regulatory agencies in your jurisdiction give notice of proposed regulations to the general public? Do they publish the text of proposed regulations before those regulations are adopted? Do they request comments on proposed regulations from the general public? Do they report on the results of the consultation on proposed regulations? And do ministries or regulatory agencies in your jurisdiction conduct an impact assessment of proposed regulations? Each of these core questions was followed by up to nine additional questions requesting details on how certain practices are carried out, for example, whether through a unified website or public meetings. The questionnaire also touched on the institutional setup for regulatory practices and legal requirements. And it gave respondents space to provide additional comments and clarifications at the end of each section. Respondents were asked to report on actual practice as well as what is required by law.
The five core questions can be seen as sequential-as tracing a possible developmental arc of good regulatory practices. The foundational transparency that comes with notifying the public of proposed regulations is essential to effectively carrying out the sequence of acts from publishing drafts, to requesting comments from the public, to reporting back on the results of that consultation. By systematically analyzing (and sharing) the possible effects of proposed regulations, impact assessments represent a further step toward ensuring that regulations meet their intended goal. Of course, the components of the process do not need to develop in the sequence shown, but we do chart a possible developmental path for countries reforming their rulemaking practices.
The questionnaire was sent to more than 1,500 experts in 190 countries worldwide. 1 It was distributed in French to respondents in French-speaking countries, in Spanish to those in Spanish-speaking countries, in Russian to those in Russian-speaking economies, in Arabic to those in Arabic-speaking countries, and in English to respondents in all other economies. (The questionnaire is available at http://rulemaking.worldbank.org/methodology.) For 144 countries experts from both the public and private sector responded to the questionnaire. A majority of the private sector respondents were corporate lawyers; however, private sector respondents also included notaries, academics and think tank researchers. The public sector respondents were civil servants identified as experts in their government's rulemaking process. For 41 countries the data come only from private sector contributors. Where necessary, we conducted extensive follow-up by phone and email to clarify and confirm the data. In many cases we also reviewed specific laws and regulations and checked websites to independently verify the information provided by respondents. When desktop research was inconclusive, additional follow-up was done by phone or email.

8
The data set is unique in two ways. First, it captures what happens in practice during rulemaking. Unlike other data sets on rulemaking practices, its data collection does not rely solely on what government officials report about their transparency and quality assurance practices; instead, it also seeks private sector input on whether the government actually publishes proposed regulations and engages with the public on them. Thus if an economy's laws or regulations stipulate that the government must engage the general public in the design and implementation of new regulations, but the government does not carry out such consultation in practice or the private sector is unaware of it, we code the economy as not conducting public consultation. Functionally, such countries fail to offer citizens a way to voice their concerns about proposed regulations. Second, the data set is global in scope. It covers countries in all regions and at all levels of income, including low-income economies. Previous studies have focused on certain economies or regions and thus do not provide the comprehensive, point-in-time global snapshot of regulatory practices given by this data set.
A significant limitation of the data set, however, is that it provides data only at the national level. Citizen engagement in rulemaking processes can differ substantially at the regional and even municipal levels. This issue could be especially relevant for big countries, such as Brazil, China, India, Mexico and the Russian Federation, as well as many others. An interesting project going forward would be to design and conduct country-specific studies comparing participation in rulemaking at the subnational level.
Another limitation is that while we know where platforms for consultation exist and are used, the data do not tell us how many people actively engage with policy makers or how much diversity there is in their views. Nor do we know the extent to which government officials take public feedback into account when finalizing regulations, unless the government reports back on the results of consultations. And while we know whether or not regulatory impact assessments are being conducted, we do not capture the quality of those assessments or their impact on the proposed regulatory changes. These questions could be additional areas of exploration in future research.

IV. Descriptive statistics
In this section, we outline the main data trends with the focus on the core components of the survey, including 1) giving notice of proposed regulations and publishing drafts, 2) requesting comments and reporting on results of the consultation process, and 3) conducting regulatory impact assessments

a. Giving notice of proposed regulations and publishing drafts
Notifying the general public of a proposed new regulation is a common practice, reported in 136 of the 185 countries included in the Citizen Engagement in Rulemaking data set. A large plurality of these economies are in the OECD high-income group or Europe and Central Asia. 2 In many countries the public notices include a wide range of information on the proposed regulation. In Estonia, for example, they provide a short summary of the proposed regulation and explain why the regulation is needed, what it is intended to change and when it is expected to enter into force. In Lithuania the notices include this same information and also detail the expected positive and negative consequences of the regulatory change and describe the consultation process planned. And in Moldova they include the general justification for the proposed regulation, contact details for civil servants responsible for the regulation, as well as the deadline and method for submitting recommendations, comments or feedback.
Providing public notice of proposed regulatory changes is part of ensuring predictability in the regulatory environment, an aspect that has long been key for firms seeking to make long-range plans and investments. Foreign investors seek insight into the rulemaking plans of the economies in which they invest, both to inform their operations there and to avoid situations where domestic actors receive preferential treatment.
otification practices vary across regions. The government gives notice of proposed regulations in more than 70 percent of countries in East Asia and the Pacific (19 of 25) and in 57 percent in South Asia (4 of 7). But the government does so in only 37 percent of economies in the Middle East and North Africa (7 of 19). It follows that in many of this region's countries trust in government institutions is low and government accountability is weak (World Bank Group 2015). Coming into the Arab Spring in late 2010, the region had promising potential for positive change-with a young and educated population, a strong resource base, low absolute poverty, and an economic resilience that helped in overcoming the effects of the 2008/09 global financial crisis. But the region has shown only limited improvement in governance, including in the quality and transparency of rulemaking institutions (Heidenhof 2014).
The practice of providing public notice of proposed regulations is by no means limited to advanced economies, nor is it followed by all wealthy countries. Notification is given in around 70 percent of low-and lower-middle-income economies in East Asia and the Pacific, including Cambodia, the Lao People's Democratic Republic and the Solomon Islands. But in the Middle East and North Africa it is the wealthiest countries that have the least transparent practices.
In the 136 countries around the world where the government provides notice of proposed regulations, the most common practice, followed in the majority of these economies, is to use a single, dedicated website to publish regulatory announcements and related materials (figure 1). This practice, in which different agencies use a single, unified website to provide notification, is widely followed among economies in the OECD high-income group as well as in Europe and Central Asia. In another 30 percent of these 136 countries, notice is given through the website of the ministry or regulatory institution leading the changes. In contrast, websites are used to share news of proposed regulations in less than 7 percent of all countries in Sub-Saharan Africa. Given global trends in internet access, it is no surprise that the use of the internet for citizen engagement in rulemaking is more prevalent among high-and upper-middle-income countries. Indeed, unified websites are used in as many as 75 percent of high-income economies, while electronic platforms are used in less than 7 percent of low-income economies.
Legal requirements may drive some of this transparency. In 51 of the 185 countries surveyed, rulemaking bodies are required by law to give public notice of their regulatory reform efforts.
Most of these economies are in the OECD high-income group or Europe and Central Asia. In Switzerland, for example, article 13 of the Publications Act requires the government to give public notice of all proposed regulations. In Hungary the government faces a similar obligation, dating from 2010, 3 and regulators comply by publishing the entire text of proposed regulations on a single, unified website. In Poland, which has adopted legal requirements over the past two and a half decades intended to ensure a more transparent and inclusive rulemaking process, 4 draft regulations are publicly accessible on the parliament's website.
Perhaps more interesting are the countries where the government faces no legal obligation to publish proposed regulations or seek public input on them but systematically does so anyway. In more than 30 of the sampled economies regulators voluntarily publish proposed regulations despite having no formal requirement to do so. Examples can be found in all regions of the world. In Australia, for instance, the consultation process is regulated mostly by "circular letters" from the Office of the Prime Minister, which are not considered legally enforceable-yet these letters are strictly adhered to. In Bosnia and Herzegovina rulemaking bodies have no legal requirement to give public notice of proposed regulations, but they do so both on websites and in the official gazette. And in Namibia, while there is no legal obligation to give notice, it is common to do so. The government often publishes draft legislation and regulations and invites stakeholders to submit comments. These examples suggest that while legal requirements may be useful in prompting reform of nontransparent rulemaking practices, they are not necessary where public consultation is deeply imbedded in the tradition and practice of rulemaking.
In 43 of the 185 countries surveyed the law stipulates a minimum period for which the draft regulations must be publicly available for review and comment. In more than half these economies this consultation period is 30 days or more. In the Republic of Korea and Montenegro it usually lasts for about 40 days, and in Mexico for 60 days. In Lao PDR and Vietnam draft regulations are available for two months, while in Hungary and the Solomon Islands they are available for a year. In Estonia, by contrast, the consultation period is 15-20 working days for draft laws and 10 working days for draft regulations. In Canada, where there is no set requirement to make draft regulations publicly available, the standard practice is to allow a comment period of 30 days, though the length can vary according to legislative requirements, international obligations and other considerations. And in India draft regulations are generally available for 30-90 days, though the length of time is not governed by law.

b. Requesting comments and reporting on results of the consultation process
The practice of seeking stakeholder input to the rulemaking process is widespread. In 123 of the 185 countries surveyed, ministries or regulatory agencies request comments on proposed regulations. By getting feedback from the groups that will need to comply with a proposed new regulation, policy makers can identify conceptual problems and shape the scope of the regulation to effect the intended outcome. They can also hear from other groups that, while not the target of the proposed regulation, may face substantial unintended harm if the draft remains unchanged.
Governments conduct public outreach through websites, through open meetings or by reaching out directly to known stakeholders. Feedback is solicited through at least one of these means in all OECD high-income economies, in 77 percent of countries in Europe and Central Asia and in 68 percent in East Asia and the Pacific (figure 2). Apart from targeted outreach to specific stakeholders, regulatory agencies request public feedback on proposed regulations in 80 percent of high-income countries globally. In contrast, comments from constituents are rarely solicited in the Middle East and North Africa, though Jordan and Morocco are notable exceptions. In Morocco comments on proposed legislation can be submitted on the same web page where the proposed text is published, in both French and Arabic. In the data collection we distinguished between fully public consultation through websites, gazettes, newspapers and public meetings and more "closed door" outreach to specific 13 stakeholders. The most common practice for soliciting feedback on proposed regulations is through targeted outreach to business associations and other stakeholders, reported in 40 percent of the countries surveyed. The least common practice for doing so is through the relevant agency's website, reported in about 16 percent of the countries ( figure 3). Yet this practice, while less frequent, can be found across all regions and in both low-and high-income countries. In both Myanmar and Spain, for example, regulatory agencies have created specific websites to receive comments from the general public on proposed regulations. In only 7 percent of the economies surveyed does the government request comments through specific outreach to identified stakeholders and not also through additional, more public means, such as through websites or meetings open to the public. Of the regulatory agencies, 33 percent conduct no consultation outside the government. Among low-and middle-income countries, another common method of soliciting comments is through discussions at public meetings. In close to a third of those in which the government requests comments on proposed regulations, it does so through open town hall meetings. Government agencies in high-income economies also use public meetings to solicit stakeholder input, often in tandem with online platforms. This approach is followed in Canada, Singapore and Spain, for example. In South Asia, unlike in other regions, the most preferred method for engaging stakeholders is by email. And among the countries worldwide where regulators request comments, 14 have a specialized government body tasked with soliciting and reviewing comments-most of them high-income countries. Open consultations are least common in the Middle East and North Africa and in Sub-Saharan Africa.
In more than two-thirds of surveyed countries where the government solicits feedback on proposed regulations, it also reports back on the results of the consultation process. In about 60 percent of these economies the government prepares one consolidated response to the comments received, while in about 30 percent it provides customized responses for different audiences. In the rest of the countries the government uses various other ways to report back on the consultation process. In Romania, for example, results are reported at public meetings, while in many other countries a synopsis is posted on the relevant agency's website. Customized responses are more common among low-income countries, while consolidated responses are more widespread in other income groups. Consolidated responses are distributed in Belgium, Peru and Vietnam, for example. In Albania both customized and consolidated responses are prepared, in the form of an explanatory note attached to the final draft of the text to be approved by the Council of Ministers. This note provides detailed information on the consultation process with the stakeholders, civic associations and the business community.
Whether consolidated or customized, reports on the results of public consultations are communicated though a variety of channels. In 18 percent of the sampled countries results are posted on a dedicated website used for all regulatory consultations, while in 10 percent they are posted on the website of the relevant agency or ministry. In another 6 percent results are printed in a federal journal or similar publication. And in 20 percent they are communicated directly to stakeholders.
Not surprisingly, online platforms tend to be used in high-income economies: in 24 of the 41 high-income countries where regulators report the results of public consultations, they use the internet to do so. Governments in these economies most often use a single, dedicated website for this purpose (figure 4). In Canada, for example, departments and agencies use such a website to provide a summary of comments received along with their responses to those comments. The website of an individual agency or ministry or a unified website is rarely used in low-or lowermiddle-income countries to disseminate the results of public consultations. Reporting the results of public consultations is required by law in 26 of the 185 countries surveyed, including 11 economies in Europe and Central Asia and 9 in the OECD high-income group. No such requirements exist in South Asia or Latin America and the Caribbean. Where results must be reported, this must happen most commonly within 15-30 days after the end of the consultation process. In Latvia the law requires the rulemaking body to publish the results within 30 days after the end of the process, on a website and, where possible, through additional public vehicles such as press releases. In Spain the rulemaking body is required to publish a consolidated consultation report within 15 days after the end of the consultation process. In practice, however, it generally takes the rulemaking body several weeks to publish the report.
In 32 of the 185 countries rulemaking bodies have no legal obligation to report the results of consultation processes but still do so in practice. In Singapore, for example, regulators generally post a consolidated report on the relevant agency's website-even though they face no statutory requirement to publish the results of public consultations. The results of a consultation are published either before or together with the final text of the legislative change.
While many governments follow such practices, it is important to keep in mind that in 108 of the 185 countries sampled the government either does not request comments from the public on proposed regulations or does not report back to the public on the results of consultations. This group consists mostly of low-and lower-middle-income economies, but it also includes Argentina, Lebanon, South Africa and Turkey.

c. Conducting regulatory impact assessments
The practice of assessing the potential impact of proposed regulatory changes on citizens, businesses and society at large is more common among higher-income economies than among lower-income ones. These assessments also vary in scope and frequency. And how they are performed, if at all, depends on legal obligations and agency capacity. In the United States, for example, under the Administrative Procedure Act and Executive Order 12866, a regulatory agency proposing regulatory changes must conduct an initial cost-benefit analysis to determine whether those changes would be economically significant. If the proposed changes meet a certain threshold, the agency must prepare an impact assessment and submit the assessment along with the underlying analysis for review by the Office of Information and Regulatory Affairs at the Office of Management and Budget. Within the European Union some member countries, including Germany, the Netherlands, Sweden and the United Kingdom, have conducted impact assessments for years, while others have only recently integrated impact assessment into their rulemaking process (De Francesco, Radaelli and Troeger 2012). And every time the European Commission proposes new legislation, it needs to evaluate the possible economic, social and environmental impacts. 5 In 98 of the 185 countries surveyed for this paper, ministries and regulatory agencies do not conduct impact assessments of proposed regulations. In those where they do, the impact assessments vary in scope, reflecting most commonly the anticipated administrative costs to the government of enforcing the new regulation and the expected compliance costs for the private sector. Measures of the possible environmental impact and potential effect on market openness are also common. Survey results show that impact assessments encompass a wide range of practices and methods. Impact assessments of proposed regulations are more likely to be conducted in richer countries: they are performed in 97 percent of the high-income countries studied but only about 18 percent of the low-income ones (figure 5). In 42 of the 87 economies where the government performs impact assessments, it is required to do so by law; most of these 42 are in the OECD highincome group or Europe and Central Asia. In 30 countries, half of them in the OECD highincome group, there are specific criteria that trigger a requirement for an impact assessment. In Norway, for example, the government conducts an impact assessment if a proposed regulation is likely to have major consequences for the economy. In Japan and Malta there are specific monetary thresholds for potential economic impacts of new regulations. In the United States the policy is to carry out an impact assessment of all "economically significant" regulations. An "economically significant regulatory action" is defined as one that is likely to result in a regulation that may have an annual effect on the economy of $100 million or more or adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or state, local or tribal governments or communities. 6 Among the countries where impact assessments are conducted, only slightly over half have guidelines on how to conduct them. In most of these countries the guidelines are published on regulatory websites. But there are also other ways to make the guidelines public. In Hungary, for example, the guidelines are part of the Decree of the Minister of Justice and Public Administration 24/201. Another 28 percent (24) of the countries with impact assessments have developed no guidelines for conducting them. These countries fall in all income groups and regions. Examples include Armenia; Hong Kong SAR, China; and Portugal.
Making the impact assessments publicly available and open for scrutiny forms part of the consultation process in many countries. In 82 percent of the countries where the government conducts impact assessments, the results are distributed online or through targeted outreach to business associations, other stakeholder groups or both. And in 65 percent the results are distributed together with the text of the proposed regulation. By providing the analysis underpinning the scope and content of the proposed regulation for comment from the general public and stakeholders, governments open their motivation and reasoning for the regulatory change to scrutiny and input.
Notably, 51 countries in the sample have a specialized government body tasked with conducting or reviewing impact assessments of proposed regulations. These oversight bodies may be responsible for determining which regulatory reforms require an impact assessment, for example. But their most common responsibility is to provide guidance to the experts conducting the assessments. They also frequently review and monitor impact assessments conducted by ministries and inform the cabinet or legislature about their compliance with the requirements (figure 6). These specialized bodies can develop expertise within the government on conducting high-quality impact assessments and simultaneously ensure that ministries are complying with impact assessment guidelines.

V. Scoring countries on the level of citizen engagement in rulemaking
To advance our analysis, we developed a composite score-the citizen engagement in rulemaking score-reflecting respondents' answers to the core questions for each economy with a few adjustments. The six components of the score reflect whether governments publish the text of proposed regulations before their adoption, publicly request comments on proposed regulations, publicly report on the results of consultation processes, conduct impact assessments of proposed regulations, have a specialized body tasked with reviewing regulatory impact assessments, and publicly distribute the results of regulatory impact assessments. These questions were chosen as best reflecting the principles of transparency, consultation and impact assessment in creating new regulations.
Two of the six component indicators (conducting impact assessments of proposed regulations and having a specialized government body tasked with reviewing regulatory impact assessments) have simple binary scoring: each economy receives a score of either 1 (the highest score) or 0 (the lowest).
For each of the other four component indicators the possible score ranges from 0 to 1, with the possible values including 0.2, 0.6 and 0.8. The score assigned to an economy depends on the level of transparency or inclusiveness reflected in how the government carries out the relevant action (if at all)-that is, publishing proposed regulations, requesting comments, reporting back on the results of the consultation or publishing the regulatory impact assessment. A score of 1 is awarded if the action is performed through a unified website, 0.8 if through individual ministry websites, 0.6 if through public meetings or publication in an official journal, and 0.2 if through targeted outreach to identified stakeholders; a score of 0 is assigned if the action is not performed. For each economy the highest possible score is assigned, even where the government uses a variety of different methods for carrying out the action. A lower score is awarded for targeted outreach to identified stakeholders because of the inherent potential for exclusion in this approach.
The composite score is the simple sum of the six component indicator scores. Table 1 provides an overview of the scoring possibilities for the Citizen Engagement in Rulemaking indicators.
Because there is no literature establishing the relative importance of transparency, consultation and impact assessment in the rulemaking process, we chose to use the common approach of assigning equal weights to all components of the citizen engagement in rulemaking score. However, the core analysis presented in this paper does not change substantially when slightly different weights are assigned to the components of the score.

VI. Discussion of the results
The composite score allows us to analyze data at the aggregate level, observing important variations by region and income level. Not surprisingly, OECD high-income countries and highincome countries in general, tend to perform well on the citizen engagement in rulemaking score ( figure 7). The exceptions are Gulf States, with little transparency or consultation around regulatory changes. The data also reveal a strong positive relationship between the level of economic development, proxied by income per capita, and the citizen engagement in rulemaking score.
The scores for the component indicators are also strongly correlated with income levels. In the scoring on publishing proposed regulations before their adoption, for example, 38 of 53 highincome countries in the sample receive the maximum score of 1, compared with only 1 of 33 low-income countries. The maximum score reflects the advanced use of unified electronic platforms. Similarly, when we examine the data on whether regulators solicit comments on proposed regulations from the general public, 22 of 53 high-income countries receive the maximum score, in contrast with only 4 of 46 lower-middle-income economies and 1 of 33 lowincome ones (see appendix table A.1). The scores for distributing regulatory impact assessments follow a similar pattern. Among 53 high-income countries, all receive the highest possible score, indicating that their government distributes regulatory impact assessments through widely accessible channels. By comparison, the maximum score is received by only 7 of 50 upper-middle-income countries, only 5 of 46 lower-middle-income countries and no low-income ones. Indeed, among 33 low-income economies, 27 receive a score of 0, while 5 get a score of 0.6 and one a score of 0.2. Overall, of the 87 sampled countries in which the government conducts regulatory impact assessments, 41 are high-income countries and only 6 are low-income ones. This is not surprising, because regulatory impact assessments can require considerable institutional capacity and expertise within the government.
Analysis relating the citizen engagement in rulemaking score to other measures of good governance supports earlier findings linking transparency, consultation and ex ante assessment in rulemaking with higher-quality regulation, lower corruption and stronger rule of law (figure 8; see also appendix table A.5 for detailed regression results). Comparator indicators include Transparency International's overall score; the World Justice Project's rule of law, effective regulatory enforcement and open government indices; and the Worldwide Governance Indicators project's measures of voice and accountability, government effectiveness, regulatory quality and rule of law. The analysis also includes measures of regulatory efficiency and regulatory quality from the World Bank Group's Doing Business project. The assumption for this analysis is that countries with high levels of citizen engagement in rulemaking tend to have better governance and stronger rule of law.
In all cases the results are statistically significant, and in most they are significant at the 1 percent level. We run regressions with and without controlling for income per capita, to ensure that we do not capture general effects of good governance associated with already high levels of economic development. The results are statistically significant in both cases. Overall, the ordinary least squares (OLS) regression outcomes presented in figure 8 appear to validate the robustness of the citizen engagement in rulemaking score, supporting its possible relevance to desired regulatory and economic outcomes. However, a rigorous analysis needs to be performed to either confirm or reject this possibility. This preliminary analysis appears to be in line with the findings of existing literature. Civil society organizations, political parties and private sector entities with a "voice" in decisionmaking help to reinforce responsive and accountable states. In such states communication channels between government officials and civic organizations are well established and highly functional (Sharma 2008). And better communication techniques and stronger formal institutions enhance people's trust in government and the rule of law (Knack and Zak 2003). Yet while our analysis suggests that economies with transparent, consultative and evidence-based rulemaking 24 also have effective government and rule of law, the reverse could also be true, of course. An alternative interpretation of the results could be that governments that are effective and follow the rule of law also actively seek to serve their citizenry-and thus engage citizens in continual efforts to create effective regulations that are easy to comply with.
After looking at the relationship between the citizen engagement in rulemaking score and relevant indicators from other data sets, we explore the relationship between this score and five distinct measures of regulatory quality from the Doing Business project. We expect inclusive rulemaking practices to be strongly and positively associated with higher regulatory quality. It is assumed that in regimes in which a wide range of stakeholders have a voice in policy making, governments benefit from that feedback and therefore produce regulations that are more relevant, easy to comply with and targeted to the desired outcome.
Results show that the citizen engagement in rulemaking score is strongly and positively correlated with all five of the Doing Business project's distinct measures of regulatory quality (figure 9; see also appendix table A.6). The results remain significant with and without introducing income per capita controls. Income per capita is highly correlated only with the quality of land administration index. As in the previous analysis, we run the same regression using the log of income per capita. Taking the log of income per capita improves the size of the coefficients without influencing the strength of the coefficients of the dependent variables.
(Regression results with the log of income per capita are presented in appendix table A.6.) We observe that countries that have higher citizen engagement in rulemaking scores also tend to have higher-quality business regulation as reflected in quality control practices for construction, in the monitoring of utilities, in the transparency of electricity tariffs and in systems of land dispute resolution. In addition, there is an apparent relationship between transparent, consultative, evidence-based rulemaking and a high-quality judiciary system: economies that perform well on the citizen engagement in rulemaking score also tend to have a judiciary with specialized courts, high levels of court modernization and automation, and an effective and transparent case management system-along with a best-practice insolvency framework. Again, we are observing simple correlations, without inferring any causal relationship. The analysis would not be complete without testing the relationship between our composite score and different degrees of democracy at the national level. As defined by Thomas Franck (1999), "Democracy, as its etymology makes clear, concerns the role of the people in their governance: the right of persons in a political community to participate meaningfully in the process by which they define and implement values, priorities and policies." Thus as the final step in our analysis we correlate the citizen engagement in rulemaking score with several measures of freedom and democracy. For this analysis we chose democracy and autocracy measures from the Polity IV data set of the University of Maryland, measures from Freedom House capturing various aspects of political rights and civil liberties and the democracy classification score of the Economist. Results from OLS regressions are presented in figure 10 and appendix table A.7.
We expected the results to show that inclusive regulatory practices go hand in hand with strong democratic forms of governance. Most of the relationships are significant at the 1 percent level and hold with and without controlling for income per capita. And all the signs of the coefficients of the Polity IV data set match our hypotheses. We find a strong positive relationship between the citizen engagement in rulemaking score and the strength of democratic regimes. The opposite is true for autocratic regimes. For the Freedom House data all the results are statistically significant. We also find a strong positive association between the World Justice Project's civic participation in politics score and our measure of inclusive rulemaking (see figure 10). Indeed, countries with a robust, well-functioning democracy also tend to have a highly inclusive rulemaking process-and vice versa. Conversely, countries that lack a well-functioning government along with political pluralism, respect for civil liberties and freedom of expression and belief also tend to exclude the public from rulemaking processes. Researchers emphasize that citizen participation in rulemaking and transparency in governance can enhance democratic legitimacy by bridging the gap between government agencies and their constituencies.
Democratic values as well as overall social welfare are directly affected by the ways in which rules are designed and implemented (Coglianese, Kilmartin and Mendelson 2009).

VII. Conclusion and recommendations for future research
This paper presents new global data on government transparency, consultation and impact assessment practices around new regulations. The data show that inclusive rulemaking processes are widespread in the developed world but sorely lacking among developing countries. Variation exists within each region and income group. Yet aside from a handful of notable exceptions, developing countries still lag significantly behind the advanced ones when it comes to better rulemaking practices.
We also find that legal requirements do not necessarily drive governments' transparency and impact assessment practices. In the majority of countries where the government publishes proposed regulations and conducts consultations, it has a legal requirement to do so. But in a third of these economies the government does so voluntarily, with no formal obligation to take these actions. Conversely, not all governments that are legally required to publish proposed regulations and conduct consultations comply with these requirements in practice.
When we compare our composite score on transparency, consultation and impact assessment in rulemaking with different outcomes measured by other relevant indicators, we find that citizen participation in the rulemaking process and stronger impact assessment practices are associated with higher-quality regulation. Broader public consultation is also linked with adherence to democratic principles of governance and stronger rule of law. Finally, we see that these better rulemaking practices are correlated with more advanced economic development.
While this paper expands the data coverage of inclusive regulatory practices, especially for developing countries, further analysis is warranted on the link between levels of development, regulatory practices and different outcomes. One question to be explored is whether there is a threshold income level below which consultation or impact assessment seems to have no impact on regulatory quality or democracy. Also interesting to explore would be differences in rulemaking at the subnational level, since the data set is currently limited to national rulemaking processes. An additional question that remains open is how, and to what degree, comments by external stakeholders change the shape of the ultimate regulation. Yet another is what the practical limits are to inclusiveness. The potential trade-offs between extensive consultation and policy making efficiency could also be explored.
What is clear from this study, however, is that much more work remains to be done by regulators to increase transparency and public involvement in the rulemaking process and deepen the analysis of possible impacts of new regulations, especially in developing economies. In so doing,     A A v e r a g e s c o r e 0 . 8 0 0 . 6 2 0 . 5 8 0 . 7 8 0 . 5 6 0 . 5 7 3 . 9 1