Effects of the internet on participation: Study of a public policy referendum in Brazil

ABSTRACT Does online voting mobilize citizens who otherwise would not participate? During the annual participatory budgeting vote in the southern state of Rio Grande do Sul in Brazil—the world's largest—Internet voters were asked whether they would have participated had there not been an online voting option (i-voting). The study documents an 8.2 percent increase in total turnout with the introduction of i-voting. In support of the mobilization hypothesis, unique survey data show that i-voting is mainly used by new participants rather than just for convenience by those who were already mobilized. The study also finds that age, gender, income, education, and social media usage are significant predictors of being online-only voters. However, technology appears more likely to engage people who are younger, male, of higher income and educational attainment, and more frequent social media users.

Although the number of examples of Internet voting is steadily increasing and the literature on the subject is burgeoning, the empirical evidence on the effects of i-voting is still limited and major questions remain open (Carter & Bélanger, 2012;Pammett & Goodman, 2013). Is i-voting used mostly by those who are already politically active and interested? Are some citizens only willing to vote via the Internet? If so, what are the socioeconomic characteristics of this group? What are the effects of Internet voting on turnout, inclusiveness, and diversity? Does introducing i-voting distract people from traditional forms of participation?
The objective of this paper is to contribute to answering these questions by providing evidence on the effects of Internet voting on participation. But apart from its empirical character, this contribution is also relevant for two other reasons. First, it assesses the effect of i-voting in an understudied field of i-voting, that is, in participatory governance (i.e., nonelectoral) processes. Second, while the majority of i-voting studies have focused on the United States and Europe, this study looks at an experience from a middle-income country, Brazil.
We present the results of a unique survey of over 22,000 Internet voters from the southern Brazilian state of Rio Grande do Sul during a referendum on state-level spending priorities. 2 This referendum is part of a large multichannel democratic innovation that simultaneously combines both online and off-line voting. This process, entitled "State System for Citizen Participation in the Budget Process," 3 allows citizens to influence the formulation of projects for the allocation of public spending and to select which of the projects will be implemented via a referendum. It is a form of participatory budgeting (PB) applied at the state level (henceforth State PB). 4 Here we analyze the last stage of the State PB, that is, the vote in the final referendum, during which the population prioritizes the projects to be funded.
More specifically, we investigate the traits and attitudes of those who cast their ballot via the Internet using a post-vote survey. The survey was implemented as an online exit poll at the time of the vote in early July 2012 and consists of 27 questions. 5 The questions aimed to identify the socioeconomic characteristics of the respondents and their media and ICT usage habits, as well as their previous level of engagement with politics and civil society (see Appendix 3). The key question we asked the participants is whether they would have voted traditionally if there has been no opportunity to vote online. This question identifies the subset of participants that self-identify as online-only voters.
The Rio Grande do Sul e-survey allows us to investigate three key research questions: (1) Does the possibility of online voting primarily attract online-only voters (mobilization hypothesis)? Or is i-voting mostly used by traditional voters for convenience (substitution hypothesis)? (2) What is the socioeconomic profile of those who self-identify as online-only voters? More specifically, are they predominantly young, educated, rich males who are familiar with the Internet-as we would expect from the literature on digital divide (reinforcement hypothesis)? Or are they predominantly marginalized individuals with low income and education (empowerment hypothesis)? (3) What is the level of preexisting engagement of these voters? Are online-only voters already engaged in the public sphere, or does the new venue of participation attract previously disengaged portions of society (engagement hypothesis)? Anticipating the key results of the analysis, our study shows that i-voting increases participation among previously nonengaged strata of the population, potentially increasing turnout and promoting the inclusiveness of the process as a whole (mobilization and engagement hypothesis). However, these new participants-the online-only nonengaged voters-are likely to be socioeconomically more privileged. The latter result rejects the empowerment hypothesis and identifies a crucial challenge for the design of multichannel democratic innovations. Introducing digital channels of participation might significantly strengthen the voice of privileged groups, and thus might overall reduce the voice of marginalized groups.
The rest of this paper proceeds as follows. Section 2 briefly presents the key research questions of the literature on i-voting and identifies testable hypotheses. Section 3 describes the introduction of i-voting in Brazilian participatory engagement processes and introduces the State PB. Section 4 presents the results of our empirical analysis. Section 5 discusses these findings in the light of the literature of convenience voting, participatory governance, and collective intelligence, concluding with implications for future practice and research.

Extant theory
Although a detailed overview of the vast literature on the variety of Internet voting mechanisms and their impact is beyond the scope of this paper, for our purposes it is important to highlight the two key issues of turnout and inclusiveness. The debate on the benefits of remote voting in increasing voter turnout started with postal voting (Kousser & Mullin, 2007;Mann & Mayhew, 2012;Moreton, 1985;Stein & Vonnahme, 2011) and has seen a series of iterations that have followed the evolution of information and communication technologies (Alvarez, Hall, & Trechsel, 2009;Bochsler, 2010;Carter & Bélanger, 2012;Trechsel, Schwerdt, Breuer, Alvarez, & Hall, 2007). The rational choice approach, assuming that the act of voting implies both costs and benefits, 6 offers a framework to support any form of convenience voting and thus can be used to support the adoption of Internet voting (Aldrich, 1993;Downs, 1957;Riker & Ordeshook, 1968). The relationship between the convenience of voting and participation is established in the following manner: holding all other factors constant, the probability of participation will be negatively correlated with the costs of participation. Because of this relationship and the growing access to the Internet in both developed and developing countries, i-voting scholars have often highlighted the Internet's potential to increase turnout by reducing its costs (Alvarez & Hall, 2004;Carter & Bélanger, 2012;Trechsel et al., 2007).
Another approach that is often used to explain why people vote, even if they have almost no chance to affect the outcomes, is rooted in various socialization theories. According to socialization scholars, participation is mostly learned. Early experiences are particularly important in shaping individuals' level of political participation (Dalton, 2008;Hooghe, 2004;Niemi & Hepburn, 1995). Using the lens of socialization theory, the adoption of i-voting should be negatively correlated with age and familiarity with new technologies.
The concept of inclusiveness in voting processes refers to the diversity of participants and the extent to which groups traditionally excluded from a process may be effectively included. In this respect, optimistic views support an "empowerment hypothesis," which states that the introduction of online modes of participation will increase the participation of citizens previously marginalized in traditional participatory processes (Norris, 2001). Although there is significant reason to believe that Internet voting has the potential to boost turnout and inclusiveness, the empirical literature finds mixed results. The effects of i-voting appear to be elusive, with some cases presenting minor results and others suggesting no effect whatsoever (Pammett & Goodman, 2013;Vassil & Weber, 2011). 7 In fact, at odds with the empowerment thesis, much of the digital divide literature suggests that unequal access to the Internet will disproportionately increase the representation of economically advantaged groups who are already politically active (Alvarez & Nagler, 2000;Bélanger & Carter, 2011;Brandtzaeg, Heim, & Karahasanović, 2011;Schlozman, Verba, & Brady, 2010). Often referred to as the "reinforcement hypothesis," such studies posit that people with the resources and motivation to participate, who are usually the better off, will be further empowered by the introduction of online modes of participation (Davis, 1998;Norris, 2001).
It is a well-established fact that Internet access is correlated with income and education (Hilbert, 2010). Although initial analyses of the digital divide focused on Internet penetration (Compaine, 2001), modern investigations in developed countries where Internet penetration has reached high percentages of the population center on citizens' capacity to effectively use the Internet (Brandtzaeg et al., 2011;Hsieh, Rai, & Keil, 2008;Sarkar et al., 2011). Overall, the trajectory of this literature has evolved from explanations based on income and education to more complex explanations that consider a variety of factors that affect the usage of ICT technologies, such as cognitive and social skills (Deursen, Helsper, & Eynon, 2014;Ferrari, 2012;Helsper & Eynon, 2013). How applicable these studies are to the specific literature on i-voting is yet to be fully explored (Carter & Bélanger, 2012). Furthermore, although most of the literature has centered on i-voting in general elections, less is known about its impact in other processes such as referenda and participatory budgeting, where turnout levels are often considerably lower.
Before moving to the next question, it is useful to highlight an important scope condition of our study. The critiques of the introduction of i-voting mechanisms are not limited to the concept of inclusiveness. There are also important debates regarding the quality of online engagement and the security of i-voting. Many refer to online participation as a form of "slacktivism," referring to an activity that, while psychologically rewarding, has little impact on political decisions (Hindman, 2009;Morozov, 2012;Smith, 2013).
A number of authors also raise questions about the security of i-voting systems (Bélanger & Carter, 2011;Wolchok, Wustrow, Isabel, & Alex Halderman, 2012). The debates on slacktivism and voting security are beyond the scope of this paper. But in the specific case of the State PB, participation in the electronic voting process was quite complex with a multipage ballot that allowed participants to affect not only the allocation of state funds to specific projects, but also more general public policy priorities. Similarly, it appears that in Rio Grande do Sul the online vote may be potentially more secure than the face-to-face process, due to the way these two mechanisms are organized. The former is centralized and managed by a unit of the state government that has no stake in the results; the latter is decentralized and managed by volunteers and community leaders who do have a stake in the results of the vote. We will briefly return to this topic in the section describing the voting procedure.
In the next sections, we will first introduce briefly the specifics of the Brazilian case, and then we explain the procedure of i-voting in Rio Grande do Sul's State PB in more detail.

ICT and participatory governance innovations in Brazil
Brazil is an international leader in participatory governance innovations (Fung, 2011). The Federal Constitution of 1988 mandated the introduction of public policy management councils, thematic councils, and management councils (Coelho, 2004;Moreira & Escorel, 2009), and introduced an array of democratic decision-making innovations (e.g., referendums, ballot initiatives, civil actions, national citizens conferences). Municipal participatory budgeting (PB) is probably the most famous of these institutions (Avritzer, 2009;Heller, Baiocchi, & Silva, 2011;Wampler, 2007). The process, initially adopted by 13 cities in Brazil in 1989, has now spread to more than a 1,400 cities worldwide (Sintomer, Herberg, Allegretti, Röcke, & Alves, 2013). Recent research has shown that PB processes have direct impacts on reducing mortality rates, promoting health-care spending, and creating civil society organizations (Gonçalves, 2014;Touchton & Wampler, 2013).
Participatory budgeting was also scaled up to the state level, 8 including the case we explore in this paper, which is one of the world's largest participatory budgeting processes both in number of participants and geographic coverage. The first instance of a state-level PB process occurred in the state of Rio Grande do Sul (RS) from 1999 to 2002. The experience was continued after the Workers Party (PT) loss of the governorship in 2003, with modifications. The new process included Internet voting for the first time and was called Consulta Popular (Goldfrank & Schneider, 2006). After PT's victory in 2010 the process was changed again to its current form, as we shall describe later.
In Brazil, the first experiments in hybridizationincluding both off-line and online venues of participation-began in the late 1990s and the early years of the new century when a few cities adopted ICT tools to present proposals to voters or conduct voting in municipal PB processes (e.g., Porto Alegre, Ipatinga). The cities of Belo Horizonte and Recife implemented some of the earliest hybrid programs. The former uses separate online and off-line PB processes that decide on the allocation of two different, specific budgets, and the latter integrates online voting into the face-to-face mechanisms (Best, Ribeiro, Matheus, & Vaz, 2010).
A number of cities around the world have adopted the online PB model of Belo Horizonte, without introducing a parallel off-line process. This new form of PB is often referred to as e-PB. This process is particularly common in Europe (Nitzsche et al., 2012). Interestingly, in many of these cities the use of ICT was introduced with the more or less explicit objective of attracting younger and more middle-class participants. Yet to date, studies on the use of ICT in PB have been inconclusive in terms of youth and middle-class inclusion (Peixoto, 2009;Sampaio et al., 2011). In the case of Rio Grande do Sul's State PB, the use of ICT is limited to the final vote of the process, where citizens can choose to cast their vote either off-line at ballot stations or via the Internet. The next section describes the process and the voting system.

Multichannel participatory governance in the state of Rio Grande do Sul
After the Workers' Party won the state gubernatorial elections in 2010, it introduced a comprehensive reform of the state-level engagement processes (Sistema Estadual de Participação Popular e Cidadã, from now on Sistema). With support from the World Bank, Sistema reorganizes and expands a preexisting array of participatory governance mechanisms. Although it is too early to understand whether this reform will achieve the ambitious goals of better integration between participatory, deliberative, and representative institutions, Sistema has unique characteristics and represents one of the most ambitious projects of state-level multichannel participatory governance in the world (Goldfrank, 2013;Oliveira & Karnopp, 2013).
One component of this system-the State PBis a participatory governance mechanism that allows citizens and CSOs to influence the allocation of a portion of the state investment budget. This mechanism is an update of the preexisting Consulta Popular introduced in 2003 (Borowski, 2012;Goldfrank, 2013). There are three changes. First, the new procedure is embedded in the entire system of participatory governance, which includes the participatory multiyear plan, a process that defines the state government's major objectives every four years.
Second, the process has a more complex ballot than the previous Consulta Popular; participants can vote on projects and more general policy questions. 9 Third, the ability of CSO representatives to oversee the implementation of projects has been strengthened. In its current version, the government provides monthly project status reports to the participatory assemblies that manage the process. The detailed mechanisms of the various stages of the State PB are described in Appendix 4. 10 In what follows we will focus only on the voting process, starting with the ballot.
Votes can be cast either in person or remotely online. The online and the off-line ballots are identical. In both processes, the voter is identified by his/her unique federal voter registration number, where each number can only be used once. Voters are not allowed to change their vote. The votes are centrally stored and tallied by the Center for Electronic Government Solutions (PROCERGS), a state government unit specifically created to manage all electronic government infrastructure and processes. The i-voting is monitored by the Regional Councils for Development (COREDES), which are composed of CSOs from their respective regions. 11 The in-person voting process is organized by coordinators who are paid by the state, and by delegates who are selected by the participants of the municipal assemblies as their representatives. 12 The delegates do not receive compensation. On average, there are three to five polling officials at each location. The bylaws of the process specify that if there is a discrepancy of 2% or more between the votes in the ballot boxes and the signatures collected, the particular ballot box is invalidated.
Overall, turnout in the State PB vote is around 15% (see Table 1) of eligible voters, reaching over one million participants on a yearly basis, making the Rio Grande do Sul PB one of the largest PB processes in the world in terms of number of participants. Most of these voters still vote in person (around 86%-88%).
Concerns about the integrity of the voting process are rarely expressed. However, nothing prevents a participant from voting both online and off-line, since the online and off-line voter lists are not compared at any point in the process, nor is there any protection against voter coercion in the remote voting. 13 It is important to note that only a few online engagement processes around the world adopt a more advanced security system that allows voters to change their votes, to discourage improper influence or use stronger authentication procedures (e.g., Estonia, Switzerland). Most of these strong e-voting security measures are limited to electoral processes. Furthermore, as the literature on election fraud has shown, electoral integrity continues to be a problem even in off-line systems (Hyde, 2011;Sjoberg, 2012).

Methodology & data
One approach to investigate the issue of turnout and the effect of digital divide on inclusiveness is to observe changes in aggregate-level turnout when i-voting is introduced. However, since electoral dynamics can change significantly from one election (at time t1) to another (t2), it is very difficult to attribute cause and effect with such a research design. The obvious problem is that there is no data on the counterfactual of no i-voting at t2. Thus in this paper we use an indirect approach. First, we identify online-only voters and then we examine the factors that explain the propensity of individuals to identify themselves as online-only voters. We can calculate the overall turnout effect attributable to the introduction of i-voting by using the proportion of self-reported online-only voters.
On July 4, 2012, we conducted an online survey during the Rio Grande do Sul PB vote. All 124,211 voters who participated in the online vote were invited to complete the survey after they had voted. The survey instrument contained 27 questions, ranging from basic demographic details to participation profile and Internet usage (see Appendix 3). A total of 22,300 people took the survey. There was substantial item nonresponse, meaning that the total sample size for the models in this paper-the number of respondents who gave an answer for each of the variables we use in the online-only voter models-is 18,235. 14 In the following we present the results obtained from the fully completed surveys from RS.
The overall response rate is 18%. 15 This is a satisfactory "first contact" response rate for both traditional (e.g., telephone) and online surveys (Cook, Heath, & Thompson, 2000, Yeager et al., 2011. We can analyze the location of each survey response using the respondent's IP-address. 16 There are 6,010 responses from the state capital of Porto Alegre, with substantial numbers of responses from the cities of Pelotas (n = 1,251), Caxias do Sul (n = 1,122), and Santa Maria (n = 1,098). Despite the reasonable response rate, concerns about nonresponse biases are still present: if respondents are missing at random, this is not a major concern, but it is also possible that there are systematic biases in nonresponse. It is therefore important to keep in mind that the results of this study are generated using a sample that may not be fully representative of the population of online voters. At the same time, however, we benefit from a very large sample drawn from across the entire state (Figure 1), and we can infer a number of things by analyzing response patterns at the municipal level.
For instance, the average response rate at the municipal level is neither correlated with the vote share of the incumbent governor, nor correlated with turnout in the preceding 2010 elections (see Figure 1). This is important since supporters of the new government might have been more inclined to participate in this process. In addition, GDP per capita is positively correlated with the survey response rate, suggesting that richer municipalities are slightly overrepresented in the survey data.
In the next section we compare the demographic characteristics of our respondents to the general demographic characteristics of the state of Rio Grande do Sul. We then present general summary statistics about the key variables that we employ to test our hypotheses.

Summary statistics
The majority of the 18,235 survey respondents are female (54.2%). 17 This largely corresponds to the gender balance in terms of historical data on participation in traditional participatory budgeting meetings in Porto Alegre. 18 Half of the respondents report they are working, while around one third report that they are studying or teaching. The number of participants that declare themselves unemployed is 7% and the proportion of retired respondents is around 5%. The unemployment rate in the state of Rio Grande do Sul is 6%, suggesting that the survey represents the general population relatively well with respect to employment status. The median age of participants is 31 years, and three quarters of the participants are between 25 and 44. The median age in the state is slightly higher, around 35 years old. Quite surprisingly, a vast majority of the participants are not very active in the community. More than nine out of ten declare that they had not participated in any discussion regarding the budget before the vote.
With regard to Internet usage, more than 97% of the participants indicated that they had used the Internet before, but one in five had not used it in the previous three months. In this sense, it seems that i-voting attracts a non-negligible number of people who are traditionally considered non-Internet users. 19 Slightly more than half of the participants said they were voting from home, while around 40% were voting from their place of work. Most important, a stunning 63.1% of all respondents identified themselves as online-only voters by answering "No" on the question about whether they would have voted if they had not been able to cast a vote online. 20

Statistical model and results
In this section we present the results of the following logistic model: where π is the probability that individual i selfidentifies as an online-only voter, and X ik are common predictors of engagement. The models include dummy variables for each of the regions (COREDES), meaning that we are only looking at variance between individuals within a region rather than explaining regional-level variation.
The design of this observational study does not allow us to establish causality, but the hypotheses we introduce have testable predictions (see Table 3) and we present evidence that is consistent with the hypotheses. We systematically address alternative explanations for the patterns we observe in the unique survey data from the 2012 PB vote. Additional robustness checks are referred to in the text and tables provided in a separate statistical appendix. 21 Figure 2 shows the marginal effects of each of the predictors on being an online-only voter from a multiple logistic regression. A one-unit change in a particular predictor, holding all other predictors at their mean, is on average associated with an effect indicated by the black dot with the 95% confidence interval indicated by the shaded bars Social media usage has the highest marginal positive effect with an increase of 3.9 percentage points. The reason for this, we hypothesize, is the combination of two factors. Social media users are likely to spend more time on the Internet and have more technological skills (Hargittai & Litt, 2011;Muñoz & Towner, 2011). In this respect, a predictor for i-voting would be the degree of Internet usage and skills. 22 Second, given that the government and a considerable number of i-voters (33.7%) share content related to i-voting on social media channels, social media users are more likely to come across content mobilizing users to participate and direct links to the i-voting platform. With regard to socioeconomic characteristics, in line with the digital divide literature, and the reinforcement hypothesis, socioeconomically privileged individuals are more likely to self-identify as online-only voters according to our analysis. Income and education both have relatively large marginal effects of nearly 2.6 and 2.9 percentage points, respectively. Because education is measured on a 10-point scale and income on a 9point scale, these effects can be substantial when considering the difference between individuals classified as low and high on these measures. Going from one extreme of the education scale to the other translates into a 56.3% increase in the probability of being an online-only voter ( Figure 3). The equivalent number for the income scale is 38.0%. Thus, our results are at odds with the empowerment hypothesis; marginalized groups do not appear to benefit from this implementation of i-voting.
With regard to gender inclusiveness, the marginal effect of being male is 2.9 percentage points ( Figure 2). Yet, combining these regression results with the fact that the majority of survey Self-reported gender, income, and education positively correlating with self-identification as an online-only voter Empowerment hypothesis: Is i-voting mostly used by marginalized individuals with low income?
Income and education negatively correlating with self-identification as an online-only voter Engagement hypothesis: Is i-voting mostly used by individuals who are not already engaged in politics?
Previous engagement in traditional and non traditional forms of politics negatively correlating with self-identification as an online-only voter Notes. Logistic regression with online-only voter as the dependent variable (Question 18, see Appendix 3). All other independent variables in the full specification (model 3 see Appendix 2) are held at their mean except for region (COREDES), which is set to be the first region (the results are robust to other region choices). The regression includes regional (COREDES) dummies to account for variation across different settings (ballots, etc.). Ninety-five percent confidence intervals are displayed. respondents were female (54.2%) offers a more nuanced understanding of the effect of i-voting in terms of gender. Women state that they are less likely to be online-only voters, but they both vote online as well as fill in the majority of the surveys. Thus, it appears that women might simply engage more frequently both in off-line and online processes than men, without necessarily having a preference between the two.
As previously mentioned, the introduction of i-voting in PB processes is motivated by the expectation of attracting young participants. Indeed, there is a substantive relationship between age and i-voting, with 69% of 18-year-olds predicted to be online-only voters compared to 56% of 40year-olds, as illustrated in Figure 4. 23 These results are in line with theories that emphasize the role of socialization as one of the main predictors of political participation (Dalton, 2008;Hooghe, 2004;Niemi & Hepburn, 1995).
The findings of this study also support the engagement hypothesis, with online voting boosting the inclusiveness of the process by mobilizing previously inactive citizens. In fact, the largest magnitude effect predicting online-only voting is the large negative effect of having previously attended a participatory budgeting meeting. This reduces a respondent's probability of being an online-only voter by 22 percentage points. The other forms of prior political participation (taking part in community meetings and contacting government) also have large negative marginal effects (10.8 and 6.8 percentage points, respectively). Findings also indicate that citizens who are already mobilized, i.e., those who take part in physical meetings and engage with the government, are not greatly affected by the introduction of online voting. In other words, there are no major substitution effects and the introduction of online voting is more likely to increase turnout among previously disengaged citizens.
For robustness we tried several different model specifications, but none of these substantively changed the main effects (see Appendix 2). In terms of alternative explanations for the observed pattern, one is that the survey may contain measurement error. The dependent variable is a selfreported measure and less privileged groups being more susceptible to a social desirability bias might explain the observed relationship. One hypothesis is that voters with less education and lower incomes could be more reluctant to reveal truth about being an online-only voter. 24 With the current survey instrument we cannot evaluate the social desirability issue. 25 Another challenge with the research design is that the counterfactual of no online voting is a hypothetical scenario in 2012. The reason this is problematic is that the introduction of online voting might have had an effect on the deliberative process building up to the vote, especially considering that the actual vote is the final step in an over three-month-long deliberative process where many stakeholders are actively engaged. Had there been no online voting, the whole campaign surrounding the vote might have looked different, which in turn could have affected voters' attitudes about participation. It is not immediately clear how this could affect the observed results. If organizations involved in voter mobilization put in less effort due to the introduction of online voting, then many people might respond affirmatively to the "online-only voter" question, even if under conditions of an off-line-only campaign they might have been mobilized to vote. 26 However, there is no evidence of there being less voter mobilization in 2012.
In general, though, social desirability biases and errors usually make respondents claim to be more participatory and engaged than they actually are, with consistently higher numbers claiming to have voted in surveys than actually turned out (Karp & Brockington, 2005, 2005Zeglovits & Kritzinger, 2013). This means that the size of the online-only vote could in reality be even larger than estimated here.
Finally, we assess the overall turnout effect of introducing online technology. In the literature it has been suggested that allowing for Internet voting increases turnout by 0.5%-2.6% (Trechsel et al., 2007). 27 Despite the reasonable response rate in the current study, any inferences to the overall population of online voters must be made cautiously. If we ignore a possible nonresponse bias and consider the sample to be roughly representative of all online voters, we can approximate how many new voters the new voting technology brought in. As almost two thirds of the survey respondents claim to be online-only voters, the introduction of online voting could be said to have brought in 78,377 new voters, given that the total number of online voters was 124,211. We estimate the total number of voters that would have voted anyway to be 952,941, combining offline voters and online voters who said they would have voted anyway. While bearing in mind the aforementioned caveats to the study, we could consider that the increase in turnout attributable to the introduction of online voting is around 8.2%. 28 Although we cannot directly observe the attributes of those who did not respond to the survey, it is reasonable to assume that nonrespondents are less participatory than respondents on average. This would mean that the calculation of the effect of i-voting on turnout is a conservative estimate, given that online-only voters are less participatory than other online voters, as shown here. Nonresponse could also be correlated with factors that make online-only voting more likely. Respondents who spend more time on the Internet or have more time free in the day might also be more willing to take an online survey. However, these effects are somewhat less clear. Both variables had a much smaller effect size on online-only voting than the effects of prior participation. All in all, it therefore seems reasonable to believe that the estimate of the overall turnout effect is conservative, while also acknowledging that the estimate has a substantial degree of uncertainty.

Concluding remarks
This paper focused on the effects of remote Internet voting in a budget priority vote in southern Brazil. With regard to the socioeconomic profile of participants, we find that youth, male gender, income, education, and social media usage are significant predictors of being onlineonly voters. We find no substitution effects with the introduction of i-voting. Instead, our findings support the mobilization hypothesis, with i-voting attracting citizens who had no prior experience of contact with the government and who were previously disengaged from the participatory budgeting process. Note that the lack of substitution effect might be an artifact of the pilot stage of the process and the lack of significant mainstream media advertising about the possibility of casting the vote online.
Quite surprisingly, our results strongly support the engagement hypothesis. Previous levels of engagement are the best predictor of participants declaring they would not have voted without an i-voting mechanism. Finally, our model can roughly estimate an 8.2% increase in turnout due to the introduction of i-voting. But our model is mainly designed to shed light on the socioeconomic characteristics of Internetonly voters. The model is only able to investigate the effect of i-voting on turnout indirectly and maintaining a number of assumptions that we cannot test. Thus, our results on the impact of introducing i-voting on turnout should be interpreted cautiously.
When put together, these findings raise a number of questions for researchers and policymakers working with i-voting and the use of ICT in citizen engagement processes. From an academic standpoint they offer an interesting contribution to one of the oldest research questions in political science: why do people vote? Our results highlight how both socialization theory and convenience voting approaches alone are not sufficient to explain the socioeconomic characteristics of Internet-only voters. Both theories predict the presence of substitution effects, something that we do not find: the majority of our respondents are Internet-only voters. The age distribution supports the socialization framework, as most of our Internet-only voters are young. However, socialization theory also predicts the presence of significant reinforcement effects that we do not detect. Most of our participants report that they have not been previously politically engaged; this finding holds even after accounting for age and is in line with the convenience voting hypothesis. Lowering the cost of participating allows previously disengaged groups to become active in politics.
As mentioned before, the novelty of a pilot process such as this one might have something to do with these results, but only further research can uncover the extent of the impact of such a factor. Overall, this study proves the opportunity that the introduction of new technologies of voting and participation offers to enrich debates in political science.
From an institutional design perspective, the fact that many of those becoming involved in the process of online voting are socioeconomically privileged people might be a cause for concern, given that traditionally the PB process has strongly emphasized its redistributive justice character (Avritzer, 2006;De Sousa Santos, 1998). Nevertheless, much of the PB design that promotes a pro-poor logic precedes the participatory stage (Marquetti, da Silva, and Campbell, 2012), such as the preallocation of budgets that prioritize poorer geographic areas and investments that favor poorer sections of society (e.g., water, basic health).
Another issue concerns the effects of i-voting on the attributes of the process in terms of collective intelligence. One rationale for participatory processes is to leverage the dispersed knowledge of citizens to shape decisions that affect them (Lévy, 2001;Ober, 2008;Peixoto, 2013). In this respect, a growing literature in the fields of epistemic democracy and decision making suggests that increasing the diversity of participants improves the quality of decisions made, as new cognitive tools, perspectives, heuristics, and knowledge are brought together (Hong & Page, 2004;Landemore, 2013;Ober, 2008Ober, , 2013Page, 2008Page, , 2014. 29 Thus, a possible hypothesis is that, through the aggregation of inputs from a more diverse group of participants, i-voting may allow for superior choices toward the collective interest. If proven through further research, this could inform the literature on collective intelligence and temper frequent reserves with regard to the use of technology in participatory processes. However, more research is required to assess if younger but privileged disengaged groups have ideas and arguments that are significantly different from privileged engaged groups. 30 There are several possibilities for future work in this area. It would be useful to conduct additional studies of i-voting in low salience elections in other contexts to discover how generalizable our findings are. Additionally, future work could use multilevel models to examine the role of aggregate factors such as poverty levels or government spending in onlineonly voting and how these interact with the individual-level factors we examine in this paper. Bearing these considerations in mind, the effect of i-voting on the final outcomes of PB is uncertain, both in terms of redistributive impact and collective intelligence. These are essentially empirical questions that can only be addressed by further research. Striking a balance between the ideals of redistributive justice and democratic inclusiveness remains a normative issue to be addressed, ideally, by Rio Grande do Sul citizens themselves. Notes 1. Oostveen and Besselaar (2004, p. 2) define i-voting as an election system that uses encryption to allow a voter to transmit his or her secure and secret ballot over the Internet. A clear distinction between i-voting mechanisms and e-petitions and e-surveys that share similar technologies does not exist in the literature. In this essay, we use the provisional distinction that ivoting produces a binding decision on the selection of candidates or the implementation of public policies, as in the case we study. 2. This survey was the fruit of a collaborative effort involving the World Bank, the Secretariat of Planning of Rio Grande do Sul Government, and the state's data-processing company, PROCERGS. 3. Translation from original Portuguese "ciclo orçamentário do Sistema Estadual de Participação Popular e Cidadã." 4. There are many definitions of what constitutes a participatory budgeting process. At an abstract level, participatory budgeting is a democratic innovation that allows citizens to affect the formulation of a budget. Most participatory budgeting processes occur at the city level. They are based on repeated negotiations between the city government and the participants, combining elements of deliberative, participatory, and representative democracy. In order to give a more precise operational definition of this democratic innovation, Sintomer et al. (2013) include five additional criteria that distinguish participatory budgeting from other similar programs: (a) the financial and/or budgetary dimension must be discussed; (b) the city level must be involved, or a (decentralized) district with an elected body and some power over administration; (c) it has to be a repeated process (one meeting or one referendum on financial issues are not examples of participatory budgeting); (d) the program must include some form of public deliberation within the framework of specific meetings/ forums (the opening of administrative meetings or classical representative instances to ''normal'' citizens are not participatory budgeting); (e) some accountability on the output is required. 5. See Appendix 3 for full list of survey questions. 6. These costs may be manifested either materially or immaterially, such as transport costs and time spent voting. Similarly the benefits might be expressive and immaterial (general elections), or might be concrete and measurable, as in the case of the allocation of public goods in participatory budgeting. 7. It is important to notice that, while in the majority of cases i-voting is offered in conjunction with traditional forms of voting, in a limited number of cases Internet voting is the only channel of participation available (See, for instance, Peixoto, 2009). 8. Participatory budgeting processes beyond the municipal level include the cases of the Poitou-Charentes region in France and South-Kivu Province in the Democratic Republic of Congo. 9. In the State PB, the ballot is divided into three sections. In the first field, the voter can select up to four regional projects from a list of 10-20 projects (see Appendix 4 for a sample). In the second field, the voter can select two regional priorities from five possibilities. The voter is also asked to express his/her opinion on a few key policy issues, which constitutes the third field. These three fields on the ballot (cédula) are defined through a series of state, regional, and municipal deliberative meetings that involve organized civil society, common citizens, and representatives of the state government. Although the first field varies from region to region, the other two are identical statewide. The latter two fields are an innovation introduced in 2011 by the State PB. 10. See Appendix 4. 11. The COREDES were introduced in 1994 as a venue to allow civil society organizations to influence development plans for each of the 28 regions of the state. See Appendix 4. 12. See Appendix 4. 13. Some of the more sophisticated e-voting systems try to minimize the risk of so-called "over the shoulder attacks" by enabling users to change their votes after they have been cast (e.g., Estonia's electoral i-voting).
Identifying solutions for mitigating the risk of multiple voting and other factors that may distort outcomes is currently one of the activities supported by the World Bank's Digital Engagement Unit. 14. This reduced sample is partially the result of high nonresponse on the question of income, a notoriously sensitive question. To test whether this nonresponse affects our models, we reran the models without income both on the restricted sample of 18,235 that we use in the paper and a larger sample of 19,696 respondents who provided complete data except for income. Exactly the same set of parameters remained significant and in the same direction across both models and there were no large changes in effect size between the two. As a result, we conclude that item nonresponse is not driving the results in this paper. 15. The response rate is calculated based on the number of respondents compared with the total number of online voters. 16. Accuracy rates for naming the city using an IP address vary between 50%-80%; see http://whatismyipaddress. com/geolocation-accuracy (accessed May 19, 2014). 17. Graphs on survey summary statistics can be found in Appendix 2.

The proportion of females in the PB sessions in Porto
Alegre in 2009 (the last year for which there is survey data), was 55.0% (Fedozzi et al., 2013 . 28. Note that this figure is calculated from the same sample that we use for the models (i.e., only respondents who answered all the questions). If we include the full sample, then the implied turnout increase is 8.1%. 29. Part of that literature emphasizes a type of collective intelligence that emerges primarily from discursive/ deliberative problem solving (Hong & Page, 2004;Landemore, 2013), which does not apply to voting processes. Yet, another part that focuses on the epistemic properties of judgment aggregation (List & Goodin, 2001;Ober, 2008Ober, , 2013Page, 2008Page, , 2014 lends support to a consideration of the potential epistemic character of processes such as that of the State PB voting. 30. In a follow-up randomized controlled trial during the 2014 PB in Rio Grande do Sul, it was established that the result of the vote does not change when the composition of voters changes, see Mellon, Peixoto, and Sjoberg (2014). 31. The sources of this section are (a) an interview with Ricardo Almeda at SEPLAG and (b) the bylaws of the process that can be found together with other official documents pertaining to the process at: http://www. portaldaparticipacao.rs.gov.br/documentos-sistema/ 32. One additional delegate is elected if there is a residual of more than 15 people.
participants. 32 These meetings are mandated by the law to last at maximum two and a half hours.
In the third phase, all municipal delegates meet in regional forums (Fóruns Regionais da Participação Popular e Cidadã). The regional forums are augmented by the members of COREDES, members of the regional coordination commission, and the regional representatives of the PPA participativo. These assemblies take the input of the previous engagement processes and construct the ballot. More precisely, they identify 10 projects for the first field of the ballot. These projects already have a specific monetary value. Then they identify five regional priorities. These priorities do not have a monetary value and will comprise the second section of the ballot. Then two representatives are elected. These representatives will participate in the Fórum Estadual da Participação Popular e Cidadã (the state-level participatory forum). The latter forum also contains representatives of COREDES, of participants in the PPA participativo, the state level coordinating assembly, and the state government. The ballot is further revised by the state bureaucracy that eliminates projects that are not feasible.
In the fourth phase, the entire population that has an electoral certificate is invited to vote. In 2012 the vote was on July 4. It lasts one day in the traditional face-to-face format, and three days online.
In the fifth and final phase, the Fórum Estadual da Participação Popular e Cidadã uses the results of the vote to define the budget. This forum also continues to monitor the implementation of the projects together with each regional COREDES.