Why is it so hard to produce a usable, well-designed ballot?

This form changed the world.

Miami-Dade presidential ballot from 2000, the "butterfly ballot"

The picture is of the so-called "butterfly ballot" from Miami-Dade County from the presidential election in 2000. It is called a "butterfly ballot" because of how the candidates for this office flow over onto the second page of a two-page spread. The designer of this  punch card ballot wanted to make the type large enough for her overwhelmingly older voting constituency. This caused the contest to flow to two pages. That caused the candidates to interlace across the two-page spread. The holes are meant for every other one to the left or every other one to the right. There are horizontal rules to call out the candidate pairs and arrows to point to the holes. If you use trifocals, and you're in a garage with bad lighting, or a high school gym where there's a lot of glare on the page, how might the alignment go for you? Also, it isn't hard to imagine a voter poking the first hole for the first candidate on the left. Then you must poke the second hole for the second candidate - right?

This intentional-but-ill-informed design caused people to vote in ways they had not intended. It caused enough voters to make mistakes that it changed the outcome of a federal election. Which, because this election happened in the US and it was to elect the president, changed the world. This is not unlike the butterfly of the Chaos Theory.



Democracy is a design problem

Whenever I tell people that I work in voting and election design, I get two questions. The first is, So, is there money to be made there? (No.) The second question is, Why is this so complicated?

The people who ask the second question usually have an answer to offer me, already. The solution, they say, is that there should be one voting system for the whole country. This would impose consistency that could be supported with standards, testing, and enforcement. But it isn't that simple.

By tradition, running elections falls to the states and counties by virtue of the 10th amendment to the US Constituion, which says that anything that isn't covered in the Constitution falls to the people. It is considered a "states' rights" issue. All the Constitution says about elections is that there will be such to elect people to offices. Later amendments say who can vote (15th - barring discrimination based on race or color; 19th - womens' suffrage; 24th - eliminating the requirement to have paid income taxes; 26th - establishing 18 years as the legal voting age). Nothing says anything about who determines what system to use. It falls to the states.

The multiplicity of voting systems is just one tiny slice of this wicked problem. As with other design problems, there are constraints. In the case of ballot design, there are several that interact: 

• Voting technology is a moving target, so standards and best practices always lag. 

• Election management systems are reprehensibly difficult to use. EMSs, into which databases of candidate filings and questions or measures must be poured to make ballots are so difficult that many county election officials just send their databases in to their voting system vendors to do the ballot layouts for them. 

• Design specifications and language for instructions are embedded in county and state election legislation. Type font, weight, and size, grid, and position of instructions are often specified in state election code. Election regulations also often include the exact wording of instructions. It's not uncommon for the instructions to have been written generations ago, in negative, threatening, passive voice. 

• Election directors are excellent public administrators but they're not trained designers. In most of the 3,000 or so counties in the US, the people who run elections are county clerks or registrars who handle vital records such as birth certificates. Most are women, who, on average have held that job for 20 years. They usually are not tech savants, but they don't fear tech, either. They are busy, burdened, and budgetless. Elections have become more and more complicated to administer. Even if they could use InDesign to lay out their ballots, they're not trained designers. For many, a "usable" ballot is one that can be counted accurately by the voting system. And they want to keep costs as low as possible. Printing, mailing, upgrades, bug fixing, translations, storage -- all this costs money. 

• Ballot templates are issued at the state level. It is typical for the secretary of state, as the head of elections, to issue what's called a "ballot template" for state and federal elections. These also come from people who aren't trained designers and don't take into account the things that can happen when county and municipal contests are added to the ballot. They might not make room for multiple languages. They rarely put ballots through usability testing before live testing on Election Day. 

• Municipal and county districts overlap to create what are called "ballot styles." For example, there are places in Washington State where you could possibly have a unique ballot. There -- as in many voting jurisdictions throughout the US -- many lower level contests are included in the ballot, from school board to cemetery commission. The boundaries for those districts have been drawn in dozens of different ways. The right combination could draw a circle around your house. And yet, the county election official must ensure that you get to vote on exactly the contests you are entitled to. For this reason, some counties end up generating hundreds of ballot styles as different levels of districts overlap.

Poor ballot design affects the outcome of elections

When ballots are badly designed, voters get frustrated. People lose confidence in elections. Supporting elections on Election Day becomes difficult for poll workers.

All voters are affected by poor ballot designs. Older voters, first time voters, some minorities, and voters who have less education are very likely to make mistakes that prevent them from voting as they intend. Even white, wealthy, educated voters make mistakes on ballots. That's what happened in 2000. 

Although the butterfly ballot became the emblem for bad ballot design, we continue to see ballot design problems, both in paper ballots and on electronic touch screen systems. Technology has introduced more design problems. It has not solved them.


Voting: the 233-year-old design problem
There are best practice guidelines, commissioned by the US Election Assistance Commission from AIGA's Design for Democracy project, that are evidence-based. Voting system manufacturers are gradually supporting more and more of the guidelines, as local election officials demand it. States are updating election code to loosen design requirements. Local election officials embrace these changes. Although change can be difficult, these particular changes can make the jobs of local election officials easier because the voter's franchise is more likely to be protected with every design improvement.

Design can change the world. This is our superpower. We can affect the accuracy and accessibility of elections. But there aren't nearly enough people who are interested in civic design. Join the movement at the most important panel in the Free World, at South by Southwest in Austin on Monday, March 14 at 3:30 Central time. (Follow #uxvote on Twitter.) There we'll have on hand Dana Debeauvoir, county clerk from Travis County, Texas with Ric Grefe, the executive director of AIGA and the Design for Democracy project, along with Larry Norden, a civil rights lawyer and senior counsel for the Brennan Center for Justice at NYU. I'll be moderating. See you there. And at poll worker training for the next election.

New York should have piloted their new voting system

It would have been so easy. So inexpensive. So quiet. Run a mock election, learn some lessons, talk with election officials from other places about how they transitioned from one voting system to another. As Larry Norden of the Brennan Center for Justice at NYU remarked on WNYC on November 2, "49 other states have done this before us." But the New York Board of Elections did none of this, and the changing of voting systems seemed to be on the legislative agenda not at all. Instead, the Board and all the counties in New York did the equivalent of live usability testing in the primary in September and November.

New York has known for several years that the state would have to ditch the generations-old mechanical lever machines for new voting systems. There was plenty of time to prepare for this historic transition. And there were plenty of people around who could have helped. Key people offered. And, there are plenty of resources the state board of elections could have tapped. There are evidence-based federal guidelines for election design. There are thousands of experts on changing voting systems across the US. Oh, and there are dozens of local design experts who the state and city boards of elections could have called on.

But none of that happened. Why? For the purposes of narrowing the discussion, let’s just focus on ballot design.


Why the New York ballot is a disaster
While on its face, ballot design seems like a simple thing to do well, it is actually a complex mix of constraints and interactions. There's the fact that the software for creating ballots is usually so difficult to use that many jurisdictions outsource it back to the voting system manufacturer. Next, most people who design ballots are not trained in design, they're trained to be election administrators. Part of that is controlling costs. Local election officials are smart, well intentioned people. But if it comes to deciding between expanding a ballot to two cards to make it easier to read and use, or keeping everything on one card because it saves the county money on printing (and mailing) by making the font smaller and the spacing tighter, many election officials are going to go with smaller, tighter type.

It would be completely unfair to blame the design of the New York ballot on local elections officials, though. In fact, a lot of ballot design is out of control of the local elections official. The responsibility here lies with regulations, history, and bureaucracy. Typefaces and wording for instructions are often specified in election regulations. Community history and norms make up traditions that are embedded in local culture. And, many ballot design decisions are made at the state level, usually without consideration of usability data. Let’s talk about just those factors for a moment.


Some design problems are embedded in legislation
According to New York election law, the ballot must be "full face". That means that everything to be voted on must appear on one side of the ballot. (The November ballot actually violated this; the ballot questions were on the back of the candidate contests.) There's some logic to this, I suppose; New Yorkers have been voting on lever machines for a long time. A major change in the look of the ballot might throw voters off completely. But I doubt that.

In addition, the instructions are legislated in the election code. This is very common across the US. Some states are more flexible on this than others, even though the wording is legislated. New York codified its instructions without regard to the final ballot design. And the final ballot design didn't take into account what the legislated ballot instructions must say. Instruction (2) reads: “To vote for a candidate whose name is printed on this ballot fill in the oval above or next to the name of the candidate.” But the bubbles for candidates appear below the names.

New York sample ballot for November 2010

Tradition also influences ballot design
New York uses fusion voting. That means that although one party nominated a candidate, other parties can endorse a candidate as theirs. For example, on the 2010 midterm ballot, Andrew Cuomo appears as the Democratic candidate for governor, but also as the nominee of the Independence party and Working Families party. If you really want Cuomo to be governor, might you mark him in all three places? That’s called a double vote. But only one counts.

Mechanical lever machines prevent double voting. Officials doubted that voters would make this mistake on paper ballots, but there’s nothing to prevent it happening. In fact in a large usability test on October 9, 2010 sponsored by UPA, AIGA's Design for Democracy project, and the Brennan Center for Justice at NYU, we found that double voting was not fiction. What about taking away fusion voting to prevent these kinds of voter errors? Fusion voting is legislated, but it is also deeply embedded tradition. Removing the option probably would have devastating effects on smaller but highly active political parties.


Bureaucracy gets in the way
The usability test on October 9 was conducted by a group of concerned volunteers, for free. Why didn’t the New York Board of Elections do a test? This is a mystery. The Brennan Center and UPA offered months ago, but the BOE declined, saying that they had spent all the money they were going to spend on the new systems. And, though the board had been warned about likely problems, the state BOE entered the state primary election in September 2010 in denial. That Election Day clearly showed the predictions of problems to be true, and fusion voting wasn’t even part of the ballot in the primary. In response to appeals for simple, inexpensive remedies, the Board said it was too late to make changes that they said required the voting system to be recertified and too late to make other changes because of how close the November federal election was.




What the design community can do about it
One of the wonderful things about working in election design is how easy it is to affect immediate, positive change. There is a woeful lack of involvement from the design community in election UX. Your country, your state, your town all need you. Here’s what to do:

Meet your local elections officials. Mayor Bloomberg and Governor Cuomo are not going to come to you. You have to go to them.  Let yourself be known. If you live outside New York County, get yourself to the county election department and introduce yourself. Ask for time with the election director. Ask questions, and then listen. Make yourself available to them without expectation of monetary profit. (There’s very little funding for design and testing in local governments right now.)

Do civic duty. Sure, you vote in every election, but have you ever worked an election? There is so much to learn by being a poll worker about how election administration works and how voters vote. Immersion holds many lessons.

In most communities, there are also citizen advisory committees that work on voter outreach and voter education. When I lived in San Francisco, I was on a committee that wrote summary digests of ballot measures for the Voter Information Pamphlet.

Yes, these things can take time away from work. But what’s more important: a little less income, or doing your part for world peace?

Read the guidelines. There’s a lot of guidance on how to design for elections. You might start with Marcia Lausen’s book, Design for Democracy. It gives a high-level very Design view of ballot design. The Brennan Center’s report, Better Ballots gives 13 case studies that show that specific ballot design elements may have affected election outcomes. AIGA’s Design for Democracy project has compiled a very good collection of tools and resources. Read through what’s there.

Get to know the LEO Usability Testing kit. The Usability Professionals’ Association realized years ago that there weren’t enough usability specialists to test all the ballots out there. What to do? Give local election officials (LEOs) tools for doing testing on their own. The LEO Usability Testing Kit is in wide use in many counties across the US, but it’s also a great tool for volunteers to use to run mock election usability tests.

Make election UX a topic at local community meetings. After you get your feet wet in the election UX world, tell the story to other designers. Have them vote a badly designed ballot and talk about that experience. Encourage them to get involved.



Where the New York ballot is going from here
After a little media attention for sloppy design that might have prevented voters from voting as they intended, counties and states usually get the message. Florida is no longer the poster child for messy elections, Ohio inherited that honor for the 2008 presidential election. Now, its new sibling New York is competing strongly to outdo Ohio. Let’s hope that New York wakes up and reforms its election board and administration.

In the meantime, the New York State Inspector General conducted an investigation of the reported issues with the September 2010 primary. As Larry Norden of the Brennan Center for Justice wrote, "It is remarkably well-written, and it is sinister, sad and comical all at once." 

After the primary, the Brennan Center filed a law suit against the New York State Board of Elections, "over a discriminatory New York State policy for counting political party votes under a procedure known as 'double voting.' ... The policy unfairly penalizes both the voters and the minor parties they support." You can read a bit about the suit on the Brennan Center web site.

- Dana Chisnell
The opinions expressed here are Dana's only, and not the officially sanctioned position of UPA.

Dear New York, please let me correct my ballot

The Usability Professionals' Association (UPA) Voting and Usability Project and the Brennan Center for Justice are urging the New York State Board of Elections to change the interaction and a message on the state's new voting systems.

Today, the New York Times joined the discussion in an editorial.

The issue: New York is changing from using mechanical lever machines to optical scan ballots. This change is good. However, with New York's particular ways of voting that include cumulative voting, it seems likely that some voters will vote too many times in a given contest. Florida saw some of this with a similar configuration; after some counties changed the system configuration to return overvoted ballots to voters, overvotes went down while the number of ballots cast remained constant.

Right now, the systems are configured to hold the ballot in the tabulator when it detects an overvote or undervote, while showing the voter a message on a small screen.

  The message voters get when they mark their ballots for too many candidates on a contest, while the tabulator holds the ballot

This would be remedied by a clear message from the tabulating machine as it pushes the ballot back out for the voter to review and change, if she wants. But these systems hold the ballot while the voter reads a poorly worded message to decide what to do. 

The New York State Board of Elections argues that the tabulator returning the ballots automatically will slow voting. The Brennan Center and UPA argue that when the machine holds the ballots, voters are more likely to let the flawed ballot be cast. This means that vote on that contest will not count, which is likely to disenfranchise thousands of voters and call results into question.

Was Alvin Greene picked by voters trying to eliminate errors on their ballots?

As a post on the Newsweek site pointed out on June 18, usability of a voting system may have influenced yet another election. This time, it's the Alvin Greene victory in the Democratic Senate primary in South Carolina last week. We think it's likely that one particular feature of the user interface may have caused some voters to change their vote: representing undervoted contests on the review screen in red.

Proximity and placement in the ballot could have made a difference

We know that placement in the order of candidates does influence the likelihood of getting votes. (According to Michael Alvarez colleagues at the Voting Technology Project at CalTech, candidates that appeared above and below Arnold Schwarzenegger in the California recall of Gray Davis in 2003 received more votes that they probably normally would have if they'd appeared elsewhere in the list of 150 candidates for governor: http://vote.caltech.edu/drupal/node/26).

Placement in the overall ballot also makes a difference. There is roll-off in later contests mainly because voters don't know much about the candidates that appear lower down on the ballot, such as judges. 

Using a computer to vote introduces usability issues with navigating ballots

Using electronic voting systems introduces another layer of usability issues in marking and casting ballots as voters must navigate the ballot differently from how they use a printed, optical scan ballot. All electronic voting systems show a summary review screen at the end of all the contests. It lists all of the contests with the choices the voter made for each. The information is color coded: On most systems a completely voted contest appears blue; an under- or un-voted contest appears red. There is mixed research about how much attention voters pay to this screen. One thing we do know is that some voters are seriously disturbed by the red messages. 

In one study, red on the summary screen of an electronic ballot caused some voters to change their votes

In a study that Janice ("Ginny") Redish and I conducted for the National Institute of Standards and Technology (NIST) in 2008 (http://www.nist.gov/itl/vote/upload/NISTIR-7556.pdf), we observed that many of our 45 study participants wanted so much to remove the "error" that appeared on the summary review screens for undervoted contests that they sometimes resorted to extraordinary measures to get the system to remove the red entries on their ballot review screens.

The voting system we used in our study behaved similarly to commercially available systems. That is, study participants could select candidates using a touch screen and move forward or back in the ballot by touching appropriate buttons. When they got to the review screen, they could touch the contest they wanted to review or change to go back to it. This is different from the South Carolina system, which forced voters to review undervoted contests before casting the ballot. Although the flow of our system was more straightforward, study participants chose to go back into the ballot to change the red contests to blue.

Some participants voted for candidates they were not otherwise interested in voting for; at least one participant entered blank write-in votes (by going to the write-in for a contest and putting in a blank space) to make the red entries in the review screens turn blue. We observed 17% of our study participants asking questions, expressing concerns, and changing votes because the red color bothered them so much.

-- Dana Chisnell

Technology as a solution in 1957: "Behind the Freedom Curtain"

This 1957 film housed in the Internet Archives in the Prelinger Archives is a tutorial on how mechanical lever machines work -- with a little chauvinistic flag-waving thrown in for good measure. It is very much a period piece (remember HUAC?), but also does a good job of explaining the kinds of mistakes that voters make on paper ballots. The Automatic Voting Machine Co. that made the film makes the case that those mistakes can't be made on the lever machine. See: http://www.archive.org/details/Behindth1957.

Go from that film to the results of a review of the November 2008 election in Florida, where 15 counties switched from direct record electronic (DRE) voting machines to paper, optical scan ballots. Twice as many ballots were rejected in 2008 than were rejected in 2004. But still, the rejection rate was 0.75%, well below the 2.9% in 2000, and below the "expected" residual vote rate of about 1% on average. From the New York Times on February 26, 2009:

The final report sent to state lawmakers showed that 0.75 percent, or 63,680 of the 8.39 million ballots cast in Florida, did not count in the presidential race. Barack Obama defeated John McCain in Florida by more than 236,000 votes. In the presidential race four years earlier, the rate of uncounted ballots was 0.41 percent, equating to about 31,000 votes.

Read the whole New York Times article here: http://www.nytimes.com/2009/02/26/us/26florida.html?nl=pol&emc=pola1.

In 2004, George W. Bush defeated John Kerry in Florida by 380,978 votes. In 2000, Al Gore received 543,895 more individual votes nationally than George W. Bush, but Bush received more electoral votes. In Florida in 2000, the margin of victory was 0.0092%, in favor of Bush. (See: http://en.wikipedia.org/wiki/United_States_presidential_election,_2000 and http://en.wikipedia.org/wiki/United_States_presidential_election,_2004.)