Corrections to citations

In my last posting I had mistakenly cited Michael Alvarez's book, Information and Elections, when I should have cited his 2002 article, "Ballot Design Options."

Also, I cited Scott Roberston, but he was actually citing Bederson, et al. 2003, "Usability of large scale public systems: Electronic voting system usability issues."

Both of these mistakes are corrected in the post below.

Common Ballot Usability Problems

While reading over the research I have gathered so far, I have come across a few sources from which I could extract some common usability problems with ballots. Here they are:

Bederson, et al. 2003, "Usability of large scale public systems: Electronic voting system usability issues."

• Unable to mark properly
• Unable to write in properly
• Readability

• Font size
• Font design
• Conspicuousness

• Comprehensibility
• Ability to associate a selection box, button, lever or punch hole with the right choice
  (This apparently happens on optical scan ballots, as well. See Alvarez, et al. 2006. The Complexity of the California Recall Election in which the authors discuss "vertical proximity" where names "neighboring" the most prominent candidates received a disproportionately large number of votes, suggesting that even "randomized" name order is not neutral and the effect is probably due to alignment problems -- all of which relates to the next two items.)
• Order effects
• Position effects
• Cues important for reminding or decision making (e.g., political affiliation)
• Voting instruction
• Error prevention and correction
• Diversity, e.g., language, disability
• Image and icon design
• Residual voting (failure to mark votes for candidates or issues)

Roth, Susan King. 1998 (reprint 2000). Disenfranchised by design: Voting systems and the election process. Information Design Journal 9,1,1-8

• Ease of use
• Accessibility
• Effective communication of information on the ballot
• Ability of the system to reduce voter error
• Potential to reduce voting fall-off (undervoting)
• Time to vote
• “…context of use – it presents information used in decision-making, under the pressure of time, in a controlled environment.”

Alvarez, 2002. Ballot Design Options.

• Amount of information (number of contests/issues) on one card/screen
• Navigating through the online ballot
• Difficulty getting an overview of the ballot and the voter’s choices

In addition, local elections officials I have talked with cite these issues as they review ballots and collect feedback from poll workers after election day:

• Undervoting
• Overvoting
  
• Not voting both sides of an optical scan ballot
• Font is too small (they provide magnifying glasses in the polling places)
  
• Not knowing how to mark the ballot (voters fail to connect arrows, or circle names or mark with Xs rather than connecting the arrow or filling in the oval)
  
• Erasing on a paper or optical scan ballot
• Have difficulty finding candidates in rotation order
• Writing in unqualified candidates
• Marking a candidate and writing them in
• Ranked choice (instant run-off) voting is problematic

The lists are similar, but what conclusions can we draw? And how can we solve these problems? I hope that upcoming research by Design for Democracy and other work sponsored by NIST will reveal solid answers.

What the process is really like

Barbara, my contact in the San Francisco Department of Elections, responded quickly to the flow diagram I sent to her for comment. She said it was "pretty accurate" and "close to what happens" but that "the beginning stages are not nearly as smooth as in the diagram. For example, right now there's a two-way flow of information between me and ES&S: I am sending them bits of updated information as I get it, and they are sending me ballot drafts with holes for the information that I still need (e.g., translations)."

I suspected this might be the case. I haven't figured out a reasonable way to represent that iterative, flexible exchange very well. I'm open to suggestions at dana@usabilityworks.net.

What local elections officials control on ballots

On August 7, 2006, I met with two people from the San Francisco Department of Elections to learn about the ballot design process. Barbara Carr, who is a clerk in the department, sees the ballot and the Voter Information Pamphlet (VIP) through the entire process from gathering information for the pieces at filing time all the way through printing and distribution. Linda Tulett is the deputy director of elections.

San Francisco uses arrow-connecting optical scan ballots in ES&S Eagle machines for regular and absentee ballots. The City uses oval fill-in optical scan ballots with an AUTOmark machine for HAVA compliance.

I had worked with Barbara in my role as a member of San Francisco's Ballot Simplification Committee.


Constraints from the state
Before our meeting, Barbara had pointed me to the sections of California state elections code that guide (and often constrain) the design and instructions on the ballot. (See Division 13 in the code.) In my light review of the code, I could see that there were some strange and interesting design constraints dictated. For example,

...immediately to the right of and on the same line as the
name of the candidate, or immediately below the name, if there is not
sufficient space to the right of the name, there shall be printed in
eight-point roman lowercase type the name of the qualified political
party with which the candidate is affiliated.

Emphasis mine.


The code goes on to allow three words to describe a candidate's "designation" such as a current office held or other role. There are several paragraphs on what constitutes a "word" in this case. For example, if the candidate wants to include a place name that is made up of two words (such as San Francisco), that counts only as one word.

These are just two examples of the type of specificity of the elections code. I was surprised that the state elections code dictates items to this level of detail, but have learned that this is true in most other states.

Constraints from the City
From the City are ordinances that cover what languages must be on the ballot, in what order. In San Francisco, there are three languages on most ballots: English, Spanish, and Chinese. All primary ballots must have at least two languages, so they are set up to be English/Spanish and English/Chinese.

When Barbara, Linda and I talked about type size, layout and order, Barbara and Linda talked about some of the trade-offs between ease-of-reading and budget concerns. If you have larger type, the ballot may flow over to more cards. The more ballot cards you have, the more it costs to print, to mail to absentee voters, and to create provisional ballots that may not be used.

There are many other local ordinances that constrain control of design of ballots, as well.


Constraints from the voting machine manufacturer
To ensure that the ballot cards are properly scanned, there must be tracking blocks at the edges to tell the scanner which end is up and what ballot card is going through. This takes up space, but generally wasn't a concern.

I was interested to find out that the City doesn't have the ballot layout software and that someone from the voting machine manufacturer lays out the ballot. Usually that person is doing that work outside the Department offices, but sometimes -- usually depending on how close to deadline time it is -- the vendor is on site at City Hall.

I suppose that the logic is that having the expert -- someone from the manufacturer -- doing the layout ensures that the process is quicker and less likely to introduce errors than if someone in the Department did the work. Learning this left me wondering though if there were things that could be done with the ballot layout software that the people in the Department didn't know about:

• Could the instructions be positioned differently?
• Could the contests be grouped differently visually?
• Could there be more images that might help reading-disabled voters?

Etc.


What the process looks like
Out of that interview, along with some helpful materials that Barbara gave me, I drafted a process flow diagram for the ballot design process. Try double-clicking the image below to see a larger version; if you want a better copy, email me. I have sent it back to Barbara for her to review. It includes the timing in the election cycle for the major steps.



It was interesting to me that "usable" to Barbara and Linda, the local elections officials, meant that the ballot met all of the legal requirements and had nothing to do with how well it might support carrying out the voter's intent.


The good news: The wording of the instructions isn't constrained
As far as Barbara and Linda could find in the code and ordinances that they have to check the ballots and VIP against, the wording of instructions can say anything they feel is appropriate. That is, that wording isn't dictated by any rules they have to follow. They were interested in doing what they could to improve instructions on ballots and liked the idea that there might someday be research to support the decisions they make about how instructions are worded.

Rumaging through others' research

For the last several days, I've been looking for primary research about ballot design. Of course, since it is handy, I've been looking on the Web for this information. A few interesting things have turned up. One irrefutable conclusion: There is no research on ballot instructions.

There have been several studies on the usability and accuracy of electronic voting systems (Bederson, et al.; Herrnson, et al.; Killam). Some cover accessibility of electronic voting systems (Burton & Uslan). One group has looked at the usability of paper ballots (Everett, Byrne and Greene). Usefully, one team has looked at issues for reading-disabled voters (Goler & Selker). The sources of many of these reports are listed in the links on the right side of this page.

Only Ginny Redish has started to look at instructions on ballots in work she did for NIST at the end of 2005 and the beginning of 2006. See her reports, available here:
http://vote.nist.gov/instructiongap.pdf
http://vote.nist.gov/032906PlainLanguageRpt.pdf

When I can, I'll post a minimal annotated bibliography with links to the reports I have.

I've also found book titles on topics that may be related to ballot design, but the printings seem to be limited; I haven't been able to find copies locally to me:

• Neimi, R. G. and Herrnson, P.S. Ballot design: How to improve life at the ballot box.
  
  
• Election Center, National Task Force on Accessible Elections; National Organization on Disability. Voting, a constitutional right for all citizens: a guidebook to assist election officials to achieve equal access for all citizens to the polling place and the ballot.

There's also a poster:
University of Minnesota Design Institute. Voting by design: A communications map of the voter

If anyone out there has access to these documents and can send them to me for review, please email me at dana@usabilityworks.net.

Why this blog

As far as I know, there is no blog about the usability and accessibility of ballots. There are a few people working on this area -- aside from doing research about voting machines -- and I invite those people doing research to talk to one another about it.