Progressive Washington State: Training local elections officials to usability test ballots

When I met Nick Handy, director of elections for Washington State, at a reception in March 2007, I almost made him spill is drink.

“So, Dana, what do you do?”

“I study how people interact with products,” I said.

Nick seemed curious. “Give me an example.”

“Well, a lot of what I do is testing web sites for how usable they are. I observe people who are like the target audience trying to use the sites to see where they get stuck. But I’m also doing similar research about ballot design.”

Nick smacked his forehead with his free hand, turned to his friend John Lindback (director of elections for Oregon), and said, “I can’t believe we’ve never thought of doing that before!”

Regional training for county elections officials

Nick was so excited that he returned to his hotel room to fire off an enthusiastic email to his staff about incorporating usability practices in the election cycle.

But first, could we include training about usability in the upcoming regional training?

Ballot design, usability, and accessibility: May 22 in Olympia and May 24 in Spokane

On each of two days in May 2007, about 70 county local elections officials gathered for regularly scheduled regional training. Libby Nieland (from the state department of elections) presented on ballot design first. The second half of the day was set aside for Patty Murphy and Kay Ramsey (also from the state DofE) to talk about accessibility in voting.

In between, in what I think is a first in the U.S. (maybe the world), I had the privilege of getting to spend two hours teaching a total of 140 local elections officials how to do usability testing. I focused on testing ballots, but the enthusiastic attendees quickly understood that they could use this tool for lots of other things, such as finding solutions to known issues they have with return envelopes or instructions on ballots.

It was a proud day for Washington State and me – and my fellow usability professionals. Get a copy of the training materials that I used. Send me feedback about them.

Developing the LEO (local elections officials) usability testing kit

A little more than year ago, Whitney Quesenbery had a crazy idea: What if someone taught local elections officials to do their own usability testing of ballots?

The MSU-UPA working group

Whitney called together a few people who thought it was an interesting idea. In April 2006, the “LEO kit” was born. LEO stands for Local Elections Official.

We worked together over a couple of days at Michigan State University’s Usability and Accessibility Center to scope out a usability testing “kit” to be used by people who have no specific training on usability or human factors engineering.

Then we went off to work on things separately and remotely, trading drafts and review comments through a wiki.

Designing a test protocol for people who aren’t usability professionals

We had a lot of questions:

• What do LEOs need to know to be able to conduct an effective usability test?
• How can we help LEOs understand the immediate and rippling benefits of usability testing their ballots?
• How can we give LEOs a great tool without adding significantly to their workloads?
• When in the elections cycle should ballots be tested?
• How many participants should LEOs include in each test?
• What constraints do LEOs have?

The pieces of the kit

We eventually agreed on three pieces for the kit:

• generic test plan that acts like a “how to” for planning usability tests of ballots
• template session script, which provides direction for moderating individual usability testing sessions, complete with what to say, when
• final report template to use to report results internally to elections departments or to the public.

We hope you’ll try them and send us feedback on how they worked for you. We will incorporate comments and post revisions regularly. If you want to be notified about updates to the LEO kit, let us know.