GAO Says 13th Congressional Machines Test OK -- but They Fail to Usability Test

The General Accounting Office (GAO) conducted functionality tests of Florida’s 13th Congressional District race, and determined that the machines appear to be functioning properly. The 13^th Congressional District came under scrutiny in 2006 when some 18,000 undervotes resulted in scrutiny from the US House of Representatives.

After determining the source code in escrow produced the same firmware, the GAO tested the function of a sampling of machines throughout the district. They concluded that “(1) the machines properly displayed, recorded, and counted the selections for all test ballots cast during ballot testing involving 112 common ways a voter may have interacted with the system, and (2) the deliberately miscalibrated machines, though difficult to use, accurately recorded the ballot selections as displayed on screen.”

The GAO notes, however, that “the large undervote in Florida's 13th Congressional District race could have been caused by factors such as voters who intentionally undervoted, or voters who did not properly cast their ballots on the iVotronic DRE, potentially because of issues relating to interaction between voters and the ballot." (Our emphasis)

We are Disappointed

UPA’s Voting and Usability project members are dismayed that the GAO missed their opportunity to promote usability testing as a means to answer those questions, as well as the likeliest method for preventing serious usability obstacles in ballot design.

Usability testing would have helped identify the likely interaction problems real voters would encounter, including the likelihood that voters might fail to vote in the US House race. Although the GAO has an expert review that suggests this problem could have been predicted, usability testing would confirm and quantify those indications.

What’s more, we wish the GAO would promote usability testing before elections to help prevent such problems. The federal Voluntary Voting Systems Guidelines and certification can only cover the capabilities of a voting system. The ballots for each election are produced by the local election officials, and require separate, easy to implement testing, such as the testing method available through the LEO Usability Testing kit.

More about the GAO report: http://www.gao.gov/new.items/d08425t.pdf

More about LEO: http://www.upassoc.org/civiclife/voting/leo_testing.html;

More about the Voting and Usability Project: http://www.upassoc.org/civiclife/voting/index.html

More about the VVSG: http://www.eac.gov/voting%20systems/
voting-system-certification/2005-vvsg

More about Banner Blindness in Ballot Design - Jakob Nielsen, Feb. 2007
http://www.useit.com/alertbox/banner-blindness-ballot-design.html