A Methodology for Testing Voting Systems

Peer-reviewed Article

pp. 7-21Download full article (PDF)

Abstract

This paper compares the relative merit in realistic versus
lab style experiments for testing voting technology. By analyzing three
voting experiments, we describe the value of realistic settings in showing
the enormous challenges for voting process control and consistent voting
experiences.

The methodology developed for this type of experiment will help other researchers
to test polling place protocols and administration. Comparing the results
from laboratory experiments with voter verification and realistic voting
experiments further validates the procedure of testing equipment in laboratory
settings.

The methodology and protocol for testing voting systems can be applied
to any voting technology. This protocol matches the real-world conditions
of voting by replicating them for the experiment.

Practitioner’s Take Away

  • Voting systems present high stakes technology whose criteria for success
    depends on usability, security, and reliability. This type of system benefits
    from testing in real-world conditions to gain better understanding of
    the issues.
  • The best practice protocol for testing voting systems in real-world
    conditions include simulation of polling place, voting systems, recognizable
    candidates, and real poll workers.
  • Voting experiments that use regular polling places as the test venue
    risk complicating the resulting data. To reduce complications, on-site
    training must be done in advance. Furthermore, voting systems must also
    be thoroughly tested for quality assurance in the voting test environment,
    not just in the lab.
  • Ballot design must be checked carefully to avoid confusion to voters
    in experimental mock elections. This includes clear instructions and discussion
    with participants regarding candidate choices.
  • Experimental protocols using regular poll workers must include clear
    training, instructions, and scripts. Studies showed that regular poll
    workers can add a level of confusion and add to lack of control with data
    sets if they are not trained correctly.