This paper compares the relative merit in realistic versus
lab style experiments for testing voting technology. By analyzing three
voting experiments, we describe the value of realistic settings in showing
the enormous challenges for voting process control and consistent voting
The methodology developed for this type of experiment will help other researchers
to test polling place protocols and administration. Comparing the results
from laboratory experiments with voter verification and realistic voting
experiments further validates the procedure of testing equipment in laboratory
The methodology and protocol for testing voting systems can be applied
to any voting technology. This protocol matches the real-world conditions
of voting by replicating them for the experiment.
Practitioner’s Take Away
- Voting systems present high stakes technology whose criteria for success
depends on usability, security, and reliability. This type of system benefits
from testing in real-world conditions to gain better understanding of
- The best practice protocol for testing voting systems in real-world
conditions include simulation of polling place, voting systems, recognizable
candidates, and real poll workers.
- Voting experiments that use regular polling places as the test venue
risk complicating the resulting data. To reduce complications, on-site
training must be done in advance. Furthermore, voting systems must also
be thoroughly tested for quality assurance in the voting test environment,
not just in the lab.
- Ballot design must be checked carefully to avoid confusion to voters
in experimental mock elections. This includes clear instructions and discussion
with participants regarding candidate choices.
- Experimental protocols using regular poll workers must include clear
training, instructions, and scripts. Studies showed that regular poll
workers can add a level of confusion and add to lack of control with data
sets if they are not trained correctly.