Usability Testing of Mobile Applications: A Comparison between Laboratory and Field Testing

Peer-reviewed Article

pp. 4-17

Abstract

Usability testing a mobile application in the laboratory seems to be sufficient when studying user interface and navigation issues. The usability of a consumer application was tested in two environments: in a laboratory and in a field with a total of 40 test users. The same problems were found in both environments, differences occurred in the frequency of findings between the contexts. Results indicate that conducting a time-consuming field test may not be worthwhile when searching user interface flaws to improve user interaction. In spite of this, it is possible that field testing is worthwhile when combining usability tests with a field pilot or contextual study where user behavior is investigated in a natural context.

Practitioner’s Take Away

  • When testing a user interface of a mobile application, field
    testing may not necessarily be the best place; mostly because it is more
    time consuming than the lab test.
  • Testing in the field requires double
    the time in comparison to the laboratory. In the field you can run half
    of the tests per day you run in the laboratory.
  • In a field test, running a pre-test or a pilot is critical: there are so many details that can go
    wrong, and you really need to check that everything is working correctly.
  • When testing in the field, be prepared that things will not go as planned:
    there may be interruptions and unexpected events more than in lab.