Abstract
Many usability practitioners conduct most of their usability evaluations to improve a product during its design and development. We call these “formative” evaluations to distinguish them from “summative” (validation) usability tests at the end of development.
A standard for reporting summative usability test results has been adopted by international standards organizations. But that standard is not intended for the broader range of techniques and business contexts in formative work. This paper reports on a new industry project to identify best practices in reports of formative usability evaluations.
The initial work focused on gathering examples of reports used in a variety of business contexts. We define elements in these reports and present some early guidelines on making design decisions for a formative report. These guidelines are based on considerations of the business context, the relationship between author and audience, the questions that the evaluation is trying to answer, and the techniques used in the evaluation. Future work will continue to investigate industry practice and conduct evaluations of proposed guidelines or templates.
Practitioner’s Take Away
- There is little guidance in the literature for the good design of a report on a formative evaluation. This is in contrast to summative evaluation reports, for which there is an international standard.
- There is wide variation in reporting on formative usability evaluations.
- The audience for a formative usability report should be carefully considered in designing the report format. The content, presentation or writing style, and level of detail can all be affected by differences in business context, evaluation method, and the relationship of the author to the audience.
- The IUSR project seeks to provide tools such as best practice guidelines and sample templates to help practitioners communicate formative evaluation results more effectively. To join the IUSR community, visit http://www.nist.gov/iusr/
