Volume 16, Issue 3, May 2021

Introduction to Volume 16, Issue 3

Our May 2021, invited essay is “The Best of Times for UX Research, the Worst of Times for Usability Research?” by Jason Buhle.  He describes how the landscape of usability and UX research has been changing and provides recommendations for increasing the academic and professional standing of usability research. 

In addition to the essay, this issue includes three methodological research papers: one on persona development, one on a guideline-based inspection method, and one summarizing the standardized usability questionnaires currently available to UX researchers and practitioners.

The first article is “Creating Personas from Design Ethnography and Grounded Theory,” by P. J. White and Frank Devitt.  They describe a method and associated case study for persona creation in three phases: Post fieldwork data management, Coding and grounded theory, and Data reduction and data display.

The second article is “GCS: A Quick and Dirty Guideline Compliance Scale,” by Lukas Lamm and Christian Wolff.  They investigated the development of compliance scales for a guideline-based inspection method, comparing problem discovery and experiential ratings between guideline reviews and conventional usability testing in an automotive case study.

In the third article, “A Review of Post-Study and Post-Task Subjective Questionnaires to Guide Assessment of System Usability,” Andrew Hodrien and Terrence Fernando summarize the standard usability questionnaires that are available for use, using this information to guide selection of which are the best to use in different research contexts. 



Invited Essay:
The Best of Times for UX Research, the Worst of Times for Usability Research?

These are booming times for the field of UX research. Looking broadly at the field of UX professionals, Jakob Nielsen (2017) estimated growth from about 10,000 people in the late 1990s to about 1,000,000 in 2017. Data from Google Trends indicates that UX research has grown even more rapidly than the broader UX field in recent years: Searches for the term “UX research” have suddenly risen several-fold since early 2016, while over the same period searches for UX have less than doubled. The result of this growth is clear—UX research is now a large field, with about 36,000 professionals on LinkedIn, and a popular one, ranked as the 39th best job in America by CNN and Payscale (Braverman, 2017). Job postings on the site Indeed portend that that the field will continue to grow rapidly, with over 1,000 open positions currently listed.

UX is a broad area, encompassing a person’s perceptions and responses when using or anticipating use of a product, system, or service (Bevan, 2009). An important component of UX is usability (Rusu et al., 2015). Usability is generally understood to describe how well users can achieve specified goals when interacting with a product, system, or service, excluding the dimensions of affect, sensation, meaning, and value that are included in UX more broadly (Law et al., 2009). While the prospects for the field of UX research as a whole appear as bright as ever, I fear that the future of usability research specifically may be dimmer. [Read More]



Creating Personas from Design Ethnography and Grounded Theory

and Frank Devitt

Personas can be an effective means of communicating and synthesizing design ethnographic field data by helping designers maintain focus on users and make sense of complex needs. Personas have however been criticized on scientific grounding and methodological weakness in their creation. Furthermore, few detailed examples of their creation have been published. It is also evident from the literature that there is a methodological gap in creating personas from ethnographic studies. 

Using a grounded theory approach, this paper offers a systematic and detailed process of creating personas post fieldwork and compares the approach against best practice. Through a case study of design ethnography in product development for people over 65, we reflect on and describe the process in three phases: 1. Post Fieldwork Data Management, 2. Coding and Grounded Theory, and 3. Data Reduction and Data Display, which leads to the final design of the persona. We conclude that using qualitative and grounded theory approaches can assist to strengthen the process of creating personas; however, the time to create and manage the personas increases. [Read More]


GCS: A Quick and Dirty Guideline Compliance Scale

Expert-based usability evaluation methods offer valuable alternatives to traditional user testing in Human-Machine Interaction (HMI) development. While general measures of usability for user-based empirical studies are well-known throughout the community of researchers, expert-based approaches often lack such general measures of usability. This research introduces the Guideline Compliance Scale (GCS), a measure that can be applied during guideline reviews to assess the overall level of usability. Several guidelines relevant for the system being evaluated are rated by the evaluators according to their compliance.

In the case study for our research, an automotive user interface was empirically evaluated in a user study as well as a guideline review with experts. The usability problem lists, which form part of the output, were made comparable by classification using the Usability Problem Classifier (UPC). An in-depth analysis revealed differences and similarities in the problem identification of both applied methods. Comparing the results of the GCS from the guideline review with the results of the System Usability Scale (SUS) from the user study, regarding the overall level of usability, showed similar results for both scales. [Read More]



A Review of Post-Study and Post-Task Subjective Questionnaires to Guide Assessment of System Usability

Usability is a key consideration when developing an interactive software application because of the various outcomes it can produce. Accordingly, numerous evaluation methods have been proposed, however, a recent review of usability methods concluded there is no current consensus on models applied to usability. While questionnaires are a commonly used measure, it is unclear which questionnaire(s) are most appropriate for a given context, so new usability researchers face confusion over which to select. To aid questionnaire selection, the appropriate type (post-study or post-task), general structure and content, specific advantages and disadvantages, aspects of usability covered, and psychometric quality should be considered. This paper presents a literature review and analysis of general post-study and post-task usability measures. Questionnaires are weighed up and discussed on each aspect, so practitioners can gain a holistic overview and identify relative strengths of individual questionnaires within their questionnaire type. Overall recommendations and specific questionnaire suggestions are provided for guidance, along with how future research can expand the review. [Read More]