Volume 15, Issue 4, August 2020

Introduction to Volume 15, Issue 4

Our August 2020 issue features an essay from Douglas Gillan, “Usability Issues in Human-Robot Interaction.” This essay describes the variety of robots with which humans currently interact and with whom we will interact in the future. To avoid the “productivity paradox” that followed the introduction of computers to the general workplace, it is very important to identify and address the usability problems that will surely arise as human interactions with robots become more common.

In addition to the essay, this issue includes three research papers, two on professional UX work practices and one on the most widely used method for assessing psychometric reliability of standardized questionnaires. […] [Read More]

 

Invited Essay: Usability Issues in Human-Robot Interaction

The term “robot” comes to us from the Czech artist Josef Capek and his playwright brother, Karel, who wrote R.U.R. (Rossum’s Universal Robots; 1921). They coined the term “robot” from the Czech word robota, which denotes labor by servants. The play, R.U.R., was about a world in which biological androids, robots, performed the menial tasks formerly done by humans. The term robot soon was used to describe any machine that did the work, especially physical labor, that otherwise might be done by humans. In recent years, robot has been generalized to include machines that perform both physical and cognitive tasks originally performed by humans. In addition, the term robot has been shortened to “bot” to indicate any autonomous system designed to behave like a human or so that humans interacting with the bot might mistake it for a human.

Although human-bot interactions are interesting in and of themselves, this paper focuses on robots as we normally think of them—machines capable of sensing the physical environment and acting on the environment by interpreting input from humans and providing feedback to humans. Within this framework, these machines (a) have various levels of autonomy, from teleoperation by a human to full autonomy; (b) may operate in various environmental contexts, for example, in air, on land, or on and under water; (c) may be fixed in place in that physical environment or may move around the environment; (d) may have an appearance that ranges from very machine-like to humanoid; and (e) have uses in health care, military operations, disaster relief, education, entertainment, transportation, or construction.

How do humans interact with robots? The answer to this question is as varied as the types of robots, environments in which they work, and the tasks that they perform. […] [Read More]

 

How Professionals Moderate Usability Tests

This paper reports how 15 experienced usability professionals and one team of two graduate students moderated usability tests. The purpose of the study is to investigate the approaches to moderation used by experienced professionals. Based on this work, we present our analysis of some of the characteristics that distinguish good from poor moderation.

In this study, each moderator independently moderated three think-aloud usability test sessions of Ryanair.com, the website of a low-fare European airline. All moderators used the same six usability test tasks. The test sessions were video recorded so that both the participant and moderator were visible.

Key observations were identified by asking other study participants to review a random video from each moderator. Each video was reviewed by five to seven study participants. With this approach, the data (not a single person, author, organizer, or moderator) determines what the key observations are.

This study documents a wide difference in moderation approaches. In this paper, we discuss several common issues in usability test moderation, including time management, prompts and interventions, moderator interaction styles, and the provision of positive participant feedback during sessions.

This study is the tenth in a series of Comparative Usability Evaluation studies. […] [Read More]

 

Positive Developments but Challenges Still Ahead: A Survey Study on UX Professionals’ Work Practices

This paper describes and gives an overview of User Experience (UX) professionals’ work practices—their environment, practices, tools, and challenges. First, we reviewed 32 empirical studies about usability and UX work to identify key issues in usability and UX work practices. For the identified key issues, we collected data from 422 UX professionals surveyed in five different countries using a comprehensive questionnaire with 62 questions. Our results show that UX professionals individually know about usability and UX concepts, methods, and tools. They typically employ between one and five Human Computer Interaction (HCI) theories on average and use one to three different techniques and tools. On the organizational level, UX is involved from early to late stages and is generally well known within all levels of the organization. On the country and community level, UX professionals generally do not report themselves as belonging to a professional community, despite the fact that the survey was administered via channels of the respective communities in the survey countries. Overall, this survey shows that UX professionals have considerable work experience and strong UX expertise self-confidence. This may be considered as indicating a positive development of the UX profession. […] [Read More]

 

On the Usage of Cronbach’s Alpha to Measure Reliability of UX Scales

If, as UX professionals, we measure the user experience of a product with a questionnaire, then scale reliability is important. If the scales of the questionnaire show low reliability, we cannot guarantee stable measurements and should not base important design decisions on such data. There are several methods available to estimate the reliability of a scale. But nearly all UX questionnaires use Cronbach’s Alpha as an indicator for scale reliability, even though this coefficient is not so easy to interpret. Several general methodical concerns and problems regarding the interpretation of Alpha are known. But these concerns are mainly discussed in highly technical publications concerning measurement theory and seem to be widely unknown by UX practitioners. In this paper, I show that there are some UX specific issues to use Cronbach’s Alpha as a measure for scale reliability. I discuss five problematic points concerning the usage of Alpha that can easily cause misinterpretations of data in practical applications and validation studies of UX questionnaires. This paper should help UX practitioners to better interpret the Alpha values published in papers describing the application and construction of UX questionnaires.In this paper, I show that there are some UX specific issues to use Cronbach’s Alpha as a measure for scale reliability. I discuss five problematic points concerning the usage of Alpha that can easily cause misinterpretations of data in practical applications and validation studies of UX questionnaires. This paper should help UX practitioners to better interpret the Alpha values published in papers describing the application and construction of UX questionnaires. […] [Read More]