Abstract
This paper outlines some of the most important of Nigel Bevan’s many contributions to the evolution of the changing concept of usability and in particular to Nigel’s input into, and leadership of, the development of usability-related international standards. It traces the concept of usability from the very early days when the focus of human-computer interaction (HCI) was predominantly on hardware ergonomics in the context of office work, through the much wider, more recent, concepts of designing for user experience, and finally designing for high quality human experience. It also presents the history as well as several freely available resources for usability professionals outlining what usability is nowadays, how it can and should be measured, and criteria for designing to support people of all ages and abilities. Finally, it acknowledges Nigel’s long-standing involvement in the UXPA, culminating in him being posthumously honored with the UXPA Lifetime Achievement Award in 2018.
Keywords
Usability, usability definitions, user experience, International
Standards Organization (ISO) standards, accessibility, UsabilityNet,
UsabilityPlanner, Usabililty Body of Knowledge
Introduction
I first met Nigel Bevan at a meeting hosted by the National Institute of Standards and Technology (NIST) in March 1998 in Gaithersburg, Maryland. The result of that meeting and subsequent work was the Common Industry Format for usability test reports (NIST, 1999). This evolved into ISO Standard ISO/IEC 25062:2006—Software product Quality Requirements and Evaluation—Common Industry Format (CIF) for usability test reports (ISO, 2006).
Nigel’s contributions to usability standards is probably what many people know about, and other articles in this special issue focus on that work. But I want to take a broader view and look at his legacy through his own publications and contributions to the usability community. Nigel was a prolific writer; I count over 50 publications in just my collection of his papers. His publications began as early as 1981 with his first article that I’ve seen (Bevan, 1981) in which he studied the effects of presenting text at different speeds on a computer screen. He found that the optimum presentation speed for understanding and retention was 10–15 characters per second. This early study reflects his focus on the user and on measurement that are hallmarks of his work.
The rest of this article will be organized around three main areas of Nigel’s publications and contributions: Defining Usability, Measuring Usability, and the Usability Body of Knowledge (BoK).
Defining Usability
People newer to the field may not realize that there wasn’t always agreement about what usability is. Although the term has been in use for many years (Sauro, 2013; Soegaard, 2012), it started to be more commonly used in the manner we use it today in the mid-1980s, perhaps largely due to the introduction of computers that individuals could afford. One of Nigel’s early papers (Bevan, 1982) reflects the terminology in common use „pre-usability,“ with a focus on human factors and ergonomics. But he clearly is talking about what we have come to characterize as usability:
As costs have decreased, computers have become widely used by people with no previous experience of them. Unfortunately, many systems are still designed more for the convenience of the programmer than for the user. This problem will become more acute as further cost reductions lead to even more widespread use of computers in professional applications… (Bevan, 1982, p. 75)
Nigel and co-authors Jurek Kirakowski and Jonathan Maissel tackled the topic head-on in 1991 with a paper entitled simply: „What is Usability?“ (Bevan, Kirakowski, & Maissel, 1991). They proposed the following:
Usability should be defined as the ease of use and acceptability of a product for a particular class of users carrying out specific tasks in a specific environment. Criterion levels for measurements of attitude and user performance determine whether the design of the product is successful in achieving usability. (Bevan et al., 1991, p. 1)
They go on to conclude the following:
Usability lies in the interaction of the user with the product or system and can only be accurately measured by assessing user performance, satisfaction and acceptability. Any change in the characteristics of the product or system, user, task or environment may produce a change in usability. A product is not itself usable or unusable, but has attributes which will determine the usability for a particular user, task and environment. (Bevan et al., 1991, p. 4)
These attributes of usability are widely recognized by most usability or UX professionals today. But they didn’t just appear out of thin air; Nigel and others helped to define them.
In 1995, Nigel built on the topic further by introducing „quality of use“:
Quality of use should be the major design objective for an interactive product: does the product enable the intended users to achieve the intended tasks? This relates usability to business objectives and elevates usability from an optional extra to the prime design goal. (Bevan, 1995a, p.1)
This relationship of usability to business objectives is something that has become critical in more recent years but was a relatively new concept in the mid-1990s. He goes on to conclude the following:
The objective of usability is to achieve quality of use. Usability requirements should be stated in terms of the effectiveness, efficiency and satisfaction required in different contexts. User-based evaluation can be used to validate achievement of these requirements. (Bevan, 1995a, p. 7)
As the field began to mature, Nigel focused more on the user-centered design process and on inclusive design, such as his chapter on „Quality in Use for All,“ in which he proposed the following:
Design for All entails both physical and cognitive accessibility. New hardware and software technologies are required to make it easier to provide physical accessibility. New integrated approaches to system development are required to make it easier to provide cognitive accessibility. Only by combining these activities can Design for All be achieved. (Bevan, 2001, p. 353)
He also elaborated on his observations of the usability and accessibility communities:
Currently, there are two separate professional communities concerned with improving product accessibility and product usability. Both of them share the objective of meeting user needs in order to achieve quality in use. However, whereas usability has been primarily concerned with the range of „typical“ or „average“ users (by implication able-bodied), accessibility is concerned with extending design to incorporate users with physical and cognitive disabilities. User-centered design can provide a common framework for enhancing current design practice, in order to meet the real needs of both these majority and minority user groups. (Bevan, 2001, p. 366)
This focus on accessibility along with usability is something Nigel continued for many years, such as his 2009 collaboration with Helen Petrie, „The evaluation of accessibility, usability and user experience“ (Petrie & Bevan, 2009). They also recognized the entry of „user experience“ into the lexicon as a more overarching term:
User experience (often abbreviated to UX) is the newest term in the set of criteria against which an eSystem should be evaluated. It has arisen from the realization that as eSystems become more and more ubiquitous in all aspects of life, users seek and expect more than just an eSystem that is easy to use. Usability emphasises the appropriate achievement of particular tasks in particular contexts of use, but with new technologies such as the Web and portable media players such as iPods, users are not necessarily seeking to achieve a task, but also to amuse and entertain themselves. Therefore the term user experience, initially popularized by Norman (1998), has emerged to cover the components of users‘ interactions with, and reactions to, eSystems that go beyond effectiveness, efficiency, and conventional interpretations of satisfaction. (Petrie & Bevan, 2009, p. 20-3)
As consensus began to emerge in the field about what usability and user experience actually are, Nigel focused more on the user-centered design processes that can help in achieving those goals. Some of this work was reflected in the UsabilityNet.org website (Bevan, 2003). Figure 1 shows the Methods Table from that website. This was the first attempt that I’m aware of for making detailed information about usability methods freely available online. Unfortunately, the site appears to have been taken down at some point in 2018, but I know that I was one of many who used the site frequently and pointed others to it. Nigel’s willingness to freely share his knowledge was at the core of his work.
Figure 1. Screenshot of the Methods Table from the UsabilityNet.org website, about 2018.
Finally, in collaboration with Xavier Ferre and Tomás Antón Escobar, even more detailed information about the selection of user-centered design methods for various stages of a project was embodied in the Usability Planner (Bevan, 2009; Ferre & Bevan, 2011; Ferre, Bevan, & Escobar, 2010). Thankfully, this tool is still available online at UsabilityPlanner.org (Figure 2).
Figure 2. Screenshot of the homepage of UsabilityPlanner.org (2019).
Measuring Usability
Anyone who knows me knows that I’m a bit of a UX metrics geek. I’m not sure if Nigel thought of himself that way, but if his publications are any indication, it’s pretty clear he was. His focus on measurement went hand-in-hand with his work on defining usability and user-centered design processes. You don’t know if something is usable unless you can measure it.
Some of Nigel’s early work on usability measurement was as part of the European initiative entitled Metrics for Usability Standards in Computing, or MUSiC (Bevan, 1992). He stated the following:
The usability of a product for a particular task for specified users and environments can be most concisely summarized by one measure in each of the following categories:
• Effectiveness: the completeness and accuracy of the goals achieved
• Efficiency: the relevant resources expended (e.g., time, money, mental effort)
• Satisfaction: the user’s feelings about the work system (Bevan, 1992, p. 124)
Of course these have become commonly accepted ways of measuring usability, reflected in ISO standards (ISO, 2006) as well as books devoted to usability/UX measurement, including one I happen to be pretty familiar with (Tullis & Albert, 2013). Nigel and his colleague at NPL, Miles MacLeod, went on to provide a much more detailed treatment of how to measure these attributes in their 1994 paper in Behavior and Information Technology, „Usability Measurement in Context“ (Bevan & MacLeod, 1994).
One of the things that impresses me about Nigel’s publications is that he didn’t limit himself to traditional human factors or HCI journals or conferences. For example, his 1995 paper on „Measuring usability as quality of use“ was published in the Software Quality Journal (Bevan, 1995b), helping to get the usability measurement message to the computer science and quality assurance communities. In that paper he spelled out some of the benefits of measuring usability:
Specifying and measuring usability as quality of use provides several potential benefits:
• Unless usability is an objective criterion in the requirements specification, there is often little incentive to put resources into designing for usability;
• Measuring against usability objectives provides a means of judging how much further work (if any) on usability is required in order to reach the objectives;
• It provides a means of establishing benchmarks and making comparisons with alternative designs, with earlier versions of a system, or with competing products. (Bevan, 1995b, p. 8)
Nigel also made sure to speak to those of us in the UX community about practical issues in measuring usability. For example, he pointed out that we need to be careful to consider various contexts for measuring usability:
Resource constraints mean that usability testing is typically carried out in the most common context of use. But many important usability issues arise only in less-common contexts:
Learnability: usability measured for the task of achieving adequate performance, for example, by completing a training course or through the use of learning materials.
Accessibility: usability measured for users with particular disabilities.
Universality: usability measured in a range of different contexts and cultures.
Risk: usability measured in situations that may have business or personal risk. (Bevan 2006, p. 43)
While much of Nigel’s early work on usability measurement focused on traditional task-oriented metrics like effectiveness and efficiency, his later work clearly incorporated a broader view of user satisfaction. For example, he stated that satisfaction as defined in ISO standards needs to be broadened . . .
. . . to include achieving pleasure through the attainment of hedonic goals and the experience of use. Satisfaction is broken down into four sub-characteristics: purpose accomplishment, trust, pleasure and comfort. The new definition should promote a broader interpretation of satisfaction. (Bevan, 2010, p. 246)
An excellent example of this broader view of measuring UX is provided by Nigel’s collaboration on a „Comparison of Kansei Engineering and AttrakDiff to Evaluate Kitchen Products“ (Bevan, Liu, Barnes, Hassenzahl, & Wei, 2016). This was also a fascinating departure from Nigel’s more traditional work in computer systems and the web to a consumer product—specifically a range hood. Kansei Engineering is a way of analyzing affective perceptions of a product and incorporating them into design (e.g., Nagamachi & Lokman, 2011). A Kansei approach tends to be somewhat resource-intensive because it is tailored to each type of product. AttrakDiff, on the other hand, is a generic 28-item semantic differential scale intended to measure two broad dimensions: hedonic quality and pragmatic quality (Hassenzahl, Burmester, & Koller, 2003). They found that there was a high correlation between the Kansei results and the AttrakDiff results in this case study, with the Kansei approach yielding richer and more specific feedback but also requiring more resources.
Usability Body of Knowledge (BoK)
The earlier discussion of Nigel’s work on UsabilityNet.org and the Usability Planner highlighted his passion to share information about usability and user-centered design methods freely and widely. That desire was carried even further by his work on the Usability Body of Knowledge, or BoK (UsabilityBoK.org). This was a project sponsored by the User Experience Professionals Association (UXPA). Many people contributed to the BoK, but Nigel was certainly one of the driving forces behind it, serving as its Managing Editor. Thankfully, as of this writing, the BoK is still available online. The homepage is shown in Figure 3.
Figure 3. Homepage of the Usability Body of Knowledge.
It’s hard to give the reader of this article a good sense of the scope of the BoK, so I encourage you to look at it yourself. But to try to convey a bit of the scope, Figure 4 shows the topics included in just the „Methods“ section of the BoK.
Figure 4. Methods section of the Usability Body of Knowledge.
The Usability Body of Knowledge is a tremendous resource that everyone from novices to experts in UX and usability can learn from and make use of. I hope that some people in the UX community will step up to updating and expanding this extremely valuable work.
Conclusion
In addition to his publications and his contributions to standards, Nigel contributed to the usability profession, UXPA, and the worldwide UX community in many ways:
- Serving on the UXPA International Board from 2002 to 2007
- Playing a key role in getting the annual World Usability Day started
- Acting as UXPA’s Regional Director for Europe, the Middle East, and Africa
- Serving as UXPA’s Co-Director for Outreach in 2004 with Elizabeth Rosenzweig
- Serving as UXPA’s Director of Professional Development
- Acting as Managing Editor of the Usability Body of Knowledge
Because of these contributions to UXPA, Nigel was presented with the UXPA President’s Award in 2014. (See Figure 5.) Finally, because of his contributions to the field in general, he was posthumously awarded the UXPA Lifetime Achievement Award in 2018.
Figure 5. Nigel accepting the UXPA President’s Award in 2014.
References
Bevan, N. (1981). Is there an optimum speed for presenting text on a VDU? International Journal of Man-Machine Studies, 14(1), 59–76.
Bevan, N. (1982). Psychological and ergonomic factors in machine translation. In V. Lawson (Ed.), Practical experience of machine translation. North-Holland Publishing Company.
Bevan, N. (1992). The MUSiC methodology for usability measurement. In Posters and short talks of the 1992 SIGCHI conference on Human factors in computing systems – CHI ’92 (p. 123). New York, USA: ACM Press.
Bevan, N. (1995a). Usability is quality of use. Proceedings of the 6th International Conference on Human Computer Interaction, Yokohama, Japan, July 1995. Anzai & Ogawa (Eds.), Elsevier.
Bevan, N. (1995b). Measuring usability as quality of use. Software Quality Journal, 4, 115–150.
Bevan, N. (2001). Quality in use for all. In C. Stephanidis (Ed.), User interfaces for all: Methods, concepts and tools (pp. 353–368). Mahwah, NJ, USA: Lawrence Erlbaum Associates, Publishers.
Bevan, N. (2003). UsabilityNet methods for user centred design. Human-Computer Interaction: Theory and Practice, 1, Proceedings of HCI International 2003 (pp. 434–438).
Bevan, N. (2006). Practical issues in usability measurement. Interactions, 13(6), 42–43.
Bevan, N. (2009). Criteria for selecting methods in user centered design. Proceedings of I-USED 2009, Uppsala, Sweden.
Bevan, N. (2010). Extending the concept of satisfaction in ISO standards. KEER2010, Paris, March 2-4, 2010, International Conference on Kansei Engineering and Emotion Research.
Bevan, N., Kirakowski, J., & Maissel, J. (1991). What is usability? In Proceedings of the 4th International Conference on HCI, Stuttgart, Germany, September 1991.
Bevan, N., Liu, Z., Barnes, C., Hassenzahl, M., & Wei, W. (2016, May). Comparison of Kansei Engineering and AttrakDiff to evaluate kitchen products. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 2999–3005). ACM.
Bevan, N., & MacLeod, M. (1994). Usability measurement in context. Behaviour & Information Technology, 13(1-2), 132–145.
Ferre, X., & Bevan, N. (2011). Usability planner: A tool to support the process of selecting usability methods. In P. Campos et al. (Eds.), INTERACT 2011, Part IV, LNCS 6949, pp. 652–655.
Ferre, X., Bevan, N., & Escobar, T. A. (2010, October). UCD method selection with usability planner. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 829–830). ACM.
Hassenzahl, M., Burmester, M., & Koller, F. (2003). AttrakDiff: A questionnaire to measure perceived hedonic and pragmatic quality. In J. Ziegler & G. Szwillus (Eds.), Mensch & Computer (pp. 187–196). Stuttgart, Germany: Vieweg+ Teubner Verlag.
International Organization for Standardization (2006). Software product Quality Requirements and Evaluation (SQuaRE) — Common Industry Format (CIF) for usability test reports. (ISO Standard No. ISO/IEC 25062:2006). Available at https://www.iso.org/standard/43046.html
Nagamachi, M., & Lokman, A. (2011). Innovations of Kansei Engineering. (Published in English in 2011). Boca Raton, FL, USA: CRC Press.
NIST (1999). Common Industry Format for Usability Test Reports (v 1.0). Retrieved February 5, 2019, from https://zing.ncsl.nist.gov/iusr/documents/cifv1.0.htm .
Petrie, H., & Bevan, N. (2009). The evaluation of accessibility, usability and user experience. In C. Stephanidis (Ed.) The universal access handbook (pp. 20.1–20.14). Boca Raton, FL, USA: CRC Press.
Sauro, J. (2013). A brief history of usability. Retrieved February 5, 2019, from https://measuringu.com/usability-history/
Soegaard, M. (2012). The history of usability: From simplicity to complexity. Retrieved February 5, 2019, from https://www.smashingmagazine.com/2012/05/the-history-of-usability-from-simplicity-to-complexity/ Tullis, T., & Albert, W. (2013) Measuring the user experience: Collecting, analyzing, and presenting usability metrics. Morgan Kaufmann/Elsevier.
