Abstract
Through the standardization of ISO 13407:1999, I became acquainted with Nigel Bevan at ISO TC159/SC4/WG6 meetings. During those meetings and at various conferences (including UPA, now UXPA, and HCI International), we discussed the concept of satisfaction, usability, and later, UX. The model of quality characteristics that I proposed in 2015 (Figure 1) had become the core of our discussion. In the model, the quality in design and the quality in use are distinguished, and the objective quality and the subjective quality are also distinguished, thus forms four quality domains. The concept of usability is included in the objective quality in design whereas the concept of UX is related to the whole of quality in use. Furthermore, the effectiveness, efficiency, and satisfaction (sub-concepts of usability in ISO 9241-11) are separated to objective quality and subjective quality.
Nigel showed a strong interest in my model and asked me to send him a copy of the article. He just sent me back a revised proposal of the figure, and we continued to discuss the structure of quality, especially its implication to the UX. We also focused our discussion on the concept of satisfaction especially in its relation to UX. Nigel was very interested in the concept of Kansei in relation to satisfaction. He invited me to organize a workshop “Kansei Engineering and Emotion Design – a Research Agenda” at the KEER 2018 conference that was held in Kuching, Malaysia. It was a successful workshop, but after the conference, he went to hiking in the northern mountain area where he passed away.
Keywords
Nigel Bevan, usability, UX, satisfaction, Kansei, ISO 9241-11,
ISO/IEC 25010, Quality Characteristics, ERM (Experience Recollection Method)
Nigel Bevan and ISO Standards
In the 1990s, I was working for Hitachi Ltd. as a user interface researcher specializing in cognitive engineering. Later, I switched to usability engineering during which time I presented a paper on “apparent usability” at ACM SIGCHI. In 1996, I became involved with standards activities via the ISO TC159/SC4/WG6. At that time, ISO 13407:1999 concerning human-centered design and usability was being drafted. I also met Nigel Bevan, Tom Stewart, Susan Harker, and others at the SC4/WG6 meeting in London. Because I was relatively new to the field and a new member of the standards committee, my contribution to the discussion at that meeting was somewhat limited. For example, I did not know the difference among WD (Working Draft), DIS (Draft International Standard), FDIS (Final Draft International Standard), and other terms.
After the ISO13407 became an International Standard and it was translated into Japanese as JIS Z8530:2000, I learned much about the standardization process and understood the content of ISO 13407, but, at the same time, I began to feel that both ISO 13407 and ISO 9241-11:1998 were not entirely appropriate. I was concerned about the structure of the usability concept that embraces the sub-concepts of effectiveness, efficiency, and satisfaction. I thought it was strange that satisfaction is deemed to belong below the overall concept of usability. To me, satisfaction is an experiential outcome of useful functionality, good performance, the aesthetic appeal of color and design, and so on, as well as high levels of usability. In other words, to me, satisfaction in the concept structure is superior to those aspects identified above. At the following meeting, I expressed my personal opinion, which other members of the committee agreed individually but did not accept at the meeting because the concept of usability had already been defined in ISO 9241-11:1998.
In 2001, I asked Nigel about the approval process in the definition of the concept of usability in ISO 9241-11. Nigel kindly located the relevant documents and facts, and he forwarded the following to me:
I have found the report of: ISO TC159/SC4/WG5, WG5 Usability Assurance sub group, London Meeting, 2-3 February 1988. The sub-group meeting was attended by Donald Anderson, Nadia Bertaggia, Nigel Bevan (editor), Eva Brenner Wallius, Fred Brigham, John Brooke, Susan Harker, Leif Hedman, Chris Marshall (chair), and David Youmans. After a lengthy discussion, he continued, the following working definition was agreed upon: The usability of a product is the degree to which specified users can achieve specified goals in a particular environment effectively, efficiently, comfortably and in an acceptable manner.
Thanks to Nigel, I realized that the concepts of effectiveness and efficiency incorporated in ISO 9241-11 were initially adverbs and that the origin of satisfaction was the adverbial phrase “comfortably and in an acceptable manner.” His reply made me think more deeply about the concept of satisfaction. This adverbial phrase means the manner of goal achievement and does not mean the concept of satisfaction in its wider sense. Hence, changing the phrase into a simple noun “satisfaction” in ISO 9241-11 was somewhat different from the original idea at the 1988 meeting. In other words, the concept of satisfaction in a broader sense could be located somewhere higher and should be treated independently in the relevant concept structure. This idea led me to propose the model shown in Figure 1.
That prompted me to begin to advocate for the importance of the concept of satisfaction and my own concept structure in academic conferences. Nigel frequently attended conferences including the annual UPA (now UXPA), ACM SIGCHI conferences as well as INTERACT, APCHI, KEER, and HCI International at which he gave tutorials related to ISO standards. Owing to his promotional activity, the usability concept of ISO 9241-11 and the concept of Human Centered Design in ISO 13407 have gradually gained recognition as a substantial standard in Europe, North America, Asia, and Oceania.
It was at the HCI International 2001 conference in New Orleans that Nigel and I embarked on a personal discussion on issues concerning usability and satisfaction concepts, and, later, the concept of UX. I found him frank and flexible but also a stern defender of the definitions in the ISO standards. We continued to discuss the concepts of usability and user engineering. I considered the latter the kind of theory that I had in mind and wrote a book about later (Kurosu, 2016).
We also discussed the possibility of a usability professional certification system with Jonathan Earthy and Asa Granlund, and we proposed that to the UPA board. Unfortunately, the board declined our initial proposal to devise a certification system because they considered the field to be too immature at that time. Subsequently, in 2009, the Human-Centered Design Organization (HCD-Net) in Japan began to tackle a certification process under the name of Certified HCD Professional. By the end of 2018, it has bestowed the title of Certified HCD Professional to 556 individuals.
Nigel drove me to the venue of an ISO meeting in Washington D.C. at which we discussed the difference between formative and summative usability evaluations as well as the difference between “small” and “big” usability in the context of the proposed Common Industrial Format (CIF). In 2010, he kindly invited me to the Dagstuhl seminar on Demarcating User Experience that finally resulted in the “UX White Paper” (Roto, Law, Vermeeren, & Hoonhout, 2011). It was the time when various definitions on UX were proposed and many different views on UX were discussed; some talked about UX from a marketing perspective, some talked from a usability perspective, some talked from a design, and so on. Although this white paper did not bring its own definition of UX, it classified the nature of UX, gave a clear temporal model of UX, and proposed what should be done for UX and how. In particular, its temporal model gave me the insight for proposing a UX evaluation method—Experience Recollection Method (ERM; Kurosu, Hashizume, Ueno, Tomida, & Suzuki, 2016). And my ideas on UX could take both the tentative form of the ERM and the spatial form of the Quality Model (Figure 1) with the mutual reference. Around that time, I refined the structure of my UX theory that I finally presented at HCI International 2015, entitled “Usability, Quality in Use and the Model of Quality Characteristics” (2015). Evidently, Nigel was very interested in it, proposing that we meet again the next morning for about an hour to discuss my conceptual model.
A Conceptual Model of Quality Characteristics
In this section, I outline my conceptual model of quality characteristics highlighting why I believe Nigel responded to it as enthusiastically as he did and then began to incorporate it into his own thinking. The model was influenced by ISO/IEC 25010:2011 as well as by ISO 9241-11:1998. It comprises, as shown in Figure 1, two major sections: one, the quality in design and the other, quality in use. Quality in design comprises a set of quality characteristics that engineers and designers might apply to ensure the best possible quality of their products, systems, and services (or the artifacts in general). It should be noted that many quality characteristics in this category have the suffix “ability” indicating that these characteristics represent a potential rather than being the result of usage. On the other hand, quality in use comprises a set of measurable quality characteristics of use that can be applied once the artifact is being used in a real context. Using the real context for evaluating the UX is quite important. The usability test is an evaluation method for usability as a part of the quality in design and cannot be used for evaluating the quality in use nor the UX. It is the evaluation method for the quality in design and uses the usability lab in most cases. But the use of a usability lab will not trigger the natural behavior of users, while the UX evaluation should be conducted in a natural setting. Furthermore, users’ motivation is different. In a usability testing situation, the artifact that will be tested will be provided by the testing side. But in a real situation, users purchase the artifact based on their actual needs and own expense. Hence, the evaluation of UX is completely different from the usability evaluation.
Other dimensions that divide those qualities are the objective and the subjective quality characteristics. Objective characteristics can be measured on a physical and objective scale such as the time and frequency of occurrences; subjective characteristics can only be assessed via psychological methods such as rating scales.
Figure 1. Model of quality characteristics that explains the location of usability, satisfaction, and UX.
As a result, usability is one of the objective quality characteristics in design, whereas UX is related to the entire set of objective and subjective quality characteristics. Thus, these concepts are completely different. It should be noted that, in this model, effectiveness, efficiency, and satisfaction are not sub-characteristics of usability; rather these include recognizability, memorability, learnability, and so on. Effectiveness and efficiency are included in the quality in use in ISO/IEC 25010. Motoei Azuma informed me that Nigel, as a member of SC7 of JTC1, insisted that these two as well as the concept of satisfaction should be included not in the quality in design (in the standard it’s called “product quality”) but in the quality in use. In my model, however, as should be clear now, satisfaction is considered not an objective quality in use but rather a subjective quality in use.
The objective quality in design simply influences the objective quality in use, and the subjective quality in design also influences the subjective quality in use. Furthermore, the perception of objective quality in design and the perception of objective quality in use influence the subjective quality in use. Satisfaction is therefore considered to be a generic concept in the characteristics that belong to the subjective quality in use. In other words, satisfaction is the result of all quality characteristics and represents the total quality of artifacts during the usage.
Regarding UX, user characteristics, satisfaction, and context of use are all related. These define the relative weighting of each of quality in design whenever the user uses an artifact. For some artifacts, a user may weight usability heavily but weight performance more heavily in a different context of use, but another user might weight these issues differently.
I see several possible reasons for Nigel’s interest in this model. One is that I accepted his proposal for ISO/IEC 25010; another is that I clearly separated the concepts of usability and UX. A third reason could be the fact that I clearly defined the concept of satisfaction as a subjective quality in use. Finally, I demonstrated a close relationship between satisfaction and UX.
Issues with the Concepts of Satisfaction and UX
Among the three usability sub-concepts, Nigel was most interested in the concept of satisfaction in relation to UX. He might have been dissatisfied with the early definition of satisfaction in ISO 9241-11:1998, namely “freedom from discomfort and positive attitudes towards the use of the product” that was used until ISO 9241-210:2010. And he re-defined the term in ISO 9241-11:2018 as the “extent to which the user’s physical, cognitive and emotional responses that result from the use of a system, product or service meet the user’s needs and expectations.” Note the similarity of the latter definition with that of UX in the same version of ISO 9241-11: the “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service.” In other words, it appears that Nigel, as the editor for ISO 9241-11:2018, might have thought that a strong relationship exists between UX and satisfaction. Coincidentally, my model (2015) emphasizes the relationship of satisfaction to UX.
Kansei Engineering and KEER 2018 Conference
Nigel’s first appearance at a Kansei Engineering and Emotion Research (KEER) conference was in 2010 in Paris. Since that time, he attended all the conferences in 2012, 2014, 2016, and 2018. He denied the word “sensibility” that can be found in the dictionary translation of Kansei which he pronounced as /kan-zei/ not as /kan-sei/. He said that “Kansei is something more complex and delicate than sensibility.” As the international conference, KEER is supported by several local communities of which one is the Japanese Society of Kansei Engineering and another is the European Kansei Research Group. He was elected to be the next chair of that group.
KEER 2018 was held in Kuching, Malaysia. Nigel suggested that he and I run a workshop entitled “Kansei Engineering and Emotion Design – a Research Agenda” together. Our suggestion that many issues concerning Kansei Engineering should be pursued in the future appears to have been borne out.
Figure 2. Nigel and the workshop participants, including me, at KEER 2018 (Kuching, Malaysia).
At the closing ceremony of the conference, he showed the manuscript of ISO/IEC WD 25010-3 v5 and pointed to a figure that included behavioral outcomes, attitudinal outcomes, and consequences of use under the heading of quality in use. He showed me that effectiveness and efficiency are included in behavioral outcomes, while satisfaction is separately included in attitudinal outcomes. He asked me if I was satisfied with this, to which I answered, “Yes! Separating satisfaction from effectiveness and efficiency is a good idea.” Quite unfortunately, this was the last conversation I had with him. A few days after the conference, he went hiking at Mt. Mulu where he sadly fell.
Involvement with Japan
Nigel frequently visited Japan and I asked him twice to give a lecture, which he willingly accepted. Those lectures took place at critical moments for the Japanese HCD activities. He gave the first talk when the concept of usability was attracting attention among industry people and the second when UX was becoming a buzzword in Japan.
Because the Japan Ergonomics Society strongly supported SC4/WG6 along with many people in industry and academia, this meant that they received breaking news about the Standard the moment it was released to the general public. As a result, many people became familiar with Nigel’s active involvement in standards work.
Personal Recollections
Once I asked him “Why don’t you write books in addition to the standards?” He replied, “For me, it is most important to be involved in the development of standards that will influence the whole world.” Indeed, Nigel did write a lot. For example, ISO 9241-11:1998 consisted of a mere 28 pages; ISO 9241-11:2018 grew to some 36 pages. He added many new concepts into these standards, such as service, accessibility, user experience, and sustainability, and he coined the term “human-centered quality.” While necessary, some of these new terms still need to be specified and referred to more in the document, but it is clear he completely devoted himself to the process of developing standards that would influence many people all around the world. It is a great thing to have done and a fantastic legacy to leave behind.
Nigel loved to hike wherever an ISO meeting took place. I also visited many places after the meetings, for example, Machu Picchu after the meeting in Lima where Nigel went into the Amazon jungle. Once I jokingly said to him that ISO actually stands for “International Sightseeing Organization,” which made us both laugh.
At the KEER 2018 conference, he invited me to visit the longhouse village of the Iban tribe before the conference, but unfortunately, my flight schedule prevented me from arriving earlier in Malaysia, and I regretfully had to decline his offer. It would have been an enjoyable hike for me too.
For me, Nigel was a man of ISO standards and a man of hiking and travelling. RIP my colleague and friend.
References
International Organization for Standardization. (1998). Ergonomic requirements for office work with visual display terminals (VDTs) — Part 11: Guidance on usability (ISO Standard No. 9241-11:1998).
International Organization for Standardization. (1999) Human-centred design processes for interactive systems (ISO Standard No. 13407:1999).
International Organization for Standardization. (2010). Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems (ISO Standard No. 9241-210:2010). Available at https://www.iso.org/standard/52075.html
International Organization for Standardization. (2011). Systems and software engineering – Systems and Software Quality Requirements and Evaluation (SQuaRE) – System and software quality models (ISO/IEC Standard No. 25010:2011). Available at https://www.iso.org/standard/35733.html
International Organization for Standardization. (2018). Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts (ISO Standard No. 9241-11:2018). Available at https://www.iso.org/standard/63500.html
Kurosu, M. (2015). Usability, quality in use and the model of quality characteristics. Proceedings of HCI International 2015 (pp.227–237). Springer.
Kurosu, M. (2016). Theory of user engineering. Boca Raton, FL, USA: CRC Press.
Kurosu, M., Hashizume, A., Ueno, Y. Tomida, T., & Suzuki, H. (2016). UX graph and ERM as tools for measuring Kansei experience. Proceedings of HCI International 2016 (pp. 331–339). Springer. Roto, V., Law, E., Vermeeren, A., & Hoonhout, J. (2011). User experience white paper – Bringing clarity to the concept of user experience. Retrieved December 2011 from http://www.allaboutux.org/files/UX-WhitePaper.pdf
