The Beauty and the Cognitive
Twenty years ago, it seemed as if functionalism had finally won. Back then, effective and efficient use had been the sole mission for practitioners of human-computer interaction (HCI). A sharp distinction was made between sober and elegant “tools” and gimmicky “toys”—the latter supposedly bloated with useless functionality and ornament. Supporting users’ task fulfillment was the holy grail, and practitioners waved ISO Standard 9241 around as if it were a magic spell. Usability engineers took pride in taking scientific knowledge and methods borrowed from cognitive psychology to their design praxis. It was a time when you may have found yourself painstakingly modelling interaction on a micro level with Goals Operators Methods and Selection Rules (GOMS, e.g., John & Kieras, 1996) to predict the variations in the performance of different design variants, only to later multiply some milliseconds with the frequency of daily, monthly, and yearly use to arrive at a fantastic amount of time and money potentially saved through usability (e.g., Bias & Mayhew, 1994). You ran usability tests to find all the problems software engineers had foolishly made in the interface. Cardinal sins included using checkboxes instead of option widgets for a mutually exclusive selection or enraging the user with obscure error messages. It was all pretty straightforward: The interface was supposed to be cognitively streamlined—functionalism ruled.
Of course, anybody who had concerned herself or himself at least a little with the history of architecture and design, would have already guessed that interactive products would not remain purely functional. For example, it didn’t take long for the praised architecture of the Bauhaus and especially Le Corbusier to be criticized as cruelly sober, bare, unwelcoming, and elitist. Functional design was overly driven by rational principles of technology and construction. It provided no room for emotionality, subjectivity, and irrationality (Mareis, 2014). “Less is a bore” (Robert Venturi) was the answer to “less is more” (Mies van der Rohe). Emerging postmodern styles, just as beloved Memphis Design, offered bursts of form, color, ornament, irrationality, and fun (Figure 1). In hindsight, it is almost sad how HCI ridiculed the supposed “follies of design” by emphasizing how all this postmodern design lacked function, while this this very lack of function was the whole point of it and the response to the devastating and inhumane functionalism of decades ago. No surprise that this design wasn’t functional—it wasn’t supposed to be.
Figure 1. A Memphis interior. Source: Zanone, Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=15773679.
Quite early on in engaging with HCI, I had this nagging feeling that I would miss out on something important by focusing on the cognitive aspects only. In the mid-1990s, I worked as a student assistant for a professor of applied cognitive psychology who specialized in HCI. He had this pet project of formulating rules for the “good” design of PowerPoint slides. (Data projectors were not around yet, so typically one designed a set of slides, printed them on transparent foil, and then projected the slides using an overhead projector.) So, we made rules about contrast (light background/dark fonts for a light room, the opposite for a very dark room), calculated optimum font sizes depending on the size of the room, determined the number of bullet points ideal for cognitive information processing, and made sure that every slide was properly numbered to give the listener a sense of progress. One day, we made a set of slides for a computer science professor who was planning to present at an external seminar. When he came back from the seminar, we were curious to know how our cognitively streamlined slides had worked. “Well well,” he said, “all fine, but I had hoped for something a little more beautiful.” Beauty! Here we were, 30 plus rules for the perfect slides and yet we missed out on something apparently important. While it is more than obvious that a presentation is a social situation, where impressing the audience is an important part of the game, the notion that this may also be a design goal for the slides didn’t feature prominently in my mindset at that time (and, to ease my shame a bit, also my professor seemed puzzled, if not a little hurt). Apparently, a psychology department focused on the cognitive is not the best context to learn more about the beauty of interactive systems. Hence, it took a while before I could revisit this theme.
In Doubt
In 1998 I started working as a usability engineer at Siemens’ Corporate Research. One of the business units we were working with was computed tomography (also known as a CT scan) and other medical imaging systems. At this time, Siemens Medical developed Syngo, a unified software for all imaging systems, that featured a visual appearance different from any standard operating system at that time (see Figure 2 for a current example). CT scanners are expensive machines. Customers expect a high-quality industrial design, and Siemens wants clear brand recognition. However, the existing software had an interface developed in OSF/Motif, which was—let me assure you—not the most beautiful sight. Syngo was meant to match the software’s look with the already existing quality of hardware design. To me, it again highlighted that focusing on usability alone may not do the trick.
Figure 2. Siemens Syngo.Via. Source: Siemens Press, www.siemens.com/press, reference number: HIM200911014-03.
Of course, I could have busied myself with diagnosing and solving usability problems emerging from testing Syngo. But I was already very interested in users’ subjective responses to interactive products. This early focus on the experience of people rather than the supposedly “objective” quality of the interactive product was due to some slightly disconcerting experiences I repeatedly had during usability testing. At that time, our tests were predominantly formative, that is, we focused on finding usability problems (as opposed to measure performance). Typically, I walked a user through some predefined tasks; she struggled, deadlocked, needed help, worried about icons, and I wrote a flurry of notes—all documenting the total inadequacy of the system. In the debriefing interview, we then talked about her general impression of the system, which should have been negative given her earlier observations. But it wasn’t. She actually found the system “interesting” and even “engaging.” I saw plenty of problems, but they did not seem to impact her opinion as much as an eager usability engineer might think they should. Obviously, there must have been other important aspects influencing her judgment beyond effective and efficient task-fulfillment. But which?
The designer working on the visual appearance of Syngo, Axel Platz, of course, took the same line. His argument was that by focusing on usability only, aspects important to him and users alike, such as novelty and beauty, go unnoticed. There is a risk to condemn novel and interesting ideas, just because they produce some minor irritations, when being used for the first time. Obviously, we needed something to more systematically address these kinds of trade-offs.
Making Tools
When going through the HCI literature at that time, it became apparent that some authors already stumbled across similar problems with usability. For example, Jack Carroll and John Thomas wrote a paper in 1988 about “fun.” They warned: “we continue to see […] the prospect of a decade of research analysis of usability possibly failing to provide the leverage it could on designing systems people will really want to use by ignoring what could be a very potent determination of subjective judgements of usability – fun” (1988, p. 23). They worried about a certain incompatibility between fun and prominent concepts of usability, such as simplicity, at the same time emphasizing the potential importance of subjective emotional experiences. In fact, in a “Funology” workshop, I co-organized with Mark Blythe, Andrew Monk, and Darren Reed at the CHI conference in Minneapolis in 2002, Randy Pagulayan from Microsoft Game Studios boiled this down to “If a usability engineer would design a game, it would be a button with a label ‘To win, press here’.” I tend to agree. We hadn’t been exceptionally good in making something fun.
One problem is self-evident: What is this elusive “fun” and, more importantly, how to address it in the context of usability engineering and especially usability evaluation? While certainly some tentative approaches already existed or appeared at that time (Draper, 1999; Jordan, 1998; Logan, 1994; Malone, 1984), it wasn’t mainstream “to go beyond” usability. We lacked tools and models.
At Siemens, we approached the challenge in quite a practical way. Based on existing models and questionnaires, we compiled a number of semantic differential items to represent usability, such as “simple – complex,” “familiar – strange,” or “comprehensible – incomprehensible.” Next, we collected a number of additional attributes that we found important but were neglected by traditional usability, such as “interesting – boring,” “exclusive – standard,” or “impressive – nondescript.” We coined this group of attributes “hedonic quality” as opposed to “ergonomic quality” (later this became re-labeled into pragmatic quality on the suggestion of a reviewer, I believe it was Jim Lewis). We began to use this semantic differential questionnaire in dedicated studies and our everyday evaluation practice. This further evolved and became the AttrakDiff2 questionnaire (www.attrakdiff.de).
Our first paper arguing for the importance of both hedonic and ergonomic (pragmatic) quality perceptions was accepted for the CHI conference 2000 in The Hague (Hassenzahl, Platz, Burmester, & Lehner, 2000). In a nutshell, this line of work showed that pragmatic and hedonic perceptions are conceptually different but both contribute to the evaluation of a system. Spare me going into details of the model and arguments in this paper, but if you are interested in details, look at Hassenzahl (2003) or even better at the remixed 2018 version of Funology 2: From Usability to Enjoyment (Blythe & Monk, 2018). The important point was that we had a simple model that went beyond usability by adding a hedonic component, a questionnaire to quantify crucial constructs, and first results showing that the hedonic contributes to a product’s appeal.
Back in 2000, this was liberating. For example, it made communication with engineers and designers much easier. They often felt mistreated by evaluation studies because usability people typically did not care about innovation, novelty, or engagement. Now we had a model that addressed both task-oriented effectiveness and efficiency as well as curiosity-driven aspects. This helped immensely in finding compromises and trade-offs. In addition, we had a measurement tool that lent itself to practical as well as more academic applications. Personally, I always found the latter to be the true advantage of our approach.
When flipping through the pages of the Funology book from 2003 (which emerged from the workshop mentioned above), I see a lot of conceptual work. Thoughts, positions, opinions, imports from many other disciplines, but only a few approaches lent themselves to application. In contrast, providing an operationalization in the form of a semantic differential allowed us to start right away to run our own studies and to replicate existing findings. This and the model’s simplicity is certainly one explanation for the fact that it is still quite popular and well-used, although slightly outdated (Diefenbach, Kolb, & Hassenzahl, 2014).
The “Why” of Product Use
I, though, went on a different trajectory, especially inspired by two pieces of work: one was Tractinsky, Katz, and Igar’s “What is beautiful is usable” (2000). Oh this paper made me angry! Here I was, fighting for the recognition of beauty and hedonic aspects as independent and important issues in HCI, and Noam (Tractinsky) annexed beauty by implying that it is systematically linked to usability. Consequently, I did my own studies, arguing that Noam and his colleagues got it totally wrong (Hassenzahl, 2004). This spawned a very interesting debate and a substantial number of empirical studies into beauty and HCI. (It also spawned quite a nice professional relationship with Noam Tractinsky, leading, for example, to the paper “User Experience – A Research Agenda” [Hassenzahl & Tractinsky, 2006], still a heavily cited overview about what User Experience [UX] might become and what not.) While in hindsight, the research on beauty turned out to be a bit confusing, the very fact that it demonstrated various relationships between usability, beauty, satisfaction, and even performance turned beauty into a concept taken quite seriously in HCI. Nowadays, it is out of question to not take visual design of interfaces into account. Interfaces have become a subject of fashion just like any other consumer product we use.
Personally, beauty was interesting to me because it provided the first, most direct link to design. The notion of the hedonic is quite conceptual. It addresses being innovative, interesting, and engaging—aspects that can be fulfilled in many ways such as novel content, interesting functionality, or engaging interaction. In contrast, beauty is more tightly linked to the actual form of an interface: its colors, graphics, typography, and so on. These are the issues that originally set my search for something beyond usability in motion (remember the PowerPoint slides, remember Syngo). Thus, my interest in beauty led me a little closer to design, although a very traditional, and craft-oriented notion of what design is.
Above I mentioned two pieces of work that heavily inspired me. The other was Gaver and Martin’s “Alternatives” (2000). In fact, when I presented our paper on hedonic and ergonomic quality on CHI in 2000, Gaver and I had been in the same session. He presented wonderfully strange conceptual designs—a whole sketch book worth of them. One example was the “dawn chorus,” which is “an artificially intelligent birdfeeder. Joining a microphone, speaker, pitch-tracker, and software, it would use behaviourist principles to teach the birds new songs, first playing an example of the tune, then progressively rewarding them as they learned to sing in response, to sing small phrases of the song, and finally to match it in tune and tempo” (p. 210). They described the intended experience to be delivered by the device as follows: “It is pleasant to be awakened by the sound of local songbirds, but how much more enjoyable it would be if they knew our favourite music” (Gaver & Martin, 2000, p. 210). If you think this is weird, I agree, and on a second thought, it might not be a very good and sustainable idea to deliberately mess with birds’ songs. However, Gaver and Martin did not just provide weird ideas, they linked their designs to particular “values” (their term). The “dawn chorus,” for example, is catering for the pleasures people derive from controlling and influencing their environment. Thus, a product that appears superfluous on first sight actually addresses one of the most fundamental human needs. And all of a sudden, I could picture myself, enjoying the feelings of power, control, and status, when being woken up by my personal “dawn-chorus” trilling Depeche Mode’s “Master and Servant.”
Gaver and Martin made me think (although it took some years to lead to anything substantial). First of all, I became much more interested in what I today would call “the Why” of product use (see Hassenzahl, 2010). Which intrinsic, psychological needs are fulfilled through interacting with a product? What kind of positive and meaningful experiences emerge from interaction and use? Instead of approaching the design of technology predominantly from the level of form and interaction (the How), I worked towards approaching it from the other end—from its meaning in everyday life (the Why).
Today, this is the backbone of my notion of (user) experience. To give an example: Some years ago, we explored potential interactive concepts for BMW. But instead of starting with driving, that is, the core functionality of cars or any given interesting interaction technology (i.e., touch, gestures, voice), we focused on psychological needs, such as relatedness, that is, the experience of feeling close to significant others (Eckoldt, Hassenzahl, Laschke, & Knobel, 2013). This approach led to concepts quite different from what is typically considered in the context of cars. For example, we developed the concept of Together Alone, a social driving application (Figure 3).
Figure 3. Together Alone concept study. Source: M. Hassenzahl and K. Eckoldt.
It connects two friends or acquaintances who are both driving but in different cars. Its functionality is simple: An interface element indicates whether selected friends are in the car at the very same moment and whether there is enough time left to have a chat. If the driver wants to chat with a friend, he or she can give a command to the system to initiate a phone call. Thus, regardless of the region or destination, the simultaneous activity of driving creates an opportunity to spend time together and talk—just as if both would be in the same car. Together Alone offers relatedness in a situation, where many people are generally alone (at least in Germany), for example, when commuting (see also Hassenzahl, Laschke, Eckoldt, Lenz, & Schumann, 2017). However, instead of delegating social activity to generic channels such as a Smartphone, Instagram, or Facebook, this concept attempts to create a highly situated social experience deeply interwoven with the activity of driving and traveling with someone through “simple” technology.
It seems as if our understanding of products is heavily shaped by the idea that a product provides a basic functionality (e.g., transport in the case of a car) and added, psychological utilities (e.g., status through owning the car; Vershofen, 1940). The basic functionality is why people supposedly buy the product. However, there are intangible, additional utilities that create a form of psychological bonus, such as the social aspects of riding in a car with someone, which is helpful for differentiating products with similar basic functionality. In this sense, pragmatic quality can be understood as basic and hedonic quality as an add-on. Similarly, Jordan (2000), for example, suggested a hierarchy, where functionality is the basis, usability the second layer, and pleasure the third. Thus, without functionality and usability, pleasure can never emerge. What I mostly disagree with is this notion of a hierarchy of basic functionality and additional utilities. I believe it to be the other way around, or to put it more bluntly: Most people don’t own cars to get from A to B; they own cars because of the psychological need fulfillment cars provide in situations of everyday life—feelings of competence, popularity, and autonomy. The positive experience, the meaning, and the need fulfillment are the true basic functionality; the actual getting from A to B is only a welcomed side effect. To me getting this wrong is at the heart of failing to understand why many people insist on using cars even though public transport or bikes are available, cheaper, and often even faster. Believing in the notion that transport is the basic functionality of a car also fails to explain why some people use their cars for joyrides. It is simply not about the basic function a car provides, but about the fulfilling experiences it is able to create and shape. All of this I believe to have learned through the weird concepts presented by Gaver and Martin in their paper. In all of their examples, the core value proposition is an experience, a feeling, a particular type of need fulfillment. However, they do not take the detour of inventing further practical usefulness just to appear rational and sober.
There is a second thing, I learned from Gaver and Martin’s paper: There is value in examples and conceptual designs, even if they are fictional. With a background in psychology, the best I could do was materializing models and ideas through empirical data. I previously have said that I believe this to be important, and I still engage in justifying and “proofing” my models. However, these models must also be generative. Design cases and examples are embodiments of these models, “materialized theory” so to say. In this sense, they are also proof of the capabilities of a given approach. Nowadays, this is the core of my work: the question of how to transform abstract notions of experiences, meaning, and psychological needs into concrete, surprising, engaging, and hopefully “better” arrangements of technologies.
Towards a Design for Wellbeing
If I should describe the trajectory of my professional development in HCI, I would say, “usability, hedonic quality, user experience, experience design, designing for wellbeing.” Design for Wellbeing (Hassenzahl et al., 2013) is the logical consequence—from the experiential as a hard-earned add-on to usability to the experiential as a central and explicit objective of design, and from a narrow concern about whether a product can be understood and operated to a much broader concern about desired and undesired experiential consequences of prolonged product use. In this respect, HCI and Interaction Design took a real leap since the mid-1990s—from the cognitive science-imbued approaches of a few experts to a mass design business that embraces beauty, emotion, experiences, persuasion, and many more concepts.
Admittedly, the speed of progress and the sheer number of models and tools can make it difficult for the practitioners of today to keep ahead. Consequently, even in 2013, one could find the question of whether it is “acceptable practice to use checkboxes as radio buttons?” in UX forums (https://ux.stackexchange.com/questions/41924/is-it-acceptable-practice-to-use-checkboxes-as-radio-buttons). In this example, there is even good old design bashing in the form of the ominous Art Director who is “convinced it makes more sense to use checkboxes because radio buttons look uglier.” And a colleague who promptly states: “Sadly, Art Directors are often the last person you want to be asking about usability issues. :)” Some things never change.
References
Bias, R. G., & Mayhew, D. J. (1994). Cost-justifying usability. San Diego, CA: Academic Press.
Blythe, M. A., & Monk, A. F. (2018). Funology 2: From usability to enjoyment. Springer International.
Carroll, J. M., & Thomas, J. M. C. (1988). Fun. ACM SIGCHI Bulletin, 19(3), 21–24.
Diefenbach, S., Kolb, N., & Hassenzahl, M. (2014). The “hedonic” in human-computer interaction. In Proceedings of the 2014 Conference on Designing Interactive Systems – DIS ’14 (pp. 305–314). New York: ACM Press.
Draper, S. W. (1999). Analysing fun as a candidate software requirement. Personal and Ubiquitous Computing, 3(3), 117–122.
Eckoldt, K., Hassenzahl, M., Laschke, M., & Knobel, M. (2013). Alternatives: Exploring the car’s design space from an experience-oriented perspective. In Proceedings of the 6th International Conference on Designing Pleasurable Products and Interfaces – DPPI ’13 (pp. 156–164). New York: ACM Press.
Gaver, B., & Martin, H. (2000). Alternatives: Exploring information appliances through conceptual design proposals. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI ’00 (pp. 209–216). New York: ACM Press.
Hassenzahl, M. (2003). The thing and I: Understanding the relationship between user and product. In M. Blythe, C. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From usability to enjoyment (pp. 31–42). Dordrecht: Kluwer Academic Publishers.
Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human-Computer Interaction, 19(4), 319–349.
Hassenzahl, M. (2010). Experience design: Technology for all the right reasons. San Rafael, CA: Morgan & Claypool.
Hassenzahl, M., Eckoldt, K., Diefenbach, S., Laschke, M., Lenz, E., & Kim, J. (2013). Designing moments of meaning and pleasure. Experience design and happiness. International Journal of Design, 7(3), 21–31.
Hassenzahl, M., Laschke, M., Eckoldt, K., Lenz, E., & Schumann, J. (2017). It’s more fun to commute – An example of using automotive interaction design to promote wellbeing in cars. In G. Meixner & C. Müller (Eds.), Automotive user interfaces: Creating interactive experiences in the car (pp. 95–120). Springer International.
Hassenzahl, M., Platz, A., Burmester, M., & Lehner, K. (2000). Hedonic and ergonomic quality aspects determine a software’s appeal. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI ’00 (pp. 201–208). New York: ACM Press.
Hassenzahl, M., & Tractinsky, N. (2006). User Experience – A research agenda. Behaviour & Information Technology, 25(2), 91–97.
John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction (TOCHI), 3(4), 287–319.
Jordan, P. W. (1998). Human factors for pleasure in product use. Applied Ergonomics, 29(1), 25–33.
Jordan, P. W. (2000). Designing pleasurable products. An introduction to the new human factors. London, New York: Taylor & Francis.
Logan, R. J. R. (1994). Behavioral and emotional usability: Thomson Consumer Electronics. In M. Wicklund (Ed.), Usability in practice (pp. 59–82). Cambridge, MA: Academic Press.
Malone, T. W. (1984). Heuristics for designing enjoyable user interfaces: Lessons from computer games. In Human Factors in Computer Systems (pp. 1–12). Norwood, NJ: Ablex Publishing
Mareis, C. (2014). Theorien des designs [Theories of design]. Hamburg: Junius.
Monk, A., Hassenzahl, M., Blythe, M., & Reed, D. (2002). Funology: Designing enjoyment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI ’02 – Extended Abstracts (pp. 924). New York: ACM Press.
Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127–145.
Vershofen, W. (1940). Handbuch dervVerbrauchsforschung [Handbook of consumer research]. Berlin: Heymann.
