Abstract
Despite the authoritative nature of international standards for usability, many of them are not widely used. This paper explains both the benefits and some of the potential problems in using usability standards in areas including user interface design, usability assurance, software quality, and usability process improvement.
Introduction
Why aren’t international standards for usability more widely used? Over the last 20 years, industry and academic experts in human-computer interaction (HCI), ergonomics, and usability have met to put together a wide range of authoritative prerequisites and guidelines for designing, developing, and evaluating usable products. Some of the most important of these standards are discussed in this paper (a more complete list can be found in Bevan, 2005a).
Different Types of International Standards
One of the main purposes of international standards is to impose consistency, compatibility, and safety. An example is standards to ensure that a cell phone will accept your SIM card, produce transmissions that are compatible with the cell networks, and not create radiation that would be dangerous to you. What about usability? The user interface of some cell phones seems unnecessarily complex. How many of the functions of your cell phone do you know how to use? Is this an area where international standards could help? While we as consumers may become frustrated, many manufacturers regard this as a design issue open for market competition.
How can one create standards for usability where there are so few absolutes? We all know that usability depends on the context of use, design environment, resources constraints, importance of usability, etc. International standards have resolved these problems in different ways in different areas. Each of the four areas below is discussed in more detail in the following sections.
- User interface design. Developing international standards that define elements of the software user interface is a challenge. An attempt to produce a standard for graphical user interface “drivability” failed long ago (IEEE, 1993). So standards for user interface design have taken the approach of providing conditional guidelines, where designers are expected to judge the applicability of each guideline.
- Usability assurance. The International Organization for Standardization (ISO) defines usability in terms of effectiveness, efficiency, and satisfaction. Usability assurance standards provide guidelines and some requirements on how to prepare, run, and document usability tests to make sure products are usable.
- Usability and software quality. Usability has also been integrated into standards for software quality.
- Human centered design process. These standards describe the activities that should be carried out in order to achieve good user interface design and good usability. They provide a basis for defining good practice in usability and have wide applications.
User Interface Design
Most of the early effort in standards for usability went into producing detailed guidelines for user interface design, both for hardware and software, in the ISO 9241 series (Stewart, 2000a). Here ISO inherited the legacy of the original Smith and Mosier (1986) guidelines. The exhaustive ISO 9241 guidelines include the presentation of information (ISO 9241-12), design of user guidance (ISO 9241-13), menus (ISO 9241-14), command languages (ISO 9241-15), direct manipulation (ISO 9241-16), and forms (ISO 9241-17). Conformance to the standards can be achieved by following all relevant guidelines and by providing justification for why particular guidelines have not been adopted (Harker, 1995).
What are the benefits?
The ISO guidelines provide a very good primer for good practice in user interface design. They can also provide authoritative evidence to cite if a user interface design decision is challenged. Interfaces can be evaluated against the guidelines (although this can be time consuming, unless the evaluator is intimately familiar with the content).
What are the problems?
While the guidelines constitute an immense body of knowledge, they are not very easy for designers to use (Carter, 1999; de Souza & Bevan, 1990). In the case of Web design (ISO 9241-151), the U.S. Department of Health and Human Services (HHS, 2006) has developed a free set of guidelines that are superior in presentation and content to the ISO equivalent (Bevan & Spinhof, 2007), which makes the HHS guidelines much more approachable for designers. Unfortunately ISO does not have the resources to develop such a professionally produced document.
What should you use?
The HHS guidelines are an excellent resource for Web design. The other ISO guidelines are good for someone who wants to learn more about usability principles when designing specific types of interfaces.
Usability Assurance
Probably the best-known definition of usability is in ISO 9241-11: “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.” The intention when this standard was first drafted in 1988 was to specify the contents of usability assurance statements that would consist of test reports giving results for effectiveness, efficiency, and satisfaction in a specified context of use (Bevan & Holdaway, 1993). Unfortunately, some leading companies at that time did not want to be forced to produce usable products. For example, a large U.S. company threatened to use its influence to ensure that the standard was rejected unless it was redrafted as guidelines rather than requirements.
It was therefore reassuring to see the same concept reinvented in the U.S. 10 years later as the Common Industry Format (CIF) for usability test reports by a group of companies frustrated by the low profile of usability in product procurement (Bevan et al., 2002). This became the U.S. standard ANSI/NCITS 354 in 2001 (produced by the National Committee for Information Technology Standards in conjunction with the American National Standards Institute) and subsequently in 2006 the international standard ISO/IEC 25062 (as part of the series of standards on software quality published jointly by ISO and the International Electrotechnical Commission).
Work started in another ISO group in 2000 to provide usability assurance for machines used by the public and for consumer products. ISO 20282 was published as a preliminary standard for review in several parts in 2006 and 2007 (but not before a large German company lobbied hard to prevent publication of a standard that could possibly regulate the usability of consumer products). Work is just starting on reviewing and revising the ISO 20282 standards to make them more useful and effective. (You can volunteer through your national standards body if you would like to participate in the revision of these standards.)
What are the benefits?
Effectiveness and efficiency are easy to relate to business objectives, as they measure whether someone can use a product and how long they take to complete tasks (Bevan, 2006). User satisfaction is also an important objective for consumer products and motivates repeat usage, for example of a Web site.
What are the problems?
User testing to measure effectiveness and efficiency requires large numbers of participants, which can be expensive.
What should you use?
Strategic usability goals can have a big impact on design, so it is worth setting targets using ISO 9241-11 and the Requirements CIF (NIST, 2007) even if it is not easy to measure the targets. Consider using ISO 20282 to demonstrate the usability of public machines and consumer products.
Usability and Software Quality
In the 1990s, battle lines were drawn between standards for software quality and standards for ergonomics. Within the ISO/IEC 9126 software quality standards, usability referred only to the design of a user interface. Some people working on the ISO 9241 standard so objected to the approach to usability in ISO/IEC 9126, that one national ergonomics committee threatened to reject an ISO 9241 standard if it contained so much as a citation of an ISO/IEC 9126 standard. But bridges have gradually been built with the broad view of usability being incorporated into software quality as quality in use (Bevan, 1999) and culminating in a joint working group to develop CIF standards in ISO.
In the new model for software quality in the draft ISO/IEC CD 25010 standard, the concept of quality in use has been broadened to embrace a wider range of issues than was common in usability (Bevan, 2009). While effectiveness and efficiency measure the positive benefits of productivity and goal achievement, the term safety has been used to refer to measures of the potential negative outcomes that could result from incomplete or incorrect output. The term flexibility has been added to refer to the need for usability in both planned and unplanned contexts of use and the need for usability for people with special needs. Flexibility can also include learnability, or how quickly and effectively a user interface can be learned. The standard also makes a distinction between usability from different stakeholder perspectives that result in different types of measures, including from the perspective of the end user achieving personal goals, the perspective of the organization achieving organizational goals, and the perspective of technical support achieving maintenance goals.
Note: The status of ISO standards is designated by the letters that precede the standard number. Draft ISO standards can include the following stages:
- CD: Committee Draft
- DIS: Draft International Standard
ISO documents below the status of a full standard include the following:
- PAS: Publicly Available Specification
- TR: Technical Report
What are the benefits?
ISO/IEC CD 25010 (and ISO/IEC 9126-1 which it will replace) provides a comprehensive structure for the role of usability as part of software quality. The broader concept of quality in use increases the business relevance of usability in many situations.
What are the problems?
These standards provide a great way to integrate usability with quality, but do not help if quality is a low priority in your organization.
What should you use?
The requirements CIF (not yet published by ISO, but available from NIST, 2007) is a good guide on how to introduce usability requirements in an organization. Use ISO/IEC 25010 or ISO/IEC 9126-1 if your organization cares about product quality.
Human Centered Design Process
One of the most well-known usability standards is ISO 13407: human-centered design processes for interactive systems. Intended as a manager’s guide, it is probably the best concise introduction to usability that is available. ISO 13407 is currently being revised (renumbered as ISO 9241-210). Ten years on, some of the recommendations in ISO 13407 have been turned into requirements in ISO DIS 9241-210. For a development process to show conformance with ISO DIS 9241-210 it must comply with requirements that include the following:
- Project planning shall allocate time and resources for the human-centred activities. This shall include time for iteration and the incorporation of user feedback, and for evaluating whether the design solution satisfies the user requirements.
- Relevant user and stakeholder groups shall be identified and their relationship with the proposed development described in terms of key goals and constraints.
- There are four linked human-centred design activities that shall take place during the design of any interactive system
- a) Understand and specify the context of use;
- b) Specify the user requirements;
- c) Produce design solutions;
- d) Evaluate.
ISO 13407 spawned two comprehensive models of human centered design (human centered rather than user centered to acknowledge the importance of stakeholders who may not be users): ISO TR 18529 and ISO PAS 18152. ISO TR 18529 broadly covers the scope of ISO 13407. ISO TR 18529 was derived from surveys of good practice procedures in industry. It has been used as the basis for assessing whether a project has adequately implemented human centered design, for assessing the usability maturity of an organization (Bevan, 2005b; Jokela & Iivari, 2001), and it provided the basis for a proposed scheme for accrediting usability professionals (Bevan, 2002).
ISO PAS 18152, originally developed in conjunction with the UK defense industry, is more ambitious in scope, covering the whole range of human centered activities involved in systems engineering. ISO PAS 18152 is divided into four categories: human-centered design activities, human resources activities, life cycle involvement activities, and human factors integration. It can be used in conjunction with the ISO/IEC 15288 systems engineering standard, which already has pointers to the essential human centered activities.
What are the benefits?
ISO 9241-210 is a powerful tool to assure a human centered design process. The comprehensive nature of these standards make them the most authoritative starting point for human centered design education, training, and practice (Earthy et al., 2001).
What are the problems?
ISO TR 18529 and ISO PAS 18152 are quite complex documents. To use these standards one needs to become familiar with the concept of processes, which are generic activities that can be implemented in different ways depending on the particular design and development needs.
What should you use?
If you are going to read only one standard, make it ISO 9241-210 or ISO 13407, as these provide the high level framework for usability work. Use ISO 18529 or ISO 18152 as a tool for improving the usability capability of your organization.
Is Cost an Obstacle?
Why have more people not used usability standards? One of the problems may be finding out where to buy standards and the cost. Even for ISO 13407, which is only 20 pages, the price is $110. It is incongruous that in an age of increasing free access to information, ISO standards remain expensive. Although the experts who write standards give their time free of charge, for national and international standards bodies, the sale of standards and associated services is their main form of income. So, international standards are not going to get any cheaper in the foreseeable future. However, buying a standard is still less expensive than trying to find out what the international consensus is yourself. International standards can be purchased as PDFs on the Web, either from ISO or your national standards body.
Using Standards in a Contract
Standards have the most impact when used in legislation or in a contract, although some ergonomics standards for hardware can be used in support of health and safety regulations in the EU (Bevan, 1991; Stewart, 2000b). Usability standards are likely to have the most influence when cited in commercial contracts. The following are examples of how the standards described in this paper could be used contractually:
- Use the CIF to establish requirements for usability (effectiveness, efficiency, and satisfaction) and to document whether the requirements have been met in a usability test (Bevan et al., 2002).
- Require an organization to demonstrate its usability capability based on ISO 18529 or ISO 18152.
- Require a design and development project to carry out activities that conform with ISO 13407 or ISO 9241-210.
- Require an interface design to comply with the user interface guidelines in the relevant parts of ISO 9241 (although for a Web site it may be more practical to require adherence to the relevant HHS 2006 guidelines).
In a large project where usability is important, a contract could include all four types of standards. Use the CIF to establish requirements and to demonstrate overall usability. Use ISO 18529 or ISO 18152 as a prerequisite for carrying out appropriate activities as part of human centered design (ISO 13407 and ISO 9241-210). Use ISO 9241 to provide guidelines for user interface design. All of these standards could be used as a basis for education and training. ISO 18529 or ISO 18152 can provide a framework for usability process improvement (Bevan, 2005b). Using standards would also provide the designer with a defense against possible legal claims where human error may have serious consequences or where the product may lead to health problems (e.g., RSI) (Earthy et al., 2001).
Why not investigate how you could use usability standards? Feel free to contact me if you have any questions.
References
ANSI (2001). Common industry format for usability test reports (ANSI-NCITS 354-2001).
Bevan, N. (1991, September). Standards relevant to European Directives for display terminals. In: Bullinger HJ (1991) Proceedings of the 4th International Conference on Human Computer Interaction. Stuttgart: Elsevier.
Bevan, N. (1999). Quality in use: meeting user needs for quality, Journal of Systems and Software, 49(1), 89-96.
Bevan, N. (2002). Proposed Competencies for Certification. Retrieved on 14 May 2009 from http://www.upassoc.org/upa_projects/body_of_knowledge/certification_project/competencies.html
Bevan, N. (2005a). International Standards for HCI, In Claude Ghaoui (Ed.) Encyclopedia of Human Computer Interaction. Idea Group Publishing. See also www.nigelbevan.com/papers/International_standards_HCI.pdf
Bevan, N. (2005b). Cost benefits framework and case studies. In R.G. Bias & D.J. Mayhew (Eds.) Cost-Justifying Usability: An Update for the Internet Age. Morgan Kaufmann.
Bevan, N. (2006). Practical issues in usability measurement, Interactions, 13(6) 42-43.
Bevan, N. (2009). Extending quality in use to provide a framework for usability measurement. Proceedings of HCI International 2009. San Diego, California, USA.
Bevan, N., Claridge, N., Maguire, M., & Athousaki, M. (2002, August 25-30). Specifying and evaluating usability requirements using the Common Industry Format: Four case studies. Proceedings of IFIP 17th World Computer Congress (pp. 133-148) Montreal, Canada. Kluwer Academic Publishers.
Bevan, N. & Holdaway, K. (1993). User needs for user system interaction standards. In Evans, Meek and Walker (Eds.) Technology standards. Butterworth Heinemann.
Bevan, N. & Spinhof, L. (2007). Are guidelines and standards for web usability comprehensive? Human-Computer Interaction, Part I, HCII 2007, LNCS 4550 (pp. 407–419) Springer-Verlag.
Carter, J. (1999). Incorporating standards and guidelines in an approach that balances usability concerns for developers and end users, Interacting with Computers, 12, 179–206.
de Souza, F & Bevan, N. (1990). The Use of Guidelines in Menu Interface Design: Evaluation of a Draft Standard. Proceedings of IFIP INTERACT’90: Human-Computer Interaction 1990 (pp.435-440).
Dresner, D. (2006).Desert Island Standards. National Computing Centre. Retrieved on 14 May 2009 from www.ictknowledgebase.org.uk/desertislandstandards
Earthy, J., Sherwood Jones, B., & Bevan, N. (2001). The Improvement of Human-Centred Processes—facing the challenge and reaping the benefit of ISO 13407. International Journal of Human-Computer Studies, 55(4), 553-585.
Harker, S. (1995). The development of ergonomics standards for software. Applied Ergonomics, 26(4), 275-279.
HHS (U.S. Department of Health and Human Sciences) (2006). Research-Based Web Design & Usability Guidelines. Retrieved on 14 May 2009 from www.usability.gov/guidelines/
IEEE (1993). Recommended Practice for Graphical User Interface Drivability. P1201.2 Balloting Draft 2.
ISO/IEC 9126 (1991). Software engineering—Product quality.
ISO/IEC 9126-1 (2001). Software engineering—Product quality—Part 1: Quality model.
ISO 9241 (1997-1999). Ergonomic requirements for office work with visual display terminals (VDTs) —Parts 12-17.
ISO 9241-11 (1998). Ergonomic requirements for office work with visual display terminals (VDTs) —Part 11: Guidance on Usability.
ISO DIS 9241-210 (2008). Ergonomics of human-system interaction – Part 210: Human-centred design process for interactive systems. ISO.
ISO 13407 (1999). Human-centred design processes for interactive systems.
ISO/IEC 15288 (2008). Systems and software engineering—System life cycle processes.
ISO PAS 18152 (2003). A specification for the process assessment of human-system issues.
ISO TR 18529 (2000). Human-centred lifecycle process descriptions.
ISO 20282 (2006/7). Ease of operation of everyday products (in four parts).
ISO/IEC CD 25010 (2009). Systems and software engineering—Software product Quality Requirements and Evaluation (SQuaRE)—Software product quality and system quality in use models.
ISO/IEC 25062 (2006). Software Engineering —Software product Quality Requirements and Evaluation (SQuaRE —Common Industry Format (CIF) for Usability Test Reports.
Jokela, T. & Iivari, N. (2001). Usability capability assessments – experimenting and developing usability maturity models. Proceedings of IHM-HCI 2001: vol. 2.
NIST (2007). Common Industry Specification for Usability—Requirements. See nist.gov/iusr
Smith, S. L. & Mosier, J. N. (1986). Guidelines for designing user interface software. Report ESD-TR-86-278. Bedford, MA: The MITRE Corporation.
Stewart, T. (2000a). Ergonomics user interface standards: are they more trouble than they are worth? Ergonomics, 43(7).
Stewart, T. (2000b). Display screen regulations: An employer’s guide to getting the best out of regulations. Retrieved on 14 May 2009 from
www.system-concepts.com/articles/office-ergonomics/guide-to-display-screen-equipment-regulations
