Beyond Specifications: Towards a Practical Methodology for Evaluating Web Accessibility

Abstract

The current set of tools and specifications for ensuring web accessibility require expert knowledge and often have a highly technical orientation, with the consequence that it is not very clear how, or even when, to make use of them. In an attempt to tackle this problem, this paper reviews the types of tools and specifications available and proposes a simple and practical methodology for web accessibility evaluation that demonstrates how these tools and specifications could be used. The proposed methodology proposes methods and processes for reaching and maintaining web accessibility, and consists of the following phases: (a) identification of user requirements and setting up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of accessibility policy. Further, in order to illustrate step (b), an example of web accessibility evaluation is described, where the domain is contemporary scientific publishing web sites. The work presented in this paper reports on issues that need to be considered by human-computer interaction (HCI) researchers, interaction design practitioners, and usability professionals for inclusive web design and are complementary to web usability engineering.

Practitioner’s Take Away

The following are the key points of this paper:

• Web accessibility concerns every user, designer, and business owner. There are many ethical, business, user, and technical arguments for designing for web accessibility.
• Current web accessibility tools and specifications are technically oriented and need expert knowledge to be understood and applied.
• Despite the work on web accessibility, most web sites are still inaccessible. For example (described in this paper) six out of ten e-publishing homepages pose major accessibility problems and only two out of ten are accessible.
• Typical web accessibility problems found include the loss of (a) information organisation, (b) navigation, (c) visibility, and (d) user control, when users access web sites in constrained contexts (e.g., with an oral browser or in a keyboard only situation).
• The paper proposes a practical methodology reaching and maintaining web accessibility, which includes (a) identification of user requirements and set up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of accessibility policy. This three step approach will help to achieve the following:
• Identify particular user requirements when designing for web accessibility and conform to a level of accessibility specifications (e.g., single-A of WCAG 1.0).
• Implement a fast and practical method for regular web accessibility inspection according to your accessibility goals. This paper illustrates a method that makes use of automated tools and heuristics that can be employed in accessibility inspections.
• Understand the appropriate time and the value of user testing. User testing is important for web accessibility, but a basic level of accessibility should be there. Testing with disabled users also increases web developers’ awareness.
• Establish and follow up a web accessibility policy. This should focus on basic rules for content update of the web site that editors of the web site should follow as well as tools for checking content on the fly or in the background. This is an area for further research development and application in the field.

Introduction

The availability of web technology does not ensure universal access to the web. As the web Accessibility in Mind (WebAIM) initiative explains “at the click of a mouse, the world can be ‘at your fingertips’—that is, if you can use a mouse… and if you can see the screen… and if you can hear the audio” (WebAIM, Introduction section, para 1). In this way, web accessibility has come to mean taking account of the needs of people with disabilities that encompass both physical and cognitive impairments. In this regard, Tim Berners-Lee famously noted that “The power of the web is in its universality. Access by everyone regardless of disability is an essential aspect” (WAI, n.d., sidebar). Indeed for some people with disabilities, access to the web is paramount.  Lynn Holdsworth, screen reader user, web developer, and programmer said, “For me being online is everything. It’s my hi-fi, my source of income, my supermarket, my telephone. It’s my way in” (European Blind Union, n.d., p. 6).

Increasingly, it has been understood that accessibility concerns every user, designer, and business owner. The ethical stand that calls for providing equal opportunities to all has for long ceased to be the main argument for ensuring web accessibility. Perhaps the most important misconception about accessibility is that it does refer only to people with special needs. Designing for accessibility addresses other user access issues as well, such as, performance for low network speed and usable access under constrained environmental technical point of view. Designing for accessibility promotes good technical design, and implementation that has obvious implications for web site maintenance and extensibility regarding content and look and feel (Koutsabasis, 2010). The business aspects of web accessibility are also significant, especially if we take into account that people with special needs are not only those that suffer from permanent disabilities but also other groups such as the ever increasing aging population. According to the United Nations Statistics on Human Functioning and Disability the percentage of people with disabilities in most western countries is estimated between 8% and 20% of the total population; for example, the results from related surveys for Canada (1996) show that a total of 15.5% of the population are people with disabilities; for Australia (1993), 18%; Germany (1992), 8.4%; New Zealand, 20%; USA (1994), 15%; and so on (2010).

Despite the large amount of work on web accessibility, the vast majority of web sites are still not accessible. Loiacono and McCoy’s (2006) study evaluated the web accessibility of a large number of web sites. The study indicated that only 23% of the U.S.’s federal homepages were accessible, while this percentage dropped to 11% for non-profit organizations and an even more disappointing 6% for corporate homepages. Because it is understood that accessibility of content seems to make the usability of sites better, there is interest in making sites more accessible, and by extension more usable, but it is difficult. Lazar, Dudley-Sponaugle, and Greenidge’s (2004) study of web masters’ perceptions about web accessibility reports that “most webmasters support the concept of web accessibility, but cited roadblocks to accessibility such as lack of time, lack of training, lack of managerial support, lack of client support, inadequate software tools, and confusing accessibility guidelines” (p. 284).

The current set of web accessibility tools and specifications have a highly technical orientation and need expert knowledge for their comprehension and application. Nor are there any widely used methodologies that encompass these into some practical form for practitioners. Also, the usability of these tools and specifications per se is an issue; as with most specialist groups, those working on accessibility have tended to develop a jargon that makes their use difficult (Kapsi, Vlachogiannis, Darzentas, & Spyrou, 2009). Thus the failure to address web accessibility can be attributed to the lack of practical integrated approaches for universal design and evaluation of web applications. As legislation for web accessibility enforcement is being developed in several countries and the web is changing to web 2.0 technologies and applications, it is important that HCI researchers, interaction design practitioners, and usability professionals have a clear understanding of the use of practical methodologies that encompass the essential issues that need to be considered for accessible web applications.

This paper contributes to both issues of awareness and use of web accessibility by (a) reviewing the current state of the art in web accessibility tools, specifications, and methods; (b) illustrating an example of practical web accessibility evaluation that shows typical accessibility problems that can be identified in contemporary web sites; and (c) proposing a practical methodological framework for web accessibility evaluation.

The structure of this paper is as follows. First the Related Work section describes the present state of affairs with regard to the web accessibility specifications, methods, and tools. In the light of this overview, the next section presents an example of the web accessibility evaluation process, discussing accessibility problems that are typical of many other contemporary web sites. However, this process is part of the proposed methodology, which is explained in detail in following section. The methodology offers a practical use of tools, specifications, and practices, and consists of the following phases: (a) identification of user requirements and setting up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of accessibility policy. The proposed methodology offers a practical means to approach and tackle the issue of web accessibility and provides readers with a framework within which to situate existing and future tools and specifications. Finally the last sections offer a summary and conclusions of this work and the practitioner’s take away.

Related Work

Research on web accessibility has produced a wide range of results that are used in mainstream web design to promote good design practice. This work can be briefly divided into specifications (and legislation based upon these specifications that aims at encouraging or enforcing the development of accessible web design), approaches, and methods for applying those specifications and tools for automatic evaluation of (aspects of) web accessibility.

Web Accessibility Legislation, Policies, and Standards

Several policies and legislation have been developed either at a national or an international level for promoting and enforcing web accessibility. Countries on both sides of the Atlantic have included eAccessibility into their laws. The U.S. has developed Section 508 of the U.S. Rehabilitation Act (for more information, see http://www.section508.gov). The most recent European policy framework for the information society and media is i2010, which promotes, with the tools available to the Commission, a European Information Society for all citizens (Europe’s Information Society, n.d.). Actions implemented under this priority of i2010 aim to ensure that the benefits of the information society can be enjoyed by everyone (eAccessibility) and encourage provision of better public services (eGovernment and eHealth). Furthermore, there are several national legislations across Europe. Probably the most well known is the Disability Discrimination Act (DDA) 1995 (updated in 2005) in the U.K. and the Act on Equal Opportunities for Disabled Persons (27 April 2002) in Germany (for more information, see http://www.w3.org/WAI/Policy). Laws promoting accessibility are often based on well established de-facto standards, like Web Content Accessibility Guidelines (WCAG) recommended by the Web Accessibility Initiative (WAI) of  the World Wide Web Consortium (W3C).

Currently, most of the countries use WCAG 1.0 as a basis for setting up their local policies. The first version of WCAG guidelines (WCAG 1.0) offers a set of techniques to remedy problem situations. However, these techniques are HTML and web technology centric. We are now at a transition phase as the second major version of WCAG has been released as a recommendation of W3C.WAI (Caldwell, Chisholm, Slatin, & Vanderheiden, 2008). The seven-year (since the first public working draft in 2001) effort to reach consensus on WCAG 2.0 produced guidelines that have tried to address problems present in WCAG 1.0, nevertheless they still present weaknesses. According to several researchers in the field (e.g., Brewer, 2005; Moss, 2006), the guidelines are presented at a very abstract level using general and vague terms; they are difficult to use because they are couched in even more obscure terminology than WCAG 1.0, and they require a great deal of explanation to become comprehensible. Kapsi et al. (2009) found several usability issues of WCAG 2.0 that they suggested could be significantly improved if usability issues could be communicated to the evaluators more clearly. Furthermore, there have been organizational policies like the U.K.’s Royal National Institute of the Blind (RNIB) that highlight some of the important accessibility issues (a subset of WCAG), which are exhaustively presented in Harper and Yesilada (2008).

Apart from the web content, web accessibility depends on two additional components of web development and interaction, including web software (tools) and web developers (people; for more information, see http://www.w3.org/WAI/mobile/experiences). Based on these, the WAI provides complementary guidelines: (a) Authoring Tool Accessibility Guidelines (ATAG and ATAG 2.0 – W3C Working Draft 21 May 2009; for more information, see http://www.w3.org/TR/2009/WD-ATAG20-20090521) and (b) User Agent Accessibility Guidelines (UAAG and UAAG 2.0 – W3C Working Draft 11 March 2009; for more information, see http://www.w3.org/TR/UAAG20). Finally, recently there is much interest on the shared web experience of disabled users, older users (Arch, 2008), and mobile device users.

Web Accessibility Approaches and Methods

W3C describes three evaluation approaches with varying levels of thoroughness and effort:

• Preliminary evaluation is a quick and simple check to identify some of the major issues on a web site. Preliminary evaluation is a formal methodology that identifies the most common web accessibility problems. This type of evaluation can be used by non-expert and non-technical evaluators and highlights some important concerns about the accessibility a web page. However, it should be noted that some of the technically based criteria are now out of date since they are based on WCAG 1.0 and older technologies (WAI, Evaluating Accessibility/Preliminary Review, 2010).
• Conformance evaluation is a comprehensive approach to determine if a web site conforms to an accessibility standard such as the W3C Web Content Accessibility Guidelines (WCAG). This is probably the most common and normative evaluation for an exhaustive evaluation of a web page based on set of criteria. The effectiveness and the problems of this type of evaluation are dependent upon the criteria themselves as noted above (WAI, Evaluating Accessibility/Conformance Evaluation, 2010).
• Involving end users goes beyond technical evaluation and includes end users. This is an approach that can provide accessibility in-context, but at the same time it requires that the web site has already addressed some basic accessibility issues so that it can be tested by (disabled) end users (WAI, Evaluating Accessibility/Involving Users in Evaluation, 2010).

Several other approaches and methods have been developed to guide the designer or evaluator to apply recommendations.

A systematic methodology for evaluating conformance with the WCAG 1.0 is the Unified Web Evaluation Methodology (UWEM; Nietzio, Strobbe, & Velleman, 2008; Velleman, Strobbe, Koch, Valasco, & Snaprud, 2007). The UWEM is the result of the combined efforts of three European projects that make up the Web Accessibility Benchmarking Cluster (WAB Cluster). Most of the countries that use WCAG 1.0 as a basis for setting up their national policies often have different interpretations of the guidelines. UWEM offers a shared interpretation of WCAG 1.0 in order to provide sufficiently robust tests. In addition UWEM aims at providing a harmonized platform for a European level—common—evaluation methodology that can be used for the development of labeling schemes (for more information, see http://www.euracert.org and ftp://ftp.cenorm.be/PUBLIC/CWAs/e-Europe/WAC/CWA15554-00-2006-Jun.pdf to see the CEN Agreement) and deals with topics such as scoping, sampling, testing, reporting, etc. Some changes are required for the maintenance of UWEM, especially regarding WCAG 2.0.

Another approach, this time aiming to achieve a common methodology on a national level, is that of the U.K.’s PAS 78 BSI Standard (for more information, see http://www.bsi-global.com/en/Shop/Publication-Detail/?pid=000000000030129227). This is intended to be a document that can help the commissioners of web sites to understand accessibility and be able to discuss development with web design project managers with reference to the WAI guidelines, usability testing, automated checking tools, etc. This approach attempts to standardise the process of achieving web accessibility, rather than the properties of web accessibility.

Web Accessibility Evaluation Tools

A large number of software tools have been developed to automatically check web accessibility (for more information, see http://www.w3.org/WAI/ER/tools/). The software tools functionality mainly includes the following:

• Generating reports of evaluation results, providing step-by-step evaluation guidance, or displaying information within web pages.
• Supporting automated checking on single web pages, groups of web pages, or web sites and automatically generated pages.
• Providing repair functionality by changing the code of the web pages, helping with captioning audio or video content, or converting document into accessible mark-up.
• Checking accessibility for several technologies like Cascading Style Sheets (CSS), XHTML, PDF, images, Synchronized Multimedia Integration Language (SMIL), and Scaleable Vector Graphics (SVG).

Because WCAG2.0 was only released a short time ago, there are only a few tools that use the criteria in that version. Presently the ATRC web accessibility checker is the only tool that claims to support it (http://achecker.ca/checker/index.php). However, Imergo® provides added functionality including integration with web 1.0 and 2.0 and advanced crawling capabilities for content management systems and a navigation consistency module (http://imergo.com/; Velasco, Denev, Stegemann, & Mohamad, 2008). Finally, the GOVMov eAccessibility Checker is interesting mainly because it supports UWEM (Velleman, Strobbe, Koch, Velasco, & Snaprud, 2007), discussed in next section.

Despite the plethora of available web accessibility tools, several researchers have identified important limitations including: limited crawling capabilities, the inability to analyze and/or repair dynamically generated Document Object Model (DOM) etc. (Velasco et al., 2008), and inconsistencies between the evaluation results of different tools (Centeno, Kloos, Fisteus, & Alvarez, 2006). This last limitation is basically due to the ambiguity of some checkpoints that in turn lead to different interpretations. This is a problem that WCAG 2.0 has tried to address (see discussion in Kapsi et al., 2009). Also, different assumptions taken from the logic of the tools (like document validation) often lead to very different reports.

In a study aimed to evaluate the effectiveness of web accessibility evaluation tools, Centeno et al. (2006), classified WCAG 1.0 checkpoints into those that were objectively automated, subjectively automated, semi-automated, and manually tested. However, it appears that a reliable coverage comparison of tools would only be possible if the tools were tested on test suites with appropriate metadata (Strobbe, Herramhof, Vlachogiannis, & Velasco, 2006). Therefore, the W3C WAI Test Samples Task Force is working towards creating and harvesting such test suites based on work done by the BenToWeb project (http://www.bentoweb.org). The result reports can then be easily compared by examining the results that are expressed in a specially developed language, the Evaluation and Report Language (EARL) (http://www.w3.org/WAI/intro/earl.php) and its metadata.

In terms of tools to support authoring of content, as opposed to the inspection of content as described above, current authoring tools offer some support to check for web accessibility. The most important effort in this respect is that of Dreamweaver (Adobe) that incorporates a number of features that promote the development of accessible web pages including a built-in accessibility validation tool and accessibility prompts (http://www.adobe.com/accessibility/products/dreamweaver/overview.html). However many web sites are not built from scratch but on top of web application frameworks, for example, content management systems. The major problem with this type of development is that these systems provide little or no support for accessible content generation. The problems with web accessibility design and evaluation tools have become even more intense with the introduction of the so-called web 2.0 technologies as well as with the advent of user generated content; now the roles of the end user and content provider are somehow blended. To try to provide guidance in this area, WAI has introduced WAI Accessible Rich Internet Applications (ARIA) Suite, which uses a semantics-based approach to try to define a way to make web content and web applications more accessible to people with disabilities based on semantics (http://www.w3.org/WAI/intro/aria.php).

Finally, we need to mention that many tools that are not classified under web accessibility evaluation tools may offer useful functionality for either evaluating specific issues of web accessibility (e.g., web developer, http://chrispederick.com/work/web-developer/) or for simulating user situations (like Vischeck, that simulates colour blindness problems http://www.vischeck.com).

A Simple Example of the Web Accessibility Evaluation Process

This section presents a simple example of the web accessibility evaluation process in order to illustrate the degree to which contemporary web sites are accessible and to bring to light typical accessibility problems. These accessibility problems are of concern to both users and designers. Web site designers are concerned with providing consistent content and services to all potential users regardless of access to technology that includes any type of browser and any kind of assistive device.

We elected to run the example of web accessibility evaluation for web sites in the domain of scientific e-publishing. This domain is one of the first that emerged with the advent of the web, and it is of particular interest for a number of reasons. First, scientific e-publishing web sites are quite mature in their development, making use of state of the art web technologies to support a wide range of services from search and browsing to alerting users for electronic subscriptions and payments. Secondly, they are used by a wide range of users with respect to age, language, culture, skills, and level of technical infrastructure, typically including academic staff, researchers, and students who are located all over the world. Furthermore, accessibility is very much an issue for e-publishing because it is likely that many of the users are disabled because studying online is a viable alternative to traditional education for those with a disability who cannot access bricks-and-mortar educational institutions. We present the accessibility evaluation of the following scientific e-publishing web sites (listed alphabetically):

• ACM (Digital library): http://acm.org/dl
• Elsevier (ScienceDirect): http://www.sciencedirect.com
• IEEE (Publications): http://www.ieee.com/Web/publications/home/index.html
• IGI Global: http://www.igi-global.com
• IngentaConnect: http://pi2.ingenta.com/content
• Nature Publishing Group: http://www.nature.com
• Springer (SpringerLink): http://www.springerlink.com
• Taylor & Francis (Journals): http://www.tandf.co.uk/journals
• Wiley (InterScience): http://www3.interscience.wiley.com
• World Scientific: http://www.worldscientific.com

The presentation of results below is made in random order because it is not an aim of this example to expose problems of specific web sites, but to illustrate the wider picture.

Method

The method used to evaluate the accessibility of e-publishing web sites includes the following two basic steps that are a part of any comprehensive web accessibility evaluation process.

• Use of automated accessibility evaluation tools that check the conformance of a web page according to the WCAG 1.0 (http://www.w3.org/WAI/intro/wcag)
• Expert (manual) inspection of accessibility on the basis of heuristics that complement automatic evaluation

For the purposes of the accessibility evaluation of scientific e-commerce web sites, the following decisions were made:

• The homepages of e-publishing web sites were evaluated in order to give an idea of the current status of accessibility. The homepage is the first page users will encounter when entering the site.
• Two widely accepted tools were selected to perform the automated tool-based aspect of the evaluation: HiSoftware® Cynthia Says™ (http://www.cynthiasays.com/) and Fraunhofer FIT’s Imergo® (http://imergo.com). The evaluation followed WCAG1.0 because there is currently no tool that can support the automatic evaluation of web accessibility according to version 2.0 of the guidelines.

• A set of heuristics to support human judgment in order to perform expert (manual) inspection of web accessibility were selected. These heuristics were proposed by W3C.WAI and are commonly used in accessibility evaluations and refer to viewing the web site under simulated constrained conditions, including
• disabled colours,
• disabled style sheets, and
• disabled images.

A practical way to perform these heuristics is to use the Mozilla Firefox browser with the web developer toolbar, which enables turning off/on various web design elements to ensure well-formed design and accessibility.

The aforementioned approach to accessibility evaluation is sufficient in order to give an overview of accessibility problems and indicate the major areas that need improvement. This approach integrates basic steps that can be applied in a time-saving way by evaluators who do not need to be proficient with web development technology. However, this approach is not a complete account of the steps required for a complete web accessibility evaluation process, which is presented later in this paper.

Summary of Accessibility Problems Found

Table 1 shows the accessibility problems of scientific e-publishing homepages identified by the automated evaluation tools used. The results of automated evaluation can vary between web accessibility tools due to their differences in the implementation of automated checks. In particular, Imergo seems to make a more detailed automated web accessibility check in comparison to Cynthia Says with respect to some aspects of WCAG double-A and triple-A priorities.

Table 1. Accessibility problems of scientific e-publishing homepages identified by automated evaluation tools (Cynthia Says, HiSoftware & Imergo, Fraunhofer, FIT).

The overall level of web accessibility for the scientific e-publishing homepages is not satisfactory. All 10 homepages examined had some accessibility problems, however these vary. Some present few accessibility problems, e.g., H2 (Homepage 2) presents five accessibility problems in total when evaluated by the Cynthia Says tool and four problems when evaluated by Imergo. On the other hand, the automated web evaluation for some of scientific publishing homepages is quite disappointing. For example, H1 presents a total of 115 problems when evaluated by Cynthia Says and 246 when evaluated by Imergo. The homepages that present many web accessibility problems are totally inaccessible and would need considerable redesign effort to reach a good level of accessibility. Overall, three sites have quite a large number of problems (ranging from 24 to 106!), five homepages have a few accessibility problems, while two homepages are clean at the single-A level. Similarly, the double-A and triple-A levels of accessibility evaluation also reveal a considerable number of problems for most web sites.

As part of the second step of the method, Table 2 provides an overview of accessibility problems identified by human judgment based on the heuristics discussed above.

Table 2. Accessibility problems of scientific e-publishing homepages identified by human judgment found after applying simple heuristics.

The accessibility problems that were found in scientific publishing homepages include the following:

• Failure to provide alternative text for graphical information: In the cases where images are solely used to convey important information for comprehension and navigation (such as links to other pages, texts, and diagrams), the information conveyed by images will not be available to constrained access contexts, (e.g., users with visual impairments, users with low bandwidth, and users with minimal content display devices).
• Failure to provide alternate and/or redundant text and required associations for supporting user input: Forms are not accessible unless they include alternative text for describing the information requested from the user and associate labels with form controls.
• Failure to allow the user to control the interaction of interactive elements of the page: One of the most traditional usability principles is that the user should be in control of the interaction at all times; this is not true with some inaccessible interactive web technologies such as animated images and auto-refreshing of the contents of a web page.
• Failure to provide consistent and effective navigation when accessed under a simulated constrained context: Many homepages suffer from significant navigation problems such as the disappearance of navigation bars (e.g., when they are designed as images with no alternative text), same colour of text with background (when colours are turned off), and others.
• Failure to provide consistent and meaningful information organisation when accessed under simulated constrained context: Some homepages suffer from significant disruption to the organisation of information, especially when style sheets are turned off.

Again, it is evident that there are significant accessibility problems in the scientific e-publishing homepages and that not one homepage was judged as adequately designed to meet all the criteria offered by the heuristics. Six out of ten e-publishing web homepages have at least one major accessibility problem. Also, all sites have minor accessibility problems. These problems are different for each homepage and mainly concern information organisation, navigation, visibility, and user control.

A comprehensive accessibility evaluation cannot stay at the level of homepages but must also include other sections of the scientific e-publishing web sites. However, given that the homepages of scientific e-publishing web sites are not accessible, it is quite safe to assume that the rest of the application will include at least the same types of problems. This example demonstrates that even the contemporary scientific publishing sites present a wide range of accessibility problems. Scientific publishing homepages will reveal several problems when accessed in constrained situations, such as, access from mobile devices, black and white print-outs, access from voice browsers, access from assistive technologies used by people with motor disabilities, and so on.

Steps Towards a Practical Methodology for Web Accessibility Evaluation and Maintenance

The example of web accessibility evaluation is an important but rather small part of the practical measures that need to be taken up to ensure web accessibility. To try to work out a practical methodology for evaluating web accessibility is very important for at least two reasons. Firstly, web accessibility is a huge, diffuse and complicated more or less theoretical field of human-computer interaction so any attempt to come up with a practical design tool and advice is valuable. Secondly legislation regulating web accessibility, especially concerning official government and federal web sites and services, are now in force and are being extended into the private corporate domain. These laws are pitched at a general and abstract level, so concrete advice and practical guidelines are urgently required by anyone who designs for the web if they are to ensure that their designs comply.

Therefore we propose that a holistic approach to the management of web accessibility evaluation should include (a) identification of user requirements and set up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of an accessibility policy.

Identification of User Requirements and Set Up of Accessibility Goals

User accessibility requirements can be identified when accessibility is included in the goals of a user-centred approach for design and evaluation. The rationale behind important work on web accessibility is that of “universal design” or “design-for-all” that includes all users without discriminating particular (types of) disabilities. On the other hand, there may be particular user accessibility requirements to be included in a web development/evaluation project that are related to specific target user groups. The general accessibility requirements of particular user groups have been reported in literature (e.g., see WebAIM, Considering the User Perspective: A Summary of Design Issues, http://www.webaim.org/articles/userperspective); however, there may be particular requirements for specific groups or access contexts that need to be identified.

When there is a need to incorporate particular requirements for specific user groups (e.g., deaf users), it is critical to remember the universal design principle and thus ensure that the web site will conform to at least a minimum level of web accessibility specifications. At present the WAI web accessibility specifications allow for levels of conformance to the guidelines. Thus, it is possible for web designers to achieve accessibility conformance at the single-A level (lowest level) and then specialise their designs for particular user groups. However, it needs to be noted that according to W3C.WAI WCAG2.0: “When a Web page is one of a series of Web pages presenting a process (i.e., a sequence of steps that need to be completed in order to accomplish an activity), all Web pages in the process conform at the specified level or better” (W3C, 2008, Conformance Requirements, number 3). Therefore, accessibility goals may be set to meet the following:

• A web accessibility specification (e.g., WCAG1.0) can occur at some level of conformance (e.g., double-A = AA).
• The requirements for (a) specific user group(s) (e.g., vision-impaired, mobility compromised, etc.) can be identified from contextual analysis provided that a minimum level of conformance to web accessibility specifications is achieved.
• Testing with well-known assistive software, such as oral browsers, screen readers, screen magnifiers, and so on, is advisable, again provided that a minimum level of conformance to web accessibility specifications is achieved.

Besides the requirements that arise from the users themselves, the web site owners can decide upon a hierarchy of services and concentrate in the first instance on making the top level accessible. For example, in the case of scientific electronic publishing a primary (top level) use might be to download a paper in a journal, whereas other services, such as the alerting service, may be given a lesser priority and be dealt with at a later stage. This does not mean that these services should not be made accessible, because by leaving them inaccessible it is not possible to claim conformance with the W3C, but more importantly it creates a two-tier user base: those who can use everything and those who can only use certain services. However for practical purposes, when faced with making a whole set of services accessible, it is reasonable to prioritise.

Web Accessibility Evaluation and Redesign Process

As the accessibility evaluation example has shown, a rapid assessment of web accessibility is possible by using free accessibility tools (automated tools that measure technical accessibility conformance to guidelines) and applying simple heuristics (manual, expert-based accessibility inspection). The use of the automated tools and the manual inspection that uses heuristics are next described as stage two of the proposed methodology. Finally, a holistic approach to accessibility evaluation and redesign also requires another important element in the process: that of user involvement through user testing. These phases of web accessibility evaluation have been also identified by other researchers such as Arch (2008), who identifies three types of web accessibility testing: automated testing, which is carried out by software tools; manual testing, which is carried out by human evaluators; and user testing, which is carried out by end-users.

Tool-based accessibility conformance to guidelines

Accessibility evaluation tools scan the source code of a web page using interpretations of either WCAG or the United States Rehabilitation Act Section 508 standard. The use of these tools is the first step for accessibility evaluation because they can quickly identify accessibility problems that can be identified at the level of the source code of a web page and produce reports with accessibility errors and warnings. These tools save the designer from the task of inspecting source code for the evaluation of accessibility and provide a first account of accessibility problems. However they cannot provide a complete account of accessibility problems mainly because accessibility is not a solely technical issue, but primarily requires human judgment. According to WebAIM (2010) of the combined 65 checkpoints in WCAG 1.0 Priority 1 through Priority 3, only 19 can be partially evaluated automatically.

A major problem for accessibility tools is that the vast majority of these tools are designed for fast evaluations of single web pages. There are no public accessibility tools that can reside on the web server, constantly monitor the web site, and are capable of crawling an entire web site to identify accessibility problems. Currently, research on the design of new generation accessibility tools attempts to address these concerns such as the MAGENTA tool (Leporini, Paterno, & Scorcia, 2006) and the BenToWeb benchmarking tools that will include the aforementioned capabilities (Herramhof et al., 2006).

Expert (manual) accessibility inspection

Manual evaluation includes a number of steps that must be followed by a designer to check the accessibility of web pages according to guidelines. These steps are another essential task of accessibility evaluation that can assess accessibility in terms of the aspects that require human judgment. Such aspects include, for example, that alternative text for images substantially describes the meaning of an image in textual form, in case this is needed (i.e., when the image conveys information and is not used for other purposes such as decoration), and that the use of colours promotes accessibility of text and images if viewed in a constrained context of use (i.e., when printed by a black and white printer).

Expert inspections of accessibility can identify a considerable number of problems that are not possible to find by using automated accessibility tools alone. Typical problems in this respect were demonstrated in the evaluation example presented in previous section. Typical inspections of web accessibility include the following:

• Inspection of human checks for accessibility according to the WCAG guidelines: WCAG explicitly refers to accessibility issues that require human check and provides techniques that can assist expert evaluators to simulate constrained access situations that users may meet due to limitations and characteristics of (assistive) technology and/or access environment. For example, disabling the colours of a web site can simulate access situations for people with colour blindness, as well as a need to view or print in black-white or in a low lighting environment.
• Inspection of accessibility following simple heuristics: There is a number of heuristics that complements those used in the example presented in section 3, (i.e., disabled images, disabled styles, and disabled colours) such as turning frames off, turning sound off, navigating without a pointing device, accessing the web site via multiple browsers, accessing the web site via text browsers, accessing the web site via a voice browser, testing with different screen resolution, and others.

The expertise required to conduct an accessibility evaluation should be wide-ranging, including both organisational and technical skills. According to the W3C.WAI an accessibility evaluator should have “an understanding of web technologies, evaluation tools, barriers that people with disabilities experience, assistive technologies and approaches that people with disabilities use, and accessibility guidelines and techniques” (2010, Evaluating Accessibility/Combined Expertise page, Recommended Expertise section).

User testing of accessibility

The involvement of users with disabilities in accessibility testing is an important aspect of accessibility evaluation. As Paddison and Engefield (2004) point out “it is not enough to follow accessible guidelines and make the appropriate technical accessibility changes… people with special accessibility needs should be considered as a distinct user profile with their own requirements, within a user-centred design process” (p. 508). It has been observed over and over that user testing with people with disabilities contributes to a better understanding of accessibility issues by all people involved, and especially web developers (Foreign & Commonwealth Office, 2010). For example, having web developers see people with disabilities accessing a web page with a voice browser helps them to identify related accessibility problems that their web site may have, such as the inappropriateness or absence of alternative text, the ordering of controls in a form, etc.

However, a user-centred accessibility evaluation will not be effective unless the site is already at a minimum level of accessibility. Even so, the inclusion of users in the accessibility evaluation process can also identify various usability, as opposed to basic accessibility, problems. Analytic methods and guidelines for involving users in accessibility evaluation include the work of Gappa and Nordbrock (2004) and Petrie, Hamilton, King, and Paven (2006).

Web Accessibility Policy

In order to reach and maintain web accessibility, web site owners need to establish a web accessibility policy that will be applied during the design, development, and operation of their web site. The accessibility policy needs to go beyond an “accessibility statement” that typically declares the specifications that web sites conform to. The policy needs to define systematic procedures, checks and tools for content updating. It has been many times observed that web sites that were accessible when first released gradually lost their accessibility when the content was updated. As an example, educators of an accessible e-learning platform or users of an accessible web log need to add content that conforms to web accessibility; this can be achieved if people are aware of accessibility issues and are able to use tools to check the content accessibility when it is inserted into the site. This is particularly important for web 2.0 applications that allow for user generated content.

Currently, web accessibility statements have been established for a number of, mainly academic and governmental, web sites and in a few e-commerce systems that are designed with the participation of organizations of people with disabilities, such as the RNIB (e.g., Gladstone, Rundle, & Alexander, 2002). An important issue for the design of the web accessibility policy is what standards, guidelines, methods, and processes should be identified from related work. In spite of the fact that there is some work in respect of accessibility policy (for a review, see Gulliksen, Harker, & Vanderheiden, 2004), we are still some way away from the recognition of a systematic accessibility policy that is implementable to the level of content generation by end users.

Summary and Conclusions

There are many reasons for web sites to be accessible. The social responsibility of web site owners requires that they provide accessible web-based information and services. The market segment for people with disabilities including the elderly is too large to be ignored; these people want to autonomously access and use the web for education, entertainment, and commerce. Furthermore, the robustness of technical development when accessibility is taken into account is another major argument for taking up this approach. Last, but not least, there are already legal frameworks in place for governmental organizations to apply accessibility to the design of their sites. Other entities are also expected to be held liable for providing information and services on equal terms for all, and if they are not punished by legal systems (for instance, the U.S. Pharmacy chain, Target (http://en.wikipedia.org/w/index.php?title=National_Federation_of_the_Blind_v._Target_Corporation&oldid=318445849 accessed February 25, 2010), then some people suggest that steps be taken to have them ostracised by public opinion, using the well known technique of “name and shame.”

This paper argues for a practical approach to web accessibility evaluation that provides recommendations for the methods and tools to be used as well as maintenance processes. This approach is required in order to address the fact that the current set of web accessibility tools and specifications need technical knowledge for their comprehension and application. Designers are continuously pressed to apply governmental legislation to web accessibility, which is often prescribed in specifications when they require practical methods, guidelines, and tools to proceed.

This paper proposes a practical methodology for evaluating web accessibility that includes (a) identification of user requirements and set up of accessibility goals, (b) web accessibility evaluation and redesign process, and (c) establishment and follow-up of an accessibility policy. The rationale behind this proposal is that of design-for-all, which incorporates the requirements of all people including those with special needs (and without discriminating types of users or disabilities) into the design process.

To illustrate the web accessibility evaluation process, this paper presents a simple example in the domain of electronic publishing and discussed the accessibility problems found that are typical of many other contemporary web sites.

The work presented in this paper reports on issues that need to be considered by HCI researchers, interaction design practitioners, and usability professionals for inclusive design of web applications that are complementary to web usability engineering.

References

• Arch, A. (2008, May 14). Web accessibility for older users: A literature review W3C working draft. Retrieved February 18, 2010, from http://www.w3.org/TR/2008/WD-wai-age-literature-20080514/
• Brewer, J. (2005, May 5). How people with disabilities use the Web: Working-group internal draft. Retrieved February 18, 2010, from http://www.w3.org/WAI/EO/Drafts/PWD-Use-Web/
• Caldwell, B., Chisholm, W., Slatin, J., & Vanderheiden, G. (2008, December 11). Web content accessibility guidelines (WCAG) 2.0.: W3C Recommendation. Retrieved February 18, 2010, from http://www.w3.org/TR/2008/REC-WCAG20-20081211
• Centeno, V.L. Kloos, C.D., Fisteus, J.A., & Alvarez, L. (2006 May, 22). Web accessibility evaluation tools: A survey and some improvements. Proceedings of the International Workshop on Automated Specification and Verification of Web sites (WWV 2005), Electronic Notes in Theoretical Computer Science, 157 (2), 87-100.
• European Blind Union. (n.d.). EBU response to the European Commission’s consultation on content online in Europe’s single market. Retrieved February 25, 2010, from http://ec.europa.eu/avpolicy/docs/other_actions/contributions/eur_blind_union_col_en.pdf
• Europe’s Information Society. (n.d.). i2010 – A European Information Society for growth and employment. Retrieved February 18, 2010, from http://ec.europa.eu/information_society/eeurope/i2010/index_en.htm
• Foreign & Commonwealth Office. (2010). Accessibility statement. Retrieved February 18, 2010, from http://www.fco.gov.uk/en/accessibility
• Gappa, H., & Nordbrock, G. (2004). Applying Web accessibility to Internet portals. Universal Access in the Information Society, 3(1), 80-87.
• Gladstone, K., Rundle, C., & Alexander, T. (2002). Accessibility and usability of eCommerce systems. In K. Miesenberger, J. Klaus, W. Zagler (Eds.). ICCHP 2002, LNCS 2398 (pp. 11–18) Berlin/Heidelberg. Springer-Verlag.
• Gulliksen, J. Harker, S. Vanderheiden, G. (2004) Guidelines, standards, methods and processes for software accessibility, Special issue editorial, Universal Access in the Information Society, (2004) 3: 1–5.
• Harper, S., & Yesilada, Y. (2008). Web accessibility and guidelines. In S. Harper & Y. Yesilada, (Eds.) Web Accessibility: A Foundation for Research (pp. 61-78). Springer.
• Herramhof, S., Petrie, H., Strobbe, C., Vlachogiannis, E., Weimann, K., Weber, & G.,Velasco, C.A. (2006, July 12-14). Test case management tools for accessibility testing. In Klaus, J, Miesenberger, K., Burger, D., & Zagler, W. Computers Helping People with Special Needs. Proceedings of 10th International Conference, ICCHP 2006, Linz, Austria.
• Kapsi, M., Vlachogiannis, E., Darzentas, J., & Spyrou, T. (2009). The usability of Web accessibility guidelines: An approach for evaluation. In Universal Access in Human-Computer Interaction. Applications and services (pp. 716–724). Berlin/Heidelberg. Springer. http://dx.doi.org/10.1007/978-3-642-02713-0_76
• Koutsabasis, P. (2010, in press). E-business and Web accessibility. In Lee (ed.) Encyclopedia of E-business Development and Management in the Global Economy. IGI Global.
• Lazar, J., Dudley-Sponaugle, A., & Greenidge, K.D. (2004). Improving Web accessibility: A study of webmaster perceptions. Computers in Human Behavior, 20, 269–288.
• Leporini, B., Paterno, F., & Scorcia, A. (2006, September). Flexible tool support for accessibility evaluation, article in press. Interacting with Computers, 18 (5) 869-890.
• Loiacono, E.T., & McCoy, S. (2006). Web site accessibility: A cross-sector comparison, universal access. Information Society, 4, 393–399.
• Moss, Τ. (2006, September). WCAG 2.0: The new W3C accessibility guidelines evaluated. Retrieved October 29, 2009, from Webcredible Web site: http://www.webcredible.co.uk/user-friendly-resources/web-accessibility/wcag-guidelines-20.shtml
• Nietzio A., Strobbe C., & Velleman E., (2008, July). The unified Web evaluation methodology (UWEM) 1.2 for WCAG 1.0. In M Austria, LNCS 5105 (pp. 394­401). Berlin-Heidelberg: Springer-Verlag.
• Paddison, C., & Englefield, P. (2004, June). Applying heuristics to accessibility inspections.  Interacting with Computers 16 (3), pp. 507-521.
• Petrie, H., Hamilton, F., King, N., & Pavan, P. (2006, April 22-27). Remote usability evaluations with disabled people. Proceedings of CHI’2006. Montréal, Quebec, Canada.
• Strobbe, C., Herramhof, S., Vlachogiannis, E., & Velasco, C. A. (2006, July). Test case description language (TCDL): Test case metadata for conformance evaluation. In Miesenberger K et al. (Eds.). Proceedings of the 10th International Conference ICCHP 2006 (Linz, Austria), LNCS 4061 (pp. 164-171). Berlin/Heidelberg: Springer-Verlag.
• United Nations Statistics on Human Functioning and Disability. (2010). Retrieved February 18, 2010, from http://unstats.un.org/unsd/demographic/sconcerns/disability/disab2.asp
• Velasco, C. A., Denev, D., Stegemann, D., & Mohamad, Y. (2008, April 21-22). A Web compliance engineering framework to support the development of accessible rich internet applications. In Proceedings of the 2008 international Cross-Disciplinary Conference on Web Accessibility (W4a). Beijing, China.
• Velleman, E., Strobbe, C., Koch, J., Velasco, C.A., & Snaprud, M. (2007, July 22-27). A unified Web evaluation methodology using WCAG. In: Stephanidis C (Ed.). Universal Access in Human-Computer Interaction. Applications and Services (4th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2007, HCI International 2007 Beijing, China.
• Velleman, E., Velasco, C.A., & Snaprud, M. (2007). D-WAB4 unified Web evaluation methodology (UWEM 1.2 Core) Retrieved February 18, 2010, from http://www.wabcluster.org/uwem1_2/UWEM_1_2_CORE.pdf
• Web Accessibility in Mind (WebAIM). (2010). Introduction to web accessibility. Retrieved February 18, 2010, from http://webaim.org/intro/
• Web Accessibility Initiative (WAI). (2010). Retrieved February 18, 2010, from http://www.w3.org/WAI/
• World Wide Web Consortium (W3C). (2008, December 8). Web content accessibility guidelines (WCAG) 2.0. Retrieved on February 18, 2010, from http://www.w3.org/TR/WCAG20

Published in: in Volume 5, Issue 4,