Investigating the Usability of E-Textbooks Using the Technique for Human Error Assessment

Introduction

Reading is a task that most people, especially college students, perform throughout their day. With the increase in mobile devices, a large portion of this reading is now done online on a variety of screen sizes. Traditional paper books are slowly being replaced by electronic reading devices, such as e-readers (i.e., Kindle, Nook, etc.) and tablet computers (i.e., iPad, Kindle Fire, etc.). Electronic versions of most published books are now available, which can be accessed from multiple devices (i.e., desktop computer, laptop, tablet, cell phone) using a variety of reading applications.

E-Book and E-Reader Usability

Several qualitative studies on the usage of e-books have been reported in the literature (Abdullah & Gibb, 2008b, 2009; Berg, Hoffmann, & Dawson, 2010). These studies have shown that users assume e-books will have similar qualities to paper books, but also have the expectation that they will provide features similar to other online resources found on the Internet (such as searching, hyperlinks, and standard navigational elements). In a usability study of four different e-books, Berg et al. (2010) reported several usability issues encountered while undergraduates completed a series of information retrieval tasks. These included difficulty navigating the e-books and using certain features (such as search) effectively despite the fact that they were very computer savvy (Berg et al., 2010).

E-reading devices vary in size and features. There are dedicated e-reader devices that are made for the sole purpose of e-book reading, and there are tablet computers that support e-book reading through an e-book reader application. Some e-readers use e-ink technology that is paper-like in quality (e.g., Amazon Paperwhite). Other e-readers have full color touch screen capabilities. These different types of devices and applications provide flexibility to their users and allow them to read almost anywhere at any time even in different illumination levels and environments (i.e., in the dark, outside).

Several studies have been published comparing the usability of different e-reader devices. Nielsen (2010) compared reading speed for short stories on the iPad and Kindle to printed text and found that reading on both devices was slower than the printed text but were equally satisfying. Jardina and Chaparro (2012) evaluated the usability of three e-reader devices (iPad, Kindle Fire, and Nook Tablet) for book reading and navigation. All three e-readers offered the ability to read books, bookmark page(s), highlight text, and take notes. Overall, the Nook Tablet was significantly preferred by participants; however, there were no significant differences between the e-readers on perceived workload or satisfaction. Each device was found to have both strengths and weaknesses. For example, participants preferred the iPad for highlighting and making annotations and the Nook for its clear menu structure. The Kindle was preferred for tasks that required changing text size and text searching.

Siegenthaler, Wurtz, and Groner (2010) investigated five e-reading devices and found that the perceived legibility of the text on the devices was comparable to a paper book upon initial exposure. However, after usage, participants rated the perceived legibility differentially based on their user experience rather than on their efficiency. Users experienced frustration while interacting with some of the devices which influenced their post assessment and satisfaction.

In a study of three touchscreen e-reading devices (i.e., iPad, Nook Color, and Kindle Fire), participants were asked to complete basic book navigation tasks, such as highlighting, bookmarking, and so forth (Jardina & Chaparro, 2012). Subjective data (i.e., task difficulty and satisfaction) and objective data (i.e., such as number of taps and time to complete tasks) were collected. Each device proved to have some strengths and weaknesses. The Nook was preferred for its menu structure, the iPad for making notes and highlights, and the Kindle Fire for changing text size and searching.

In a study examining longer term usage of e-reader devices, Clark, Goodwin, Samuelson, and Coker (2008) conducted focus groups with participants who used a Kindle device for a month. The participants reported the device to be adequate for leisure reading, but limited for academic use due to the limited number of books available. Participants also reported they were not willing to pay for the device and were not impressed with the graphics.

E-Textbook Usage in the Classroom

Many schools and universities have experimented with the use of e-textbooks as a replacement or alternative to traditional paper textbooks (Colgrass, 2011). In one of the first documented cases, Reed College integrated Kindle DXs into the classroom. Students liked the portability of the devices, along with the paper savings, and legibility of the textbook material. Students did not like that the device did not support all of their course materials and that they were unable to view complex and color material due to insufficient resolution and the grayscale display. They also reported the device to be slow to load content and to navigate using the manual control button. Multiple textbooks also could not be referenced easily. As a result, students concluded that a multifunctional device, like a tablet computer, may be better for the academic environment (Marmarelli & Ringle, 2009). In a follow-up study, the college gave students an iPad for use in the classroom. Students reported that tasks involving highlighting and annotating were easy to do, and that searching and switching between texts was easier and faster than on paper (Marmarelli & Ringle, 2011).

McKelvain (2011) investigated the use of features such as highlighting and annotating on an e-textbook for writing and studying tasks. Their results showed that students highlighted frequently, but would often use outside materials (e.g., notecards, notepads, word processing document, etc.) instead of the annotation feature of the e-textbook. This has been shown to be true for a variety of devices for textbook access, including the iPod Touch and cell phone (Chao & Chen, 2009; Johnson, Levine, Smith, & Stone, 2010).

Differences exist in how people read digital text versus printed material. Liu (2005) found that when reading digital text, users reported that they are more likely to browse the material and scan for keywords but highlight and annotate less. Berg et al. (2010) reported that undergraduates adopt a less linear strategy when reading an e-book than a printed book. Reading strategy also differs when reading for leisure (i.e., novels) than when reading a textbook. During leisure reading, readers typically read from beginning to end and do not focus on marking or annotating specific material. During textbook usage, readers are more likely to skim, scan, and navigate non-sequentially through the textbook to find information (Berry, Cook, Hill, & Stevens, 2011; Horney & Anderson-Inman, 1994; Wandersee, 1988). Table of contents, index, and search capability are important features for navigation. Abdullah and Gibbs (2008b) found that readers consistently and successfully used an index when reading a paper book but did not realize an index was even available when using the e-book. In a study examining reading strategies on an e-textbook on a Nook e-reader, Schugar, Schugar, and Penny (2011) found that students were less likely to use highlighting and to take notes on the Nook than they were on the traditional textbook. Bookmarking pages, however, was reported to be done about as much as folding the corner of a page in a traditional book. The authors attribute the lack of annotation and highlighting activity to a “steep learning curve” of how to use the features on the Nook. As more schools and students switch to using e-textbooks instead of traditional textbooks, it is important to understand how the user interface design and implementation of advanced features facilitates the strategies students employ when studying.

E-Textbook Reader Applications

As mobile device usage increases, more e-textbook reading applications and associated mobile apps have been developed by both textbook publishers (i.e., McGraw-Hill) and online book providers (i.e., Amazon, Barnes & Noble, Apple, Chegg, Inkling). Some of these applications offer a variety of features, such as the ability to make handwritten notes, use different colors for highlighting, and the ability to share notes and bookmarks through social networks.

Students often choose the cheapest option and ignore the features of the reader application. While the book content is the same in each case, the user interface of the e-textbook reader application (or application “look-and-feel”) often differs. Users must learn how to make a note, search for a key term, highlight, and navigate the text in each application. While there have been some suggestions to standardize e-textbook design guidelines (Wilson & Landoni, 2002), adherence to these guidelines does not appear to be mandated and as a result, the interfaces vary considerably. Furthermore, the guidelines put forth by Wilson and Landoni (2002) were largely based off of evaluations of e-books on webpages and e-book readers that are now outdated.

Jardina and Chaparro (2013) compared the usability of two e-textbook applications (i.e., Inkling and Kindle) for tasks related to studying for a quiz (e.g., bookmarking, highlighting, searching for terms, etc.). Issues were discovered that influenced overall ease of use for both applications. For example, the table of contents was more difficult to navigate on the Kindle application and finding a specific page in the textbook was more difficult using the Inkling application.

Purpose

Given the diversity of interfaces offered in e-textbooks, the aim of this paper is to compare and contrast the features offered in eight popular e-textbook reading applications and to evaluate their usability from an error analysis perspective. The eight e-textbook applications included Inkling, Kno, CourseSmart, VitalSource, Chegg, iBooks, Kindle, Nook.

Table 1 lists a series of tasks students commonly complete when using an e-textbook and whether it was present in each of the eight applications. It can be seen that all applications offered a search capability, a notebook feature, and text highlighting. Sharing, changing the text size, and navigating directly to a specific page were less frequent features offered.

Table 1. Features Present on the E-Textbook Reading Applications

In addition to the presence of the features, we were interested in how the features were implemented because previous research suggests that aspects of the user interface design influence user performance and satisfaction.

Methods

To compare features of the eight e-textbook applications the Technique for Human Error Assessment (THEA) method was used to identify errors and consequences. THEA is a structured human error identification (HEI) approach that incorporates Norman’s action model in identifying user interface errors (Pocock, Fields, Harrison, & Wright, 2001). It uses a scenario analysis to consider context and then employs a series of questions in a checklist based upon goals, plans, performing actions, and perception/evaluation/interpretation. THEA is different from other methods, such as a Cognitive Walkthrough, in that it provides the reviewer a number of prompts related to the application goals, plans, and actions (Stanton, Salmon, Rafferty, Walker, Baber, & Jenkins, 2013).

THEA Procedure

A THEA is completed by an expert or a group of experts for a particular task. The four tasks examined using this approach were bookmarking, making a note, locating a note with the notebook, and searching for a word. For each task, the researcher started on an open page in the middle of an e-textbook. If a main menu was displayed upon opening the e-textbook, it was left open; however, if the main menu was only opened by tapping on the screen it was not opened at the start.

Before a THEA analysis can begin, several tasks must be completed, such as understanding the functionality of the device (in this case an iPad) and system (e-textbook applications), and completing a task analysis in order to fully understand all the goal and sub-goals of each task. While completing the THEA, the analyst keeps in mind these goals and sub-goals to identify where errors may arise and what consequences may occur due to the errors. The questions the analyst answers to carry out a THEA assist the analyst in discovering errors (Pocock et al., 2001). An example of a THEA with the standard questions for one application for one task can be seen in the Appendix.

The items in the THEA are split into four categories based on failures users are likely to make within Norman’s action cycle. The four categories are (a) Goals, Triggering, Initiation; (b) Plans; (c) Performing Actions; and (d) Perception, Interpretation, and Evaluation. Each category assumes a certain kind of cognitive failure. A cognitive failure in Goals, Triggering, and Initiation assumes a lost, unachievable, or conflicting goal. A faulty, wrong, or impossible plan is a cognitive failure in Plans. Forgetting or carrying out an action incorrectly is a cognitive failure in Performing Actions, and failing or misperceiving information is a cognitive failure in Perception, Interpretation, and Evaluation (Pocock et al., 2001).

For each task there are multiple yes or no questions located in the Questions column. The reviewer answers each question and then provides feedback on why they answered it in that particular way in the Causal Issues column. In the Consequences column, the reviewer states the consequences of the feedback given in the Causal Issues column. Consequences refer to negative aspects, like not being able to complete the task, completing the task incorrectly, a delayed response, and so forth. The consequences column is crucial because it explains why a user may not be able to complete a task.

Materials

An iPad 3 (running iOS 7.1.2) was used to run all the e-textbook applications used in this study. (Note: The applications can be run on multiple platforms [e.g., Galaxy Tab, iPad, etc.], but for the purpose of this study all applications were only tested on one platform.) The eight e‑textbooks applications were chosen based on their dominance in the educational e-textbook domain. (Note: The applications chosen for usage in the study were e-textbook and e-book applications.) If a user could purchase an e-textbook using the application, it qualified for being chosen for analysis on this study. However, some applications, such as Chegg and Inkling, are used solely for e-textbooks, while applications such as Kindle and iBooks can be used for leisure e-book reading as well as e-textbooks. For the purpose of this study, only e-textbooks were investigated on all of the applications; therefore, all applications are referred to as e-textbook applications.

Results

A THEA was conducted for all four tasks on all the e-textbook applications. For each task, examples of the interface demonstrating both positive and negative aspects of the features were identified. See the Appendix for a sample THEA output.

Bookmarking

Bookmarking is an important task for e-textbook readers because it allows them to mark a point of interest to which they can quickly return. For the Bookmarking task, the researcher completed the process of adding and saving a bookmark. For the bookmarking task, Kno provided an intuitive bookmark icon and required the fewest taps to access (Figure 1). Inkling used an unconventional icon to access the bookmark (Figure 2). In CourseSmart, the bookmark icon was located in the bottom left corner instead of the traditional top right and presented a 2- to 3-second delay before showing that the bookmark was recorded. Chegg changed the appearance of the bookmark icon once set but used a low-contrast color that made it more difficult to view (Figure 3). A summary of the positive and negative aspects of each e-textbook application can be seen in Table 2.

Table 2. Positive and Negative Aspects of the Bookmarking Feature

Figure 1. The bookmark feature on Kno without the main menu open (1) and with the main menu open (2). When the user taps on the bookmark icon, the bookmark turns orange (3).

Figure 2. The bookmark feature on Inkling. When a user taps on the arrow (1), a pop-up menu appears (2).

Figure 3. The bookmark icon on Chegg before the bookmark is made (1), and the bookmark icon on Chegg after a bookmark is made (2). The arrow menu to access the bookmark on Inkling (3), and the unconventional star to indicate a bookmark has been made on Inkling (4). The bookmark icon on iBook before the bookmark is made (5), and the bookmark icon after the bookmark has been made (6).