Abstract
Eye-tracking has been employed in usability engineering for many years because, among other things, it affords usability practitioners information about where users focus their attention. It helps practitioners identify the extent to which the visual display elements presented on many interactive products enhance or detract from the user experience. Eye movement data offer system developers and usability engineers information about visual attention, visual search efficiency, and visual information processing while users interact with a system.
In this study, we tracked participants’ eye movements as they viewed newspaper and TV-oriented news Websites. We used several visual attention measures (number of fixations, fixation duration, gaze time, and saccade rate) and scan path analysis to investigate whether ocular behavior differed by type of news site.
We found that newspaper and TV-oriented site types did not influence measures of visual attention. However, the areas where participants fixated differed by site type. In addition, there was greater across-user variability in the viewing of newspaper homepages compared to TV homepages. Finally, we report on the utility of examining visual attention using scan path analysis and string-editing methods. These methods were especially useful for identifying fixation areas as well as variability in participants’ scan paths.
Practitioner’s Take Away
The following were the main findings of the study:
- There was greater across-user variability in the viewing of newspaper homepages compared to TV-oriented homepages.
- Possibly because newsprint homepage layouts resembled newspapers and employed headlines as primary navigation to news stories instead of text link groupings, visual traces were less concentrated and less efficient.
- Text link groupings and content directed visual attention. Text link groupings should be placed consistently across sites.
- Visual traces showed that news story text links in the body of the page captured visual attention and served as a central point of regard.
- Newspaper sites may reduce variability in visual attention by grouping links consistently in upper regions of the page.
- Participants initially focused attention in the upper regions of the page and then moved to the page body. The upper regions of the page appeared to serve an orienting function and as navigational cues.
- Initial fixations occurred in the browser bar and in the site branding areas. To support a more efficient information search, place link groupings directly below these areas.
- String-editing and OMA provided data about fixated areas of the display. They were useful methods for identify areas of fixation in a display as well as scan path variance within and across users.
- Eye-tracking provides a valuable tool by itself or in conjunction with other usability methods to help usability practitioners examine users’ visual attention allocation and search efficiency when using interactive systems.
Introduction
Print and television are two dominant media outlets for the news. Unlike radio, they are predominately visual, although television provides both visual and auditory information. Newspaper and television news organizations each have long-established traditions for reporting news in their respective media, which engenders distinct human behaviors that shape how people obtain news. For instance, television broadcasts present short video-based stories that are sequenced linearly and fit within a specified timeframe for on-air viewing. One’s access to and the sequence of such stories is controlled by the news organization. There are no archives of stories immediately available, unless the viewer records the broadcast. Conversely, newspapers primarily offer text content intended for in-depth reading whereby the reader selects a story of interest and reads it for however long he or she desires; thus, readers control the access to the information presented in the paper. Additionally, the printed paper can be archived for later reading.
Increasingly, news organizations employ the Web as an outlet to accommodate a growing number of people who seek news online. More than fifty million Americans utilize the Internet daily to keep informed about local, national, and international events (Horrigan, 2006), a trend that will likely continue. Millions of people seek news through newspaper affiliated Websites (Jesdanun, 2009; Society for New Communications Research, 2007) and TV-oriented sites associated with television news stations, both of which pervade the Web.
While the Web is another vehicle for disseminating news, it represents a highly dynamic interface characterized by a proliferation of motion and static media and interactivity that supersedes what is found in either traditional newsprint or on television. It is not yet clear how it shapes the way users attend to news information. In addition, newspaper and television news organizations often design sites to underscore their newsprint or television traditions and reporting methods, resulting in distinctive information and graphical layouts that will likely influence user behavior. For example, the homepage of The New York Times, a newspaper-oriented site, reflects a newsprint layout that engenders reading. It presents a minimalistic design with headlines and article summaries dispersed throughout the page, similar to a newspaper. CNN, a TV-oriented site, features minimal text and more concentrated listings of links, many with corresponding video camera icons indicating that video is available. Visitors spend between 25 and 35 seconds on a Website homepage before leaving (Nielsen & Loranger, 2006) and they typically read pages by scanning. Because The New York Times features headlines and story summaries distributed throughout the homepage, and high density text, one might expect a user’s visual attention to be more dispersed and browsing to be prolonged compared to a site like CNN, which presents limited text and a concentration of navigation links in the upper portion of the display. Moreover, people perceived newspaper and TV-oriented sites as different, with TV-oriented sites receiving more positive ratings in terms of screen layout, design, and overall rating (Gibbs, Bernas, & McKendrick, in press). In a survey that examined newspaper, newsweeklies, and TV-oriented sites in the Houston, Texas area, almost half of the sites that attracted 10% or more of the immediate market were TV-oriented (The Media Audit, 2005).
The aforementioned factors are compounded by the fact that a convergence of newspaper and television media are occurring, dramatically increasing the complexity of the visual landscape. On American television, it is common for news programs to use visual treatments such as split-screens and animated text that are typically associated with the Web (Josephson & Holmes, 2008, p. 387). The Web has adopted elements of television. While a provider’s site may reflect its media origins, it will likely integrate characteristics not typically associated with that form of media. For instance, users can watch video on a newspaper site such as USA Today and The New York Times. In the United States, video is available on 92% of the major 100 newspaper Websites (Society for New Communications Research, 2007). Alternatively, users can read news articles on a TV-oriented site such as CNN, as well as participate in blogs about various topics (Gibbs, 2008).
Better understanding of how people allocate visual attention on newspaper and TV-oriented sites is an important area of inquiry for several reasons. First, print and television media and associated reporting approaches pervade traditional media outlets and the Web. Millions of people use these sites daily. Second, major newspapers and TV providers each represent their traditional media origins online in unique ways and these representations are perceptual to users. The extent to which they support or distract user attention is not yet clear but should be examined given the pervasiveness of these sites. Third, the Web has emerged as a distribution channel for the news. Compared to newspapers and television, it affords distinct interaction modalities that shape how users access and attend to newsprint and television media. For instance, TV news broadcasts, prepared as video for the Web, can be made nonlinear and user controlled but they are often segmented to increase accessibility, which constitutes a dramatically different experience of obtaining the news compared to watching a continuous newscast on television. Text articles can be presented online without segmentation, as in a newspaper. However, users tend to scan text online so these articles may be read differently from those in newsprint.
With greater understanding of where and how users allocate visual attention, developers and information architects can begin to design sites to augment information access and improve Web designs and services overall (Chi, Pirolli, Chen, & Pitkow, 2001; Heer & Chi, 2002). This is especially important for news sites where users actively seek content that changes continuously.
Eye-tracking and Visual Attention Metrics
Diagnostic eye-tracking is becoming increasingly prevalent in interface usability testing (Duchowski, 2002). We used eye-tracking to study whether newspaper and TV-oriented sites influenced ocular behavior differently. Based on a review of literature, we selected several metrics to monitor visual attention: (a) number of fixations overall; (b) fixation duration, the average eye fixation duration; (c) gaze time, the percent of time participants spent fixating instead of saccade time; (d) saccade rate, the number of saccades per second; and (e) scan path analysis using a string-edit method and Optimal Matching Analysis (OMA). In the following paragraphs, we provide a brief overview of eye-tracking in general and then summarize the metrics used in this project.
Eye-tracking
It is not yet clear how users view Web pages (Pan et al., 2004) and what directs their attention, specifically visual attention. Eye movement studies on news sites can offer valuable information about visual attention during information seeking. Eye movements reveal moment-to-moment cognitive processes during task execution (Rayner, 1998) and provide reliable information about where one allocates attention when presented a visual stimulus (Pieters, Rosbergen, & Wedel, 1999). For instance, Pan et al. (2004) reported that participants fixated longer on the first page of a Website than on subsequence pages, and the type of Website being viewed influenced changes in eye movement direction and magnitude. Goldberg, Stimson, Lewenstein, Scott, and Wichansky (2002) found that when participants navigated portlets (screen areas) with at least two columns, they made more horizontal than vertical eye movements. They contend that between-column (horizontal search) preference should be used to determine screen layout to improve usability. The researchers also found that participants visited the body of pages prior to page headers, suggesting that the header bars were not viewed consistently for navigational cues. Similarly, the Stanford Poynter Project (2000) examined reading behavior on news sites and found that news readers preferred text over graphics, with most readers attending to text first rather than artwork.
With the exceptions of the Stanford Poynter Project and Josephson and Holmes (2002, 2008) few studies have examined eye movement behavior on newspaper and TV-oriented sites and, as Goldberg et al. (2002) pointed out, there has been insufficient work that links usability with eye-tracking results. Given the widespread use of newspaper and TV-oriented sites and the proliferation of media on them, “…it is imperative to understand how users view different Web pages in order to provide a cognitive basis for interface design” (Pan et al., 2004, p. 147).
Visual attention measures
When looking at a visual stimulus, the eyes make continuous and rapid movements known as saccades. With durations of roughly 150–200 milliseconds for planning and execution (Muir & Richardson, 2005), a saccade directs attention to the visual target and during which information processing is restrained (Pan et al., 2004; Rayner, 1998). Occurring between saccades, fixations are relatively motionless gazes associated with cognitive processing that take place over some minimum duration (Jacob & Karn, 2003; Josephson & Holmes, 2008). Viewers take in information when the eye is motionless, during fixations (Stanford Poynter Project, 2000). At least three processes take place while fixating: (a) visual stimulus encoding, (b) peripheral field sampling, and (c) preparation for the next saccade (Pan et al., 2004; Viviani, 1990). According to Pieters, Rosbergen, and Wedel (1999) fixations are an important characteristic of visual attention and their duration, position, and pattern should be determinants when assessing a visual display’s potential to gain attention.
The number of fixations in a particular area of a visual display is an indication of the importance a viewer affords to the display area. Conversely, a higher number of fixations overall suggests an inefficient information search that may be due to a poorly designed interface (Jacob & Karn, 2003).
Fixation duration is a brief glance lasting between 100-300 milliseconds, although some researchers (e.g., Bojko, 2006; Palmer, 2002; Rayner, 1998) document fixation ranges from ~50-~500 milliseconds. As noted by Josephson and Holmes (2002, 2008), researchers who study images in print and television (e.g., Baron, 1980; Fischer, Richards, Berman, & Krugman, 1989) classify a fixation as a pause in eye movement with a minimum duration of 100 milliseconds, although there is no definitive research that establishes 100 milliseconds as the minimum fixation duration. Fixation duration is a measure of information complexity and/or one’s difficulty discerning information from a display, and fixation patterns between displays indicate how efficiently visual elements are arranged (Fitts, Jones, & Milton, 1950; Jacob & Karn, 2003). Salvucci and Anderson (2001) point out that fixation durations are contingent on task difficulty and they can range from roughly 100 milliseconds to more than 1 second. In a Web search study, Cutrell and Guan (2007) reported mean fixation durations of over 2 seconds as participants viewed highly ranked search results on a search engine results page (SERP). The durations decreased as participants viewed search results ranked lower in the list.
Gaze time, or the portion of time spent looking at a display element, may suggest the importance a participant associates to an element. Similar to gaze time, ascertaining the percentage of participants who fixate in specific areas of a display can be indicative of an interface element’s ability to attract visual attention (Jacob & Karn, 2003).
Saccade rate and task difficulty are inversely related, signifying that as cognitive demands increase saccade rate decreases (Nakayama, Takahashi, & Shimizu 2002). When examining ocular behavior and task difficulty, Nakayama, Takahashi, and Shimizu found that the saccade rate (1.0 saccade/second) was similar when participants did not perform a mental task and when they responded quickly and correctly to a task. However, as participants responded slowly or incorrectly perhaps due to increased mental effort, saccade rate decreased (0.8 saccade/second). Pan et al. (2004) found that the first pages of business Websites were more cognitively demanding than second pages as indicated by an increased saccade rate on second pages. Conversely, they found that the second pages of search sites required more cognitive effort as indicated by a reduction in saccades.
A scan path is a repetitive succession of eye fixations (Brandt & Stark, 1997). Scan path theory (Brandt & Stark, 1997; Norton & Stark, 1971) purports that when a person is first presented a visual stimulus, he or she visually scans it and then retains in memory a spatial model of the succession of fixations that constitute the scan path. Upon re-exposure to the stimulus, the participant’s eye movements follow the scan path established during the initial viewing, facilitating recognition (Josephson & Holmes, 2002). Josephson and Holmes (2008) used string-edit method and OMA in which scan paths are assigned a code and then sequence pairs are compared to observe variations. They found that participants had preferred scan paths and that different participants exhibited similarities in eye movement sequences suggesting that features such as the complexity of the Web page design may be an important determinate in scan path variations.
Purpose of the Project
In this research, we tracked participants’ eye movements as they viewed newspaper and TV-oriented news homepages to investigate whether ocular behavior differed. We anticipated that measures of visual attention would differ by site type, with newspaper sites having greater numbers of fixations, longer fixation duration, higher gaze times, and lower saccade rates. We expected that overall visual attention on newspaper homepages would be more dispersed over the page and that there would be more scan path variation among users. We based these expectations on the following factors: (a) the likelihood of high text density on newspaper sites compared to TV-oriented sites; (b) layouts that reflect a newspaper orientation, with headlines dispersed throughout the page; (c) the way users read online, preferring to scan text rather then read thoroughly; and (d) the fact that reading is more difficult on a computer screen.
The research sought to answer three main questions: (a) Do measures of visual attention differ by the type of news site (newspaper and TV)? (b) What areas of news site homepages attract visual attention and do those areas differ by site type? and (c) Do scan paths vary as a function of site type? Among other things, answers to these questions can provide valuable insights regarding the types of news sites and site elements that support or impede visual attention, areas of the display that draw attention (salient features of display areas), and the extent to which users vary in how they attend to a display. For news sites, small variance may suggest, for example, an interface that affords efficient and consistent use.
Modeled after previous research (e.g., Brandt & Stark, 1997; Josephson & Holmes, 2002, 2008; Pan et al., 2004), we first examined the effect of news site type (newspaper and TV) on four metrics of ocular behavior, three of which were employed by Pan et al. (2004): (a) number of fixations overall; (b) fixation duration, the average eye fixation duration; (c) gaze time, the percent of time participants spent fixating instead of saccade time; and (d) saccade rate, the number of saccades per second. We then used the string-editing method and OMA to examine differences among scan paths including: (a) scan path variation within and across users and (b) percentages of participants who fixated in specific areas of the display. To the best of our knowledge, this is one of only a small number of studies that examined ocular behavior and news sites using these methods. A final aim of the study was to examine the utility of a string-edit approach and OMA in conjunction with eye-tracking data.
Method
The following sections provide information about the participants, apparatus, and stimuli used in the study.
Participants
Graduate and undergraduate students from Duquesne University, Pennsylvania, USA participated in this research project by locating information on news provider Websites. Fourteen individuals, 6 females and 8 males, with an average age of 23 years, comprised the participant group. They reported that they were experienced with Web browsing and obtaining news online. Most indicated that they read news Websites each day or several times daily and that the Web was their preferred medium for obtaining news followed by television and then newspapers. For most participants, their primary areas of the study were Journalism and Multimedia Arts, although some individuals majored in Advertising and Public Relations and English. All were proficient computer users who had normal or corrected-to-normal vision.
Apparatus
We recorded eye movements with a ViewPoint eye-tracking system designed by Arrington Research, Inc., an established company that has been making eye-tracking equipment since 1995. The ViewPoint system provides corneal and pupil reflection eye-tracking and monitors eye movement, gaze, and pupil size under infrared lighted conditions. The eye-tracker sampled at 30 Hz with accuracy of 0.25°-1.0° visual arc (~10-40 pixels). We calibrated the system for each participant trial and performed re-calibrations when necessary. Participants viewed visual stimuli (Websites) on a LCD 19-inch monitor with a 1024 X 768 pixel resolution. An adjustable head positioning device stabilized viewing position and restricted head movement.
Stimuli
The following six Websites obtained from popular news organizations that have tradition in either newspaper (e.g., The New York Times) or television (e.g., CNN) were chosen for this research:
- TV-oriented news Websites
- CNN (http://www.cnn.com)
- Fox News (http://www.foxnews.com)
- MSNBC (http://www.msnbc.msn.com)
- Newspaper-oriented news Websites
- Pittsburgh Post Gazette (http://post-gazette.com)
- The New York Times (http://nytimes.com)
- USA Today (http://usatoday.com)
We chose these sites based on the following criteria: (a) site type, the site could be classified as either television or newspaper; (b) popularity, the sites characterized prevailing online news outlets from which many people obtain the news; (c) the sites presented Web pages with distinct categories of visual imagery and high information density; and (d) we felt that major characteristics of the sites (layout, link concentration) were indicative of many popular newspaper and TV-oriented sites.
The eyes fixate on important information in the first few seconds of viewing a stimulus (Loftus, 1976). Studies of ocular behavior on Websites (e.g., Josephson & Holmes, 2002, 2008; Pan et al., 2004) analyzed data obtained from the first 15 seconds of participants’ viewing. Brandt and Stark (1997) who employed a string-editing analysis used 10 seconds exposure for collecting eye data. Consistent with these studies, we analyzed ocular data from the time a participant began viewing a homepage until he or she clicked a news story link, an average of 20 seconds.
Procedure
We loaded the homepages of the six news sites prior to the start of a study session so that when a participant viewed a site he or she began at the homepage. Participants used Internet Explorer with the toolbar displayed across the top and the browser window expanded to full screen. The sequencing of the site presentations was counterbalanced to reduce order effects. A moderator gave an overview of the study and the equipment, after which time he calibrated the eye tracker.
The moderator instructed participants to browse each of the news sites freely and to find a story of interest. Participants clicked links to go to the full story. Upon accessing the full story page, participants could stop browsing and moved on to the next site. They repeated this process for each of the six sites. The entire study session lasted approximately 20 minutes.
Defining Scan Path Sequences
From the eye-tracking data, we extracted X and Y coordinates or direction-of-gaze coordinates. The gaze was normalized relative to the X- and Y-axes such that coordinates of 0.0, 0.0 indicated a position of gaze at the top-left corner while 1.0, 1.0 indicated a position of gaze at the bottom-right corner of the display window (Arrington Research, 2005). After carefully reviewing each Website for layout and content, we defined six regions common to all sites, assigned each region a letter, and constructed a grid (see Figure 1) that defined target areas over which fixation patterns were superimposed. In other words, using X and Y display coordinates for each fixation, we identified the grid region where the fixation occurred. If the fixation coordinates happened to be X- 0.10 and Y- 0.05, the fixation occurred in region A and it got labeled as A. In this way, we constructed the scan path sequences. For instance, if on the CNN homepage a person fixated in region B first followed by a fixation in region A, followed by three consecutive fixations in region E, the scan path sequence would be B, A, E, E, E for that person on CNN. We defined scan path sequences for each participant’s viewing of every site. The regions depicted in Figure 1 represented (a) A—browser bar, Internet Explorer’s toolbar; (b) B—site branding, header, and site navigation; (c) C—left-hand navigation rail; (d) D—headline news content; (e) E—miscellaneous content and advertisements; and (f) F—browser window area with no content.
Figure 1. Site grid for defining fixation regions
We then compared the coded sequences using OMA, which generates a numerical index (Levenshtein distance [LD]) of the variation between sequence pairs. We compared each scan path to every other (CNN, FOX, MSN, POST, NYT, and USA) for each participant. For each comparison, we obtained the LD. The LD is the “…smallest number of insertions, deletions, and substitutions required to change one string into another” (National Institute of Standards and Technology, 2007). If a person had a scan path for FOX news as CADEBA, and a scan path for CNN as CADEEA, the LD between his or her FOX and CNN scan paths would be 1 because these two strings differ by one character, as follows:
Scan path 1 (FOX): CADEBA
Scan path 2 (CNN): CADEEA
The length of scan paths varied. To adjust for length, the resulting cost of transformations was normalized by dividing it by the length of the longer of the two sequence pairs (Josephson & Holmes, 2008). Each participant had six scan paths, one for each news Website. To obtain the within-user average variance, the LD was calculated so that each participant’s scan path for each of the six Websites was compared to every other. Scan paths were then compared across participants to obtain an across-user variance.
Results
We first examined the effect of site type (newspaper or television) on the four metrics of ocular behavior: (a) number of fixations overall, (b) fixation duration, (c) gaze time, and (d) saccade rate. We then examined scan paths to identify where participants fixated in the display area and whether there were scan path variations within and across users.
Number of Fixations, Fixation Duration, Gaze Time, and Saccade Rate
A t-test for dependent means was conducted for each of the visual attention measures enumerated in Table 1. Differences in the measures for newspaper versus TV-oriented news Websites were not found to be statistically significant.
| TV-oriented
news Website |
Newspaper print-oriented
news Website |
|||||
|---|---|---|---|---|---|---|
| Mean | Standard deviation | Mean | Standard deviation | t * | p | |
| Number of fixations | 14.06 | 6.45 | 14.94 | 6.07 | -.69 | .50 |
| Fixation duration(s) | 1.25 | .82 | .99 | .34 | 1.25 | .23 |
| Gaze time(s) | .91 | .05 | .93 | .12 | -.50 | .62 |
| Saccade rate(s) | 1.01 | .47 | 1.13 | .39 | -1.43 | .18 |
| Total viewing time(s) | 13.38 | 1.82 | 12.86 | 1.67 | 1.24 | .24 |
* df = 13
Fixations and Areas of Interest
When number of fixations was examined in terms of target regions of the display (see Figure 1), region D had the highest percent of fixations. Participants fixated mostly in screen areas D (36%) and E (27%), which are areas for headlines and content, respectively (see Figure 1). A chi-square test indicated that fixations in the various screen areas varied by type of site, ?2(5) = 49.89, p < .001. To determine where the specific differences lie, a follow-up chi-square test was performed for each screen area. As shown in Table 2, the number of fixations on screen areas B (site branding, header, and navigation), C (left-hand navigation), and D (headline news content) were significantly more common in TV-oriented Websites than in newspaper print-oriented news Websites. On the other hand, fixations on screen area E (miscellaneous content and advertisements) were significantly more evident in newspaper Websites than TV-oriented ones.
We identified the area of the initial fixation when a homepages loaded. Most (40%) of the first fixations occurred in region A (browser bar), followed by regions B (20%), D (20%), and E (18%). Two percent of initial fixation occurred in region F (browser window area with no content) and 0% in region C.
| Screen area | TV-oriented news Website | Newspaper print-oriented news Website | Total number of fixations in each screen area | X2 | p |
|---|---|---|---|---|---|
| A (browser bar) |
78 13%) |
67 (11%) |
145 (12%) |
1.31 | .25 |
| B (site branding, header, and navigation) |
99 (17%) |
65 (11%) |
164 (14%) |
9.32 | .002 ** |
| C (left-hand navigation rail) |
28 (5%) |
14 (2%) |
42 (3%) |
4.73 | .03 * |
| D (headline news content) |
233 (40%) |
198 (32%) |
431 (36%) |
6.61 | .01 * |
| E (miscellaneous content and advertisements) |
110 (19%) |
211 (35%) |
321 (27%) |
37.30 | .000 *** |
| F (browser window area with no content) |
37 (6%) |
54 (9%) |
91 (8%) |
2.39 | .12 |
| Total number of fixations | 585 | 609 | 1194 |
* p < .05; ** p < .01; *** p < .001
Scan Path Analysis
We calculated the within-user and across-user scan path variances. A t-test for dependent means was conducted and as shown in Table 3, only the across-users average variance/cost between scan paths in newspaper versus TV-oriented Websites was found to be statistically significant, t(13) = 2.27, p = .04. Variance in scan paths between users was significantly higher in newspaper sites (M = 24.98, S.D. = 7.57) than in TV-oriented ones (M = 21.90, S.D. = 8.37).
| TV-oriented news Website |
Newspaper print- oriented news Website |
|||||
|---|---|---|---|---|---|---|
| Mean | Standard deviation | Mean | Standard deviation | t * | p | |
| Within-user average variance/cost between scan paths |
11.67 | 4.87 | 11.65 | 4.25 | .03 | .98 |
| Within-user average variance/cost between scan paths (normalized) |
.70 | .15 | .70 | .11 | -.05 | .96 |
| Across-users average variance/cost between scan paths |
21.90 | 8.37 | 24.98 | 7.57 | 2.27 | .04 ** |
| Across-users average variance/cost between scan paths (normalized) |
.72 | .04 | .73 | .04 | 1.10 | .29 |
* df = 13; ** p < .05
The Most Central Scan Path Sequence
In an effort to examine the influence of homepage design on eye traces, we adopted a method used by Josephson and Holmes (2008) whereby the “… most central sequence has the lowest mean LD from other sequences and therefore may serve as a representative sequence for that group” (p. 398). In other words, for each news site, we identified the individual with the most central scan path based on the lowest LD relative to other scan paths for that site. Figures 2 through 7 represent participants’ 15 second eye movement traces of the most central fixation sequences.
The designs and associated elements (i.e., photographs, text) influenced eye movements as evident by the location, form, and density of traces. A number of traces toward the far-right side of the screen represent participants locating the scrollbar. Overall, the figures show high trace density on text links within the document body suggesting that visual attention was captured by reading or skimming textual information. For the most part, traces conform to text areas and bypass salient elements such as the dominant photograph, headers, and advertisements, which get attention but not to the same extent as text. It is also noteworthy that based on these figures, traces appear more dense on TV homepages than on newspaper homepages. For example, in Figure 2 (CNN), visual attention is concentrated on the link list to the right of the main photograph. The traces on The New York Times as well as the other newspaper sites are more spread out and less concentrated. Distributed throughout the body of the document are headlines that serve as the primary navigation links to the full story. Below the headline is a summary of the story and possibly a byline. On the other hand, the TV homepages present a list of text links that are, for the most part, concentrated in one area. These approaches to homepage layout influenced eye movement, with the text link lists seemingly the primary area of importance.
Figure 2. Eye path trace of most central fixation sequence for CNN
Figure 3. Eye path trace of most central fixation sequence for Fox News
Figure 4. Eye path trace of most central fixation sequence for MSNBC
Figure 5. Eye path trace of most central fixation sequence for NYT
Figure 6. Eye path trace of most central fixation sequence for Post Gazette
Figure 7. Eye path trace of most central fixation sequence for USA Today
Discussion
It is important to point out that the findings presented here are not conclusive due to the limitations and scope of this study, which looked at specific news Websites with a unique participant group. In addition, the findings are based on participants performing a free-form browsing task (i.e., looking for a topic of interest). Findings may vary with different types of visual searches. Additional research is required to examine ocular behaviors across larger groups with a greater variety of news Websites and tasks, and for longer time durations.
We sought to answer three main questions: (a) Do measures of visual attention differ by the type of news site (newspaper and TV)? (b) What areas of news site homepages attract visual attention and do those areas differ by site type? and (c) Do scan paths vary as a function of site type?
Visual Attention Allocation Measures and Site Type
We anticipated that the measures of visual attention would differ by type of site, but this expectation was not supported by the data. Previous research (Pan et al., 2004) reported similar findings indicating that different types of Websites did not influence fixation duration. While our results show slight differences between site types, they were not significant. For example, number of fixations, a possible indication of an inefficient information search (Jacob & Karn, 2003) was higher on newspaper sites. Conversely, TV-oriented sites exhibited a tendency toward information and task complexity as measured by fixation duration and saccade rate. We thought that indices of information and task complexity might be higher on the newspaper sites used in the study because of their layouts, with headings and text summaries dispersed over the page.
Given the information task, browsing a home page for a news story of interest, and the metrics used, newspaper and TV-oriented sites influenced ocular behavior comparably. This finding refutes our initial expectation that site type will have distinct influence on ocular measures, with participants experiencing less efficient search and increased information and task complexity on newspaper sites. However, it is plausible that the free-form browse task did not sufficiently stress the visual system and it introduced variability among participants resulting in the lack of differences across site types. Perhaps if participants were instructed to locate a defined visual target in a specific amount of time (e.g., a visual search task), then more pronounced differences would have been observed for these measures.
Areas of Visual Attention and Site Type
We were interested in identifying areas of the display where participants first fixated. Unlike the findings of Goldberg et al. (2002), most participants (60%) fixated initially in the upper portions of the homepages (regions A and B). At the same time, 38% of initial fixations occurred in regions D and E, the body of the document. This suggests that on news homepages while some users’ visual attention may go directly to the body of the document, most will use the navigation or header regions to orient visual searches and/or cue navigation.
Where participants fixated on the display area differed by site type. Moreover, the most central scan paths (see Figures 2 through 7) show eye movement variations illustrated by the shape, density, and location of visual traces. This intimates that participants’ visual attention varied across sites as a function of layout design. With the site homepages used in the study, participants’ eyes attended primarily to text links and text areas in the document body, which is consistent with the findings of the Stanford Poynter Project (2000). Possibly, because users forage for information by navigating Web links (Chi, Pirolli, Chen, & Pitkow, 2001). Visual characteristics of link areas (e.g., positioning, grouping, length, color, underlining, etc.) may have provided salient visual referents that aided information access, despite shifts in layout. Participants attended primarily to news story links in the body of documents rather than site navigational links at the top and left sides of pages.
On TV-oriented homepages, participants fixated most often on regions B, C, and D whereas newspaper homepages had more fixations in region E. In regions B, C, and D, the TV homepages presented a logo and global navigation across the top of the page. Positioned to the far-left or middle-left of the page, the homepages featured a dominant photograph that related to the major news story. In close proximity to the photograph, to its immediate right or left and below it, were listings of links. The sites concentrated these elements in regions B, C, and D.
The layouts of newspaper homepages, to some extent, mirror a newspaper. They presented headlines with summaries and bylines throughout the homepage, which engendered a dispersed visual scan, as depicted in the eye trace figures. The observation that participants fixated in region E more often on newspaper homepages was likely influenced by several factors. First, on homepages, visual attention may be especially drawn to listings of text links (news or top stories) because users seek an efficient means to review and access available stories. The New York Times homepage presented lists of links under More News and On The Blogs that were positioned in region E. In addition, similar to the way TV sites presented links, USA Today listed headlines in close proximity to one another in region E. Finally, the Post Gazette presented a listing of links (top stories) in a scroll box at the top-center of the page and a dominant image directly below the list. To the right of the image and also below the links, headlines with summaries ran down the page into region E. As participants scrolled down, they encountered additional listings of text links located in this same region. Interestingly, of all the sites, the Post Gazette had the fewest number of fixations in regions A and B. Possibly, fewer participants attended to these regions because the Post Gazette presented top story links in a scroll box rather than listing them in a more conventional manner as was done on TV homepages. TV sites, on the other hand, grouped news story links in one general area. It seems plausible that the observed differences with respect to fixation area are mainly influenced by (a) visual attention being attracted to areas with concentration of news story text links in the document body (the TV homepages presented those areas in the upper regions of the page); (b) newsprint homepages used headlines as primary navigation links, which were spread over the page; and (c) with the possible exception of USA Today, the placement of link groupings on newspaper sites varied from site to site more so than on TV homepages. For instance, The New York Times presented news story links in the lower-middle portion of the page, whereas the Post Gazette presented them in the upper-middle.
If number of fixations is considered in conjunction with highly fixated areas, it suggests that TV homepages had central areas of importance and a free-form search was more efficient compared to newsprint pages. Frequent fixations in a specific display area indicated how important viewers perceived that area while increased numbers of fixations overall may signify an inefficient information search (Jacob & Karn, 2003). Based on the visual traces, number of fixations, and results identifying high fixation areas, participants’ visual attention on TV homepages concentrated around link listings in the upper regions of the pages, signaling those areas as highly important. However, when viewing newsprint pages, participants had slightly more fixations but visual traces were less concentrated, which intimates that, unlike TV homepages, there was no central area of importance and a free-form information search was less efficient.
Scan Path Variation
Scan path variance can be used to, among other things, address whether a user’s eye movement remains consistent on different news sites. Additionally, it provides an indication of whether the eye movements of different users are similar across sites. There was no difference in the within-user variance indicating that an individual’s scan paths did not vary significantly across the six news homepages. However, the across-user scan path variation was greater on newspaper homepages. This finding suggests that the design layouts of newspaper and TV-oriented homepages influenced eye movements differently. Scan paths on newspaper homepages had more variability than those on TV-oriented homepages. The finding is consistent with the visual traces (see Figures 2 through 7) that depict less dense traces on newsprint homepages. It appears that as participants looked for news stories they focused in the document body and on text links. When links were concentrated in one general area, as on TV homepages, those areas capture visual attention, perhaps resulting in less scan path variability. Conversely, on the newsprint sites, designs resembled a newspaper with prominent headlines throughout the page, possibly making link groupings less salient and resulting in across-user variability.
The Utility of String-editing and OMA
We used string-editing and OMA as means to identify where participants fixated on the homepages and to observe scan paths variations within and across users. As noted by Josephson and Holmes (2002) these methods afforded ways of identifying, assessing, and categorizing scan path sequences. They were especially appropriate because, like other research (e.g., Josephson & Holmes, 2002; Pieters, Rosbergen, & Wedel, 1999) the goal of this project was to understand where users directed their visual attention and how often rather than specifying exact eye fixations. In the study, the utility of the string-edit and OMA methods depended on defining layout regions of the display. In this case, the Websites had analogous layouts overall and information appoQArtioned to specific regions was similar across sites. Had there been greater variability in layouts and associated content, the definition of regions would have been more difficult. Nevertheless, we feel that string-editing and OMA are value approaches that can provide practitioners and researchers insight about users’ visual attention.
Conclusion
Major newspapers and TV news providers each uniquely represent their traditional media origins online. While these representations may be perceptual to users, we observed no differences in measures (number of fixations, fixation duration, gaze time, and saccade rate) of ocular behavior resulting from type of site. We expected the measures to differ by site type but this expectation was unfounded, at least for a free-form browsing task. However, this is an area for additional research because it is possible that differences in these metrics would be observed had participants performed a visual search for a specific target under a time constraint.
As one might expect, participants fixated mostly in content areas of the page, areas D and E of Figure 1. On TV-oriented sites most fixations concentrated in regions B, C, and D. TV-oriented homepages presented a high concentration of text links in the upper portion of the screen and, generally, their placement was consistent across sites. Consequently, participants fixated in those areas. Conversely, when participants search newspaper sites they had more fixations in region E, where content and links were located. Moreover, newspaper sites use headlines throughout the page as a primary navigation to news stories. From these observations we surmise the following: (a) even for free-form browsing, the placement of content and the means of accessing it (link groupings) were the primary factors that directed attention, more so than site navigation and extraneous content; (b) for all participants, most initial fixations occurred in the browser bar (region A) and in the site branding area (region B), which participants appeared to use to orient themselves to the page; and (c) participants’ initial fixations occurred in the upper portions of the screen and thus it seems probable that one might support a more efficient information search for news stories by placing link groupings directly below these areas. This seems especially important for newspaper sites that use headlines as primary navigation, which are dispersed over the page.
Scan paths on newspaper homepages exhibited more across-user variability than those on TV homepages. This finding revealed differences in how newspaper and TV-oriented homepages support navigation to news content (top stories). The TV-oriented homepages presented lists of text links whereas the newspaper homepages used headlines as primary navigation, which produced less concentrated visual attention.
If we can assume that the purpose of a site homepage is to provide users the most efficient access to a limited number of top news stories, then the observed variability intimates that the information search was less optimal on newspaper sites. Researchers (Chi, Pirolli, Chen, & Pitkow, 2001) have noted that users forage for information by navigating Web links. Concentrating links in one area of the screen, as on TV-oriented sites, appeared to serve several useful functions. First, as observed, link groupings attracted user attention and served as a focal point of navigation. Second, the link listings grouped items of similar functionality into a single prominent area of the display and they conveyed to users what stories the news editors deemed as most newsworthy. Third, the links were easily scanned (visually) and their proximity to each other may have provoked users’ interest in stories that they would have otherwise missed. It is also worth noting that the total viewing time was slightly longer on TV-oriented sites. While speculative, it is possible that link groupings offer a focal point for navigation and, at the same time, engender more thorough reading. On the other hand, using headlines as primary navigation, as on newspaper sites, induced variability in how different users directed their attention. This resulted in more dispersed visual traces, with the eye traversing a wider area of the display. Newspaper sites could potentially reduce variability in visual attention and provide an easily scanned list of top news stories by grouping links consistently in upper regions of the page. This navigation structure could be used in addition to the current headlines navigation and it may provide a more efficient means of navigation and perhaps more focused reading. Future research would be needed to examine the extent to which users use link grouping versus headlines navigation.
In general, we believe that eye-tracking and the methods used in the study to observe attention allocation patterns offer usability practitioners valuable tools to assess product usability. Overall, eye-tracking helps practitioners evaluate the extent to which the visual display elements presented on many interactive systems enhance or detract from the user experience.
Usability engineers have used eye-tracking for many years because, among other things, it affords product developers information about where users focus attention as well as what fails to get their attention. Eye movement data provide developers information pertaining to the efficiency of visual searches, how users process visual information while interacting with systems, and the factors contributing to failed searches (Bojko, 2006). These data can help usability practitioners investigate the efficiency with which experts and novices process visual information and how users progress from novice to expert. Moreover, eye-tracking offers a quantifiable means to verify observations obtained with other usability methods. For example, when conducting usability tests on news Websites, we often use software to record mouse-clicks, Web page changes, and time on tasks, among other things. These techniques, used in conjunction with eye-tracking data, enable us to observe, for example, what attracts user attention between mouse-clicks or the visual elements that may have influenced a user’s decision to click one link over another. Between mouse-clicks, pages changes, and other such events, users’ attention shifts frequently as they process extensive amounts of information. Eye-tracking data help make these events salient, which ultimately enables practitioners to improve the usability of products and the overall user experience.
References
Arrington Research, Inc. (2005). Data Collection. In ViewPoint EyeTracker®: PC-60 Software User Guide (pp.56-57). Scottsdale, Arizona: Arrington Research, Inc.
Baron, L. (1980). Interaction between television and child-related characteristics as demonstrated by eye movement research. Educational, Communication and Technology: A Journal of Theory Research and Development, 28(4), pp. 267-281.
Bojko, A. (2006). Using eye tracking to compare Web page designs: A case study. Journal of Usability Studies, 3(1), pp. 112-120.
Brandt, S. A., & Stark, L. W. (1997). Spontaneous Eye Movements During Visual Imagery Reflect the Content of the Visual Scene. Journal of Cognitive Neuroscience 9(1), pp. 27-38.
Chi, E. H., Pirolli, P., Chen, K., & Pitkow, J. (2001). Using information scent to model user information needs and actions and the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 490-497) Seattle, Washington, USA. CHI ’01. New York, NY: ACM.
Cutrell, E., & Guan, Z. (2007, April 28-May3). What are you looking for?: An eye-tracking study of information usage in Web search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 407-416) San Jose, California, USA. CHI ’07. New York, NY: ACM.
Duchowski, A. T. (2002). A Breadth-First Survey of Eye Tracking Applications, Behavior Research Methods. Instruments, & Computers (BRMIC), 34(4), pp.455-470.
Fischer, P M., Richards, J. W., Berman, E. F., & Krugman, D. M. (1989). Recall and eye tracking study of adolescents viewing tobacco ads. Journal of the American Medical Association, 261(1) pp. 90-94.
Fitts, P. M., Jones, R. E., & Milton, J. L. (1950). Eye movements of aircraft pilots during instrument-landing approaches. Aeronautical Engineering Review, 9(2), pp. 24–29.
Gibbs, W.J. (2008). Examining users on news provider Websites: A review of methodology. Journal of Usability Studies, 3(3), 129-148.
Gibbs, W.J., Bernas, R.S., & McKendrick, J. E. (2008). Observations of information seeking behavior on news provider web sites, Manuscript submitted for publication.
Goldberg, J. H., Stimson, M. J., Lewenstein, M., Scott, N., & Wichansky, A. M. (2002, March 25-27). Eye tracking in web search tasks: design implications. In Proceedings of the 2002 Symposium on Eye Tracking Research & Applications (pp.51-58) New Orleans, Louisiana, March. ETRA ’02. New York, NY: ACM.
Heer, J., & Chi, E.H. (2002). Separating the swarm: Categorization methods for user sessions on the Web. Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves (pp. 243-250). Minneapolis, Minnesota, USA.
Horrigan, J. B. (2006, March 22). For many home broadband users, the Internet is a primary news source. Retrieved December 12, 2007, from http://www.pewinternet.org/PPF/r/178/report_display.asp
Jacob, R. J. K., & Karn, K. S. (2003). Eye tracking in Human-Computer Interaction and usability research: Ready to deliver the promises, In J. Hyönä, R. Radach, & H. Deubel (Eds.), The mind’s eye: Cognitive and applied aspects of eye movement research (pp. 573-605). Amsterdam: Elsevier.
Jesdanun, A. (2009). Newspapers getting more web visits. The Associated Press, Retrieved January 27, 2009, from http://abcnews.go.com/Technology/wireStory?id=6743900
Josephson, S., & Holmes, M.E. (2002). Visual attention to repeated Internet images: testing the scanpath theory on the world wide web. In Proceedings of the Eye Tracking Research & Applications Symposium 2002 (pp. 43-49). New York, ACM.
Josephson, S., & Holmes, M.E. (2008). Eye monitoring studies on TV news and the Web converge: A tale of two screens. In R.I. Hammoud (Ed.), Passive Eye Monitoring: Algorithms, Applications, and Experiments. Berlin Heidelberg: Springer.
Loftus, G. R. (1976). A Framework for a Theory of Picture Recognition. In Monty, R. A. and Senders, J. W. (Eds.), Eye Movements and Psychological Processes (pp. 499-514). Hillsdale, N.J.: Lawrence Erlbaum Associates.
Loftus, G. R. (1976). A Framework for a Theory of Picture Recognition. In Monty, R. A. and Senders, J. W. (Eds.), Eye Movements and Psychological Processes. John Wiley and Sons.
The Media Audit (2005). New Research Shows Local Newspaper Websites Still Dominate Metro Markets, Retrieved August 14, 2008, from http://www.themediaaudit.com/0505tvweb.pdf
Muir, L. J., & Richardson, I.E. (2005). Perception of sign language and its application to visual communications for deaf people. The Journal of Deaf Studies and Deaf Education, 10(4), pp. 390-401.
Nakayama, M., Takahashi, K., & Shimizu, Y. ( 2002). The Act of Task Difficulty and Eye-movement Frequency for the ‘Oculo-motor indices’. In Eye Tracking Research & Applications (ETRA) Symposium (pp. 43-51) ACM.
National Institute of Standard and Technology (2007). Levenshtein distance. Retrieved August 14, 2008, from http://www.nist.gov/dads/HTML/Levenshtein.html
Nielsen, J, & Loranger, H (2006). Prioritizing Web Usability. Berkeley, California: New Riders Publishing.
Norton, D., & Stark, L. W. (1971). Scanpaths in Saccadic Eye Movements While Viewing and Recognizing Patterns, Vision Research 1 (I), 929-942.
Palmer, S. E. (2002). Vision science: Photons to phenomenology. Cambridge, MA: MIT Press.
Pan, B., Hembrooke, H.A., Gay, G. K., Granka, L.A., Feusner, M.K., & Newman, J.K. (2004). Eye Tracking Research & Application. Proceedings of the 2004 symposium on Eye tracking research & applications (pp. 147-154). San Antonio, Texas.
Pieters, R., Rosbergen, E., & Wedel, M. (1999). Visual attention to repeat print advertising: A test of scanpath theory. Journal of Marketing Research, 36(4), pp.424-438.
Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422.
Salvucci, D.D., & Anderson, J. R. (2001). Automated Eye-Movement Protocol Analysis. Human-Computer Interaction, 16(1), pp. 39–86.
Society for New Communications Research. (2007, August 15). New Industry Research & Market Updates. Retrieved May 31, 2008, from http://www.newcommreview.com/?p=779
Stanford Poynter Project. (2000). Terminology. Retrieved July10, 2008, from http://www.poynterextra.org/et/i.htm
Viviani, P. (1990) Eye movements in visual search: cognitive, perceptual, and motor control aspects. In Kowler, E. (Ed.), Eye movements and their role in visual and cognitive processes. Reviews of Oculomotor Research V4 (pp. 353-383) Elsevier.
