Abstract
This study examined user performance for Web sites in which a critical navigation link may have changed over the course of two visits. These second visits occurred either immediately after the first visit, one week later, or three weeks later. A shortcut link to the information was either consistently present, consistently absent, or varied between user visits. Results indicated when a pertinent navigation link was removed users searched through more pages and were less accurate in finding target information. When a link was added after the first visit, only about half of the users used it; however, even users who didn’t use the added link still showed a performance improvement in their subsequent visit, suggesting that completing a single previous search on a site will lead to relatively long-term memories that can influence subsequent searches.
Practitioner’s Take Away
The following were the main findings of the study:
- By the second time users visit a site, their performance improves compared to their first visit (decreased page counts and search times), even after delays of up to three weeks.
- Even small changes, like the addition or deletion of a single link, can have large impacts on user performance.
- Presence of a shortcut link to vital information dramatically increases the probability that a user will find that information. Compared to a site without the link, users will be faster and visit fewer pages.
- When a pertinent navigation link is removed, users search through more pages and are less accurate in finding target information.
- When users have clear memory of a site, deletion of a link causes users to exhaustively examine the primary navigation structure; this behavior persists as the delay between visits increases for at least up to three weeks.
- When a link is added after the first visit, only about half of the users will use it. The rest will continue to use the site in the way they have done in the past.
- Even users who don’t use an added link still show a performance improvement in their subsequent visits, suggesting that secondary paths to information should not be deleted simply because a top-level navigation link has been added.
- Even if the link is removed after a user has been exposed to it, users are no less accurate than if the link had never been present.
Introduction
The Web has evolved into an important tool in everyday modern life (Haythornthwaite & Wellman, 2002). From the sharing and gathering of information to the business of e-commerce, typical users visit hundreds of pages every week (Hawkey & Inkpen, 2005). Even though the content of all these pages is as varied as the interests of the users who seek them out, each of these pages shares a common element—links. While some pages are simply a collection of links, or links imbedded in content, most pages employ some common, persistent collection of links that are collectively referred to as navigation links. In western language Web sites, navigation links are generally located near the top and left of Web pages (Weinreich, Obendorf, Herder, & Mayer, 2006) and serve to guide the user to the content of interest on any given site.
Because the Web was designed to easily share information, it is constantly changing. New pages are added, old pages are taken down, and content is continually modified. Over the course of 3 years, the number of new sites added to the Internet more than doubled to 108 million (Netcraft, 2006). Almost 40% of the pages on the Web are changed in some way over the course of a week, with 23% being changed daily (Cho & Garcia-Molin, 2000). This degree of change is not surprising, as these measures included changes to content, and many of the most popular sites are those used to shop or gather frequently updated news information (Adar, Teevan, & Dumais, 2008). While changes to the content of a site may be beneficial to users, changes to the navigational links used to access that content can prove to be more problematic, especially when considering that approximately 53% of visits to Web pages are repeat visits (Tauscher & Greenberg, 1997). Kohler (2002) found that over a 1-year period approximately 18% of Web pages undergo changes in which links are renamed or deleted. Over a 4-year period the number of pages undergoing this kind of change rises to nearly 39%. Adding links to pages is less common, with approximately 8% of the Web pages adding links over the course of a year, with this number increasing to 30% over 4 years.
How do these changes impact users? It is known that changes to user interfaces can have detrimental impacts on users (Tamborello, Chung, & Byrne, 2008). It is also known that wholesale changes to Web sites (site redesigns) adversely impact user performance, at least in the short term (Jones, Farris, & Johnson, 2005). Even if an old site had significant usability issues associated with navigation, there will be some degree of relearning involved for users who had previously used the site in its original form. Often times, however, site navigation does not change as part of a complete redesign. Rather, in the course of normal maintenance and content update, small navigation changes are implemented. What are the impacts of these smaller changes? Changing a single, crucial link may be a more significant issue than changing a large number of rarely used navigation links.
Consistency of navigation is one of the design elements that is supported by the leading evidence-based Web design guidelines, such as the U.S. Department of Health and Human Services Research based Guidelines (2006). This design recommendation is almost always present even when overall Web design guideline agreement tends to be low (Bevin, 2005). However, one of the difficulties in trying to maintain navigation consistency is trying to decide when a change in the navigation structure is enough to be considered major.
Scharff (2006) demonstrated that there were indeed performance decrements when a single pertinent link was removed from the navigational structure. Scharff had users look for a piece of target information in the first visit to a site and then had them repeat the task immediately following completion of this first task. There were four conditions: two consistent conditions in which the pages presented were identical across visits and the pertinent link was either always present or always absent, and two inconsistent conditions in which the link was either present then absent or absent then present. In all cases, the information could be found using other navigational paths that were three clicks in length. Her results showed that users generally took longer to find the information and searched more pages in the condition where the link was present on the first visit and absent on the second. Overall, the addition of the pertinent link in the inconsistent condition improved user performance, due to a subset of users who noticed the added link and used it to directly navigate to the target page. The time between the first and subsequent visit was extended to one week in a subsequent study by Kortum and Scharff (2007) and the results were similar. These results suggest that even a single visit to a site can establish relatively long-lasting path memories in users that will impact their behaviors on subsequent visits to a site.
Because users may visit certain kinds of sites consistently but infrequently, it could be that changes to navigational structures have less impact than these earlier studies might suggest if the time between visits is longer than those previously studied and memory of the navigational structure has faded. This paper extends the previous work by including longer times between a user’s first and subsequent visits to a site. We describe the results of an experiment in which an important link in the main navigational menu was altered by adding or deleting a single link in between a user’s first and subsequent visits. The length of time between the first and second visit varied from an immediate revisit to as long as three weeks.
Methods
The following sections discuss the experiment design, the participants, the task, and the materials used.
Experiment Design
This study had users look for target information on a site over the course of two visits. The second visit occurred either immediately after the first visit, one week later, or three weeks later. A shortcut link to the information was either consistently present, consistently absent, or varied between user visits. Users’ performance was measured on the number of pages viewed, the time it took to locate the target information (or give up trying), and the accuracy with which they found the target information.
The study used a 2 (search task: first or second) x 2 (shortcut link presence consistency: yes or no) x 2 (presence of shortcut link on second search task: yes or no) x 3 (time delay: no delay, one-week delay, or three-week delay) between-participants design. When the shortcut link was present, it was always in the same location in the left-side navigation. Participants completed two search tasks, with the time delay between the two searches varying across participants in a blocked manner across different semesters. All of the participants were assigned to one of the four consistency x link presence conditions in a random fashion.
Participants
Thirty-eight undergraduate student participants from a mid-sized state university in east Texas completed the no delay study. Sixty-nine undergraduate student participants completed the one-week delay study; thirty-nine of the participants were students from a private university in southeastern Texas, and thirty of the participants were from the mid-sized state university. Fifty-five undergraduate students from the state university completed the three-week delay study. The participants tested in this experiment had moderately high self-reported Web literacy skills. On a scale of 1 to 5, where 1 indicates higher comfort, participants rated their general comfort in finding information on other sites at 1.84 (SD = .66). They further rated their average success in navigating Web sites at 1.98 (SD = .66) on a 5point scale where 1 indicates higher success. Regardless from which institution or in which of the three studies the students participated, all received course credit or extra credit for participating. Some of the students participated individually while others participated in larger groups as part of a class or seminar. In all cases, consistent instructions were given and students had access to a dedicated task computer and did not interact with any other participants. Total testing time for each participant was less than 30 minutes for the no delay experiment, and less than 15 minutes for each part of the one-week delay and the three-week delay experiments.
Task
When beginning the search task, participants started at the front page of a test Web site and were then allowed to freely explore the entire site until they found the answer to the question or chose to give up the search. When the shortcut link was available, the target information was only one click away. Even though the search task question and the remainder of the content on the Web pages were the same for both the first and second visits, the answer to the task question was changed to prevent users from using memory from their first visit to complete the task on the second visit. Once the user found the target information, they could click an Answer Question button, which was always present in a frame at the bottom of all of the pages (see Figure 1). If the user believed that the target information was not located on the site, the user could also chose an Information Not Available button at any time.
For every task, we recorded search times (in milliseconds). Start time was recorded once a user clicked the Begin Task button and the front page of the site had begun to load. Time recording stopped when the user clicked the Answer Question button or the Information Not Available button. A page log was also created that counted and identified the pages the user visited. This page count included the front page of the site and all pages that were viewed up to and including the page that was displayed when the Answer Question or Information Not Available button was clicked. User input into the task answer box was recorded so that we could measure the accuracy of the answers given to the task question.
Following the first search task, participants were asked to rate how comfortable they were finding information on Web sites in general and how successful they believed they were at navigating through Web sites in general. They responded using a 5-point Likert scale (always, usually, occasionally, rarely, never). They were also asked if they had previously explored the site before. Following those questions, participants were then given the 10-item System Usability Scale (SUS) (Brooke, 1996) to measure the usability of the portions of the site that were used by a participant. Following the second search task, participants were asked if the site changed from the first time they had used it and, if yes, how so. They were also given an additional opportunity to give comments about the experimental tasks or the Web site.
Materials
The experiment used a modified version of the Stephen F. Austin Teaching Excellence Center Web site. If the shortcut link was not present in the left-hand navigation of the site, then the target information was at minimum three clicks from the front page. Several of the navigation links that led outside the Teaching Excellence Center pages were disabled in order to retain full control over the data collection. There were two versions of the site. In one of the versions, a shortcut link was present in the left-hand navigation (fourth from the bottom in a list of ten links). In the other version there were only nine links in the left-hand navigation, with the shortcut link being removed. Other than the presence or absence of the link and the altered answer to the task question, these two test sites were identical. Figure 1 shows the front page of the site. This is the link-present version of the site. The shortcut link, while highlighted with the red arrow in this figure (the red arrow was not present during the study), was consistent with the other navigational links in the menu during the actual experiment.
Figure 1. The site presented to users in the experiment. This is the version of the site with the shortcut link present, as indicated by the red arrow. The red arrow did not appear to the participants.
The participants were first presented with a task screen that asked them the following question: When the Teaching Excellence Center opened, Dr. Steve Davis spoke at the opening ceremonies. What item did Dr. Davis donate at this event? On the site with the shortcut, the link was labeled Opening Speaker Event.
A bottom frame (1-inch height x screen width) contained both the search question (in order to reduce the memory load of the user) and the two response buttons (Answer Question and Information Not Available); the bottom frame was always present on every page during the study. When a user clicked the Answer Question button, a response box was shown in the same space that allowed the user to type in an answer. The page that the user was viewing when he or she clicked the Answer Question button remained displayed so that any errors in the response would not likely be due to the user forgetting the answer. If the user clicked the Information Not Available button, the user was prompted to continue to the debriefing questions. The answer to the first task was Handbook of the Teaching of Psychology and for the second task the answer was Guide for Successful Lectures.
Results
For each participant for each search, we recorded accuracy, search time, link use, navigation path, and page count. Ratings of comfort and success did not significantly correlate with accuracy, search time, or page count; thus, these ratings were not included as covariates in the subsequent analyses. SUS scores were within the acceptability ranges outlined by Bangor, Kortum, and Miller (2008), so this variable was also dropped from further consideration.
Prior to any multivariate analyses, we performed a simple comparison of the effect of link presence versus link absence. For the dependent variable of search time, users were significantly faster when the link was present (M=90.8 sec) than when it was not present (M=158.6 sec), t(226) = 7.98, p<.001. For the dependent variable of page count, users viewed fewer pages when the link was present (M=3.7) than when the link was not present (M=10.1), t(194) = 12.1, p<.001.
To better understand the interaction between variables, for both search times and page counts a 2 (search task) x 2 (consistency) x 2 (link presence on second search) x 3 (delay) mixed ANOVA was performed. Search task (first or second) was the within variable and the remaining were between variables. Search times were measured with millisecond accuracy but are reported in seconds. Figures 2 and 3 show means for all conditions collapsed across the three delays.
For search times, there were several significant main effects. However, these were modified by significant interactions. On average, the first search was slower than the second, F(1,152) = 74.61, p<.01; consistent searches were faster than inconsistent searches, F(1,152) = 15.70, p<.01; the three-week delay led to the slowest searches, F(2, 152) = 9.49, p<.01 (M = 118.24, 108.28, 146.11 secs for immediate, one-week delay, and three-week delay, respectively); there was no effect for link presence on the second search. Search task, consistency, and delay interacted significantly so that all groups, except the one-week delay with inconsistent link presence group, showed a decrease in search times on the second search, F(2, 152) = 3.46, p = .034. Finally, search task, consistency, and link presence on the second search interacted significantly, F(1, 152) = 46.32, p<.01. Users showed the largest improvement (decrease in search times) in the second search when a link was added (NY condition, shortcut link was present [Y] or not [N]), but they were also faster on the second search with the consistent conditions (NN and YY). Those users who first had a link and then had it removed showed an increase in search times for the second search. See Figure 2 for a plot of this interaction. Overall, delay did not change the pattern of responses, and users with the condition where the link was consistently present (YY) had the fastest search times (M=84.5 sec).
Figure 2. Search times (seconds) and standard errors for each condition collapsed across delay. Users showed improvements for all conditions except when the shortcut link was removed (YN condition). Letter pairs along the axis indicate whether the shortcut link was present (Y) or not (N) with the first letter representing the first search and second letter the second search.
For page counts, there were two significant main effects. Consistent searches led to fewer page counts than inconsistent searches, F(1,152) = 6.97, p<.01, and having a shortcut link on the second search led to significantly fewer page counts, F(1, 152) = 20.46, p<.01. There was no effect for search (first or second) or for delay (M = 6.64, 6.59, 7.31 pages for immediate, one-week delay, and three-week delay, respectively). However, these were modified by significant interactions. Similar to the results with search time, there was a significant 3-way interaction between search, consistency, and link presence on the second search, F(1, 152) = 49.04, p<.01. Specifically, there was a significant increase in the number of pages users viewed when the link was removed on the second search (YN) and a significant decrease in the number of pages viewed when the link was added (NY). Users in both the NN and YY conditions showed a non-significant decrease in the number of pages on the second search. Users in the YY condition explored the fewest pages overall (M=3.3). See Figure 3 for a plot of this interaction. Again, delay did not change the pattern of responses.
Figure 3. Page counts and standard errors for each condition collapsed across delay. Removal of the link led to a significant increase in the number of pages and addition of the link led to a significant decrease in the number of pages. Letter pairs along the axis indicate whether the shortcut link was present (Y) or not (N) with the first letter representing the first search and second letter the second search.
Page count statistics alone do not tell the entire story. While there is clearly a significant difference in the page counts for the inconsistent conditions, the pattern of page visits also appears to vary to an extent noticeable by participants. Indeed, the participants in the inconsistent conditions were more likely to report that the site had changed in some way than the participants in the consistent conditions. Using a color chart showing page viewing hot spots for the site, Figures 4 through 7 show the different viewing patterns for the consistent and inconsistent conditions. These figures reflect the averages of page views for all users in the three-week delay condition. The results are similar for the no delay and the one-week delay conditions. The top row of boxes in each figure shows the links available in the left-hand navigation, with the far left box representing the home page link and the second-to-left box representing a link to Services and Workshops. The layout of the other boxes indicates the organization of the deeper pages that are accessed when clicking on links. In all figures, the star indicates the page where the target information was located.
Figure 4 shows the viewing pattern from the consistent NN condition. As can be seen, viewing patterns remain relatively consistent between the first and second viewings. Figure 5 shows the patterns from the consistent YY condition. As we expect, the patterns are different than the NN condition, with most of the page views confined to the top level pages. Within the YY condition, however, the viewing patterns are again consistent between the first and second viewings. In contrast, Figures 6 and 7 show the different patterns of page views in the inconsistent conditions. In Figure 6 the pattern is shown for the NY condition, where users did not have the link in the first visit and then did in their second visit. The pattern of viewing is tighter and more localized in the second condition, as users found the shortcut link and utilized it to navigate directly to the target material. Figure 7 shows that the opposite is true in the YN condition. Here, users had the link in their first visit, but the link was removed on the return visit. As can be seen, the pattern of page views (especially the use of multiple left-navigation links) spreads out as users look for the target pages or the links leading to it. Also note that in all N conditions, regardless of whether they were the first or second task, the left-navigation link Speakers and Workshops is red. In contrast, in all Y conditions, it is either blue or green because many of the users avoided the path by jumping directly to the target page using the shortcut link.
Figure 4. Color temperature chart of pages visited for the NN consistent condition. Colors indicate the average number of times a page was viewed in the given condition. The top diagram is the first visit (link not present) and the bottom diagram is the second visit (link not present) three weeks later. The no delay and one-week delay conditions have similar patterns. The star shows the location of the target information.
Figure 5. Color temperature chart of pages visited for the YY consistent condition. Colors indicate the average number of times a page was viewed in the given condition. The top diagram is the first visit (link present) and the bottom diagram is the second visit (link present) three weeks later. The no delay and one-week delay conditions have similar patterns. The star shows the location of the target information.
Figure 6. Color temperature chart of pages visited for the NY inconsistent condition. Colors indicate the average number of times a page was viewed in the given condition. The top diagram is the first visit (link not present) and the bottom diagram is the second visit (link present). This figure shows the three-week delay, but the no delay and one-week delay conditions are similar. The star shows the location of the target information.
Figure 7. Color temperature chart of pages visited for the YN inconsistent condition. Colors indicate the average number of times a page was viewed in the given condition. The top diagram is the first visit (link present) and the bottom diagram is the second visit (link not present). This figure shows the three-week delay, but the no delay and one-week delay conditions are similar. The star shows the location of the target information.
For the dependent variable of accuracy, users were significantly more accurate overall when a link was present (M=94%) than when it was not (M=63%), t (223) = -6.84, p<.001. More specifically, users were significantly more accurate when a link was added for the second search (NY), t (39) = -4.15, p<.001, and significantly less accurate when the link was removed (YN), t (50) = 4.14, p<.001. Importantly, however, even if a link was removed after the first search (YN), users were no less accurate than if the link had never been present, t (106) = 1.33, p>.05. See Figure 8 for a plot of this interaction. As with search times and page counts, delay did not affect the pattern of results.
Figure 8. Accuracy (in percent) for each condition collapsed across delay. Letter pairs along the axis indicate whether the shortcut link was present (Y) or not (N) with the first letter representing the first search and second letter the second search.
An investigation of link use showed that approximately half of the users used the link in the second search if it was not present during the first search (NY condition). However, regardless of link use, users showed improvement across the two searches. For search times, both those who did and those who did not use the added link showed a significant decrease in their search times, t (28)=, p<.001 and t(15)=3.53, p<.002, respectively. Similarly, for page counts, both those who did and those who did not use the added link showed a significant decrease in the number of pages viewed, t(28)=5.54, p<.001 and t(15)=3.53, p<.002, respectively.
Discussion
Web sites must occasionally make navigation changes, and it is important to understand the impact of such changes on the user.
The data presented here show that, even after a single prior visit to a site, users’ performance in searching for target information can be hindered by the loss of a single important link. Somewhat surprisingly, this performance decrement is evident even after an extended period of time after the user first used the site. This means that Web designers need to take extra caution when altering the navigation structures for pages, even when the users of those pages might be defined as occasional users. Our study indicates that users maintain some knowledge of site and link structure for at least three weeks, even if the users only visited the site a single time before. Web designers need to insure that the links that are removed are those that traffic statistics support as being no longer important.
In reviewing the site diagrams, it is clear that when users are looking for their lost link, they do so by navigating progressively wider using the remaining navigation links, rather than performing a narrower search closer to the page where they believed the link to be. They also tend to be less able to find the information on the second visit, giving up before they can reacquire the information. This has implications for providing alternate pathways to critical information and insuring that these alternate pathways remain if the top level link must be deleted.
Happily, the converse is also true; the addition of a single link can enhance performance if the link is germane to the task at hand, even if the user had established a navigation path to the target information on a previous visit. This means that if an analysis of site traffic indicates that a page needs to have a top-level navigation link, there is a good chance that even previous users (about half) will find, see, and use the added link. Even users who don’t find the added link will still exhibit improvements in their performance, reinforcing the fact that secondary paths to information should not be deleted simply because a top-level navigation link has been added.
There are several potential limitations of the study that may have impacted the results. First, the study used college students as participants, and these students were in general quite Web literate. It is possible that users with less Web experience might have different performance on this kind of information searching task. Second, the design of the study itself might have had an impact on users’ performance. Because the information search task was an isolated task completed in a single session, the task might have been more memorable than if the task had been performed as part of a series of Web tasks. Certainly in the no delay task, the lack of intervening stimuli would allow maximum access to the participants’ previous memories of the page. It is likely that the delay task was less affected by this design, because the participants would have likely had significant Web use in the intervening interval.
Recommendations
The current data reflect Web site interaction behaviors that the users performed as they searched for the target information. It is possible that physiological data, such as eye movements, could yield further insight into the performance measures that we report here. It is possible that eye movements would reveal that there is significantly more confusion than the page counts might seem to indicate. For example, a user might exhaustively scan a Web page looking for the shortcut link, but then either abandon the task or take a single path from the page into other content. This means that the page counts might be under-representing the cost of a missing link. While this kind of behavior may be reflected in the search time measure, search times do not reveal where a user spent time looking when a change was made. Having access to eye scan data would allow a better understanding of the users’ reactions in the different conditions.
It may also be of interest to further examine the location of the link in relation to the performance decrements and enhancements that we observed. The link in this task was set to be in a location of moderately high importance (in the clearly identified navigation structure that was located near the top left of the page), but it did not occupy the top or bottom position of the list where higher saliency would be expected. Observing these kinds of super-saliency locations as well as those with much lower prominence, like the body or lower right areas, might yield important information regarding the generalizability of the data. It may also be important to investigate user behavior using a multi-site/multi-task paradigm and increase the number of visits to more than two, in order to provide a more complete understanding of site design on the memorability of the navigation.
References
Adar, E., Teevan, J., & Dumais, S.T. (2008, April 5-10). Large scale analysis of web revisitation patterns. Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Florence, Italy.
Bangor, A., Kortum, P., & Miller, J.A. (2008). The system usability scale (SUS): An empirical evaluation. International Journal of Human-Computer Interaction, 24(6), 574-594.
Bevin, N. (2005). Guidelines and standards for web usability. Proceedings of HCI International 2005. Lawrence Elebaum.
Brooke, J. (1996). SUS: a ‘quick and dirty’ usability scale. In P.W.Jordan, B. Thomas, B.A. Weerdmeester, and I.L. McClelland (Eds.) Usability Evaluation in Industry (pp. 189-194). London: Taylor and Francis.
Cho, J. & Garcia-Molina, H. (2000). The evolution of the web and implications for an incremental crawler. In Proceedings of the 26th International Conference on Very Large Databases.
Hawkey, K. & Inkpen, K. (2005). Web browsing today: The impact of changing contests on user activity. CHI ’05 extended abstracts on Human factors in computing systems. Association of Computing Machinery.
Haythornthwaite, C. & Wellman, B. (2002). Moving the Internet out of cyberspace. The Internet in everyday life: An introduction. In C. Haythornthwaite and B. Wellman (Eds). The Internet in Everyday Life. Malden, MA: Blackwell.
Jones, K., Farris, S., & Johnson, B. (2005). Why does the negative impact of inconsistent knowledge on web navigation persist? International Journal of Human-Computer Interaction, 19(2), 201-221.
Kohler, W. (2002). Web page changes and persistence – a four year longitudinal study. Journal of the American Society for Information Science and Technology, 53(2), 162-171.
Kortum, P. & Scharff, L.V. (2007). The effect of small changes in web page navigation links on the performance of users who revisit sites. Proceedings of the Human Factors and Ergonomics Society (pp. 1138-1142). Santa Monica, CA: Human Factors and Ergonomics Society.
NetCraft Web Server Survey (2007). Retrieved February 2007, from http://news.netcraft.com/archives/2007/02/02/february_2007_web_server_survey.html
Research-based Web Design and Usability Guideline (2006). U.S. Department of Health and Human Services. Washington DC.
Scharff, L. (2006). Who moved my link? Search performance on repeated tasks. Presented at the Human Factors and Ergonomic Society Houston Chapter Conference, Houston, TX.
Tamborello, F. P., Chung, P. H., & Byrne, M.D. (2008, April 5-10). Where no interface has gone before: what can the phaser teach us about label usage in hci? Proceedings of the SIGCHI conference on Human Factors in Computing Systems. Florence, Italy.
Tauscher, L. & Greenberg, S. (1997). How people revisit web pages: Empirical findings and implications for the design of history systems. International Journal of Human Computer Studies, Special issue on World Wide Web Usability, 47(1), 97-138, Academic Press.
Weinreich, H., Obendorf, H., Herder, E., & Mayer, M. (2006). Off the beaten tracks: Exploring three aspects of web navigation. WWW 2006 Proceedings of the International World Wide Web Conference Committee, Edinburgh, Scotland.
