Insights for the TV Interface from the Mobile Phone Interface

Peer-reviewed Article

pp. 166-177

Abstract

We reviewed the service structure, needs analysis, user interface model, and interaction analysis for television (TV) and mobile phones. Due to the increasing use of services such as electronic program guide (EPG), digital video recorder (DVR), and pay-per-view (PPV), we concluded that text input for the TV interface will be inevitable. Furthermore, jumping interaction will remain as the main interaction for TV. Based on the successes and failures of various interaction technologies for mobile phones, we present a prediction of new input paradigms for the TV interface. Finally, cooperative design by TV manufacturers and service operators will be significant for the success of advanced interactive TV services.

Practitioner’s Take Away

Because user interface design elements for both TV and mobile phone were comparable, user interface design practitioners for TV can take advantage of user interface design practice for the mobile phone.

  • As web browsing is not a popular task for TV and EPG, DVR, and PPV are being important, the jumping interaction will remain as the main interaction for TV. Thus, input devices which effectively support the jumping interaction and which provide text input capability should be introduced for the TV remote interface.
  • The touch screen interface seems to be a good input device for mobile phones according to the recent success of touch screen phones, but this will not be applicable for TV.
  • Cooperative design in the mobile industry has become a new key for the success of mobile services (Williams, 2006). Because of the similar service structure between TV and mobile phones, cooperative design of TV interfaces between manufacturers and service providers will be a key for the success of TV services as well.

The Changes of TV Paradigm

Television (TV) technology has experienced two fundamental changes since its introduction to the consumer market. The first fundamental change was the introduction of color TV that supplanted existing black and white TV broadcasts. With the second fundamental change, the evolution from analogue to digital transmission introduced more channels along with higher quality images and sound. Because digital transmission can employ data compression and send more information using a given bandwidth, interactive applications and services have become available to the TV viewers. Thus, TV sets supporting digital systems have become more powerful and sophisticated to support advanced services such as high definition TV (HDTV), electronic program guide (EPG), and digital teletext service (Peng, Cesar, & Vuorimaa, 2001). These changes have increased the complexity of the user interface of the TV sets.

Although the type and number of TV services have changed significantly, few studies have addressed the TV user interface even though there are clear challenges in the use of TV by consumers. It seems that TV viewers are having difficulty adapting to the new TV paradigm, because they are used to the limited and one directional service of traditional TV. For example, Internet TV failed in the market soon after its release. Also, the buying cycle for TV sets is longer than that of mobile phones, so updating to newer systems and different user experiences is very slow. Furthermore, most TV manufacturers are in far-east Asia with no major manufacturer in the U.S. Thus, researchers in the U.S. who are concerned with user interface design have limited access or opportunities to affect the TV user interface.

Mobile Phones as the User Interface Leader

Meanwhile, the mobile wireless phone was originally designed primarily to support voice interaction, much like its stationary, wired counterpart. However, current mobile phones have evolved to support text messaging, Internet browsing, interactive gaming, high-resolution cameras, MP3 audio, and mobile TV services. Like TV, the mobile phone has become a fundamental artifact of the modern lifestyle, and it is experiencing continuous changes. Due to the increase of device complexity, the usability of mobile phones has become a key differentiator for the success of mobile phone devices and services (Williams, 2006). Unlike TV, much effort and numerous studies have been dedicated to the improvement of the user interface for mobile phones. U.S., European, and Asian device manufacturers as well as researchers have modified multiple device characteristics and conducted many research studies during the last 10 years. This may be because the mobile phone market was a brand new market, or because the mobile phone is relatively smaller and cheaper with more rapid design changes, or because the mobile service provider has the buying power to influence the device manufacturers. These factors are not present in the TV system design, production, and usage models.

Can TV Learn Lessons from the Mobile Phone?

The dominant usage of both TVs and mobile phones is based around the display device, and the capabilities of these displays are not homogeneous. The resolution of the mobile phone display includes Quarter Common Intermediate Format (QCIF) (176 x 220), Quarter Video Graphics Array (QVGA) (240 x 320), and Video Graphics Array (VGA) (480 x 640), while that of TV includes 480i, 480p, 720p, and 1080i (the i is the abbreviation for pixels in interlaced mode and the p is the abbreviation for pixels in progressive scan mode). The input devices for PCs have been well established with the mouse and keyboard from the inception of the technology. There is less standardization in input devices for mobile phones than PCs, however, four-way directional keys with an OK (Select) button, small discrete thumb-joysticks, soft keys1 (left and right soft keys for mobile phones), and labeled designated buttons are common (Amant, Horton, & Ritter, 2007). For TV navigation, color soft keys (four color buttons for TV), four-way directional keys, and numeric keypads are common (Eronen & Vuorimaa, 2000). Both TV and mobile phones try to convey information, especially information on the Internet; however, both of these systems are struggling to solve issues in interface design for Internet service. TV and mobile phones are very different, but they have similarities in the user interface.

Because there are reasonable similarities between TV and mobile phones, we believe that the significant amount of mobile phone user interface studies, design process, and lessons learned from the mobile phone industry can guide the direction of the user interface design effort for the TV industry. Thus, we have conducted a comparison of user interfaces between TV and mobile phones to provide insights for designers of TV user interfaces. To analyze the structure of service and user interface for this comparison, the usability engineering lifecycle process (i.e., requirement analysis, design/testing/development, and installation) by Mayhew (1999) was utilized.

Service Structure of TV and Mobile Phones

Although it is not apparent to most consumers, there are similarities between the main organizational entities affecting the user experience for both TV and mobile phones. These entities can be classified as program and content provider, service operator and provider, and device manufacturer. Currently, there are three different types of TV service, and the organizational entities affecting user experience for each service in the U.S. are summarized along with mobile phone services in Table 1.

Table 1. Organizations Affecting TV and Mobile Phone User Experience by Service Type in the U.S.
Service
type
Program/content
provider
Service operator Device manufacturer
Terrestrial TV network TV network channels CBS, NBC, ABC, FOX TV manufacturer (Samsung, Sony, LG, etc.)
Satellite TV TV network channels
Cable TV channels
Satellite TV channels
DirecTV, DISH
Network, etc.
Set-top box manufacturer
TV manufacturer
Cable TV TV network channels
Cable TV channels
Satellite TV channels
Time Warner, COX,
Comcast, etc.
Set-top box manufacturer
TV manufacturer
Mobile phone Content provider
portal
Service provider
(AT&T, Verizon,
Sprint, T-Mobile, etc.)
Mobile phone manufacturer
(Samsung, Nokia, LG, Motorola,
Blackberry, etc.)

The TV user experience is integrated and provided by the program and content provider, the service provider, and the device manufacturer. The mobile phone user experience is similarly integrated. Because user interface designers in each organizational body try to implement their own design to optimize user experience, sometimes the user experiences between different organizations can be incompatible. Thus, the need for a collaborative user interface design between service operator and device manufacturer in the mobile phone industry has led to guidelines for cooperative design as discussed in Williams’ (2006) work. Because most mobile phone users are also TV viewers, the user experience of mobile phones affects the experience of TV and vice versa.

1 A button that performs a function dependent on the text shown on the display above or next to the button.

Needs Analysis

The following sections discuss user analysis, task analysis, and system analysis.

User Analysis

TV content (programming) is usually targeted for more than one person. The user’s age is very important for TV programming in relation to age-based ratings that describe the level of sexual or violent content. Also, there are programs tailored or targeted for specific user groups such as single women, business men, or children. Meanwhile, TV manufacturers design and produce the devices based on the assumption that various types of users and more than one user can watch the TV at the same time. Thus, the user interface of TV is not designed for specific user groups.

For mobile phones, some services are designed for specific user groups and some are designed for every user group, although the main services related to voice interaction are designed for all users. Unlike TV, mobile phone manufacturers design the user interface according to the user profile they are targeting. Thus, the user interface included by the device manufacturer is personalized to some extent. Table 2 compares characteristics of TV and mobile phone users.

Table 2. User Characteristic Comparison between TV and Mobile Phones
  User characteristic TV Mobile phone

Manufacturer

Target user

All ages

All ages or specific ages

Personalization

Group

Individual

Usage environment

Stationary

Mobile

Usage/operation distance

3 m

Handheld

Programming/ service provider

Target user

All ages or specific ages

All ages or specific ages

Personalization

Group

Individual

Usage environment

Stationary

Mobile

Usage/operation distance

3 m

Handheld

Task Analysis

The most important task for TV is watching televised programs. Thus, other peripheral services or on-screen display menus should not interfere with the task of watching a program. The main task of mobile phones is having voice conversation. Users should be able to talk on the phone no matter what other peripheral tasks are being performed. The TV can let users do multi-tasking of the main task by providing multiple pictures on one screen. Multi-tasking for mobile phones, for example, is to let users watch video or listen to music while talking on the phone.

Because the number of functions implemented in electronic products is increasing and the convergence of various products is being accelerated, structuring tasks is getting more complicated. Structuring tasks can be done in various ways based on certain criteria, but one example is shown in Table 3. This method of structuring tasks classifies tasks into installation, use of basic functions, use of applications, setup, and maintenance based on the life cycle of the product. This method can be easily applied to any other electronic product.

Table 3. Task Analysis of TV and Mobile Phones
Task TV Mobile phone

Installation

Connect antenna
Connect wires
Connect power lines
Install battery for remote
Mount on the wall

Subscribe to service provider
Update phonebook
Install battery

Basic functions

Turn on/off
Navigate channels
Volume control and mute
External input control
Watch TV

Turn on/off
Send/receive voice calls
Send/receive video calls
Send/receive text messages
View calling history
Update phonebook

Applications

Reserve program
Previous channel
Favorite channels
Multi-language sound
Caption
Caller ID
EPG
DVR
PPV

Taking and view pictures
Download and listen to music
Watch video
Email
Internet browsing
Download and playing game

Setup

Screen setup
Sound setup
Channel setup
Time setup

Screen setup
Sound setup
Lock the phone
Application setup

Maintenance

Clean up
Move

Recharge battery
Exchange battery
Clean up

System Analysis

The display resolution of digital TV is significantly greater than that of analog TV. Digital TV uses one of two formats: 1280 × 720 pixels in progressive scan mode (abbreviated 720p) or 1920 × 1080 pixels in interlace mode (1080i), and each utilizes a 16:9 aspect ratio. For the digital terrestrial television, there are digital TV sets with built-in digital tuners, while the digital TV sets without built-in digital tuners need a separate tuner to receive terrestrial digital television broadcasts. For digital cable and satellite television, special set-top boxes are required for receiving digital television broadcasts (Figure 1). In this case, users have two separate remote controls for the TV and the set-top box.

The display resolution of mobile phone screens has increased to the level of QVGA (320 x 240) and VGA (640 x 480). Also, the CPU capability of mobile phones has become powerful enough to provide multi-tasking with a camera, MP3s, mobile TV, and games. One significant difference between a mobile phone and TV comes from the size of the screen. The small screen-size of mobile phones presents a major limitation in their usability. Menus and applications on a mobile phone must compensate for the small size of the screen (Ahmadi & Kong, 2008; Burigat, Chittaro, & Gabrielli, 2008), whereas TV has enough space to present complex menus and applications. However, TV must also use space carefully so that the main task of watching programs is not compromised.

Figure 1

Figure 1. Illustration of TV system with user

User Interface Model

IBM (1992) and Mandel (1997) suggested there are three models to configure the user interface: the user’s mental model, the programmer’s model, and the designer’s model. It is important to understand each of the three different perspectives for user interface design. Thus, if we revise IBM’s user interface model to apply to TV and mobile phone usage, the three entities would be user, designer, and developer (Figure 2). This model will allow us to view both TV and mobile interfaces in three different perspectives in order to incorporate the design differences for positive impact.

Figure 2

Figure 2. User interface model revised from IBM (1992)

User’s Model

TV users are in a passive mode while watching TV. Thus, TV user interface design should reflect the changes in the passive mode of the user. The traditional TV task is for users to watch programs displayed on a TV screen. In this context, the user’s only active task is to change channels or control volume. Digital TV provides additional interactive services so that users need to be in active mode to fully utilize them. However, it is not easy for users to adapt their attitude toward the active use of TV. The success of traditional TV is based on the fact that viewers do not need to do anything other than relax and watch. Also, viewers are not motivated to search and select program contents if these services are not offered at a reasonable price (Wyss & Vong, 2001). Thus, service providers’ effort to reduce the cost of using interactive digital TV service will be necessary to encourage the active participation of viewers. With the advancements in technology, more functions are implemented in electronic products that require active user participation. Thus, the user acceptance level of newly introduced functions should be studied carefully.

For mobile user interface design, there are two different perspectives in device design. These perspectives are due to the convergence of various functions into the device. The first perspective is to consider voice conversation as the main function of the device, and the other added functions such as MP3, camera, or mobile TV are the sub- or supplementary functions. The other perspective is to look at all the functions at the same level of importance, which means any function including voice conversation, MP3, camera, or mobile TV can be the main function of the device at any time. For example, the phone function of the iPhone seems to be an extra function built into an iPod Touch, which is mainly an MP3 player, because both iPhone and iPod Touch have the same basic look and interface structure, and because the iPhone evolved from the iPod.

We should also observe that users only recognize an integrated experience. So, users are not really interested in recognizing who (manufacturer or service provider) provides which function or service of TVs or mobile phones. Thus, it raises the important issue of cooperative design of user interface among different organizational entities (such as device manufacturer, service provider, and content provider) to optimize the integrated user experience that is presented through the user interface.

Designer’s Model

To effectively design interfaces, designers must understand both users and manufacturers. Designers are tasked with making compromises between the two incompatible goals of being easy to use and easy to make. TV and mobile phones are more limited than PC products in terms of design. While a PC can solve various usability problems with the help of software, TV and mobile phones must allocate most resources to their main tasks.

Identity is one of the most important elements in design. Device manufacturers want to maintain their identity regardless of a service provider. In other words, mobile phone manufacturers want to supply their phones to as many service providers as possible. Conversely, service providers want to maintain their identity regardless of device manufacturers. After all, the same effort by the two different parties can produce negative user experience. To improve the integrated user experience, the needs of cooperative design between manufacturer and service provider become essential.

Developer’s Model

This model is based on the perspective of development and technology. Developers focus on the appointed date of delivery and quality of their products. From this point of view, the difference between the device manufacturers and service providers is more significant than the difference between TV and mobile phones. Device manufacturers are more resistant to changes because they have the process of manufacturing and evaluation, while service providers are more willing to change to accommodate user desire for enhanced services. Manufacturers perform more evaluation and undergo a verification process when they introduce a new design of user interface or technology. Iterative design is a critical process for successful interface design; however, the cost of an iterative design process in a manufacturing firm is significant.

Comparison of Interaction

Interaction design is a more extensive concept than user interface design. However, the interaction design referred to in this paper will only be limited to the actual and specific interactive behavior between products and users. Foley et al. (1990) introduced six types of interaction tasks including Select, Position, Orient, Path, Quantify, and Text. With the Select task, the user makes a selection from a set of alternatives such as a menu. With the Position task, the user indicates a position on the interactive display using input devices or type-in coordinates. The Orient task requires the user to control orientation angles of an entity. The Path task is a series of positions or orientations over time. The Quantify task asks the user to specify a value, and the Text task prompts the user to input a text string.

Input interaction indicates input navigation, while input device means a physical device to provide data or control signals. Input interaction includes pointing interaction, jumping interaction, and recognition interaction based on the electronic products available in the market. Pointing interaction is based on the moving cursor imposed over the applications on screen independently. Typical input devices used for pointing interactions are a mouse, a touch pad, or a track ball. The jumping interaction uses a cursor or a highlighted box subject to the applications on screen and they only move around the designated areas. This interaction typically performs the selection task. The recognition interaction performs Select, Quantify, and Text tasks through voice recognition, handwriting recognition, or gesture recognition.

The main interactions used in TVs are the Select and Quantify tasks, whereas the main interactions used in mobile phones are the Select, Text, and Quantify tasks. The jumping interaction is used as input interaction for both TV and mobile phones. The pointing interaction was adopted for early Internet TVs in the 1990s; however, it has been replaced with the jumping interaction. Also, the jumping interaction is the most popular method used in mobile interfaces. Recently, manufacturers released mobile phones with a touch screen using the pointing interaction; however, service providers provide applications and services based on the jumping interaction to support non-touch screen phone users. The interaction task, input interaction, and input device for both TV and mobile phones are summarized in Table 4.

Table 4. Interaction Characteristic for TV and Mobile Phones
Interaction TV Mobile phone

Interaction task

Select
Quantify

Select
Quantify
Text

Input interaction

Jumping interaction

Jumping interaction
Pointing interaction (touch-screen phone)

Input device

Remote
Numeric keypads
Directional keys + OK button
Color soft keys

Numeric keypad
QWERTY keyboard
Directional keys + OK button
Left and right soft keys
Click-wheel on the side
Finger trackball
Isometric joystick
Touch screen
Voice recognition

TV and mobile phones share a very similar approach in input interaction and input device. This typical approach is different from the interaction provided by recently popular touch screen phones such as the iPhone and LG Prada. Navigation has not been a popular task for TV usage. However, as a result of the addition of various functions and services such as EPG, DVR, and PPV, directional keys with an OK button have become a basic component of TV remotes. These features are in addition to the channel and volume control buttons on remote controls as well as the color soft keys with four different colors (red, green, yellow, and blue). Basically, each color key represents the function indicated on the bottom of the TV screen as a soft key similar to left and right soft keys on mobile phones (Figure 3).

Figure 3

Figure 3. Examples of directional keys and soft keys

Although mobile phones have utilized various ways to support navigation tasks, the four-way directional keys with an OK or a Select button have become the popular method with the left and right soft key for non-touch screen phones. Recently, Blackberry introduced a finger trackball for their new models instead of directional keys and a click-wheel on the side of the device. The click-wheel, which is usually implemented on the right side of the device, can be very intuitive and quick for the jumping interaction, but it may not be a good solution for left-handed users. In contrast, a trackball can be used by either hand. Also, there was an attempt to adopt an isometric joystick (Chau, Wobbrock, Myers, & Rothrock, 2006), which has been used for Thinkpad laptops instead of a touchpad. The input devices already present in the market make it difficult for the emergence of new input paradigms. For example, the Prada phone by LG, which mainly uses a touch screen as the input device, also adopted directional keys with an OK button on the screen in case users still want to use this approach instead of the touch screen pointing interaction. Thus, new input devices face challenges because users are accustomed to the previous input devices.

Future of the TV Interface

The following sections discuss the interaction task, the input interaction, and the input device.

Interaction Task

Because of the popularity of text messaging on mobile phones (which is driven mainly by younger generations), the Text task has become the dominant interaction task, displacing the Select and Quantify tasks. Thus, mobile phones have adopted physical or on screen QWERTY keyboards to support the Text task. For TV, Select and Quantify remain the primary tasks, and no Text tasks are supported. Thus, the typical TV remote control does not have a text keyboard. However, the emergence of EPG, DVR, and PPV services may drive a requirement for a text keyboard for TV interaction. For example, entering a text string for searching a program or channel would be significantly faster than navigating through numerous channels, genre categories, and so on. Searching by actor name, director name, production year, nationality, etc. can also be easily supported by the Text task.

Input Interaction

While the jumping interaction has been the primary mechanism for both mobile phones and TVs, the new trend of touch screen phones has made the pointing interaction another primary input interaction for mobile phones. However, it is obvious that a touch screen would not be a viable solution for TV, because the users are 3 meters away from the device. Touch screen remote controls for TV would not be a good solution, because users control the handheld remote while watching TV so tactile feedback is important. Tactile feedback is a significant weakness of the touch screen interface. The pointing interaction on TV actually exists through devices such as the remote used in the Nintendo Wii video game console; however, the pointing interaction using arm and wrist movement at a distance is not an easy task as compared to the pointing interaction based on finger movements. Thus, it is unlikely that the pointing interaction will be introduced for TV users, and the jumping interaction will remain as the primary input interaction for TV.

Input Device

As shown in Table 4, various input devices have been introduced and adopted for mobile phones, while TV has relied on traditional remote control with a numeric keypad, directional keys, and recent color soft keys. Because the pointing interaction would not be a viable solution for TV, the isometric joystick and touch screen are unlikely to be adopted as TV input devices. The finger trackball could be a good input device to support jumping interactions as in the Blackberry menu, and it can support the pointing interaction very well. The click-wheel on the side could be a very good supplementary input device for TV remote, because it provides excellent tactile feedback of jumping one step at a time and it provides fast performance. Voice recognition is a difficult solution for TV, because there are already significant sounds in the usage environment from the TV or home theater speakers. A summary of the probable near future TV interfaces are shown in Table 5.

Table 5. Interaction Characteristic for TV in Near Future
Interaction TV

Interaction task

Select
Quantify
Text

Input interaction

Jumping interaction

Input device

Remote
Numeric keypad
QWERTY keyboard
Directional keys + OK button
Color soft keys
Click-wheel on the side
Finger trackball

 

Recommendations

The motivation of the comparison between mobile phones and TV is to provide insights for TV user interface designers and industry from the lessons learned and literature studies of mobile interface researchers and industry. It was shown that user interface design elements for both TV and mobile phones were comparable, and the experience with the mobile interface will be an important source for TV interface design. Also, because most mobile users are also TV viewers, the user experience of mobile phones affects the user experience of TV and vice versa. Thus, consistency and coordination across products will be important design factors for the TV interface.

The user experience is an integrated one affected by both manufacturers and service providers and operators. The combined user interfaces optimized by each organizational entity may not necessarily be the global optimum. As the cooperative design of mobile interfaces by manufacturers and service providers has become a new key for the success of mobile services, the cooperative design of the TV interface by manufacturers and service providers will be a valuable process for the success of digital TV services. For example, newly released TVs in the market have started to feature stock, news, and weather services built-in by manufacturers. This service may not be compatible with the remote interface of service operators using set-top boxes.

In summary, the following are general recommendations for user interface researchers:

  • Because TV users are also mobile phone users, TV interface designers should consider consistency with the mobile phone interface.
  • New input devices for TV may well be introduced and tested for TV such as the click-wheel, the trackball, or a QWERTY keyboard on the remote control.
  • Cooperative design of the TV interface between manufacturers and service providers will be an essential process for the success of digital TV services.

Conclusion

In this paper, we reviewed the service structure, needs analysis, user interface model, and interaction analysis for both the TV and mobile phone. Due to the increasing use of EPG, DVR, and PPV services, text input for TV interface will be inevitable and jumping interaction will remain as the main interaction for TV. Based on the success or failure of various input devices for mobile phones, a prediction of new input devices for TV interface was presented. Also, because the service structure for both TV and mobile services are comparable, cooperative design by TV manufacturers and service operators will be essential for the success of advanced interactive TV services.

References

Ahmadi, H., & Kong, J. (2008). Efficient web browsing on small screens, Proceedings of the Working conference on advanced visual interfaces. Napoli, Italy: ACM.

Amant, R.S., Horton, T.E., & Ritter, F.E. (2007). Model-based evaluation of expert cell phone menu interaction. ACM Trans. Comput.-Hum. Interact., 14(1), 1.

Burigat, S., Chittaro, L., & Gabrielli, S. (2008). Navigation techniques for small-screen devices: An evaluation on maps and web pages. Int. J. Hum.-Comput. Stud., 66(2), 78-97.

Chau, D.H., Wobbrock, J.O., Myers, B.A., & Rothrock, B. (2006). Integrating isometric joysticks into mobile phones for text entry, CHI ’06 extended abstracts on Human factors in computing systems. Montreal, Quebec, Canada: ACM.

Eronen, L., & Vuorimaa, P. (2000). User interfaces for digital television: a navigator case study, Proceedings of the Working conference on advanced visual interfaces. Palermo, Italy: ACM.

Foley, J.D., Wallace, V.L., & Chan, P. (1990). The human factors of computer graphics interaction techniques. In J. Preece & L. Keller (Eds.), Human-Computer Interaction (pp. 67-121). Upper Saddle River, NJ, USA: Prentice Hall.

IBM Corporation. (1992). Object-Oriented Interface Design: IBM Common User Access Guidelines. New York: QUE.

Mandel, T. (1997). The Elements of User Interface Design. New York: Wiley & Sons, Inc.

Mayhew, D.J. (1999). The usability engineering lifecycle: a practitioner’s handbook for user interface design. San Francisco, CA: Morgan Kaufmann Publishers Inc.

Peng, C., Cesar, P., & Vuorimaa, P. (2001). In Integration of applications into digital television environment (pp. 266-272). Paper presented at the 7th International Conference on Distributed Multimedia Systems, Taipei, Taiwan.

Williams, D.M. (2006). Co-design, China, and the commercialization of the mobile user interface. interactions, 13(5), 36-41.

Wyss, B., & Vong, M. (2001). The Future of Digital Entertainment: Interactive TV Revisited Telecommunications Research Project. Hong Kong.