The Interplay Between Perceived Usability and Quality in Visual Design for Tablet Game Interfaces

Abstract

Research in human-computer interaction (HCI) has shown inconsistencies in the relationship between users’ perception of usability and the quality of visual design in digital products. Research is lacking in the gaming domain for visual design in user interfaces on tablet screens. Despite extensive research on visual aesthetics and perceived usability, best practices offer limited guidance for game interfaces from a user-centered design perspective.

The objectives of this study are twofold: to employ a design-oriented methodology to create a real iOS tablet game app from start to finish using ideation, focus groups, iterative prototyping, usability testing, and empirically evaluating game participants’ experiences; and, to use experimental methods to examine the effect that the quality of visual design in a tablet game interface has on perceived usability and user engagement.

In Phase I, we designed and developed a real iOS game; the user interface was rendered into two visual design conditions for hypothesis testing. In Phase II, we recruited 56 participants to play each game condition for 10 minutes for a within-subjects study. We administered the Multidimensional Mood, AttrakDiff, and User Engagement Scale (UES) questionnaires to collect data.

Findings demonstrate that high-quality visual design does not necessarily promote perceived usability; although, both low- and high-quality visual designs showed significant influence. Participants rated their perceived usability of the game conditions to be equivalent. Findings also demonstrate that participants experienced a higher level of user engagement in the game interface with high-quality visual design.

Keywords

Tablet game design, usability test, visual design, iterative prototyping, user-centered design principles, game learnability

Introduction

The advent of touchscreen computing has revolutionized the field of Human Computer Interaction (HCI) as it is now part of our everyday life and experience (Schiphorst, 2009). As the usage of conventional console-based games has shifted to tablets and mobile games (Oshita & Ishikawa, 2012), the computing industry has made tablet devices more affordable to a wider demographic of users (Ziefle, 2010). In 2021, tablet gamers in North America accounted for the following electronics market share: 26% Apple® (iPad®) and 20% Google™ (Android™). The revenue from tablet sales in the United States has increased from 8.94 billion dollars in 2018 to 16.03 billion dollars in 2021, and the number of tablet users is projected to rise to 232.64 million by 2025 according to Statista (https://www.statista.com/). In addition, 43% of tablet gamers in the US play the action-adventure game genre. Digital games comprise a broad range of genres, classifications, and categories, and game media and interaction have been transformed by technology with touch screen interfaces evolving from the Vectrex gaming consoles released in 1984 to the current iPads. The mobile platform has tremendous scope for developers to create new types of games and broaden the demographic of users (Feijóo et al., 2010).

Designing digital games for a broader audience with specific genre preferences is a challenging task for game designers and developers. Creating user-centric gaming applications based on user needs and satisfaction is one strategy that engages end-users. Hassenzahl (2004) explains that a digital product is characterized by two facets of design: pragmatic (usability and functionality) and hedonic (enjoyment and visual design). Understanding how these two facets influence user experience during gaming is necessary. Although the concepts of usability and visual aesthetics have been widely discussed in the field of HCI, they have not been applied in a coherent and comprehensive way to the HCI sub-category of tablet gaming, which is distinct from other HCI areas. Critical research on usability and visual aesthetics in tablet-based games is lacking.

The main research objective of this study is to investigate if perceived game usability is affected by different levels of quality in visual design for tablet game interfaces. This study is relevant to the field of User Experience (UX) because the relationship between quality in visual design and perceived usability has not been tested in the domain of tablet gaming to evaluate the effects on user engagement. Our research discussion addresses the discord between visual design and usability (Hassenzahl, 2003; Mahlke, 2008; Sonderegger & Sauer, 2010; Silvennoinen et al., 2014), which are integral to the two essential facets of a digital product.

Within this study, the term perceived usability is analogous to apparent usability, which is defined as a subjective assessment of usability by game participants. Alternately, inherent usability is defined as practical product use, with goals such as efficiency, effectiveness, and user satisfaction in a specific contextual use (ISO 9241-11).

This paper explores these concepts further and outlines the research gap identified from an extant literature review. We present hypotheses developed on the literature review, and the Methods section outlines the development of the tablet game interface used in an experimental research design to test the hypotheses. We present the findings and include a discussion of the implications for designing tablet games.

Literature Review

User-Centered Design

User-centered design (UCD) originates from the work of Norman and Draper (1986); it is defined as the “active involvement of users for a clear understanding of user and task requirements, iterative design and evaluation, and a multi-disciplinary approach” (Vredenburg et al., 2002). The UCD approach involves users at different stages in the planning, design, and development of a product. Applying its principles in product design improves the usability, and usefulness, of the product; furthermore, involving users in the design and development of the product results in a more effective, efficient, and safer product (Abras et al., 2004). The goal of UCD is therefore to produce usable products that meet users’ needs because they are always at the center of the process (Rubin & Chisnell, 2008). As Vredenburg et al. (2002) indicate, the most common methods utilized in UCD practices are iterative design, usability evaluation, task analysis, informal expert review, and field studies.

In this research, we adopted UCD principles to design and develop the tablet game apps in an agile environment by creating several sprints (multiple versions of iterative game prototypes). In UCD, the focus is on the construction of the artifact that serves a specific purpose based on an evaluation process (i.e., usability evaluation). We, as researcher-practitioners, worked in collaboration with a game developer and gameplay experts to design and build a real iOS game based on a predefined set of criteria. The intention of the UCD approach was not to produce generalizable knowledge to be shared or applied to a different digital product, even though the findings of our user experience evaluations may be applied to similar projects.

Visual Aesthetics

Our research makes a distinction between visual design and aesthetics. Visual design is the top-most plane in a digital product, the visible surface layer composed of user interface elements. According to Garrett’s (2011) five planes of elements of user experience, aesthetics is often associated with “sensory phenomena” through visual, auditory, haptic, and embodiment” (Niedenthal, 2009). Visual design is therefore a subset of aesthetics. According to Bullot and Reber (2013), an aesthetic experience begins with exposure to the artifact, an object intentionally devised to provide a response. By interacting with the artifact, a sensory experience occurs which is internalized as both a primary emotional response and a cognitive mental state at its basic level. Engholm (2010) broadly defines aesthetics as “sensuous qualities, the emotions, moods, and experiences” that take place when a user interacts with a product. As a subset of aesthetics, visual design refers to the visceral appearance of the user interface, depicted as the top-most visible surface layer of the UX model (Garrett, 2011). For example, visual elements such as color and two-dimensionality in interfaces enhance visual usability (Silvennoinen et al., 2014).

The topic of aesthetics has played a pivotal role in the field of HCI because it is an integral part of user experience. However, the findings of the relationship between perceived visual design and usability in many studies have shown methodological and theoretical inconsistencies (Hassenzahl & Monk, 2010). In the study “What is Beautiful is Usable,” Tractinsky et al. (2000) demonstrated that the relationship between the perception of visual aesthetics and the perception of usability were correlated. The outcomes of the study were aligned with the results of the Automated Teller Machine experiment by Kurosu and Kashimura (1995). Their study exhibited that the visual aesthetic qualities of a digital product impacted its perceived usability. Similarly, the study performed by Sonderegger and Sauer (2010) using simulated mobile phones on a computer screen supported the findings of Tractinsky et al. (2000), revealing a relationship between perceived visual aesthetics and usability. Research participants rated appealing mobile devices as highly usable using subjective measures. However, there is a likelihood that the subjective rating had a halo effect, a phenomenon showcased by Thorndike (1920), leading to potential biases in user judgment. Halo effects are opinions that may influence product evaluation by a previous judgement (Nielsen & Cardello, 2013).

Visual Design and Usability

Malhke’s (2008) model of the components of user experience indicates that instrumental qualities, such as usability, functionality, utility, and practicality, and non-instrumental qualities, such as aesthetics and symbolism, are inherent in a product. These qualities independently influence user experience, and they combine with emotional responses to determine overall judgements based in human-product interaction. The concept of user experience moves beyond instrumental quality (e.g., usability) to embrace hedonics, visual design, affect, emotion, and “experiential” technology-interaction (Forlizzi & Ford, 2000; Hassenzahl & Tractinsky, 2006; Desmet & Hekkert, 2007). The roles of instrumental and non-instrumental qualities in digital products have been studied by several scholars (Hassenzahl, 2003; Mahlke, 2008; Sonderegger & Sauer, 2010; Silvennoinen et al., 2014). These studies suggest no consensus has been reached on the relationship between usability and visual aesthetics in the domain of product design and interactive design. Some researchers have discussed the concept of “what is beautiful is usable,” signifying that a “beautiful” object influences usability (Lavie & Tractinsky, 2004). Yet, other groups of researchers have demonstrated that no relationship exists between perceived usability and visual aesthetics (Hassenzahl & Monk, 2010; Mahlke, 2008). And the theory “what is beautiful is usable” was not shown to be supported in a longitudinal study of web page aesthetics (van Schaik & Ling, 2009).

Studies have also shown a theoretical dissonance between usability and visual aesthetics. For instance, a study on mobile phone visual aesthetics did not find any effect on perceived usability (Hamborg et al., 2014). In direct contrast, the researchers found that a usable digital product was perceived to be beautiful. However, the researchers did not specify the usability levels (low, moderate, or high) of the stimuli used in the experiment for the digital device to be deemed beautiful.

Other studies did not find any correlation between visual design and usability. For example, Hassenzahl’s (2004) study of MP3 player skins did not reveal any correlation between the perception of usability and visual design. Considering extant research, sampling is likely a contributory factor to the reported findings. Sample units were defined as either the research participants or the prototypes. Even though participants might have been sampled randomly, prototypes were assigned based on user preferences to each group. Furthermore, two experimental studies led by Mahlke and Lindgaard (2007) used portable audio players as stimuli to examine two independent variables. The portable audio players were manipulated by modifying the level of usability and visual design to create four conditions. Variations of usability and visual design components were found to have independent effects on users’ perception of usability and visual design. In this case, a deficiency in the methodology might have failed to identify a relationship between visual design and usability, such as how the sample units were defined, which could have been due to either the research participants or to the prototypes. Even though participants might have been sampled randomly, the prototypes were assigned based on user preferences to each group. Only one kind of prototype (an MP3 player and portable audio player) was used as a stimulus in the above studies, so the results may not be applicable to the domain of tablet gaming. Therefore, to address the sample unit limitations identified in previous studies, the present study sought to utilize real iOS game prototypes.

Moreover, extant research did not find any direct relationship between the visual design quality of a product and perceived usability (Lindgaard & Dudek, 2003; Mahlke, 2008; van Schaik & Ling, 2009; Grishin & Gillan, 2019). It is possible that a mediator variable moderated the independent and dependent variables; the prototypes used as stimuli tended to lack the necessary aesthetic quality to appeal to the senses. Thus, the study of “beauty” plays a key role in understanding and defining UX research (Hassenzahl & Monk, 2010). HCI researchers have examined “beauty” using objective measurements through a psychological approach, whereas art historians have evaluated “beauty” through a subjective lens (Norman, 2004; Palmer et al., 2013). In other words, psychologists generally study the perception of beauty using scientific methods, whereas art historians focus study on the historical evolution of broadly agreed-upon conceptions of beauty. Ashby and Johnson (2003) advocate for the peculiarity between a good design and a superior design: a good design may be usable and functional whereas a superior design also makes the user happy (Norman, 2004). User perception is a complex phenomenon to measure in usability studies, yet aesthetics has been found to affect users’ perceptions of product use (Zhang et al., 2000). So, the current study adopts an experimental method to evaluate subjective components associated with the perception of visual aesthetics and perceived tablet game usability to gain deeper insight into user engagement.

The term “beauty” has been used interchangeably with aesthetics and visual appeal (Norman, 2004; Frohlich, 2004). For example, beauty has been described from three different perspectives (Reber et al., 2004). First, the objectivist view infers that an object is composed of certain properties such as form, symmetry, color, and simplicity, which make up beauty. This philosophy explains that beauty can be aggregated by a combination of these elements or properties. Second, the subjectivist view interprets beauty in such a way that any object can be beautiful if it appeals to the senses of an individual. Therefore, this implies that personal preferences influence beauty in the beholder’s mind. Third, the interactionist approach describes how the notion of beauty emerges from the interaction between a user and an object. This is in fact a combination of the two previous philosophies—objectivist and subjectivist. The interactionist view parallels perceptual theories in the domain of aesthetics and emotional design (Lavie & Tractinsky, 2004; Hassenzahl, 2008; Lindgaard & Whitfield, 2004; van Schaik & Ling, 2012; Norman, 2004).

This study examines users’ aesthetic experiences in the context of the enlightenment of Moshagen and Thielsch (2010) and their interactionist approach to beauty that combines objective and subjective design appraisal, referred to as the perception of visual aesthetics.

Perceived Usability

Another area of relevant literature is the Technology Acceptance Model, originally presented by Davis (1989), which explains that perceived ease of use and perceived usefulness are key product characteristics that affect user acceptance of a novel digital product. Behavioral engagement forms part of user engagement and predicts users’ intention to continue using a product. Therefore, user engagement can determine the success metrics of a digital product. A tablet game interface connects a player’s experience to the gaming system; therefore, it is crucial to understand how a visual and dynamic game interface influences user engagement. User engagement is defined as a user-product relationship comprising emotional, cognitive, and behavioral components that prevail over time (Attfield et al., 2011). User engagement encompasses the initial reaction of users towards technology (Sutcliffe, 2010) as well as the continuous use, or re-engagement, with the system over time (O’Brien & Toms, 2008; Jacques, 1996). Engagement is determined by factors such as visual aesthetics, system usability, user involvement, and evaluation of the experience (O’Brien & Toms, 2008). User attitude towards the system is part of engagement, which focuses on the thoughts of individual users and “the degree of activity” according to Laurel (1993) as well as “feelings” according to Jacques et al. (1995) during product interaction (Norman & Draper, 1986). O’Brien and Toms (2010) devised the User Engagement Scale (UES) to measure user engagement by assessing the perception of usability, reward, focused attention, and aesthetics. The aesthetics of interaction is a phenomenon experienced through the pleasure of using a digital product (Djajadiningrat et al., 2004; Mottus, 2017). There is, however, limited research that explores perceived usability and visual aesthetics with continued use over time (Grishin & Gillan, 2019). Investigations into perceived dimensionality highlighted users’ preferences towards 2D over 3D graphics in mobile interfaces (Silvennoinen et al., 2014), and yet another study found that a flat visual design style can impede tablet usability (Moran, 2017).

In summary, visual aesthetics research has been focused on general attributes (Porat & Tractinsky, 2012; Silvennoinen et al., 2014) such as novelty, typicality, and fluency. Therefore, research insights on specific visual attributes, such as color, shape, symmetry, and arrangement of visual elements, need attention. Seo et al. (2016s) argue that to obtain accurate and meaningful results, different levels of visual design, such as high-quality, intermediate-quality, and low-quality visual design, should be incorporated in the stimuli using experimental design.

The demonstrated inconsistencies in research findings into the influences on user perceptions between perceived usability and aesthetic design in extant literature have therefore steered the current research development. Traditional methods, such as using paper or low fidelity prototypes, may have skewed findings of participant engagement; and employing screen captures in anticipated use-phases to evaluate mobile interfaces may not have provided reliable and valid results. Reported findings have failed to distinguish between the levels of visual aesthetics of products and their effects on perceived usability. Extant research has not adequately evaluated mobile user interface using real products as stimuli.

This research is therefore focused on examining if visual attributes like full color (high-quality visual design) versus monochromatic color (low-quality visual design) in tablet game interfaces impacted participants’ perceived game usability. A real iOS tablet game was developed in response to the hypotheses in this study.

Hypotheses Development

The central research question of the present study is this: Does the quality of visual design in a tablet game interface influence perceived usability and user engagement?

This study adapted a 2.5D graphic style (2D game assets within a 3D virtual environment) to depict the user interface elements. To do this, a real iOS game stimulus was developed using UCD principles to create stimuli (Kokil, 2019), and it included two levels of visual design quality. Previous studies have shown that web usability is influenced by visual complexity, which has a significant effect on perception and cognitive load (Geissler et al., 2001; Wang et al., 2014). In the present study, we posit the following hypothesis.

Hypothesis H1: High-quality visual design in a tablet game interface is perceived to be more usable.

Studies exploring interaction design and product design have researched the relationship between visual design and usability, but these studies have revealed theoretical and methodological inconsistencies. The different models garnered from the literature review can be summarized as follows: 1) Beautiful products are perceived to be usable (what is beautiful is usable). 2) Usable products are found to be attractive (what is usable is beautiful). 3) No direct relationship exists between perceived usability and visual design, but such a correlation can exist through a mediator variable. 4) Usability and visual design can be independent of each other. Based on the above outcomes, the quality of visual design may affect usability.

Hypothesis H2: There is no correlation between low-quality visual design and perceived game usability.

From Dewey’s (1938) perspective, the four types of experience threads are sensual, emotional, compositional, and spatiotemporal (McCarthy, 2004). The most relevant thread in this study is sensory engagement through visual means and touch mode interaction. Engholm (2010) describes aesthetics as the “sensuous qualities, the emotions, moods, and experiences” that occur during human-technology interaction. A repetitive emotional occurrence prolongs into mood (Brave & Nass, 2002). Moods last longer and are more dispersed, and they do not respond to a stimulus or event (Pekrun, 2006; Rosenberg, 1998), whereas emotions are brief with intense interludes in response to an event or situation (Rosenberg, 1998). Research has shown that a colorful object has an impact on “feelings, attention, judgments, and decisions” (Bellizzi & Hite, 1992; Babin et al., 2003; Noiwan & Norcio, 2006), and, in fact, a colorful prototype can capture and hold a viewer’s attention for a longer duration than one rendered in gray scale (Bonnardel et al., 2011). A colorful palette elicits a higher level of arousal (Geslin et al., 2016), a warm color scheme generates excitement, and a cool color scheme is more likely to cause relaxation (Gorn et al., 2004). Moreover, Norman (2004) discusses the concept of behavioral aesthetics that can occur through product interaction.

In the present study, we devised three different types of tactile interaction (swipe, tilt, and tap) to complement the overall game aesthetics. The aesthetics of an interface cannot be judged by its visual design quality alone; the tactile feedback plays a key role during user interaction (Jiang et al., 2016).

Hypothesis H3: High-quality visual design in a tablet game interface significantly enhances player engagement.

Methods

We conducted research in two phases.

In Phase I, using the participatory design approach, we systematically employed the UCD principles in the creation of a real iOS tablet game, which served as stimuli for users’ experience. A series of qualitative research methods included: mind-mapping, diary-study, personas, focus groups, and iterative usability testing.

In Phase II, we employed a one-way within-subjects analysis of variance (ANOVA) design to reduce error due to individual differences because the same participants played each game condition for 10 minutes. For hypotheses H1 and H2, the dependent variable was perceived usability, and the independent variables were the two levels of visual design quality. For hypothesis H3, the dependent variable was aggregated: aesthetics, perceived usability, reward, and focused attention, and the independent variables were the two levels of visual design quality.

We obtained Internal Review Board (IRB) approval (1516/341) prior to collecting data from participants. We provided an overview of the experimental procedure to each participant who read and signed the consent form prior to taking part in the study; they were not compensated.

Phase I

The game app was modified into two visual design conditions. We adopted an asymmetrical design to depict the high-quality visual design in the interface, giving rise to harmony and unity among the visual elements. To create the low-quality visual design condition, we violated the principles of design, as explained in the following studies (Tuch, 2011; Wong et al., 2010; Mahlke, 2008). In this study, we modified the user interface into a monochromatic color scheme, with reduced material texture and a low number of graphic bits, to create the low-quality visual design in the tablet game interface. Whereas a red-orange color palette (warm color scheme) was employed to depict the high-quality visual design condition for it is deemed to be more appealing with a propensity to capture viewers’ attention.

In Phase I, we modified the game prototypes into two visual design conditions, low- and high-quality (see Appendix 1). We gave a classical design questionnaire (Lavie & Tractinsky, 2004) (see Appendix 2) to a sample of nineteen participants who had design backgrounds as a first step to evaluate the perceived quality of visual design in the game prototypes on iMac® computers. Classical design refers to the traditional design guidelines that evaluate contrast, orderliness, hierarchy, symmetrical design, and grid system. The classical design questionnaire has five Likert scale items, and all five items are computed to obtain the mean score. Following the design and evaluation of the game prototypes, we proceeded to build the games for the iPads.

During the game development process, we conducted more than twenty-four rounds of iterative user testing by inviting six game participants to diagnose usability issues, including functionality, and to debug the game prototypes. Following the prototype development for the iPad, two versions of the iOS game application were finalized as stimuli for research experiments in Phase II.

Phase II

We recruited 56 participants (20 females and 36 males) on a university campus in the United States using expert sampling, a type of purposive sampling (Etikan & Bala, 2017). This sampling technique was necessary because participants were screened based on mobile game proficiency. Of the recruited sample, 66% self-reported that they played mobile games for 1-3 hours a week, 23% indicated they played less than 1 hour a week, and 1% indicated they played for more than 4 hours a week. The number of participants was determined based on the within-subjects research design experimental study; the sample was counterbalanced to account for order effects (Brooks, 2012). Fifty percent of the participants were randomly assigned to play the game with the low-quality visual design condition first, and 50% were randomly assigned to play the game with the high-quality visual design condition to increase internal reliability. Each participant was randomly assigned to play a game condition to create equivalent groups and safeguard the reliability of the research design. Carry-over effects were minimized by requesting the participants take a 15-minute break between playing each game condition. Consequently, experiment effects were reduced by following the same formal experimental procedure for each participant. Two participants did not complete the survey, so their data were discarded.

In Phase II, the Multidimensional Mood State Questionnaire was used, which consisted of 24 adjectives characterizing various mood states (Steyer et al., 1997) to measure the baseline mood of participants (Appendix 4). The instrument includes three constructs related to mood: valence (good–bad), arousal (awake–tired), and calmness (calm–nervous), of which valence and arousal were determined to be applicable to the study.

The AttrakDiff instrument, a semantic differential scale comprised of a six-point bipolar rating scale (Appendix 3), has been extensively utilized to study perception of usability, hedonic, and visual design in products (Christou, 2014; Hamborg et al., 2014; Hassenzahl & Monk, 2010). Two self-report questionnaires of the AttrakDiff instrument (Hassenzahl et al., 2003), namely the Pragmatic Questionnaire (PQ) and the Attractive Questionnaire (AT), were administered to measure perceived usability and perceived visual design on a scale of 1-6. The AttrakDiff instrument has high internal consistency: PQ with Cronbach’s Alpha = .88, and AT with Cronbach’s Alpha = .91 (Isleifsdottir & Larusdottir, 2008). The PQ questionnaire measures perceived usability with seven bipolar items; the AT questionnaire measures perceived visual design, consisting of six bipolar items. Additionally, we employed O’Brien’s (2010) UES questionnaire to evaluate engagement levels in both game conditions. O’Brien et al. (2018) reported that the UES constructs had high reliability. With a 95% confidence interval (CI), the Omega reliability estimate of the UES constructs is as follows: perceived usability (.92), aesthetics (.90), focused attention (.92), and reward (.87).

At the beginning of the experiment, we administered a mood questionnaire to assess the baseline mood of the participants. Each participant was then randomly assigned to play a 10-minute game with either the low- or high-quality visual design condition. After playing each game condition, participants completed the two dimensions of the AttrakDiff instrument (PQ and AT). Participants also completed the UES questionnaire (Appendix 5) for each game condition to measure user engagement.

At the end of each gameplay session, we randomly administered a semi-structured questionnaire with open-ended questions to 25 participants about their gameplay experience. The questionnaire asked the following: 1) Which features in the low-quality visual design condition made gameplay challenging? 2) Which features in the high-quality visual design condition made gameplay gratifying? The game scores for each condition were recorded.

Results

Quantitative Data Analysis

We conducted a debriefing session with participants, and we analyzed the quantitative and qualitative data.

Phase I

Evaluation of Game Prototypes on an iMac 22” Monitor

The first step was to design and test the game prototypes on an iMac 22” computer monitor to ensure they met the requirements of low- and high-quality visual design. We report the data analysis of the classical design questionnaire in this study.

A paired-sample t test was conducted to compare the perceived quality of visual design in both conditions. There was a significant difference in the mean values between the high-quality visual design ( = 4.25, s = 0.731) and the low-quality visual design ( = 2.83, s = 0.714) in t(18) = 1.356, p < .05. The results in Figure 1 confirm that an interface with high-quality visual design was perceived to be significantly more attractive and appealing. After that, the iOS game prototype was built for the iPad using Apple’s game development engine, the SprikeKit™ framework.

Figure 1. Mean scores of perceived quality in the visual design of game prototypes.

Evaluation of iOS Game Apps on iPad 9.7” Screen

The next step was to confirm that the two visual design conditions of the tablet game interfaces were significantly different from each other. The perception of visual design data was primarily inspected for normality of distribution and verified for ANOVA assumptions. Upon initial data inspection, the boxplot of the residuals of the dependent variable data, perception of quality in visual design, were skewed to the right; the standard deviation shows more variability in the game sample with high-quality visual design. Since the ratio of the largest group variance (s² = 0.942) was not more than four times larger than the smallest group variance (s² = 0.311), the assumption of homogeneity of variances was not violated. This implies Type 1 and 2 errors were minimized, and the variance between the two groups was equivalent.

Table 1. Descriptive Statistics of Perceived Visual Design, Mean Scores—AttrakDiff (AT)

	N	Mean,	Std. Deviation, s
Perceived Low-Quality Visual Design	54	3.969	.558
Perceived High-Quality Visual Design	54	4.871	.971

As shown in Table 1, the mean value of the perceived high-quality visual design sample ( = 4.87, s² = 0.94) was greater than the low-quality visual design sample ( =3.96, s² = .31), as measured by the semantic differential scale of the AttrakDiff AT (Appendix 3). This implies that the high-quality visual design condition was perceived to be moderately attractive, and the low-quality visual design condition was perceived to be fairly attractive. The bipolar rating scale provides a mean score between the range of 1-6.

Phase II

The outcome of a within-subjects ANOVA test, F(1,53) = 49.488, p < .005, η_p² = 0.483, indicated that there was a significant difference of perceived quality in the visual design between the two manipulated conditions. This supported the main objective of the study to examine how the variation of quality in the visual design of tablet game interfaces affects users’ perception of the game’s usability. The perception of usability (PQ) dataset was inspected for normality of distribution and verified for ANOVA assumptions. Both high and low perception of usability datasets were normally distributed, as assessed by the Shapiro-Wilk test, at a conservative significance level of .05. Table 2 shows the mean values of the perceived usability of games with low- and high-quality visual design conditions.

Table 2. Descriptive Statistics of Perceived Game Usability, Mean Scores—AttrakDiff (PQ)

	N	Mean,	Std. Deviation, s
Perceived Usability of Low-Quality Visual Design	54	4.868	.720
Perceived Usability of High-Quality Visual Design	54	4.908	.758

Hypothesis H1: High-quality visual design in a tablet game interface is perceived to be more usable.

A one-way within-subjects ANOVA test (Girden, 1992) revealed that perceived game usability in the two conditions was not statistically significant, F(1,53) = .138, p = .711. This indicates that there was no significant difference in the perception of game usability as experienced by the participants interacting with interfaces that had low- ( = 4.868) and high-quality ( = 4.908) visual design. Perceived game usability was therefore perceived to be equivalent in both conditions. This confirms that participants did not find that the game with the high-quality visual design condition was more usable than the game with the low-quality visual design condition.

Examining the relationships of the two components, the perceived visual design quality and perceived game usability, in each game condition was important. Prior to conducting a regression analysis, preliminary analyses ensured no violation of the assumptions of normality, linearity, and homoscedasticity. A scatterplot of perceived game usability against perceived visual design quality with superimposed regression lines depicted linearity. Cohen’s (1988) classification determined the strength of the effect as follows: r of .1 for small effect size, r of .3 for medium effect size, and r of .5 for large effect size.

Hypothesis H2: There is no correlation between low-quality visual design and perceived game usability.

We conducted a linear regression analysis to examine the relationships of the two dependent variables, perceived quality in visual design and perceived game usability, in both game conditions. Both low- and high-quality visual design conditions accounted for significant and similar amounts of variation in perceived usability.

Preliminary analyses ensured there was no violation of the assumptions of normality, linearity, and homoscedasticity. Visual inspection of the scatterplots indicated a linear relationship between the variables. Homoscedasticity and normality of the residuals were present. The datasets were standardized by transforming them into z-scores. A linear regression analysis predicted perceived game usability from the independent variable, perceived quality of visual design in the game user interface, in both the game conditions.

Low-quality visual design had a significant influence on the perception of game usability, F(1,53) = 4.089, p = .048, accounting for 7.3% of the variation in the perceived usability with adjusted R2 = .055, a small effect size (according to Cohen’s classification (1988) of 0.01, 0.09, and 0.25 for small, medium, and large effect size respectively in the case of adjusted R2; and a standardized beta coefficient, r = .270 (95% CI, .002 to .538). Based on this observation, low-quality visual design in the game interface had a significant and small positive correlation with perceived game usability.

High-quality visual design significantly predicted the perception of game usability, F(1,53) = 5.05, p = .029, accounting for 8.9% of the variation in perceived usability with adjusted R2 = .071. A small size effect according to Cohen’s (1988), with standardized beta coefficient, r = .298, (95% CI, .025 to .440) occurred. A high-quality visual design in the game interface had a significant and small correlation with perceived game usability.

The two above coefficients, r1 = .270 (Z1 = .277) and r2 = .298 (Z2 = .307), were converted to z-scores to determine . Since -1.96 ≤ = -.1514 ≤ +1.96. This implies that there was no significant difference in the strength of the correlation between perceived usability and visual design in each condition. Both low- and high-quality visual design conditions resulted in the same amount of perceived usability in a positive association; perceived high-quality visual design is associated with higher perceived usability by a negligible amount.

Figure 2. Correlation between the quality of visual design and perceived usability.

Hypothesis H3: High-quality visual design in a tablet game interface significantly enhances player engagement.

We conducted a one-way repeated measures ANOVA of the UES data to examine the levels of user engagement in the two game conditions. The analysis revealed a statistically significant difference of overall user engagement between the two game versions, F(1,33) = 42.84, p < .005, η_p² = 0.565. We aggregated the overall UES by averaging the mean values of each of the four UES dimensions: reward, focused attention, aesthetics, and perceived usability. Table 3 illustrates users were more engaged playing the game interface with the high-quality visual design condition, with an overall UES mean value of 3.34.

Table 3. Comparison of UES Dimensions in Both Conditions

UES Dimensions	N	Mean: LQ	Mean: HQ	p-value .05	F-Statistics
Overall UES	34	= 2.97 (s = 0.46)	= 3.34 (s = 0.50)	p < .005***	F(1,33) = 42.84, np2 = 0.565
Reward	34	= 3.07 (s = 0.48)	= 3.347 (s = 0.35)	p < .005***	F(1,33) = 15.09, np2 = 0.314
Focused Attention	34	= 2.43 (s = 0.89)	= 2.491 (s = 0.90)	p = .477	F(1,33) = 0.518, np2 = .015
Aesthetics	34	= 2.83 (s = 0.57)	= 3.84 (s = 0.64)	p < .005***	F(1,33) = 50.74, np2 = 0.606
Perceived Usability	34	= 3.54 (s = 0.52)	= 3.67 (s = 0.49)	p = .128	F(1,33) = 2.43, np2 = .069

*(p < .05), **(p < .01), ***(p < .001)

The result also shows that only two dimensions of the UES scale, namely aesthetics and reward, were significantly higher in the high-quality visual design version, which implies that participants found this version of the game more rewarding and aesthetically pleasing (Figure 3). Alternately, the ratings for the other two dimensions, perceived usability and focused attention, were non-significant and therefore equivalent in both conditions.

Figure 3. Boxplots of overall user engagement showing mean scores of low- and high-quality visual design conditions (p < .05).

We individually analyzed the four dimensions of UES in each condition as shown in Tables 4 and 5 respectively. We measured participants’ baseline mood using the two constructs, valence (good-bad) and arousal (awake-tired) of the Multidimensional Mood State Questionnaire. We conducted a partial correlation between the four UES dimensions by controlling the two mood covariates, mood-GB (valence) and mood-AT (arousal), to minimize the effects of the confounding variables. In both conditions, there was linearity as assessed by partial regression plots and a plot of studentized residuals against the predicted values. No extreme outliers were detected in the individual boxplots. Homoscedasticity was present, as assessed by visual inspection of a plot of studentized residuals versus unstandardized predicted values. All variables were normally distributed, as assessed by Shapiro-Wilk’s test (p > .05).

Table 4. Partial Correlation Coefficients of UES Dimensions (Low-Quality Visual Design), Controlling for Mood Variables

	Focused Attention	Perceived Usability	Reward
Perceived Usability	.256
Reward	.462**	.462**
Aesthetics	.611**	.109	.596**

**p < .01; *p < .05

In Table 4, a Pearson partial correlation showed that all the UES dimensions were significantly correlated, excluding: perceived usability and focused attention; and aesthetics and perceived usability. It is important to note that the correlation between perceived usability and aesthetics, as measured by the UES instrument, was not significant in the low-quality visual design condition.

Table 5. Partial Correlation Coefficients of UES Dimensions (High-Quality Visual Design), Controlling for Mood Variables

	Focused Attention	Perceived Usability	Reward
Perceived Usability	.05
Reward	.737**	.413*
Aesthetics	.424*	.391*	.681**

**p<.01; *p<.05

In Table 5, a Pearson partial correlation indicated that all the UES dimensions were significantly correlated in the high-quality visual design condition, except for perceived usability and focused attention. The relationship between aesthetics and perceived usability, as measured by the UES instrument, in the game with the high-quality visual design condition, showed a significant, moderate, and positive correlation, r = .424′.

A linear regression determined the effect of game aesthetics on perceived usability. The linear regression established that game aesthetics could statistically significantly predict perceived usability, F(1,33) = 5.486, p < 0.05, and aesthetics accounted for 17.9% of the explained variability in perceived usability. The regression equation predicted perceived usability = 2.65 + .264 x (aesthetics).

Qualitative Data Analysis

As recommended by Braun & Clarke (2008), we analyzed the open-ended questions of the semi-structured questionnaire by first reading the data responses thoroughly, grouping the thematically similar data, and categorizing them related to theoretical concepts (Guest et al., 2012). Data that did not support the theories were also highlighted in this process. We made connections between meaningful themes and research questions. Themes converged from the analysis of 23 of the 25 participants. Data highlighted that the participants preferred to play the game version with high-quality visual design. Participants reported that the crisp and high-quality visual design graphics made gameplay playful and engaging, enticing them to play multiple times. Participants also reported they had a more pleasurable experience playing the high-quality visual design version. The colorful graphics enlightened their mood and made gameplay more gratifying. They could identify the obstacles and enemies easily and felt more in control.

In contrast, only a few participants explained that they could make a connection with the game environment rendered in a monochromatic color scheme, which was used in the low-quality visual design game version. In the game version with low-quality visual design, participants experienced a subtly higher level of challenge to distinguish game assets due to the low color contrast and monochromatic color scheme. Additionally, the game version with low-quality visual design appeared overly simplistic and unattractive to game participants.

Discussion of Results

The purpose of this study was to examine the role of quality in visual design on perceived game usability and user engagement.

Phase I

We manipulated the game interface into conditions having either low-quality visual design or high-quality visual design; the inherent usability of each game condition was unaltered. In Phase I, we evaluated the two visual design conditions for users’ perception of the quality of the visual aesthetics using a total sample of nine participants. Visual perception is a factor of both, “the perceptual and the conceptual gist” (Harper et al., 2009). The perceptual gist is the factual information extracted from the stimuli such as color, texture, shape, and volume; the conceptual gist is the assimilation and interpretation of the stimuli presented. To violate the design principles, we modified the low-quality visual design into a monochromatic color scheme with subdued contrast between the visual game elements and the background; the graphics were rendered in an 8-bit format. Users perceived a significant difference of quality in the visual aesthetics between the two game conditions. A series of user testing and debugging eliminated the usability issues in both game conditions, resulting in equivalent inherent usability in both game conditions. Thereafter, the game stimuli were adequately developed to test the three hypotheses formulated for the experimental phase of the study.

Phase II

Hypothesis H1, in Phase II, posited that a tablet game interface with high-quality visual design is perceived to be more usable. The results of the current study did not support hypothesis H1. The findings of the present study show that perceived game usability was practically equivalent in both game conditions (Figure 2). Therefore, a game interface with high-quality visual design does not necessarily improve perceived game usability. This result was substantiated by the performance of participants in each game condition as there were no significant difference between the scores. The qualitative findings from the open-ended questions reveal that a game interface with high-quality visual design, including the attractive heads-up display, has the propensity to attract user attention during gameplay; however, it does not necessarily promote perceived game usability. The quality of visual design in the game interface might have had an initial, visceral impact on players, but with time, users shifted their focus to the dynamic gameplay components.

Although low-quality visual design in the game interface could have impeded gaming activity to some extent (e.g., making it less rewarding), participants were able to complete the low-quality visual design version with a level of accuracy. Open-ended questions, administered post-gameplay, recorded that participants mentioned the monochromatic color scheme had impaired visual clarity but that there was a trade-off between the level of difficulty and the learning curve of the game activity. Besides the quality of visual design, a plausible explanation is that game mechanics played an important part in sustaining gameplay because both game versions had the same inherent usability. Participants became familiarized quickly with the game mechanics and rules as the game interface features demonstrated moderate to high utility in the low-quality visual design condition. Utility is a function of interface, context, and task (Toomim et al., 2011), and it is determined by the usefulness of the functionality of the game. According to Shackel (1991), utility is defined as the fulfillment of user needs regarding the functionality of the product, whereas usability concerns the ability to use the feature sets and functionality. Findings of this study reveal that quality in visual design is important to game participants at the beginning of the activity but, once they are engaged in the gaming action, their perception becomes more resilient to low-quality visual design over time because they focus more on game mechanics.

Hypothesis H2 posited that low-quality visual design does not correlate with perceived game usability. The findings did not support hypothesis H2. The results showed a significant correlation between visual design and perceived game usability. Findings from the AttrakDiff instrument showed that both game interfaces employing low- and high-quality visual design significantly correlate with perceived game usability (Figure 2). Unlike previous studies, in which the influence of visual design on perceived usability in digital products were not supported (Lindgaard & Dudek, 2003; Mahlke, 2008; van Schaik & Ling, 2009; Grishin & Gillan, 2019), the present study empirically shows that both low- and high-quality visual design had an influence on perceived usability in the domain of tablet gaming. Interestingly, it was found that the game interface with low-quality visual design had a small, positive, and significant correlation with perceived game usability. This implies that even a game interface with low-quality visual design can ease game operation. Analyses of open-ended questions, administered post-gameplay, found that the monochromatic color rendering of the low-quality visual design condition drew participants closer to the game environment, which relates to game fantasy.

Aesthetic experience draws on visual evaluation of artifacts (Palmer et al., 2013; Cinzia & Vittorio, 2009) and is dependent on higher level cognitive processes for visual analysis, according to Zhou et al. (2016). The findings of the present study do not fully align with the notion of “what is beautiful is usable” (Tractinsky et al., 2000), as this theory claims that only attractive products are perceived to be usable. The visual elements of the game interfaces have an influence on players’ behavior and interaction. Game aesthetics is characterized by challenge, fantasy, story, and senses (Hunicke et al., 2012). It can be argued that other characteristics of game aesthetics, besides visual design, provided necessary motivation and impulse for players to achieve their goals in the low-quality visual design version. The visceral layer of the interfaces transmits the necessary hedonic experiences of the game aesthetics at the behavioral level during gameplay. Thus, gameplay gives rise to an aesthetic experience, which entails sensory engagement (tactile user interaction and musical sound effects), and a balance between individual skills and game challenge. We concluded that the aesthetics of interaction was more prominent through playfulness, dynamism, and smooth mechanics than aesthetics of appearance of game user interaction. The aesthetics of an interface cannot be judged by its visual design quality alone, but also by user interaction (Jiang et al., 2016).

Hypothesis H3 posited a significant and positive association between a tablet game interface with high-quality visual design and overall user engagement. The findings supported hypothesis H3. Game participants experienced a relatively higher level of engagement interacting with the high-quality visual design condition. This is demonstrated by the mean values of the two UES dimensions, aesthetics and reward; the two constructs were shown to be higher in the high-quality visual design condition and significantly different in the two game conditions (Table 3).

UES Constructs in Game Conditions

In the low-quality visual design condition, the correlation between the aesthetics and perceived usability was non-significant using the UES instrument (Table 4). The themes that emerged from the semi-structured questionnaire data in the low-quality visual design condition indicated that participants experienced anxiety and had difficulty focusing on targets. Participants had to make extra effort discerning the visual game elements. Participants could not accurately focus and target game assets in the low-quality visual design condition even though the game mechanics and the inherent usability in both versions were similar. The low-quality graphics with a condensed visual density posed certain limitations in the monochromatic color scheme that might have increased user cognitive load, which in turn compounded focused attention.

The partial correlational analysis showed that focused attention did not significantly correlate with perceived usability. Reward and perceived usability were moderately and positively correlated in the low-quality visual design condition because participants were able to accomplish their goals by completing the game. This shows that participants were intrinsically motivated to continue the gaming activity because they felt the game experience and interaction were rewarding. In fact, a player’s interest to continue an activity is sustained by motivational factors. Literature on motivation theories describe two types of motivation for achieving one’s goal: intrinsic and extrinsic motivation (Lin et al., 2012; Ryan & Deci, 2000). Participants in this study voluntarily participated in the game study; so, we can rule out external reward or incentives. We can confidently say that game participants were intrinsically motivated because they sought inner-satisfaction and self-smugness (Bittner & Shipper, 2014); and self-interest and enjoyment (Deci & Ryan, 2000), though not necessarily because of external reward or pressure (Lin et al., 2012; Banyte & Gadeikiene, 2015). In addition, focused attention significantly correlated with the aesthetics variable. Perhaps, the different types of touch gesture interaction such as swipe, tilt, and tap increased their enthusiasm even though visual design quality was inferior.

In contrast, the correlation between perceived usability and aesthetics in the high-quality visual design condition was significant and moderate to strong, with a Pearson coefficient of r = .681 (Table 5). Findings from qualitative data suggest that the 2.5D graphics style rendering along with the analogous color scheme in the high-quality visual design played a prominent role in capturing participants’ attention in the gameplay. This is indicated by the strong and positive correlation between aesthetics and focused attention, with Pearson coefficient of r = .424 (Table 5). Furthermore, the crisp colorful game graphics appealed to the visual senses, thereby creating an engaging visual experience. Sophisticated graphics from the attractive interface augmented player engagement as it evoked a heightened level of valence, and pleasurable experience, which reflects Tellegen et al. (1999) findings. The goal is to elicit emotional responses such as curiosity, wonder, and surprise for a player (Lazzaro, 2008). The high-quality visual design version of the game provided clear goals and objectives, which clearly promoted gameplay. As participants found the game to be usable, it provided them with necessary directions to understand the game mechanics and gameplay. Participants reported that the high-quality visual design version provided a more rewarding gaming experience. This is substantiated by the strong and positive correlation between esthetics and reward, with a Pearson coefficient of r = .681 (Table 5). Thus, the visual elements of the high-quality visual design condition were more gratifying and fun to use. Consequently, the mean value for the reward dimension was higher.

A user may experience different levels of involvement during product interaction: engagement, engrossment, and total immersion (O’Brien et al, 2018). Engrossment is also a determinant of Sweetser & Wyeth’s (2005) game flow model to measure game enjoyment. Besides high-quality visual design elements, game narrative proved to be an important component in strategy-based tablet games, given the exploratory nature of gameplay. Participants recommended that a dynamic twist in the game story line should be incorporated every time a player returns to play the game. The engagement derived from high-quality visual design elements aligns with the theory explaining how visual design appeals to the senses, influencing user perception to make a positive design appraisal (O’Brien et al., 2010; O’Brien & Toms, 2008). The theory of design engagement rationalizes that a user is quickly drawn into an interface with high-quality visual design by its attractive graphics and actively engages with the interactive game artifact.

Thus, the contribution to knowledge from the present study is that a game interface with high-quality visual design empowers users to engage deeply by promoting gameplay, but it does not necessarily enhance perceived usability. Findings suggest that there are other hedonic factors such as game aesthetics that contribute to player engagement. The impact of perceived quality in visual design plays a crucial role at the beginning of gameplay to capture players’ attention but, over time, its effect wanes and participants tend to focus more on game mechanics.

Limitations of the Study

• The sample recruited for this study was not selected using probability sampling due to resource constraints. To increase internal reliability, participants were assigned randomly to each game condition.
• The carry-over effect in the experimental condition might have led to a practice effect when playing one game condition was followed by the second one, despite the short break interval.
• The carry-over effect might have also led to a context effect; testing in one condition might have altered participants’ perception of the stimulus in the later condition.
• The sample recruited on a university campus may not be representative of the actual population of tablet game players.
• There is always a risk that the game interface was not appraised with accuracy given that not everyone had an affinity to judge the level of visual aesthetics in the products.
• The researcher was the sole observer during the iterative game design and development process; important information may have been missed. A moderator to observing the design and testing process would be able to observe participants’ gestures and behaviors during play tests.

Conclusion

The approach of this study incorporates user-centered principles and techniques into the design process. The main contribution to knowledge is that the two components of user engagement, aesthetics and rewards, empower user engagement because they promote gameplay. It is deduced that the quality of visual design in a game interface alone is not a determining factor to judge perceived usability; there are other important components such as game mechanics and game narratives that render gameplay holistically meaningful. Additionally, both low- and high-quality visual design can influence perceived usability in the domain of tablet gaming. Excluding other confounding factors from the visual design variable, particularly on perceived usability in the low-quality visual design condition, remains a challenge. This study established that even a game interface with low-quality visual design could connect the player to gameplay through learnability and familiarity because inherent usability was equivalent in both conditions.

Although a game interface with high-quality visual design does not necessarily improve perception of the game’s usability, it does enhance user engagement. The high-quality visual design condition empowered user engagement as sophisticated graphics have the propensity to appeal to the senses, create a more rewarding experience, and impact user experience. We recommend future research to explore visual design styles that corroborate with genres already shown to be effective in mobile-game user interfaces. It is also necessary to pursue further research to understand the effect of game duration, its corresponding narrative, and its aesthetics related to perceived usability and game performance.

Tips for Usability Practitioners

• The usability inherent in the game is a fundamental component of the game mechanics that enables gameplay; in contrast, apparent or perceived usability plays a secondary role. It is the inherent game usability that sustains player experience.
• Aesthetics of interaction is more prominent through playfulness, dynamism, and game mechanics than the visual design component, which is apparent at the surface, or visceral, level.
• Players seek to identify symbolic meanings of the game’s visual elements that are included in the game’s narrative so that they can relate at the reflective level and make an emotional connection, which motivates them to continue to play.
• Visual design quality is important at the beginning of the activity; once the game participants are engaged in the gaming action, their perception becomes more resilient to the (low-quality) visual design.
• Artistic styles conveyed in game interfaces are unique forms of artistic, visual expression that play a crucial part in captivating users’ attention and interest. In this study, the rendition of 2.5D graphics accentuated the illusion of visual volume and game assets appeared to stand out from their background. Assessing artistic quality in game interfaces using objective methods such as eye tracking and electrodermal activity is recommended.

References

Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-centered design. In W. Bainbridge (Ed.), Encyclopedia of human-computer interaction (Vol. 2, pp. 763–768). SAGE Publications.

Ashby, M., & Johnson, K. (2003). The art of material selection. Materials Today, 6(12), 24­–35.

Attfield, S., Kazai, G., Lalmas, M., & Piwowarski, B. (2011). Towards a science of user engagement. In Proceedings of WSDM’11, Workshop on User Modelling for Web Applications, Hong Kong, China. AMC.

Babin, B. J., Hardesty, D. M., & Suter, T. A. (2003). Color and shopping intentions: The intervening effect of price fairness and perceived affect. Journal of Business Research, 56(2), 541–551.

Banyte, J., & Gadeikiene, A. (2015). The effect of consumer motivation to play games on video game-playing engagement. Procedia Economics and Finance, 26, 505–514.

Bellizzi, J. A., & Hite, R. E. (1992). Environmental color, consumer feelings and purchase likelihood. Psychology and Marketing, 9(5), 347–363.

Bittner, J. V., & Shipper, J. (2014). Motivational effects and age differences of gamification in product advertising. Journal of Consumer Marketing, 31(5), 391-400.

Bonnardel, N., Piolat, A., & Le Bigot, L. (2011). The impact of colour on website appeal and users’ cognitive processes. Displays, 32(2), 69–80.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101.

Brave, S., & Nass, C. (2002). Emotion in human-computer interaction. In A. J. Jacko & A. Sears (Eds.), The human-computer interaction handbook (pp. 81–96). Erlbaum Associates Inc.

Brooks, J. L. (2012). Counterbalancing for serial order carryover effects in experimental condition orders. Psychological Methods, 17(4), 600–614. https://doi.org/10.1037/a0029310

Bullot, N. J., & Reber, R. (2013). The artful mind meets art history: Toward a psycho-historian framework for the science of art appreciation. Behavioral and Brain Sciences, 36(2), 123–137.

Christou, G. (2014). The interplay between immersion and appeal in video games. Computers in Human Behavior, 32, 92–100.

Cinzia, D. D., Vittorio, G. (2009). Neuroesthetics: A review. Current Opinion in Neurobiology, 19(6), 682–687.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates Publishers.

Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.

Desmet, P. M. A., & Hekkert, P. (2007). Framework of product experience. International Journal of Design, 1, 57–66.

Dewey, J. (1938). Experience and education. Collier Books.

Djajadiningrat, T., Wensveen, S., Frens, J., & Overbeeke, K. (2004). Tangible products: Redressing the balance between appearance and action. Personal and Ubiquitous Computing, 8(5), 294–309.

Engholm, I. (2010). The good enough revolution—The role of aesthetics in user experiences with digital artefacts. Digital Creativity, 21(3), 141–154. https://doi.org/10.1080/14626268.2010.488809

Etikan, I., & Bala, K. (2017). Sampling and sampling methods. Biometrics & Biostatistics International Journal, 5(6), 215–217.

Feijóo, C., Gómez-Barroso, J. L., & Ramos, S. (2010, October 11–14). An analysis of mobile gaming development: The role of the software platform. In Proceedings of IEEE 14th International Conference on Intelligence in Next Generation Networks, Berlin, Germany. IEEE Xplore.

Forlizzi, J., & Ford, S. (2000). The building blocks of experience: An early framework for interaction designers. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, New York, USA. ACM.

Frohlich, D. M. (2004). Beauty as a design prize. Human Computer Interaction, 19(4), 359–366.

Garrett, J. J. (2010). The elements of user experience: User-centered design for the web and beyond (2^nd ed.). New Riders.

Geissler, G., Zinkhan, G., & Watson, R. (2001). Web home page complexity and communication effectiveness. Journal of the Association for Information, 2(1), 1–47.

Geslin, E., Jégou, L., & Beaudoin, D. (2016). How color properties can be used to elicit emotions in video games. International Journal of Computer Games Technology, 2016, Article 5182768. https://doi.org/10.1155/2016/5182768

Girden, E. R. (1992). ANOVA: Repeated measures. SAGE Publications, Inc.

Gorn, G. J., Chattopadhyay, A., Sengupta, J., & Tripathi, S. (2004). Waiting for the web: How screen color affects time perception. Journal of Marketing Research, 41(2), 215–225.

Grishin, J., & Gillan, D. J. (2019). Exploring the boundary conditions of the effect of aesthetics on perceived usability. Journal of Usability Studies, 14(2), 76–104.

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18(1), 59–82.

Hamborg, K-C., Hulsmann, J., & Kaspar, K. (2014). The interplay between usability and aesthetics: More evidence for the “What is usable is beautiful” notion. Advances in Human-Computer Interaction, 2014, Article 946239. https://doi.org/10.1155/2014/946239

Harper, S., Michailidou, E., & Stevens, R. (2009). Toward a definition of visual complexity as an implicit measure of cognitive load. ACM Transaction on Applied Perception, 6(2), Article 10.

Hassenzahl, M. (2003). The thing and I: Understanding the relationship between user and product. In M. Blythe, C. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From usability to enjoyment (pp. 31–42). Kluwer Academic Publishers.

Hassenzahl, M., Burmester, M., & Koller, F. (2003). AttrakDiff: A questionnaire to measure perceived hedonic and pragmatic quality. In J. Ziegler & G. Szwillus (Eds.), Mensch & computer 2003: Interaktion in Bewegung (pp. 187–196). B. G. Teubner.

Hassenzahl, M. (2004). The interplay of beauty, goodness, and usability in interactive products. Human Computer Interaction, 19(4), 319–349.

Hassenzahl, M. (2008). User experience (UX): Towards an experiential perspective on product quality. In Proceedings of the 20th International Conference of the Association Francophone d’Interaction Homme-Machine (pp. 11–15). ACM.

Hassenzahl, M., & Monk, A. (2010). The inference of perceived usability from beauty. Human Computer Interaction, 25(3), 235–260.

Hassenzahl, M., & Tractinsky, N. (2006). User experience—A research agenda. Behavior and Information Technology, 25(2), 91–97.

Hunicke, R., Le Blanc, M., & Zubek, R. (2004). MDA: A formal approach to game design and game research. Retrieved on July 03, 2021, from http://www.cs.northwestern.edu/~hunicke/MDA.pdf

Isleifsdóttir, J., & Lárusdóttir, M. (June 2008). Measuring the user experience of a task oriented software. In Proceedings of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (pp. 97–102). Institute of Research in Informatics of Toulouse (IRIT).

International Organization for Standardization. (1998). Ergonomic requirements for office work with visual display terminals (VDTs)—Part II: Guidance on usability (ISO Standard No. 9241-11:1998, ISO/IEC 9241-11).

Jacques, R. D. (1996). The nature of engagement and its role in hypermedia evaluation and design [Unpublished doctoral dissertation, South Bank University].

Jacques, R., Preece, J., & Carey, T. (1995). Engagement as a design concept for multimedia. Canadian Journal of Learning and Technology, 24(1), 49–59.

Jiang, Z., Wang, W., Tan, B. C. Y., & Yu, J. (2016). The determinants and impacts of aesthetics in users’ first interaction with websites. Journal of Management Information Systems, 33(1), 229–259.

Kokil, U. (2019). The effect of visual design quality on player experience components in tablet games [Doctoral dissertation, De Montfort University]. De Montfort Open Research Archive.

Kurosu, M., & Kashimura, K. (1995, May 7–11). Apparent usability vs. inherent usability: experimental analysis on the determinants of the apparent usability. In Proceedings of CHI ’95: Conference Companion on Human Factors in Computing Systems (pp. 292–293). ACM.

Laurel, B. (1993). Computers as theatre. Addison-Wesley.

Lavie, T., & Tractinsky, N. (2004). Assessing dimensions of perceived visual aesthetics of websites. International Journal of Human–Computer Studies, 60(3), 269–298.

Lazzaro, N. (2008). The four fun keys. Game usability. In K. Isbister & N. Schaffer (Eds.), Advancing the player experience (pp. 315–344). Elsevier.

Lin, H-H., Wang, Y-S., & Chou, C-H. (2012). Hedonic and utilitarian motivations for physical game systems use behavior. International Journal of Human–Computer Interaction, 28(7), 445–455. https://doi.org/10.1080/10447318.2011.618097

Lindgaard, G., & Dudek, C. (2003). User satisfaction, aesthetics and usability: Beyond reductionism. In Proceedings of IFIP 17th World Computer Congress (pp. 231–246).

Lindgaard, G., & Whitfield, T. W. A. (2004). Integrating aesthetics within an evolutionary and psychological framework. Theoretical Issues in Ergonomics Science, 5(1), 73–90.

Mahlke, S., & Lindgaard, G. (2007). Emotional experiences and quality perceptions of interactive products. In J. Jacko (Ed.), Human-Computer Interaction: Interaction Design and Usability. HCI 2007. Lecture Notes in Computer Science (Vol. 4550, pp. 164–173). Springer. https://doi.org/10.1007/978-3-540-73105-4_19

Mahlke, S. (2008). User experience of interaction with technical systems [Doctoral dissertation, Berlin University of Technology]. https://depositonce.tu-berlin.de/handle/11303/2090

Mahlke, S. (2008, July 13–16). Visual aesthetics and the user experience. In Dagstuhl Seminar Proceedings of the Study of Visual Aesthetics in Human Computer Interaction (pp. 1–5). Leibniz-Zentrum fuer Informatik.

McCarthy, J., & Wright, P. (2004). Technology as experience. MIT Press.

Moran, K. (2017). The aesthetic-usability effect. Retrieved August 21, 2020, from https://www.nngroup.com/articles/aesthetic-usability-effect

Moshagen, M., & Thielsch, M. (2010). Facets of visual aesthetics. International Journal of Human–Computer Studies, 68(10), 689–709.

Mottus, M., Lamas, D., & Kukk, L. (2017). Aesthetic categories of interaction: Aesthetic perceptions on smartphone and computer. In Proceedings of Federated Conference on Computer Science and Information Systems (Vol. 11, pp. 1249–1256). http://dx.doi.org/10.15439/2017F408

Nielsen, J., & Cardello, J. (2013, November 09). The halo effect. Retrieved October 25, 2020, from https://www.nngroup.com/articles/halo-effect/

Niedenthal, S. (2009). What we talk about when we talk about game aesthetics. In A., Barry, K., Helen, K., & Tanya (Eds.), Breaking new ground: Innovation in games, play, practice and theory. In Proceedings of the 2009 digital games research association conference. Brunel University.

Noiwan, J., & Norcio, A. F. (2006). Cultural differences on attention and perceived usability: Investigating color combinations of animated graphics. International Journal of Human-Computer Studies, 64(2), 103–122.

Norman, A., & Draper, S. W. (1986). User centered system design. Lawrence Erlbaum.

Norman, D. (2004). Emotional design: Why we love (or hate) everyday things. Basic Books.

O’Brien, H., Cairns, P., & Hall, M. (2018). A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. International Journal of Human Computer Studies, 112, 28–39.

O’Brien, H., & Toms, E. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology, 59(6), 938–955.

O’Brien, H. L. (2010). The influence of hedonic and utilitarian motivations on user engagement: The case of online shopping experiences. Interacting with Computers, 22(5), 344–352.

O’Brien, H., Toms, E., Kelloway, K., & Kelley, E. (2010). The development and evaluation of a survey to measure user engagement, Journal of the American Society for Information Science and Technology, 61(1), 50–69.

Oshita, M., & Ishikawa, H. (2012, November). Gamepad vs. touchscreen: A comparison of action selection interfaces in computer games. In Proceedings of Workshop SIGGRAPH Asia WASA’ 12 (pp. 27–31). ACM.

Palmer, S.E., Schloss, K. B., & Sammartino, J. (2013). Visual aesthetics and human preference. Annual Review of Psychology, 64(1), 77–107.

Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Physical Review, 18, 315–341.

Porat, T., & Tractinksy, N. (2012). It’s a pleasure buying here: The effects of web-store design on consumers’ emotions and attitudes. Human Computer Interaction, 27(3), 235–276.

Reber, R., Schwarz, N., & Winkielman, P. (2004). Processing fluency and aesthetic pleasure: Is beauty in the perceiver’s processing experience? Personality and Social Psychology Review, 8(4), 364–382.

Rosenberg, E. L. (1998). Level of analysis and the organization of affect. Review of General Psychology, 2(3), 247–270.

Rubin, J., & Chisnell, D. (2008). Handbook of usability testing: How to plan, design, and conduct effective tests (2nd ed.). Wiley Publisher.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–

78.

Schiphorst, T. (2009, April). Soft(n): Toward a somaesthetics of touch. CHI ’09 Extended Abstracts on Human Factors in Computing Systems, 2427–2438. https://doi.org/10.1145/1520340.1520345

Seo, K., Lee, S., & Chung, B. D. (2016). Effects of perceived usability and aesthetics on emotional responses in different contexts of use. International Journal of Human-Computer Interaction, 32(6), 445–459. https://doi.org/10.1080/10447318.2016.1160537

Shackel, B. (1991). Usability—Context, framework, design and evaluation. In B. Shackel & S. Richardson (Eds.), Human factors for informatics usability (pp. 21–38). Cambridge University Press.

Silvennoinen, J., Vogel, M., & Kujala, S. (2014). Experiencing visual usability and aesthetics in two mobile application contexts. Journal of Usability Studies, 10(1), 46–62.

Sonderegger, A., & Sauer, J. (2010). The influence of design aesthetics in usability testing: Effects on user performance and perceived usability. Applied Ergonomics, 41(3), 403–410.

Statista. (2021). Tablet market in the US. Retrieved September 2, 2021, from https://www-statista-com.us1.proxy.openathens.net/study/10519/us-tablet-pc-market-statista-dossier/

Steyer, R., Schwenkmezger, P., Notz, P., & Eid, M. (1997). The Multidimensional Mental State Questionnaire: Manual. Hogrefe.

Sutcliffe, A. (2010). Designing for User Engagement: Aesthetic and Attractive User Interfaces. Synthesis Lectures on Human-Centered Informatics, 2(1), 1–55. Morgan & Claypool publishers.

Sweetser, P., & Wyeth, P. (2005). Gameflow: A model for evaluating player enjoyment in games. Computers in Entertainment, 3, 1-24.

Tellegen, A., Watson, D., & Clark, L. A. (1999). On the dimensional and hierarchical structure of affect. Psychological Science, 10(4), 297–303.

Thorndike, E. L. (1920). A constant error in psychological ratings. Journal of Applied Psychology, 4, 25–29.

Toomim, M., Kriplean, T., Portner, C., & Landay, J. (2011). Utility of human-computer interactions: Toward a science of preference measurement. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems (pp. 2275–2284).

Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beautiful is usable. Interacting with Computers, 13(2), 127–145.

Tuch, A. N. (2011). Visual website aesthetics in human computer interaction: Determinants of beauty and the effects of interface-aesthetics on users’ experience [Doctoral dissertation, University of Basel].

Van Schaik, P., & Ling, J. (2009). The role of context in perceptions of the aesthetics of webpages over time. International Journal of Human–Computer Studies, 67(1), 79–89.

Van Schaik, P., & Ling, J. (2012). A cognitive experiential approach to modelling web navigation. International Journal of Human Computer Studies, 70(9), 630–651.

Vredenburg, K., Mao, J. Y., Smith, P. W., & Carey, T. (2002, April 20–25). A survey of user centered design practice. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems (pp. 471–478). ACM. https://doi.org/10.1145/503376.503460

Wang, Q., Yang, S., Liu, M., Cao, Z., & Ma, Q. (2014). An eye-tracking study of website complexity from cognitive load perspective. Decision Support Systems, 62, 1–10.

Wong, C. Y., Khong, C. W., & Thwaites, H. (2010). Mobile user interface for seniors: An impact of ageing population on mobile design. Design Principles and Practices—An International Journal, 4(4), 231–248.

Zhang, P., von Dran, G., Small, R., & Barcellos, S. (2000). A two-factor theory for website design. In Proceedings of the 33^rd Annual Hawaii International Conference on System Sciences (pp. 1–10). IEEE Xplore.

Ziefle, M. (2010). Information presentation in small devices: The trade-off between visual density and menu-foresight. Journal of Applied Ergonomics, 41(6), 719–730.

Zhou, B., Pöppel, E., Wang, L., Yang, Y., Zaytseva, Y., & Bao, Y. (2016). Seeing without knowing operational principles along the early visual pathway. Psychology Journal, 5(3), 145–160.

Appendix 1

Figure 4: Prototype with high-quality visual design.

Figure 5: Prototype with low-quality visual design.

Appendix 2

Classical Design Questionnaire (Lavie & Tractinsky, 2004)

Classical Aesthetics	Strongly Disagree	Disagree	Neither Agree not Disagree	Agree	Strongly Agree
Clear design
Aesthetic design
Pleasant design
Clean design
Grid design

Appendix 3

AttrakDiff Semantic Differential Questionnaire (Hassenzahl et al., 2003)

 

INSTRUCTION

Please indicate how you felt about the three aspects of the game user interface

1. PQ – Usability features of interface
2. AT – Attractiveness of the visual interface

When you played the game for each of the following items by applying an X in the cell, as per the example.

Example:

If you felt that game was moderately “user-centered,” then you would apply a cross to the corresponding cell as shown below:

 

PQ	Extremely	Moderately	Fairly	Neutral	Fairly	Moderately	Extremely
Technical						X		User Centered

 

 

START

 

PQ	Extremely	Moderately	Fairly	Neutral	Fairly	Moderately	Extremely
Technical								User-centered
Complicated								Simple
Impractical								Practical
Cumbersome								Straightforward
Unpredictable								Predictable
Confusing								Clearly Structured
Unruly								Manageable

 

 

AT	Extremely	Moderately	Fairly	Neutral	Fairly	Moderately	Extremely
Unpleasant								Pleasant
Ugly								Attractive
Disagreeable								Likeable
Rejecting								Inviting
Repelling								Appealing

 

Appendix 4

Mood Questionnaire (Steyer et al., 1997)

Valence	Definitely Not	Not	Not Really	A Little	Very Much	Extremely
Good
Unhappy
Discontent
Happy
Wonderful

 

Arousal	Definitely Not	Not	Not Really	A Little	Very Much	Extremely
Sleepy
Alert
Fresh
Exhausted
Wide Awake

Appendix 5

User Engagement Scale (UES Questionnaire) (O’Brien et al., 2018)

Questionnaire items and instructions for scoring.

Code	Item	Strongly Disagree	Agree	Neither Agree Nor Disagree	Disagree	Strongly Disagree
FA-S.1	I lost myself in this experience.
FA-S.2	The time I spent using the Application just slipped away.
FA-S.3	I was absorbed in this experience.
PU-S.1	I felt frustrated while using this Application.
PU-S.2	I found this Application confusing to use.
PU-S.3	Using this Application was taxing.
AE-S.1	This Application was attractive.
AE-S.2	This Application was aesthetically appealing.
AE-S.3	This Application appealed to my senses.
RW-S.1	Using this Application was worthwhile.
RW-S.2	My experience was rewarding.
RW-S.3	I felt interested in this experience.

 

B1. Scoring the UES

• Reverse code the following items: PU-S1, PU-S2, PU-S3.
• If participants have completed the UES more than once as part of the same experiment, calculate separate scores for each iteration. This will enable the researcher to compare engagement within participants and between tasks/iterations.
• Scores for each of the four subscales can be calculated by adding the values of responses for the three items contained in each subscale and dividing by three. For example, “Aesthetic Appeal” would be calculated by adding AE-S1, AE-S2, and AE-S3 and dividing by three.

An overall engagement score can be calculated by adding all of the items together and dividing by twelve.

Published in: in Volume 17, Issue 3,