Abstract
In this paper, we report the findings from the usability testing of the Finnish Skiing Game with 24 Japanese elderly participants at the Sendai City Health Promotion Center in Japan. The main objective of the study is to investigate the participants’ feedback towards the usability of the Skiing Game, which was originally designed and developed for elderly Finnish people. We also investigated the Japanese elderly participants’ game experiences during and after the gameplay. The findings show that the Skiing Game was an easy and user-friendly game for the elderly participants. Their gameplay experiences during and after the game session were fairly positive. Most of the participants were interested in the gameplay, and they agreed that digital games could be an effective way of exercising. In this study, we observed that the Finnish Skiing Game is a suitable game for the elderly Japanese people because of its simple interface, easy game action, and friendly game context. In this paper, we provide the usability guidelines and recommendations that we learned from this study.
Keywords
usability, user experiences, human-computer interaction, physical exercise game, game localization
Introduction
According to the World Health Organization (WHO, 2011), being engaged in physical activities can improve the elderly’s physical fitness, functional health, and cognitive ability. Basically, physical activities include leisure activities (e.g., gardening, dancing, and walking), activities of daily living (e.g., meal preparation and dressing), playing games, and participating in regular exercises routines. Bherer, Erickson, and Liu-Ambrose (2013) advocated that physical exercise is an effective non-pharmaceutical intervention for the elderly to prevent cognitive decline and neurodegenerative disease (e.g., Alzheimer’s disease). The effects of physical exercises on the elderly are not only prevention of age-related diseases (e.g., heart disease and diabetes) but also an improvement in the quality of life (e.g., independence; Sun, Norma, & While, 2013).
Participation and engagement in regular physical exercises can decline when a person becomes old (Factora, 2013). Milanović et al. (2013) indicated that the ageing process can reduce the physical fitness of the elderly such as strength, endurance, agility, and flexibility. As a result, the elderly may encounter difficulties in the activities of daily living and functional ability. The American Psychological Association (2016) pointed out that most elderly people experience natural and normal age-related changes that include physical and functional decline. This can impact the quality of life for the elderly.
People from different areas such as healthcare, digital media, and physiotherapy, have become interested in helping the elderly to improve their active participation in regular physical exercises by using the latest technologies. For example, in recent years, healthcare practitioners and researchers have become interested in digital games as an alternative way of physical exercises for the elderly (Alankus, Lazar, May, & Kelleher, 2010; Gerling, Livingston, Nacke, & Mandryk, 2012; Uzor & Baillie, 2014). Different terms are used to define the concept of game-based exercises for the elderly such as serious games for healthcare (Tashiro, 2009), exergames (Brox, Burkow, Evertsen, Åsheim-Olsen, & Vognild, 2014), and gamification for healthcare (Brauner, Valdez, Schroeder, & Ziefle, 2013).
To date, many researchers have studied the usability and usefulness of existing games and technologies for the elderly’s healthcare such as Nintendo Wii Fitness games (Kahlbaugha, Sperandioa, & Ashley, 2011; Theng, Chua, & Pham, 2012), Kinect for Xbox Fitness games (Ganesan & Anthony, 2012), and PlayStation PlayMove Fitness games (Pyae, Luimula, & Smed, 2016). Furthermore, virtual reality and augmented reality games are also being used to improve the elderly’s motivation and engagement in their daily exercises (Halton, 2008; Pyae, Tan, & Mark, 2013). IJsselsteijn, Nap, & de Kort (2007) mentioned that the digital game market for the elderly is significantly large because game developers primarily focus on adolescent users, and commercial games do not meet the needs, abilities, and limitations of elderly people. The authors of this paper also pointed out that the elderly encounter more usability problems in digital games, compared to younger users because of their limited functionality and lack of technological experiences (IJsselsteijn et al., 2007).
There are many studies that investigated the usability of digital games for the elderly and their user experiences (Marin, Lawrence, Navarro, & Sax, 2011). Silva and Nunes (2010) also pointed out that although there are an increasing number of studies on usability guidelines for users in various contexts, little is known for usability guidelines for the elderly in digital games. To date, there are some promising and insightful findings from the usability testing of digital games for the elderly (Gerling, Schild, & Masuch, 2010; Nawaz et al., 2015). However, this is an on-going process, and it is important to study more widely in this area. The better we investigate the usability of digital games for the elderly, the more we will gain insightful design guidelines that will also help the future research in related areas.
In this paper, we present the background of our project called Gamified Solutions in Healthcare (GSH). We explain the concept of the Skiing Game and its technology. Then, we mention the design and procedures of the usability testing of the Skiing Game, which is conducted with 24 Japanese elderly participants in Japan. After that, the results from the usability testing are reported in the Data Analysis section. The findings from this usability testing are reported in the Discussion section. We also recommend usability design guidelines that can be useful not only for our future design, development, and usability testing but also for practitioners in the same area.
The main objectives of this paper are
- to report the findings from the usability testing of the Finnish Skiing Game with Japanese elderly people, and
- to recommend usability design guidelines for our future development and other practitioners in a related area.
Background
Gamified Solutions in Healthcare (GSH) is a Tekes-funded research project, which is the collaboration between Turku University of Applied Sciences and University of Turku, Finland. We are closely working with our industry partners such as GoodLife Technology Finland and Puuha Group Finland. We also have public and private sector partners such as Attendo Group and the City of Turku that helped us recruit elderly participants in our usability testing. The main objective of GSH is to provide gamified services and solutions for the elderly to improve their quality of life in terms of active ageing, cost-effective healthcare solutions, better socialization, and their safety (Raitoharju, Luimula, Pyae, Pitkäkangas, & Smed, 2014).
The core concept in this project is called Virtual Nursing Home (VSN), and it includes four major services for the elderly: Socialization, Entertainment, Counseling, and Rehabilitation. The Socialization service focuses on promoting an elderly persons’ socializing experiences through gamification. The Entertainment service aims at promoting the elderly’s recreation through gamified activities. The Counseling service includes interactive counseling services such as information on banking, healthcare, and safety for the elderly. The Rehabilitation service utilizes gamified exercises for the elderly’s physical fitness and rehabilitation.
In this paper, we only emphasize the Rehabilitation service. The main objective of this service is to provide game-based exercises for the elderly to improve their engagement and motivation in daily or weekly physical exercises. Furthermore, we aim at providing effective exercises for the elderly who are going through rehabilitative training. Figure 1 shows an overview of the VSN concept (Pyae et al., 2015a, 2015b).
Figure 1. The overview of VSN concept.
Before we designed and developed digital games for the Rehabilitation service for the elderly, we conducted preliminary studies to understand the elderly’s requirements for physical activities and the potential of commercial and existing games for physical exercises for the elderly. The findings from the preliminary studies were reported in the previous research publications: a literature review on digital games for the elderly (Raitoharju et al., 2014), understanding motivational factors for the elderly (Pyae et al., 2015a), a review of existing game technologies and commercial games (Raitoharju et al., 2014), usability testing of commercial games, and SportWall game developed by Puuha Group Finland (Pyae et al., 2015a, ; Pyae, Luimula, & Smed, 2016; Pyae, Raitoharju, Luimula, Pitkäkangas, & Smed, 2016). The findings from the preliminary studies are useful and insightful for our game design, development, and future usability testing of games for GSH.
The game industry today is no longer targeting a particular user group in a specific country or region. Game localization and culturalization have become a trend in today’s digital game market. According to Skoog (2012), localization (as it pertains to gaming) is defined as the adaptation of digital games for different markets of specific regions (e.g., Asia, Europe, and America). Localization includes changing game components, not only graphics, but also cultural references. Localization is a means to provide a game for players in different regions to enjoy without losing the originality of the games. It can also add local cultural values while being in compliance with the legal and regulatory requirements of a specific country.
David, Curran, and Simon (2005) advocated that to expand the sales of a particular game across different countries, the effort for localization needs to be more sensitive to local customs, tastes, rules, and regulations. Salen and Zimmerman (2006) also pointed out that games can reflect the specific values of a particular society or cultural group, and a game can miss important cultural values unless the game designers or artists engage the games with the cultural representation. Chandler and Deming (2011) defined that “culturalization” is more than a simple localization; it takes a closer and deeper look into a game’s contents and assumptions, and looks at the viability in both multicultural marketplace and different geographical locations.
One of the main objectives of the GSH project is to evaluate our games with not only the Finnish elderly but also the elderly in different countries. Although our game is designed for the Finnish elderly, we would like to investigate whether the game is also suitable and user-friendly for the elderly living in different countries and different cultural groups. One of the goals of our project is to partner with other foreign organizations to understand how we can localize our games in different countries. For instance, we are closely working with Japanese game companies and social organizations that specialize in the elderly’s well-being. The objectives of the usability testing of the Skiing Game with the Japanese elderly are
- to investigate the Japanese elderly’s feedback towards the usability of the Skiing Game,
- to investigate the Japanese elderly’s game experiences during and after the gameplay, and
- to understand their general feedback towards the game.
Skiing Game
In this study, we developed a physical exercise game called Skiing Game, which was designed for the Finnish elderly to improve their motivation and engagement in regular physical exercises. The main objectives of the Skiing Game are to provide a user-friendly, simple, easy, and effective exercise game for the elderly and to encourage them to engage in regular physical activities.
Based on the findings from the requirements gathering (Pyae et al., 2015a, 2015b; Raitoharju et al., 2014), we found out that game context and background should be familiar to the elderly because the elderly prefer game context and background that are close to real-life environments and activities. For instance, they like to play card games, sport-based games, and recreational games (e.g., dancing and singing games). Moreover, we realized that they like user-friendly and natural game actions, which can be familiar to them (Pyae et al., 2015b). For example, they like tennis, bowling, and dancing game actions. According to the observations from the previous usability testing (Pyae, Luimula, & Smed, 2016), we found out that simple, easy, and effective interaction techniques are important for the elderly when playing a game. They prefer controller-free natural movements when playing games (e.g., a motion detection device). The elderly participants also recommended simple and easy in-game instructions to follow. Furthermore, they preferred simple and clear feedback that makes the game easy to understand.
Based on the design guidelines mentioned above, we designed a skiing-based exercise game where we chose snowy mountains and a forest as the game context and background. The main reason we chose this type of sport is that skiing as an activity is familiar to most Finnish elderly people. The gameplay is simply based on steering a pair of skiing poles (double pole skiing technique). This game is relatively easy and is a familiar exercise technique to the Finnish elderly who have previous experiences with cross-country skiing. We used a traditional webcam based controller-free interaction. We also designed simple game labels, instructions, and the ability to provide feedback in the game.
The Skiing Game was designed and developed using a Unity 3D game engine. The player has to complete a simple calibration before the gameplay. When a player starts playing the game, he or she needs to ski through gates and to reach the finishing line in a given time. There are many obstacles to avoid along the way, which can cause the player to fall down in the game. To control the game the player needs to hold both hands (double pole skiing motion) as the way he or she is steering a pair of skiing poles, left and right. To move forward, the player needs to continuously move both hands forward and backward. To accelerate the movement, the player has to perform this action faster. The more the player moves both hands, the faster the movement becomes. It is important for the player to follow the skiing trail displayed in the game. To move left or right, the player simply needs to move his or her body to the left or right. In the uphill and downhill sections of the game, the player needs to accelerate more or to slow down. When the player reaches the finishing line, the score screen will pop up, showing the player’s time and the number of gates the player successfully went through. See Figure 2 for screenshots from the Skiing Game.
Figure 2. Screenshots from the Skiing Game.
For the game interaction, we used a webcam with Extreme Reality (XTR3D) Technology to track the player’s upper-limb movements. Extreme Reality includes the technology that supports software-based motion analysis. It can control any computing device by using a traditional web-camera (Extreme Reality Technology, 2015). XTR3D includes a motion capture engine that detects the 3D position (X, Y, and Z) of a player’s skeletal position just in front of the camera in every frame. After that, it creates a live 3D model of the player that is analyzed by the software to recognize the gestures from the skeletal positioning of the player. The overview of XTR3D is shown in Figure 3.
Figure 3. The overview of XTR3D. Reprinted with permission from Extreme Reality Technology, 2015.
Usability Study
To test the Skiing Game with the elderly participants in different countries, we collaborated with our Japanese counterparts: the Sendai City Health Promotion Center and the Sendai-Finland Wellbeing Center in Japan. The Sendai City Health Promotion Center, operated by the Sendai City Health and Welfare Organization, is a rehabilitation center that provides rehabilitative training for the Japanese elderly. The Sendai-Finland Wellbeing center is a technology hub for the Finnish health informatics, and it also has a nursing home that provides elderly care including physical, social, and healthcare activities. Both organizations are interested in utilizing digital game-based intervention for the elderly to improve their physical exercises.
Elderly Participant Recruitment
The recruitment of the Japanese elderly participants in this study was mainly done by the physiotherapists from the Sendai City Health Promotion Center. They advertised the recruitment at their nursing home. The selection of the participants was based on certain inclusion criteria. For instance, the age requirement for the participants was between 60 to 85 years. The health condition of the participants needed to be stable. Moreover, they needed to be physically and mentally capable, with no neurological or cognitive deficits, of playing the game for at least 15 minutes. The wheelchair-bound elderly were excluded in our study because the game required players to stand during the gameplay. Participants needed to have the capability of independent standing and walking 10 meters. The physiotherapist from the center selected the elderly participants carefully to meet the needs of the study. We also requested the consent from each and every participant in this study.
Usability Test Design and Procedure
After recruiting 24 Japanese elderly participants, we conducted the two-day usability testing. On the first day, 16 participants played the game and answered the questionnaires. The remaining eight participants joined the usability testing on the second day. There were a total of five researchers who conducted the usability testing: three Japanese physiotherapists from the Sendai City Health Promotion center, one project officer from the Sendai-Finland Wellbeing Center, and one Finnish researcher from Turku University of Applied Sciences, Finland. The usability testing took place at a therapy training room at the Sendai City Health Promotion center. The whole study was conducted by using the Japanese language. Our Finnish researcher guided and monitored the whole study. There were three stations in three different rooms. At station one (room 1), one Japanese physiotherapist from the Sendai City Health promotion center conducted the pre-study interview that included the participants’ demographic information, health background, their physical exercise activities, and their consent. It took about 5 to 10 minutes to ask the questions.
Station two (room 2) was operated by the two Japanese physiotherapists from the Sendai City Health Promotion center. At this station, the participants played the Skiing Game. One therapist guided participants on how to play the game, and the other monitored the participants to minimize their health risks. With regard to the gameplay by the participants at station two, one physiotherapist guided them on how to play the game before they played it. While they were playing the game, the therapist gave support if they needed it. The total time for station two took about 20 minutes. In station three (room 3), the project officer from the Sendai-Finland Wellbeing Center asked (in Japanese) the participants to complete the post-gameplay questionnaires, which were also in Japanese. One student from the Sendai National College of Technology helped in videotaping the whole study. The detailed procedures of the whole study are mentioned in Table 1.
Table 1. Skiing Game Usability Testing Design and Procedures
Activity |
Duration |
Station |
Introduction to the usability testing |
10 mins |
1 |
Pre-gameplay interview and consent taking |
|
|
Game Tutorial guided by the physiotherapist |
20 mins |
2 |
Skiing Game played by participants |
||
Break-time |
5 mins |
|
Post-gameplay questionnaire |
10 mins |
3 |
Total |
45 mins |
System Usability Scale (SUS)
To investigate the participants’ feedback towards the usability of the Skiing Game, we used the System Usability Scale (SUS), which is easy and simple to understand for the elderly. SUS includes a 10-item questionnaire that gives the subjective assessments of the usability of a particular system (Brooke, 1996). SUS is based on the 5-point Likert scale from Strongly disagree (1) to Strongly agree (5). SUS can be used to evaluate a variety of products and services, which includes software and hardware systems, mobile devices, applications, and websites (“System Usability Scale (SUS)”, n.d.). We can use this scale for a small sample size, and it is easy to apply to participants. Sauro (2011) mentioned that while SUS is intended to measure perceived ease-of-use of a system, it can also provide not only usability but also learnability dimensions. According to Brooke (2013), SUS is technology-neutral, and it has been continuously used as technology develops over the years without reinvention of questionnaires. Bangor, Kortum, and Miller (2009) mentioned that SUS is an effective tool for measuring the usability of a system or product, and it provides a score from 0 to 100, which is easy to understand for participants. Bangor, Kortum, and Miller (2008) advocated that SUS is a highly robust and flexible tool for usability practitioners. In this study, we modified the SUS questionnaires to suit our study. For instance, the original version of SUS refers to a particular system, and we modified it to a game system and its usability. Table 2 shows the revised version of the SUS questionnaires that are used in our usability testing of the Skiing Game.
Table 2. System Usability Scale (SUS) Modified for This Study
Questionnaires |
1. I think that I would like to play this game frequently. |
2. I found this game unnecessarily complex. |
3. I thought this game was easy to play. |
4. I think that I would need the support of a technical person to be able to play this game. |
5. I found the various functions in this game were well integrated. |
6. I thought there was too much inconsistency in this game. |
7. I would imagine that most people would learn to play this game very quickly. |
8. I found this game very cumbersome to play. |
9. I felt very confident playing this game. |
10. I needed to learn a lot of things before I could get going with this game. |
Game Experience Questionnaires (GEQ)
In this usability testing, we used GEQ to understand the participants’ experiences during and after the gameplay. We used two GEQ modules to assess the player’s experience: in-game and post-game (IJsselsteijn et al., 2015a). The GEQ in-game module is used to assess a player’s game experience during a game-play session. It includes seven components: Sensory and Imaginative Immersion, Flow, Competence, Positive Affect, Negative Affect, Tension, and Challenge (de Kort, IJsselsteijn, & Poels, 2007; Poels, de Kort, & IJsselsteijn, 2007). Norman (2013) advocated that GEQ is a reasonable and applicable questionnaire in exploring a player’s experiences with a game. Norman also advocated that the GEQ questionnaire has been widely and successfully used in many studies internationally (2013).
Johnson, Wyeth, Sweetser, and Gardner (2012) mentioned that the components—Competence, Tension, Negative Affect, and Positive Affect—are easily understood by their titles. Competence refers to a player’s skills and success in playing the game (e.g., I felt skillful). The component Tension is about a player’s frustration in playing the game (e.g., I feel frustrated). Positive Affect is when a play feels positive (e.g., I felt good), whereas Negative Affect refers to the negative feeling a player may experience while playing the game (e.g., I felt bored). The component Challenge means the effort, challenge, and pressure felt by a player in the game (e.g., I had to put a lot of effort into it), and Sensory and Imaginative Immersion means how interested a player is in the game’s story (e.g., I was interested in the game’s story; Johnson et al., 2012). The component Flow is about how engrossed a player feels while playing the game (e.g., I felt completely absorbed). All GEQ in-game components and response options from the questionnaires are mentioned in Table 3.
Table 3. In-Game GEQ Components and Response Options from the Questionnaires
In-game GEQ components |
Response option |
Sensory and Imaginative Immersion |
I was interested in the game’s story. I found it impressive. |
Flow |
I forgot everything around me. I felt completely absorbed. |
Competence |
I felt successful. I felt skillful. |
Positive affect |
I felt content. I felt good. |
Negative affect |
I felt bored. I found it tiresome. |
Tension |
I felt frustrated. I feel irritable. |
Challenge |
I felt challenged. I had to put a lot of effort into it. |
Regarding the GEQ post-game questionnaire, it is used to assess how players felt after they have played the game (IJsselsteijn et al., 2015a, 2015b). It has four components: Positive Experience, Negative Experience, Tiredness, and Returning to Reality. Basically, the Positive Experience component is about a player’s positive experiences after playing the game such as satisfaction, victory, and power (e.g., I felt satisfied). In contrast, Negative Experience is about a player’s bad experiences after he or she has played the game (e.g., I felt bad). The component Tiredness refers to a player’s exhaustion in the gameplay (e.g., I felt exhausted). Returning to Reality is about a player’s disorientation after he or she has played the game (e.g., I found it hard to get back to reality). In this study, for both GEQ in-game and post-game questionnaires, we use a 5-point Likert scale from Not at all (0) to Extremely (4). All GEQ post-game components and response options from the questionnaires can be seen in Table 4.
Table 4. Post-Game GEQ Components and Response Options from the Questionnaires
Post-game GEQ components |
Response option |
Positive Experience |
I felt revived. It felt like a victory. I felt energized. I felt satisfied. I felt powerful. I felt proud. |
Negative Experience |
I felt bad. I felt guilty. I found it a waste of time. I felt that I could have done more useful things. I felt regret. I felt ashamed. |
Tiredness |
I felt exhausted. I felt weary. |
Returning to Reality |
I found it hard to get back to reality. I felt disoriented. I had a sense that I had returned from a journey. |
Post-Gameplay Interview Questions
The post-gameplay interview questions were based on the Senior Technology Acceptance & Adoption Model (STAM; Renaud & van Biljon, 2008).The questions include the following four items that investigate the elderly participants’ perceptions after playing the game:
- Perceived Usefulness was used to understand the effectiveness and usefulness of playing digital games.
- Perceived Ease-of-Use was used to find out if playing the Skiing Game was easy for them.
- Gerontechnology Self-Efficacy was used to find out if the player could easily play after receiving the game instructions and to determine if the game instructions were adequate.
- Gerontechnology Anxiety was used to investigate if the player was afraid of making mistakes during the gameplay.
Table 5 shows the post-gameplay interview questions. Figure 4 shows a photo taken during the gameplay.
Table 5. Post-Gameplay Questionnaires
Post-gameplay questions |
|
Perceived Usefulness |
Could playing digital games be an effective and effortless way of exercising? |
Perceived Ease-of-Use |
Was playing the game easy? |
Gerontechnology Self-Efficacy |
Were you able to play the game after receiving instructions? Would the user instructions have been adequate? |
Gerontechnology Anxiety |
Were you afraid of making mistakes when playing the game? |
Figure 4. Skiing Game usability testing.
Data Analysis
From the pre-gameplay interview sessions, we observed that all the elderly participants in this study performed regular exercises daily and weekly. They were active in physical exercises, which included walking, stretching, playing golf, and gardening. They exercised either at home or at the sports club. In this study, half of the participants (12 out of 24) had not played digital games before, and they did not have knowledge about digital games. The remaining 12 participants had prior experiences with digital games. However, six of them had negative game experiences. They claimed that games are difficult to play and are not interesting. The other six participants who had prior gameplay experiences enjoyed playing digital games, and they played often. They normally used a PC, tablet, or notebook to play digital games. The digital games they played were card games and memory games.
We present the findings from the analysis of in-game and post-game GEQ questionnaires data in Table 6 and Table 7. The average score of each component in both the in-game and post-game questionnaires was calculated according to the original GEQ assessment method (IJsselsteijn et al., 2015a). As shown in Table 6, the highest score of the in-game module can be seen for the Flow component (M = 3.0), followed by the second highest for the Positive Affect component (M = 2.9). In contrast, the Negative Affect component produced the lowest score (M = 0.4). With respect to the Tension component, it was the second lowest among other components (M = 0.8). The mean scores of the Competence component (M = 2.6), Sensory and Imaginative Immersion (M = 2.7), and Challenge components (M = 2.4) were greater than the average score (M = 2.0). Figure 5 shows the graphical presentation of the GEQ in-game mean scores and the standard errors.
Table 6. In-Game GEQ Mean Scores and the Standard Errors
In-game GEQ Questionnaire |
Mean score (M) |
Standard error (SE) |
Flow |
3.0 |
0.21 |
Positive Affect |
2.9 |
0.14 |
Sensory and Imaginative Immersion |
2.7 |
0.17 |
Competence |
2.6 |
0.16 |
Challenge |
2.4 |
0.21 |
Tension |
0.8 |
0.16 |
Negative Affect |
0.4 |
0.10 |
Figure 5.The in-game GEQ mean scores and the standard errors.
Regarding the findings from the post-game GEQ questionnaires, we can see that the Positive Experiences of the participants after the gameplay was about the average score (M = 2.1), whereas the Negative Experiences and Tiredness had the least average scores (M = 0.5 and M = 0.4, respectively). The Returning to Reality was also considerably low (M = 0.9).The results from the post-game GEQ questionnaires in this study are shown in Table 7. Figure 6 illustrates the post-game GES mean scores and the standard errors.
Table 7. Post-Game GEQ Mean Scores and the Standard Errors
Post-game GEQ components |
Mean score |
Standard error |
Positive Experience |
2.1 |
0.21 |
Returning to Reality |
0.9 |
0.12 |
Negative Experience |
0.5 |
0.10 |
Tiredness |
0.4 |
0.10 |
Figure 6. The post-game GEQ mean scores and the standard errors.
We analyzed the correlations between the Positive and Negative Affect in the in-game GEQ, and the Positive and Negative Experiences in the post-game GEQ. There are no correlations between the in-game and post-game GEQ variables except for the Positive Affect in the in-game GEQ and Positive Experience in the post-game GEQ. They have a positive correlation, as expected, and the correlation coefficient value is 0.72. Table 8 shows the correlations between the in-game Positive and Negative Affect, and the post-game Positive and Negative Experiences. Figure 7 shows a scatterplot diagram that illustrates the correlation between the in-game Positive Affect and the post-game Positive Experience.
Table 8. Correlations Between In-Game and Post-Game GEQ Variables
|
Negative affect |
Positive affect |
Positive experience |
Negative experience |
Negative Affect |
1 |
|
|
|
Positive Affect |
-0.31 |
1 |
|
|
Positive Experience |
-0.24 |
0.72 |
1 |
|
Negative Experience |
0.03 |
0.11 |
0.3 |
1 |
Figure 7. A scatterplot diagram of the correlation between the in-game Positive Affect and the post-game Positive Experience.
To report the participants’ feedback towards the usability of the Skiing Game, we analyzed the SUS questionnaires data based on the calculation method mentioned in the original test (Brooke, 1996). We gave a score, which ranges from 0 to 100, to the individual participants. Basically, a SUS score lower than 68 was ranked as Below Average, and a score higher than 68 was considered Above Average (Sauro, 2011, “System Usability Scale (SUS),” n.d.). However, the ranking does not represent the meaning of the actual score. Therefore, we adopted the Adjective Rating Scale for SUS that has a range from Worst Imaginable to Best Imaginable (Bangor et al., 2009). This type of scale has seven adjective ratings (Brooke, 2013). Table 9 presents the individual elderly participant’s SUS score and adjective ranking of their feedback towards the usability of the Skiing Game.
Table 9. Individual’s SUS Score
Participant ID |
SUS score |
Adjective ratinga |
4 |
95 |
Excellent |
6 |
90 |
Excellent |
10 |
85 |
Excellent |
11 |
90 |
Excellent |
12 |
87.5 |
Excellent |
15 |
95 |
Excellent |
16 |
92.5 |
Excellent |
21 |
95 |
Excellent |
1 |
80 |
Good |
2 |
80 |
Good |
5 |
77.5 |
Good |
8 |
75 |
Good |
9 |
77.5 |
Good |
19 |
77.5 |
Good |
20 |
77.5 |
Good |
17 |
80 |
Good |
3 |
62.5 |
OK |
7 |
55 |
OK |
13 |
65 |
OK |
14 |
65 |
OK |
18 |
70 |
OK |
23 |
70 |
OK |
24 |
67.5 |
OK |
22 |
47.5 |
Poor |
a Based on the SUS scores, none of the adjective ratings fell into the Worst Imaginable, Awful, or Best Imaginable adjective ratings.
As shown in Table 9, eight participants rated the game as Excellent (SUS score between 85 and 99). The percentage of the number of participants who rated the game Excellent is 33.33%. Eight participants rated the game as Good (SUS score between 73 and 84), which is 33.33% of the participants. The total number of participants who evaluated the game OK was seven (SUS score between 52 and 72), which is 29.17% of the participants. There was only one person who rated the game Poor (SUS score between 38 and 51), which is 4.17% of the participants. Figure 8 shows an overview of the SUS scores in adjective rating.
Figure 8. The overview of the SUS score in adjective ratinxg.
According to the participants’ responses to the post-gameplay questionnaires, there were 21 participants who agreed that playing digital games could be an effective and effortless way of exercising (Perceived Usefulness), whereas three out of 24 participants disagreed. With regard to the question of Perceived Ease-of-Use, all participants agreed that the game was easy to play. Furthermore, all participants agreed that they were able to play the game after receiving instructions. They also mentioned that the game instructions given in the game were adequate. There were six participants who mentioned that they were afraid of making mistakes during the gameplay, whereas 18 participants were not afraid of making mistakes. Table 10 shows the post-gameplay questionnaires and the participants’ responses.
Table 10. Post-Gameplay Questionnaire
Post-gameplay questions |
No. of participants |
||
|
|
Agree |
Disagree |
Perceived Usefulness |
Could playing digital games be an effective and effortless way of exercising? |
21 |
3 |
Perceived Ease-of-Use |
Was playing the game easy? |
24 |
0 |
Gerontechnology Self-efficacy |
Were you able to play the game after receiving instructions? Would the user instructions have been adequate? |
24 |
0 |
Gerontechnology Anxiety |
Were you afraid of making mistakes when playing the game? |
6 |
18 |
Discussion
From the pre-gameplay interview, we observed that the Japanese elderly participants in this study were active in physical activities. They did physical exercises daily and weekly. In the pre-gameplay interview session, we found out that 12 participants did not have prior experience playing digital games, whereas the other 12 participants did have experience playing digital games.
According to the findings from the in-game GEQ questionnaires, we discovered that the participants had the highest positive experience with the Flow component, meaning that they were quite absorbed in the game. With regard to the Positive Affect, we observed that the participants were fairly positive about their experiences while they were playing the game. The Sensory and Imaginative Immersion component had the third largest score which means that the participants were fairly interested in the game during the gameplay. With regard to the Competence component, it can be said that the participants felt that they gained some achievements and skills in their gameplay to a certain extent. Regarding the component Challenge, participants reported that the game was moderately challenging, and they put effort into the gameplay. Conversely, the participants had noticeably lower Tension and Negative Affect scores while they were playing, which means that the participants did not feel frustrated and bored in their gameplay. Based on these findings, we can state that the participants were fairly engaged and interested in the game. Moreover, they felt challenged and put effort into the gameplay. More importantly, they felt content and good about the game. Therefore, we can say that the Japanese elderly participants in this study had fairly positive experiences while they were playing the Skiing Game.
Regarding the post-game GEQ questionnaires, we discovered that the participants had moderately positive experiences after they played the game. This outcome means that the participants were moderately satisfied and energized after the gameplay. In contrast, their negative experiences with the gameplay were noticeably low, which refers to the fact that the participants reported that they did not feel bad or discouraged after they played the game. Regarding the component Tiredness, we observed that the average score was very low, and we can say that the participants were not tired at all during the gameplay. However, this result can be linked to the fact that the gameplay session was relatively short, taking about 15 to 20 minutes to complete. They played a single game in the whole session, which might have led them to think that they did not get tired after playing the game. For the component Returning to Reality, which had a noticeably low average score, participants reported that they did not feel disoriented after they played the game. Based on these findings, we can generally say that the elderly participants were moderately positive about their experiences after playing the Skiing Game.
With respect to the correlation between the in-game and post-game GEQ components, there was a significant correlation between in-game Positive Affect and post-game Positive Experience. This correlation means that when the player had a more positive affection during the gameplay, he or she had a more positive experience after the gameplay. Based on the findings from both in-game and post-game GEQ questionnaire, we can say that the Japanese elderly participants’ experiences in playing Skiing Game were fairly positive.
Concerning the usability of the game, although 12 out of 24 participants did not have prior experience in digital gameplay, their feedback towards the usability of the Skiing Game was noticeably positive: 16 out of 24 participants gave positive feedback (Excellent or Good). Some participants, 7 out of 24, commented that the game was OK. Notably, only one participant reported that the usability of the game was Poor. The participants who regularly played digital games gave the Skiing Game positive feedback (Excellent or Good). The participants who had previously reported negative impressions of digital games also gave relatively positive feedback for the game (Excellent, Good, and OK), except one participant who rated the game Poor. Based on the overall feedback from the participants, we can say that the usability of the Skiing Game for the Japanese elderly participants was fairly good and user-friendly.
With respect to the post-gameplay questionnaire, we observed that 21 out of 24 elderly participants mentioned that playing digital games could be an effective way of exercising for them, whereas the remaining participants did not recommend it. All participants commented that the game was easy to play, and they could easily follow the instructions. Of the 24 participants, 18 reported not feeling anxiety if they made a mistake during their gameplay, whereas the other six participants mentioned that they were nervous because they thought they’d make a mistake in their gameplay. Based on the findings from the post-gameplay questionnaire, most of the participants agreed that the Skiing Game was easy and an effective way to exercise.
According to the general feedback given by the participants, they mentioned that the game was interesting, easy to play, and they would like to play it again. They were also interested in playing digital game-based exercises regularly at the center. The only concern that the participants mentioned was that there was only one game and no option to choose to play. Moreover, the session was quite short. They recommended having more than one game to play and having it available for a longer time.
Based on the analysis of the participants’ feedback towards the GEQ, SUS, and post-gameplay questionnaires, we highlight the following findings:
- The usability of the Skiing Game was fairly usable and user-friendly for the Japanese elderly participants.
- The participants’ overall experience of playing the Skiing Game was fairly positive although most of them never played digital games before.
- Most of the participants agreed that the Skiing Game was easy and effective for them.
Based on the facts mentioned above, we can say that the Skiing Game is well-accepted by the Japanese elderly. The findings also highlight the potential of future game localization outside of Finland. However, further investigation on game localization and culturalization in a specific cultural group is still needed to improve the game.
The limitation of this study was that there was only the Skiing Game in this study. Furthermore, the duration of the usability testing was short, and the number of participants in this study was relatively small.
Usability Guidelines
We provide two kinds of guidelines for usability. First, we have recommendations for usability guidelines that we learned from the usability testing of the Skiing Game and the previous usability testing (Pyae et al., 2015a, 2015b; Pyae, Luimula, & Smed, 2016; Raitoharju et al., 2014). Second, we discuss usability issues that the participants encountered in this study.
Recommendations
The game context and content should be familiar and related to the elderly so that they can easily engage in the game. According to the findings from the previous usability testing, we found out that the elderly prefer game context and environment that are related to their personal lifestyles such as recreational activities and physical activities (Pyae et al., 2015a, 2015b; Raitoharju et al., 2014). Most elderly participants in this usability testing also advocated that they could relate to the context and contents of the Skiing Game, and thus, they could engage in the game.
We recommend that game actions in a particular game system should be natural and elder-friendly. The findings from the previous usability testing suggested that the elderly prefer natural game actions that they can easily perform such as bowling, tennis, and golf (Pyae et al., 2015a, 2015b; Pyae, Raitoharju, et al., 2016; Raitoharju et al., 2014). In this usability testing, the elderly participants advocated that the skiing-based game action was simple and easy. As a result, they could play and engage in the gameplay easily.
According to our previous usability testing, we observed that a simple and clear game interface reduces distractions for the elderly player (Pyae et al., 2015b). Furthermore, less but effective game contents can make the elderly more engaged in the game (Pyae et al., 2015b). Most elderly participants in this usability testing pointed out that the Skiing Game had an easy and simple game interface as well as uncluttered game contents. Therefore, they did not feel any distraction while they were playing the game. Based on this point, we recommend that a simple and clear interface, as well as uncluttered game contents, should be taken into account when designing games for the elderly.
We observed in the previous usability testing that excessive audio feedback could make the elderly distracted from the gameplay (Pyae, Luimula, & Smed, 2016). In the Skiing Game, we used a simple and effective audio feedback so that the participants could not be distracted from the gameplay. Thus, there was no issue of audio feedback in the game. Based on this point, we recommend that game song and audio feedback should be simple but effective for the elderly.
Controller-free and motion-based interaction is easy for the elderly (Pyae, Raitoharju, et al., 2016). The findings from the previous usability testing also suggested that natural and motion-based interaction was an effective way for the elderly when they played digital games (Nakai et al., 2015, Pyae, Raitoharju, et al., 2016). In this usability testing, the Japanese elderly participants also pointed out that motion-based interaction to play the Skiing Game was effective and effortless. Based on these statements, we recommend that motion-based interaction is an effective way for the elderly to play a game.
Most of the elderly participants in this usability testing did not have prior experiences in playing digital games. Thus, we used simple and easy game actions in the Skiing Game such as the skiing action. Consequently, the elderly participants did not encounter major challenges in the gameplay, and they advocated that the game was easy to play. Therefore, we recommend that in designing games for novice elderly players, it is important to take into consideration that the game actions should be simple and effective.
Usability Issues
Visual cues are important for elderly players. In the Skiing Game, there were game obstacles for players to avoid while they were playing the game such as trees, stones, and fallen trunks on the skiing trail. We observed that some elderly participants did not notice these obstacles. As a result, it made them fall in the game, and some participants felt frustrated after he or she had encountered such a problem a few times. In this case, it would be a better game design if we displayed some visual cues for players to notice, as they were novice players.
The other usability issue is the repetition of a single action throughout the game. For instance, the participants played the game by moving their hands forward and backward to perform skiing actions. Some participants criticized that after they had played this single action for a certain time, they felt bored, and they would like to perform multiple actions so that the gameplay would be more interesting.
In Skiing Game, when a player hits an obstacle, the player falls down in the game. Some participants expressed their frustration when they experienced this for a number of times. Therefore, when we design a game for the elderly, it is important to design the game to reduce the frustration for elderly players when they consistently fail a particular game task.
According to the findings from the participants’ general feedback, we found out that they preferred having a competitor or co-player option in the game to increase their engagement with the game. In the Skiing Game, there was not an option to play a competitor or co-player. In the future design of this game, including an option for a competitor or co-player would increase the appeal of the game for elderly players.
Lastly, we observed an important finding from this study that most of the elderly participants expressed their concern that they were afraid of falling while they were playing the game. Thus, it is worthwhile to question how we can design a game for the elderly to play alone while there is no one around, and show them how to prevent a fall.
Conclusion
In this study, we evaluated the Finnish Skiing Game with Japanese elderly participants to investigate the usability of the game and their user experiences in the gameplay. We conducted a usability test at the Sendai City Health Promotion Center in Japan with 24 Japanese elderly participants. We used the System Usability Scale (SUS) to investigate the participants’ feedback towards the usability of the Skiing Game. We also used in-game and post-game GEQ questionnaires to understand the participants’ user experiences during and after the gameplay. Based on the findings from the usability testing, we observed that the participants had fairly positive game experiences during and after the gameplay. The participants’ feedback towards the usability of the Skiing Game is noticeably positive. Therefore, we can state that the Finnish Skiing Game is an easy and user-friendly physical exercise game for the Japanese elderly. Furthermore, they were interested in playing the game, and they would like to play it again. Most participants agreed that playing digital games are an easy and effective way of exercising.
Based on the findings mentioned above, we conclude that the Skiing Game designed for the Finnish elderly is usable and user-friendly for the Japanese elderly because of its simple game context, gameplay, and ease-of-use. From this study, we also recommend the game design improvements and usability guidelines discussed in this paper for our future game development as well as for other designers and practitioners in the related areas. As a future work, we will compare and contrast the findings between the Finnish and Japanese usability testing of the Skiing Game. Furthermore, we will conduct a similar study in other countries (e.g., Singapore).
Tips for Usability Practitioners
In this usability study, we observed the following game design recommendations and usability guidelines for elderly players that can be insightful and useful not only for our future game design and development but also for usability practitioners working in the areas of game usability and HCI:
- Take into consideration that the game interface, context, and contents should be simple and uncluttered so that elderly players are not distracted from the gameplay.
- Do not use excessive in-game instructions and unnecessary audio feedback which can cause too much distraction in playing a game designed for elderly players.
- Provide visual cues, which are important in the game especially for novice elderly players.
- Use controller-free and gesture-based interaction, which is effective for the elderly.
- Use natural and familiar game actions to make the game more engaging for elderly players.
- Provide options that reduce frustration in playing the game, even if they did not achieve a particular task. For example, provide positive, constructive, and encouraging feedback.
- Prevent and reduce the risk of falling, which is always important in designing games for the elderly’s physical activity.
Acknowledgements
We would like to thank the funding agency, Finnish Funding Agency for Technology and Innovation (Tekes), City of Turku, our industrial partners, and the experts from the Turku Game Lab (TUAS) Finland. We also thank all Japanese partners including the Sendai City Health Promotion Center and the Sendai-Finland Wellbeing Center. More importantly, we thank the Japanese participants who took part in the usability test in Japan.
References
Alankus, G., Lazar, A., May, M., & Kelleher, C. (2010). Towards customizable games for stroke rehabilitation. CHI ’10 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2113–2122). New York, NY: ACM.
American Psychological Association (2016). Older adults’ health and age-related changes. Retrieved from http://www.apa.org/pi/aging/resources/guides/older.aspx
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594.
Bangor, A., Kortum, P. T., & Miller, J. T. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123.
Bherer, L., Erickson, K. I., & Liu-Ambrose, T. (2013). A review of the effects of physical activity and exercise on cognitive and brain functions in older adults. Journal of Aging Research, 2013. Article ID: 657508. doi:10.1155/2013/657508.
Brauner, P., Valdez, A. C., Schroeder, U., Ziefle, M. (2013). Increase physical fitness and create health awareness through exergames and gamification. In A. Holzinger, M. Ziefle, M. Hitz, & M. Debevc (Eds.), Human Factors in Computing and Informatics (Vol. 7946 of Lecture Notes in Computer Science, pp.349–362). Berlin Heidelberg: Springer.
Brooke, J. (1996). SUS: A quick and dirty usability scale. In: P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability Evaluation in Industry (pp. 189–194), London, UK: Taylor & Francis.
Brooke, J. (2013). SUS: A Retrospective. Journal of Usability Studies, 8(2), pp. 29–40.
Brox, E., Burkow, T., Evertsen, G., Åsheim-Olsen, H., & Vognild, L. (2014). Experiences from long-term exergaming with elderly. Proceedings of the 18th International Academic MindTrek Conference: Media Business, Management, Content & Services (pp. 216-220). New York, NY: ACM.
Chandler, H. M., & Deming, S. O. M. (2011). The game localization handbook. Burlington, MA, USA: Jones & Bartlett Learning.
David, M., Curran, S., & Simon, B. (2005). The complete guide to game development, art, and design. Cambridge, UK: ILEX.
de Kort, Y. A. W., IJsselsteijn, W. A., & Poels, K. (2007, October). Digital games as social presence technology: Development of the social presence in gaming questionnaire (SPGQ). Proceedings of PRESENCE 2007, The 10th International Workshop on Presence, Barcelona, Spain (pp. 195–203).
Extreme Reality Technology. (2015). Retrieved from http://www.xtr3d.com/about-us/overview/
Factora, R. (2013, May). Aging and preventive health. Cleveland Clinic, Center for Continuing Education. Retrieved from http://www.clevelandclinicmeded.com/medicalpubs/diseasemanagement/preventive-medicine/aging-preventive-health/ .
Ganesan, S., & Anthony, L. (2012). Using the Kinect to encourage older adults to exercise: A prototype. Proceedings of CHI’12 Conference on Human Factors in Computing Systems (2297–2302). New York, NY: ACM.
Gerling, K., Livingston, I., Nacke, L., & Mandryk, R. (2012). Full-body motion-based game interaction for older adults. CHI ’12 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1873–1882). New York, NY: ACM.
Gerling, K. M., Schild, J., & Masuch, M. (2010). Exergame design for elderly users: The case study of SilverBalance. ACE ’10 Proceedings of the 7th International Conference on Advances in Computer Entertainment Technology (pp. 66–69). New York, NY: ACM.
Halton, J. (2008). Virtual rehabilitation with video games: A new frontier for occupational therapy. Occupational Therapy Now, 9(6), pp. 12–14.
IJsselsteijn, W.A., de Kort, Y.A.W., & Poels, K. (2015a). Development of the Game Experience Questionnaire (GEQ). Manuscript in preparation.
IJsselsteijn, W.A., de Kort, Y.A.W., & Poels, K. (2015b). The game experience questionnaire: Development of a self-report measure to assess the psychological impact of digital games. Manuscript in preparation.
IJsselsteijn, W., Nap, H. H., & de Kort, Y. (2007). Digital game design for elderly users. Proceedings of the 2007 conference on Future Play (pp. 17–22). New York, NY: ACM.
Johnson, D., Wyeth, P., Sweetser, P., & Gardner, J. (2012). Personality, genre and videogame play experience. Proceedings of the 4th International Conference on Fun and Games, Toulouse (pp. 117–120). New York, NY: ACM.
Kahlbaugha, P. E., Sperandioa, A. J., & Ashley, L. (2011). Effects of playing Wii on well-being in the elderly: physical activity, loneliness, and mood. Activities, Adaptation & Ageing, 35(4), pp.331–344.
Marin, J. G., Lawrence, E. M., Navarro, K. M. F., & Sax, C. (2011). Heuristic evaluation for interactive games within elderly users. Paper presented at eTELEMED 2011: The Third International Conference on eHealth, Telemedicine, and Social Medicine, Gosier, Guadeloupe, France.
Milanović, Z., Pantelić, S., Trajković, N., Sporiš, G., Kostić, R., & James, N. (2013). Age-related decrease in physical activity and functional fitness among elderly men and women. Clinical Interventions in Aging, 8, pp. 549–556.
Nakai, A., Pyae, A., Luimula, M., Hongo, S., Vuola, H., & Smed, J. (2015). Investigating the effects of motion-based Kinect game system on user cognition. Journal on Multimodal User Interfaces, 9(4), pp. 403–411.
Nawaz, A., Skjæret, N., Helbostad, J. L., Vereijken, B., Boulton, E., & Svanaes, D. (2015). Usability and acceptability of balance exergames in older adults: A scoping review. Health Informatics Journal, SAGE Publications. doi: 10.1177/1460458215598638
Norman, K. L. (2013). GEQ (game engagement/experience questionnaire): A review of two papers. Interacting with Computers, 24(4), pp. 278–283.
Poels, K., de Kort, Y. A. W., & IJsselsteijn, W. A. (2007). “It is always great fun!”: Exploring dimensions of digital game experience using focus group methodology. Proceedings of the 2007 Conference on Future Play, Toronto, Canada (pp. 83–89). New York, NY: ACM.
Pyae, A., Luimula, M., & Smed, J. (2015a). Investigating the usability of interactive physical activity games for elderly: A pilot study. 6th IEEE International Conference on Cognitive Infocommunications (pp. 185–193). IEEE.
Pyae, A., Luimula, M., & Smed, J. (2015b). Understanding stroke patients’ motivation for motivation-driven rehabilitative game design. In R. Giaffreda, R-L Vieriu, E. Pasher, G. Bendersky, A. J. Jara, & J. J. P. C. Rodrigues…(Eds.), Internet of Things. User-Centric IoT (pp. 99–111). Springer International.
Pyae, A., Luimula, M., & Smed, J. (2016). Pre-studies on using digital games for the elderly’s physical activities. Paper to be presented at The 6th International Conference on Well-being in the Information Society.
Pyae, A., Raitoharju, R., Luimula, M., Pitkäkangas, P., & Smed, J. (2016). Serious games and active healthy ageing: a pilot usability testing of existing games. International Journal of Networking and Virtual Organisations, 16(1).
Pyae, A., Tan, B. Y., & Mark, G. (2013). Understanding stroke patients’ needs for designing user-centered rehabilitative games. Proceedings of the 7th Annual International Conference on Computer Games Multimedia and Allied Technologies, (pp. 151–156).
Raitoharju, R., Luimula, M., Pyae, A., Pitkäkangas, P., & Smed, J. (2014). Serious games and active healthy ageing: A pre-study. Proceedings of the International Conference on Well-being in the Information Society, WIS.
Renaud, K., & van Biljon, J. (2008). Predicting technology acceptance and adoption by the elderly: A qualitative study. Proceedings of the SAICSIT 2008, Wilderness Beach Hotel, Wilderness, South Africa.
Salen, K., & Zimmerman, E. (2006). The game design reader. Cambridge, MA: MIT Press.
Sauro, J. (2011). Measuring usability with the system usability scale (SUS). MeasuringU Retrieved from http://www.measuringu.com/sus.php.
Silva, P. A., & Nunes, F. (2010). 3 x 7 Usability testing guidelines for older adults. Proceedings of the 3rd Mexican Workshop on Human Computer Interaction, Vol. 2 (pp. 1–8).
Skoog, K. E. (2012). When to forgo the culturalization of video games: Contextualizing globalization within the mobile marketplace. Gamasutra. Retrieved from http://www.gamasutra.com/blogs/KarinESkoog/20120702/173371/When_to_Forgo_the_Culturalization_of_Video_Games_Contextualizing_Globalization_within_the_Mobile_Marketplace.php.
Sun, F., Norma, I. J., & While, A. E. (2013). Physical activity in older people: A systematic review. BMC Public Health, 13(499), doi: 10.1186/1471-2458-13-449.
System Usability Scale (SUS). (n.d.). Retrieved from http://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
Tashiro, J. (2009). What really works in serious games for healthcare education. Conference on Future Play on @ GDC Canada. Vancouver, British Columbia, Canada (pp. 3–4). New York, NY: ACM.
Theng, Y. L., Chua, P. H., & Pham, T. P. (2012). Wii as entertainment and socialization aids for mental and social health of elderly. Proceedings of CHI’12 Extended Abstracts (pp. 691-702), New York, NY: ACM.
Uzor, S., & Baillie, L. (2014). Investigating the long-term use of exergames in the home with elderly fallers. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’14, (pp.2813–2822). New York, NY: ACM.
World Health Organization (WHO). (2011). Physical activities and older adults. Retrieved from http://www.who.int/dietphysicalactivity/factsheet_olderadults/en/