A System in the Wild: Deploying a Two Player Arm Rehabilitation System for Children With Cerebral Palsy in a School Environment

by Raymond Holt, Andrew Weightman, Justin Gallagher, Nick Preston, Martin Levesley, Mark Mon-Williams, Bipinchandra Bhakta

Peer-reviewed Article

pp. 111-126

Abstract

This paper outlines a system for arm rehabilitation for children with upper-limb hemiplegia resulting from cerebral palsy. Our research team designed a two-player, interactive (competitive or collaborative) computer play therapy system that provided powered assistance to children while they played specially designed games that promoted arm exercises. We designed the system for a school environment. To assess the feasibility of deploying the system in a school environment, the research team enlisted the help of teachers and staff in nine schools. Once the system was set up, it was used to deliver therapy without supervision from the research team. Ultimately, the system was found to be suitable for use in schools. However, the overriding need for schools to focus on academic activities meant that children could not use the system enough to achieve the amount of use desired for therapeutic benefit. In this paper, we identify the key challenges encountered during this study. For example, there was a marked reluctance to report system issues (which could have been fixed) that prevented children from using the system. We also discuss future implications of deploying similar studies with this type of system.

Tips for Usability Practitioners

Schools present a challenging environment for doing research, particularly where the system must be left unattended. We recommend that any usability practitioners intending to carry out a similar field study should consider the following:

Keep feedback requirements to a minimum. Teachers already have very demanding jobs, so the effort required for their participation beyond the use of the system being tested—such as providing feedback or returning data—should be kept to a minimum.
Use the system to gather data, when possible, to minimize the teachers/staff feedback responsibility and to simplify the logistics of data collection.
Include remote error tracking for unsupervised systems to alleviate the reluctance to report faults during deployment.
Discuss advanced planning needs so that all staff that are using the system are aware of the basic requirements of the system, including delivery and collection requirements.
Understand that a negative outcome of a study can be just as valuable as a positive one. Taking a hands-off approach allowed us to find that schools are not an effective environment for rehabilitation. That finding allows us to seek more promising environments and will help other researchers avoid expending resources that can be more useful applied elsewhere.

Introduction

Cerebral palsy (CP) is the most common cause of severe disability among children in Europe, affecting 2.1/1,000 live births (Johnson, 2002). CP is an umbrella term covering a range of permanent movement and postural disorders arising from non-progressive brain injury prior to birth or during infancy. The effects can vary greatly between individuals, depending in part on which parts of the brain have been affected (Rosenbaum, Paneth, Leviton, Goldstein, & Bax, 2007). Effects may be restricted to a single limb, several limbs, or the entire body. Effects can also include hypertonia (muscle stiffness causing restricted movement), involuntary movement, impaired coordination, and sensory deficits (such as impaired vision or hearing). While CP is not progressive, the impairments it causes can adversely affect the trajectory of child development by restricting opportunities to develop social and motor skills. CP can create problems in later life as strategies used to compensate for impairments can place undue strain on other parts of the body (Cox, Weze, & Lewis, 2005). There are a range of organizations whose aim is to support people with CP, such as Scope in the UK (www.scope.org.uk) or United Cerebral Palsy in the US (www.ucp.org). These organizations provide a good range of resources on the condition and its varied effects and support available to address them.

Where the condition affects one or both upper limbs, the ability to reach and manipulate objects is affected. Movements in an arm affected by CP exhibit slower and more variable movements (Jaspers et al., 2011; Utley & Sugden, 1998), which combined with weakness and sensory deficits, can significantly impair the ability of individuals with CP to carry out daily activities and can create significant social barriers (Imms, 2008). It is clearly desirable to improve upper limb function in children with CP, but the best strategy remains an open question, with many proposed treatment modalities demonstrating improvements in upper limb function (Boyd, Morris, & Graham, 2001). A common adjunct to all of these treatment modalities is the recommendation that children practice appropriate arm exercises. Use of the affected limbs has been shown to significantly offset the impact of CP (Kluzik, Fetters, & Coryell, 1990). However, a lack of physiotherapy resources means that such exercise is often delivered through a self-managed home exercise program with only occasional expert supervision. Exercises are frequently dull and repetitive, and children often lack the motivation to carry out these regimes, leading to poor compliance with the prescribed plan (Chappell & Williams, 2002).

There has been little research on how much exercise is required for therapeutic benefits to show, but it is generally agreed that the affected limb needs to be pushed to the point of fatigue for this to happen. Successful programs have required children to exercise for 20 to 45 minute sessions three times a week (McBurney, Taylor, Dodd, & Graham, 2003) and 75 minute sessions three times a week (Knox & Evans, 2002), representing a significant time commitment.

One solution to the problem of a lack of motivation is the use of interactive computer play-based therapy (Sandlund, McDonough, & Hager-Ross, 2009), where therapy is delivered as a game through a computer interface. This approach has been growing in popularity due to the increased popularity of video gaming as a pastime in the last few decades. The development of consoles that use movement-based interaction with videogames, most notably the Nintendo WiiTM, has led to great interest in their use as a means of encouraging physical activity among children and making rehabilitation enjoyable (Anderson, Annett, & Bischof, 2010; Chang, Chen, & Huang, 2011; Deutsch, Borbely, Filler, Huhn, & Guarrera-Bowlby, 2008). The use of off-the-shelf videogame consoles in rehabilitation has many benefits: The consoles are mass produced and do not require specialist development, and the games are designed first and foremost to be enjoyable. However, the game systems have some limitations that reduce their effectiveness as therapeutic devices. First, such systems may be unusable and frustrating for individuals with significant arm impairments. Some systems do not verify whether the actions performed are therapeutically useful, and while one could argue that any use of an affected limb is potentially beneficial, systems such as the Wii are easily operated with small sharp movements rather than the smooth coordinated movements required for therapy (Levac et al, 2012). Second, hands-on therapy from a physiotherapist provides support to push reaching motions to their limit and extend reach beyond that which the individual could achieve on their own, compensating for the weight of the individual’s arm.

An alternative to the existing commercial, unassisted technologies are assisted movement devices (AMDs), a term that encompasses any rehabilitation technology that applies assistive force while promoting therapeutically beneficial movements. Such treatment devices offer benefits not only to children with CP, but also to stroke patients (Jackson et al., 2007) and children with developmental coordination disorder (Snapp-Childs, Mon-Williams, & Bingham, 2013). While this requires the development of more complex specialist devices than existing games consoles, the devices are able to promote adherence to desirable trajectories—controlling both spatial and temporal components through the application of force (at either an endpoint or around the upper and lower limbs to control joint position). A conventional force-feedback joystick (i.e., those available commercially) is not appropriate for this purpose, as its control is based around fine wrist movements rather than the wide workspace needed for these sort of exercises (ensuring full arm movement). Moreover, the force-feedback available in commercial joysticks is not sufficient to provide the forces needed by an AMD. Such systems also require knowledge of target points to be reached in order to calculate the appropriate force and trajectory, and it is important to ensure that the ordering of these target points (and the motions required to reach them) are therapeutically appropriate. Accordingly, this means developing not only specialist hardware, but also specialist games that reflect these characteristics rather than using existing commercial games.

Such technology has been utilized in clinical settings (Fluet et al., 2010, Jackson et al., 2007; Krebs, Ladenheim, Hippolyte, Monterroso, & Mast, 2009) and home environments (Weightman et al., 2011). In response to feedback gathered from the Weightman et al. (2011) project, we developed a two-player system with the aim of deploying it in a school environment. Social interaction (such as cooperation and competition) in games has long been identified as a motivator for playing (Malone & Lepper, 1987) and continues to be recognized as an important aspect of making games enjoyable (Sweetser & Wyeth, 2005). However, it does raise significant challenges in computer play-based therapy, as different players will have different levels of impairment (and in some cases, none at all), making it difficult to create a level playing field. It also means deploying a system in an environment where time and space are constrained, and where teachers supervising the use of the system have little expertise in therapy and robotics. The goal of the research presented here was to determine whether such a complex system could be deployed under the “real life” conditions of a school environment. This paper represents the first reported deployment of such a system. In this paper we discuss the results of deploying the system in terms of the usage it received and the barriers encountered during its use.

Mehtods

The goal of the research was not to assess the clinical efficacy of the AMD approach but to determine whether it was feasible to utilize a dual-player system in a school environment, and if not, what prevented this from taking place. This section outlines the hardware and software that made up the system as deployed, the process used to deploy it in both single- and two-player modes, and the data gathering plan to evaluate its usage.

Hardware

Our research group designed 4 two-user AMD computer game systems. The systems were designed and manufactured to provide safe physical assistance to children with arm impairments while undertaking therapeutic exercises. We developed these systems to be an extension of the single-user technology that had been previously deployed for home use (Weightman et al., 2011). We designed the original system using a Microsoft^TM Force-Feedback Sidewinder II joystick that was modified to provide the required assistive forces and that would fit in an appropriate workspace. The system was connected to a PC via USB. We loaded specially designed therapeutic games onto the PC. Originally, we envisioned that the new two-player system would take a similar form, with up to six joysticks plugging into a school PC to allow multiplayer gaming, which would have the benefit of being portable and easy to store. However, several factors made this approach not feasible. On a technical front, the Sidewinder joystick was not able to provide as much force as desired. This required a move toward a new design with larger motors and customized control software developed in LabVIEW^TM and delivered via a National InstrumentsTM compact Reconfigurable Input-Output (cRIO) controller. However, the move from a home to a school environment also entailed a number of significant alterations to the design.

To ensure that the system was acceptable to the school staff supervising the students’ activity with the system, we held group meetings at three different schools, with a total of seven teachers in attendance. At the meetings, we wanted to address teachers’ concerns and questions and to obtain feedback on the design concepts. It quickly became apparent that while portability and ease of storage were considered important, this was secondary compared to ease of use and a quick setup. Teachers felt that they couldn’t accommodate any system that took more than a minute or two to set up during break times. A longer setup time would leave too little time for the system to be used and would be disruptive while students waited for the teacher to get the system organized. The teachers were enthusiastic about the system and felt that time could be made in the school day for its use, through a combination of lunchtime clubs and afterschool sessions. One teacher suggested that for it to be included in the classroom environment, the games would need to have an educational angle.

Accordingly, we designed the system as a self-contained unit with two joysticks, two monitors, a PC, and cRIO controllers as shown in Figure 1. Figure 2 shows a close-up of the joystick component. We designed the system to be portable in a school environment. To ensure an acceptable footprint before we built the system, we measured doorways and aisles in a range of schools. The deployed system had a footprint of 135 cm x 67 cm (53 in. x 26 in.) and was 116 cm (45 in.) tall. We put wheels on the system so that it could be easily moved between classrooms or out of the way as needed. The cRIO and PC had separate power supplies that needed to plug into the main power supply. All system components, including the monitors, were connected to a four-way power extension so that only a single external socket was required to power the system. In case of emergency, a large safety push button (not illustrated in Figure 1) that disconnected the power from the assistive haptic interfaces was located on the front top of the system. Each haptic interface incorporated a handgrip sensor such that no assistive forces could be applied when a user was not holding the handgrip. The system included another handgrip with a push button that was used to access some of the game functions to ensure that the game was bimanual. In this way, the children with CP had something to do with their unimpaired arm, reducing the temptation to use it (rather than their impaired arm) to operate the joystick.

Figure 1. The AMD system as deployed

Figure 2. Close-up of the joystick used in the system

The monitors and cRIO would turn on automatically once power was supplied. A single press of the PC power button is all that was required to turn the PC on. This done, the system would boot up and the teacher could launch its initialization process by selecting an icon from the desktop, after which the system was ready to launch the games. In this first version of the system, initialization entailed the teacher following a short series of onscreen prompts to make sure that the joysticks were in a central position. To register that the joystick position was acceptable, the teacher pressed the trigger button for each joystick. This ensured that the system was not starting near the edge of its working range.

Software

We developed a suite of games that required therapeutically appropriate movements to play and permitted a combination of competitive/collaborative multiplayer and single player experiences. All games were based on the same outward and inward movement from the body. The goal of the movement was to follow a defined path between targets so that the assistive force controller could identify how to assist the movement. Furthermore, all of the games were based on the same scenario delivered by a cutscene when the system first loaded. The cutscene showed the players adopting the role of “monkeys whose friends have been kidnapped by a hungry crocodile and are being taken to his lair as dinner.” The premise of each game is based on the monkeys trying to rescue their friends, which is in line with Malone and Lepper’s (1987) recommendations of the use of fantasy as a motivator in playing games. We designed each of the games such that they could be operated in a single user mode. The multiplayer aspect of each game varied depending on whether it was sequential (players taking turns to move their characters) or simultaneous (both players’ characters moving at the same time), and whether it was cooperative (either both players win or both players lose against the computer) or competitive (one of the players wins and the other loses). The following are descriptions of the four games:

Sequential Cooperative Play—The Puzzle Game. The players work together to navigate their way across a series of islands, taking turns to follow bridges and pressing buttons to make new bridges appear. The goal is to reach a trapped monkey.
Sequential Competitive Play—The Chase Game. The players take turns to race across a series of islands and bridges to reach a trapped monkey. The entire maze is navigable from the start, and time penalties are invoked for touching the water. The player with the quickest time wins.
Simultaneous Competitive Play—The River Race Game. The players travel along a winding river at a constant rate, collecting their own color bananas to score points as they go. Players can leave the confines of the river, but lose points for doing so.
Simultaneous Cooperative Play—The Van Chase Game. The players play simultaneously collecting their own color bananas and firing them at the van to achieve the common goal of stopping the evil crocodile as it races away with the captured monkeys. Once enough bananas have been fired, the van is destroyed and the game ends. Scoring is based on a collaborative effort of hitting the van.

We developed the four games with input from a user group of children with cerebral palsy who had participated in the previous home-based project (Weightman et al., 2011). This group was recruited because their prior knowledge of the system meant that they would not be participating in the upcoming school-based trial, and consequently, there was no concern that their exposure to the games might confound later stages of the study. Meetings took place at the University of Leeds. This gave the children the opportunity to offer feedback and make comments on early iterations, to evaluate the initial concepts and game-play proposals, to give feedback on early prototypes, and to carry out informal usability testing to help identify bugs and check the clarity of instructions. Sample screens from each game are shown in Figure 3.

Figure 3. Screenshots of the four game types, clockwise from top left: the Puzzle Game, the Chase Game, the River Race Game, and the Van Chase Game

In addition to the games, software was required to manage the amount of assistance the joystick provided each player. We called this the Adaptation to Player Performance Algorithm (APPA), and it served two purposes. From a therapeutic perspective, the goal is to gradually reduce the amount of assistance provided as players’ motor skills improve—but such a change has to be based on therapeutic progress, rather than simply based on the amount of time played. In addition, there is the problem of creating a level-playing field between players who have different levels of impairment; scaling the amount of assistance provided to a player’s abilities provided a convenient way of doing this. To assess players’ abilities during the field deployment, every time the system started, each player completed an assessment task in which each player had to collect as many bananas as possible in a minute. This unassisted task was used to determine how much assistance a child would receive during the play session. Each child using the system performed this assessment, regardless of whether they had CP, although the system only provided assistance for the children with CP.

In addition to the assessment routine, a child’s score on a given day was compared with his or her score on this task for previous days. Where the performance scores increased by more than 10% of the previous mean score, the assistance was reduced. Where the performance score decreased by more than 10% of the previous mean score, assistance was increased. In this way, the APPA fine-tuned the assistance to the most appropriate level for a given player on a given day.

Deployment

Our goal was to create the most realistic test possible for the system, not to assess the clinical efficacy of the system. We wanted to see whether it could be deployed in a school environment and whether it would sustain usage over an extended period of time. This section outlines the participants in this study, the process used to deploy the system, and type of evaluation data that we wanted to gather.

Participants

There were 11 children with upper limb impairments due to cerebral palsy at the schools (one at each school, except for two schools with two such children each); they were the target users for the system. These comprised eight males and three females, with a mean age of nine years (SD = 1 year 11 months). Given the two-player nature of the system, other children, without CP, at the participating schools also played the games. But for this study, we were only interested in how much therapeutic exercise the children with CP received via the system. It is these children’s usage data that we discuss in this paper.

We asked all children recruited for the study to provide written consent after we presented an age appropriate explanation of the study. All children were told that they were free to withdraw from the study at any time. The children all received age appropriate information sheets as did the parent(s)/guardians of the children. Ethical approval was granted by Leeds (East) Research Ethics Committee (REC reference 09/H1307/48).

Process

We deployed the four systems in the following three stages:

Stage 1. Four schools received the system. Five children with CP used the system.
Stage 2. Three schools received the system. Three children with CP used the system.
Stage 3. Two schools received the system. Three children with CP used the system.

Each school received one system during each deployment. The spare systems in Stages 2 and 3 remained at the University. Each stage comprised two 4-week phases separated by a minimum of three weeks to ensure that therapeutic effects from the first deployment did not influence performance on the second deployment. In one of two phases, we deployed the system in single-player mode; in the other phase, we deployed the system in two-player mode. The phases were counterbalanced such that five of the schools received the system in the two-player mode first, and the other four schools received the system in the single-player mode first.

Our goal was for the children to use the system approximately 30 minutes a day. This was an important target. We discussed this goal with the head teacher of each school before the school agreed to participate in the project. We also discussed it with school staff on two occasions: once when they first agreed to participate in the project and again upon delivery of the system. We provided participating staff with a copy of the information sheet supplied to parents when consenting for their children to participate in the study. This information sheet stated that children would use the system for up to an hour a day. Initially this was met with some disbelief by school staff, but we explained that the hour was a maximum (included to avoid problems with ethics compliance should sessions inadvertently exceed 30 minutes), and that 30 minutes was the target. The staff accepted this with some relief. It is worth noting, however, that we did not provide the staff with any written indication of the 30-minute target, and with hindsight, including this in the written information may have been beneficial.

On delivery of the system to the school, we asked each school to designate a teacher to supervise the system (except in School H, see Table 1, where the designated staff member was the Special Educational Needs Coordinator, rather than a teacher). We showed the supervising teacher how to use the system: plug in the system to the power supply, start the PC, start the real-time sub-system software (icon on the desktop), move the haptic interface joints to set positions, and then start the game sub-system (icon on the desktop) software. After this initial setup, the onscreen software instructions guided the users. We demonstrated the emergency shutdown procedure, and then demonstrated how to restart the system if it did not initialize. We left an instruction manual with the system. The instruction manual included contact numbers to reach project team members if any problems occurred. We pointed this out to the supervising teacher or staff member as part of the initial briefing when we delivered the system. This represented some conflict with the desire to deploy the system without any input from our team, but we felt that this represented the kind of technical support that a school might receive in practice. It was better for problems to be fixed and noted and to identify any other barriers to using the system rather than have the system sit unused for four weeks because of a straightforward technical problem.

We gathered evaluation data from three sources: a record of any calls received from the schools to report problems or to request assistance, usage data stored on the systems for each child during each phase of the trial, and feedback questionnaires completed by the participating children and their teachers. As a result of difficulties in getting time in teachers’ schedules for interviews in the early phases of the project, we agreed with the participating staff that the most effective way of gathering information was to provide feedback questionnaires on paper. We provided a stamped, self-addressed envelope so that the school could return the questionnaires to the researchers at their convenience, rather than attempting to arrange face-to-face feedback interviews.

Results

This section reports the results of three sources of data collected: the error logs and callouts that indicated problems that occurred during system deployment, the amount of usage the systems received, and the qualitative feedback from participating staff and children.

Errors and Callouts

No calls were received from the schools during any of the deployments. However, when we collected the systems after the first deployment, teachers from all four schools reported that the systems were prone to crashing when first being initialized and that this cut into the amount of therapeutic time available. Teachers had struggled to remedy the situation—even turning the PC off and back on again did not correct the problem. Because of these issues, in some cases, there was insufficient time to carry out the required sessions. On other occasions, the systems appeared to work without difficulty. Analysis of the error logs confirmed this, with a large number of Transmission Control Protocol (TCP) errors, indicating that the PC was not communicating with the cRIO properly. The same problem was reported by all four schools in the first deployment, though the systems continued to be used and no schools contacted the team for technical support during this period. This situation caused us some frustration, as the systems could have received greater usage during this deployment had the schools contacted us for assistance when problems first arose. However, we were also conscious that the staff participating in the project received no direct benefit from their participation and were doing us a great service by taking the time to use the system at all. Time taken to report problems would be over and above this, and we felt that it was important to be sensitive to the demands we were making on teacher’s time, particularly given the need to maintain good relationships with them for this and future projects. For this reason we did not take issue with the staff for the lack of contact, but simply reminded them at the start of the second deployment that they could contact us if problems persisted.

We eventually identified the problem as being the system timing out during initialization if the trigger buttons for the joysticks were not pressed within 30 seconds of the system starting up. As the cRIO was powered separately to the PC, turning the PC off would not correct this problem because the cRIO would continue to report an initialization error. Turning the system off at the main power supply, however, would cause the cRIO to reset the next time it was turned on so the system could initialize properly. This explained the intermittent nature of the problem: If a teacher initialized the system within 30 seconds of it first being turned on, it would run without difficulty; if not, it wouldn’t run again until it had been switched off and back on at the main power supply, which often wasn’t until the next day when the system was due to be used again.

After the first phase of the first deployment, we addressed this problem by extending the time-out period to two minutes and offering more explicit instructions on the importance of initializing the joysticks within this period. But the problem still persisted into the second phase of the first deployment—though this was again only reported at the end of the deployment. Accordingly, we changed the time-out period to infinite so that teachers could initialize the joystick at their leisure. This solution resolved the problem for the rest of the study.

Usage Data

The system was able to log the length of each game and the date and time on which it took place. We only analyzed data for children with CP, as these were the target users of the system. The system recorded the raw amount of therapeutic play that the children received. The numbers presented here do not include time for setting up or waiting between games.

No allowance was made for days where the schools were closed due to, for example, bad weather, teacher training, public holidays, or where class activities such as school visits might have rendered it impossible or impractical to use the system. The aim was to provide a realistic snapshot of how the system would be used, and these are all factors that would affect the system in real usage. However, one issue worth noting is that Child 9 and 10 (both at School H) did not use the system at all in their second phase. The participating teachers reported that this was due to pressure in preparing for Standard Assessment Test (SAT) exams and was therefore a function of the time of year at which the deployment took place, rather than any disinclination towards the single-player mode. While this represents a genuine usage pattern, it is important to bear this in mind when comparing overall usage in single versus multiplayer modes, as it will make the gap between the two appear larger than it might have been had the deployment occurred at a different time of year. It is also worth noting that in Schools C and H, which each had two children participating in the study, the children with CP did not play against each other in two-player mode, but instead they played with other friends. Because the combination of frequency and intensity of exercise is important, rather than the raw amount of exercise undertaken, Table 1 reports the number of days on which the system was used in its single player and two player deployments (from a possible 20 days in each case, as no allowance was made for school closures due to, for example, bad weather or teacher training days). Table 1 also reports the mean length of sessions when it was used in each deployment. Note that where the system was used multiple times in the same day, the session was counted as usage for one day, with session lengths treated separately because multiple short sessions in a day do not equate to a single full-length session. Session length therefore covers the total time spent carrying out therapeutic play in a single sitting with the system.

Table 1. Amount of Usage per Child

Taken across both single and multiplayer deployments, the system was used on a mean of 22.4 days (SD=8.49 days) of a possible 40 (the sum of the 20 possible days in the single-player deployment and the 20 possible days in the two-player deployment). The mean session length was 11.0 minutes (SD=6.76 minutes). Table 1 shows how the session lengths varied dramatically from child to child, for example, Child 3, 4, and 6 used the system more than other children. When looking at all of the children, there was little difference between single- and two-player usage. The system was used on a mean of 9.55 days (SD=5.68 days) in single-player mode compared with a mean of 12.8 days (SD=4.47 days) in two-player mode, both from a possible 20 days. The mean session lengths were 12.0 minutes (SD=7.51 min) for the single-player mode and 10.4 minutes (SD=6.09 min) for the two-player mode. If the results for Child 9 and 10 are excluded on the basis that their lack of play in single-player mode is an artifact of timetabling problems rather than a disinclination towards the mode itself, this becomes a mean of 11.6 days (SD = 3.53 days) in single-player mode and 13.67 days (SD = 4.27 days) in two-player mode, with mean session lengths of 12.0 minutes (SD=7.51 min) in single-player mode and 10.4 minutes (SD =6.09 min) in two-player mode. Such small differences are not meaningful in a therapeutic sense, but do indicate that providing a two-player option did not present a barrier to using the system, despite the requirement for children who did not need therapy taking the time to participate.

In all cases, the average times were substantially below the target 30 minutes per day. In fact, 30 minute sessions were only ever achieved on seven occasions and then only by three of the children (three times by Child 3, three times by Child 8, and once by Child 1). The lowest target that might be considered acceptable (twenty minutes, three times a week) was not achieved by any of the children.

The most common time for usage was during the morning registration period (the first 15 minutes of the 9 a.m. to 10 a.m. slot) and then afternoon registration (the last 15 minutes of the 1 p.m. to 2 p.m. slot). It is worth noting that the children whose usage was concentrated around lesson times (Children 3, 4, and 11) or lunchtime (Child 6) were those who achieved the longest sessions.

Qualitative Feedback

As scheduling interviews with staff at the participating schools was difficult, given the huge demands already existing on their time, to gather teacher and child feedback we used questionnaires that were comprised of open questions on the system’s usage. Through these questionnaires we asked if there were any barriers to its use and asked for suggestions for future improvements. However, getting staff to complete and return the questionnaires and ensuring that the children returned the feedback questionnaires proved extremely problematic.

We sent the questionnaires by mail one week before the end of the deployment. All children who used the system and any staff who supervised their use were instructed to complete the questionnaires and leave them with the system for collection. We telephoned the schools to explain this process when the questionnaires were mailed to the school. We telephoned again the day before the systems and questionnaires were to be collected, as a reminder to complete the questionnaires. However, it was rarely possible to speak with the teachers themselves, and we were obliged to leave messages with secretarial staff to be passed on to the teachers.

Despite our prompt and original information letters and briefing sessions emphasizing the importance of this feedback to evaluating the system, very few schools provided any of the questionnaires when we collected the system. We made follow up phone calls and sent additional copies of the questionnaires with pre-paid, self-addressed envelopes to the schools that had not completed them. We received a total of 21 questionnaires from the staff through a combination of direct collection and by mail with at least one from each school and in some cases two or three. It was not always clear which school each questionnaire came from, particularly when the questionnaires were returned by mail. We received 14 questionnaires from the children, although the usage data recorded on the systems showed that at least 38 children used the system during the deployment. Because the questionnaires were often not labeled correctly (such as not including a name or not including their surname), we were not always able to determine which questionnaire came from children with CP. As the original questionnaires had been intended to be left with the system, we assumed that we would know which school they had come from and could label them accordingly. This problem would have been prevented if the questionnaires had been pre-labeled with the participating school. Similarly, asking children to provide forename and surname would at least have allowed us to identify the children with CP. Initial return rates might also have been higher if a simple set of closed questions had been used (open answer questions require significantly longer to complete). Although the value of this would have been limited—we had already gathered usage data from the system, and our interest was in what had facilitated or prevented that usage.

Feedback comments from the school staff responsible for the system focused consistently on the size of the system and ease of setup. One school indicated that maneuverability was not important, as the size of the system meant that it could not be moved anywhere else. The other schools indicated that being able to move the system around easily was very important, as it needed to be used in different classrooms or moved out of the way when not in use. Every school was able to find space for the system. No school felt it was too big, but all agreed that a system any larger than this would be untenable.

Ease of setup was also an issue. While all schools praised the ease with which the system could be plugged in and booted up, those in the first deployment reported that the system sometimes failed to initialize properly the first time it was switched on each day—as had already been identified through the callouts made during the deployments. While they were always able to resolve this by restarting the system, School B in particular pointed out that this cut significantly into the short periods of time that they were able to find for use.

All schools indicated that the major problem in the systems’ usage was in relation to timetabling. Only School C took regular time out of lessons to utilize the system. For the other schools, the most convenient time for using the system was during morning or afternoon registration. This immediately restricted playing sessions to no more than 10 or 15 minutes, allowing for time to let the children settle down to their task. Other schools took time out of lessons only where it could be conveniently accommodated, such as during times where a story was being read to the whole class. Schools E, F, G, and H, in particular, all indicated that exam preparation cut into the amount of time available for therapy, particularly in the second phase of deployment (the multiplayer phase for Schools E and F and single players for School G and H). Of these, only School H failed to find any time for the system’s use during this phase, as indicated by their usage data.

Finally, several respondents mentioned the need for more games to maintain interest, citing the concern that a small library of games might maintain interest for a limited deployment (such as the eight total weeks of this study), but beyond that boredom may become an issue.

These comments confirmed our initial findings from working with teachers when developing the system. Maneuverability, ease of setup, and a small footprint are all important for a school environment. This demonstrates the merit of the revised concept of a self-contained unit over the initial notion of up to six independent joysticks, but also the importance of engaging with users to properly understand their needs. Nevertheless, it also demonstrates the inherent difficulty in finding time for utilizing such a system in the school day. There was a fundamental mismatch between the schools’ goal of educational achievement and our goal of delivering the required therapy. Participating teachers were all sympathetic to the need for children to undertake therapeutic exercise, but as they will ultimately be assessed on the academic performance of their pupils, there was an active disincentive to make class time available for therapeutic use.

Discussion

The system deployment was successful insofar as it was usable by school staff and children without direct intervention from the research team. The early problems with setup and initialization may have impacted the amount of use, but the system was still used. When these problems were resolved in later deployments, there was not a marked increase in the amount of usage. The major barrier to use of the system was not its technical complexity, but finding sufficient time in the school day for children to use it. None of the children came close to the target level of usage, and the tendency to use registration periods inevitably limited the amount of therapy that could be delivered in one session. Those children making greatest use of the system were those where dedicated lesson time or the lunchtime of the school day was given over to use of the system. The lack of system use was most pronounced around the time leading up to exams, which is not unreasonable on the part of the schools. It is worth noting that the therapy here is intended to replace or supplement a home exercise plan, rather than replace existing time with a physiotherapist and is therefore moving therapy that would otherwise have taken place at home (if at all) into the school environment.

Despite this limitation, the two-player version of the system was used slightly more frequently than the single-player version, albeit for slightly shorter sessions than in single player. Some of the reduction in therapeutic exercise time was caused by the need to set up a second player and carry out the assessment task to determine the level of assistance twice. The system may have value in schools as an adjunct to a home exercise plan—though whether this would be worth the cost (especially if only one or two children in the school were to use it for therapeutic purposes) is another matter. Our future research will focus on home deployments of the system.

Working with schools presents a range of significant practical challenges, not least the fact that rehabilitation research will inevitably be secondary to children’s education. Several issues were raised in this deployment that are worth bearing in mind for future research, particularly where a system is to be deployed unsupervised.

Firstly, schools demonstrated a marked reluctance to contact the research team and report problems. They only mentioned these problems at the end of the deployment, despite the fact that the problems interrupted use. While this was consistent with our aim to leave the system without direct intervention, it did mean that a problem that could have been corrected in the field went unaddressed for several weeks. In this case, the impact does not appear to have been serious, but it does demonstrate the value of maintaining an error log on the system to keep track of problems for review post-deployment, rather than simply relying on self-report from users in the field. In future deployments, we suggest that it would be worth having the system relay its usage and error logs back to the research team via wireless Internet on a regular basis so that problems could be identified and corrected quickly. Better that than finding that a four-week deployment had been wasted because of an unreported technical problem.

Secondly, obtaining feedback was particularly challenging. While staff were generally enthusiastic and happy to cooperate with the project, any involvement was inevitably over and above their already demanding job. As schools are busy, the systems often had to be delivered and collected either before children arrived in the morning or after they had left in the evening. In many cases, the participating staff were not present when the system was collected at the end of the deployment. This schedule meant that the only feedback available were the completed questionnaires, which often were not completed when we collected the system. It is important to note, that completing a questionnaire after a significant amount of time has passed may affect the accuracy of one’s recall. With hindsight, it would have been better to deliver the questionnaires upon collecting the system to ensure that they were labeled or coded correctly so responses could be tied to a particular school even if they were not fully completed.

Finally, having multiple teachers involved in using the same system (or using the space in which the system was located) did sometimes present a problem in terms of delivery and collection. We only gave system and study instructions to the selected supervising staff member; not all staff members that would be using the system were given instructions. Because the system was used by multiple staff members at different times (and/or the system was housed in an area used by multiple classes), this sometimes led to confusion when the system was being delivered or collected, i.e., some staff members did not have the benefit of knowing what we needed from them because they were not debriefed by us or the supervising staff member.

Recommendations and Conclusions

The deployment described here has demonstrated that the system may be feasible for use without supervision in some contexts, but that the need to emphasize academic performance means that schools are not an appropriate environment for delivering this therapy. Accordingly, the next steps in developing this system are the following:

Develop a single-player version of the self-contained system to be more suitable for deployment in a home environment.
Evaluate the clinical efficacy of a home system through a randomized control trial.
Develop additional games to extend interest and to appeal to more adult audiences, such as stroke patients.

Refernces

Anderson, F., Annett, M., & Bischof, W. F. (2010). Lean on Wii: Physical rehabilitation with virtual realist Wii peripherals. Studies in Health Technologies and Informatics, 154, 229-34.
Boyd, R. N., Morris, M.E., & Graham, H. K. (2001). Management of upper limb dysfunction in children with cerebral palsy: A systematic review. European Journal of Neurology, 8(5), 150-166.
Chang, Y.-J., Chen, S.-F., & Huang, J.-D. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 32(6), 2566-70.
Chappell, F., & Williams, B. (2002). Rates and reasons for non-adherence to home physiotherapy in pediatrics. Physiotherapy, 88(3), 2-11
Cox, D., Weze, C., & Lewis C. (2005). Cerebral palsy and ageing: A systematic review. Scope. Available from http://www.scope.org.uk/help-and-information/cerebral-palsy/ageing-and-cerebral-palsy.
Deutsch, J. E., Borbely, M., Filler, J., Huhn, K., & Guarrera-Bowlby, P. (2008). Use of a low-cost, commercially available gaming console (Wii) for rehabilitation of an adolescent with cerebral palsy. Physical Therapy, 88(10), 1196-1207.
Fluet, G. G., Qiu, Q., Kelly, D., Parikh, H. D., Ramirez, D., & Adamovich, S. V. (2010). Interfacing a haptic robotic system with complex virtual environments to treat impaired upper extremity motor function in children with cerebral palsy. Developmental Neuro-rehabilitation, 13(5), 335-345.
Imms, C. (2008). Children with cerebral palsy participate: A review of the literature. Disability and Rehabilitation, 30(24), 1867-1884.
Jackson, A. E., Holt, R. J., Culmer, P. R., Makower, S. G., Levesley, M. C., Richardson, R. C., Cozens, J.A., Mon-Williams, M., & Bhakta, B.B. (2007). Dual robot system for upper limb rehabilitation after stroke: The design process. Proceedings of the Institution of Mechanical Engineers Part C: Journal of Mechanical Engineering Science, 221(7), 845-857.
Jaspers, E., Desloovere, K., Bruyninckx, H., Klingels, K., Molenaers, G., Aertbelien, E., Gestel, L.V., & Feys, H. (2011). Three-dimensional upper limb movement characteristics in children with hemiplegic cerebral palsy and typically developing children. Research in Developmental Disabilities, 32, 2283-2294.
Johnson, A. (2002). Prevalence and characteristics of children with cerebral palsy in Europe. Developmental Medicine and Child Neurology, 44, 633-640.
Kluzik, J., Fetters, L., & Coryell, J. (1990). Quantification of control: A preliminary study of effects of neurodevelopmental treatment on reaching in children with spastic cerebral palsy. Physical Therapy, 70, 65–76.
Knox, V., Evans, A. L. (2002) Evaluation of the functional effects of a course of Bobath therapy in children with cerebral palsy: a preliminary study. Developmental Medicine & Child Neurology, 44, 447–460.
Krebs, H. I., Ladenheim, B., Hippolyte, C., Monterroso, L., & Mast, J. (2009). Robot-assisted task-specific training in cerebral palsy. Developmental Medicine and Child Neurology, 51(4), 140-145.
Levac, D., Pierrynowski, M. R., Canestraro, M., Gurr, L., Leonard, L., & Neeley, C. (2012). Exploring children’s movement characteristics during virtual reality video game play. Human Movement Science, 29(6), 1023-38.
Malone, T.W., & Lepper, M.R. (1987). Making learning fun: A taxonomy of intrinsic motivations for learning. In R.E. Snow & M.J. Farr (Eds.), Aptitude, learning and instructions: Cognitive and affective process analyses (Vol. 3, pp. 223-253). Hillsdale, NJ, US: Laurence Erlbaum Associates.
McBurney, H., Taylor, N.F., Dodd, K. J., & Graham, H.K. (2003). A qualitative analysis of the benefits of strength training for young people with cerebral palsy. Developmental Medicine & Child Neurology, 45 (10), 658-663.
Rosenbaum, P., Paneth, N., Leviton, A., Goldstein, M., & Bax, M. (2007). A Report: The definition and classification of cerebral palsy April 2006. Developmental Medicine and Child Neurology Journal Supplement, 49, 8-14. doi: 10.1111/j.1469-8749.2007.tb12610.x
Sandlund, M., McDonough, S., & Hager-Ross, C. (2009). Interactive computer play in rehabilitation of children with sensorimotor disorders: A systematic review. Developmental Medicine and Child Neurology, 51(3), 173-179.
Snapp-Childs, W., Mon-Williams, M., & Bingham, G. P. (2013). A sensorimotor approach to the training of manual actions in children with developmental coordination disorder. Journal of Child Neurology, 28(2), 204-12.
Sweetser, S., & Wyeth, P. (2005). Gameflow: A model for evaluating player enjoyment in games. ACM Computers in Entertainment, 3(3), 3A.
Utley, A., & Sugden, D. (1998). Interlimb coupling in children with hemiplegic cerebral palsy during reaching and grasping at speed. Developmental Medicine and Child Neurology, 40, 396-404.
Weightman, A. P. H., Preston, N., Levesley, M. C., Holt, R. J., Mon-Williams, M., Clarke, M., Cozens, A. J., & Bhakta, B. B. (2011). Home based computer-assisted upper limb exercise for young children with cerebral palsy: A feasibility study investigating impact on motor control and functional outcome. Journal of Rehabilitation Medicine, 43(4), 359-363.

Published in: in Volume 8, Issue 4,