This article is mentioned in:

Introduction

Stroke remains a major cause of morbidity and disability (1). In the USA, stroke is the most frequent cause of adult-onset disability and the cost of care for stroke patients is among the fastest-growing expenses for Medicare (2). Therapy can improve recovery and reduce long-term disability in patients who have suffered a stroke (3). Post-stroke rehabilitation services are generally targeted to the initial post-acute phase, predominantly the first several weeks or, perhaps up to a few months. This limited period of intervention derives from the perception of deficit permanence after a 3–6-month ‘critical window’ of enhanced neuroplasticity after stroke (4), and thus, expectations of no subsequent improvement. Remarkably, many studies have demonstrated a gradient of rehabilitation therapy responsiveness in stroke patients that extends beyond a year (5). New rehabilitation strategies enabled by new technological developments have been shown to yield performance gains in chronic stroke patients (6). Serious games (7,8) and robotic devices (9) have emerged as promising rehabilitation tools for the improvement of motor function and quality of life.

Serious game-based interventions immerse patients in enriched, stimulating computer simulated environments in which they interact with virtual objects while performing functional task-specific activities. Real-time feedback, predetermined task goals, and avatars are used to increase a patient's motivation and engagement. The amenability of such gaming interventions to being accessed via commercially available game alleviates cost while maximizing convenience in that rehabilitation protocols can be followed at home. Several studies have shown modest advantages of virtual reality-based interventions compared to traditional repetitive task practice protocols (10).

Clinical studies have shown that robot-assisted therapy can enhance neurological recovery (11). The main advantage of rehabilitation robots is that they can deliver high-dosage, high-intensity training, which is especially beneficial for patients working to improve motor system dysfunction due to stroke or spinal cord disease. Incremental improvements in clinically observed performance following intensive robotic therapy, although small, are statistically significant and promising for a variety of patient population, including chronic stroke patients (12).

The aim of the present study was to examine whether and to what extent the motor performance of chronic stroke patients can be improved with a combination of a robot-assisted therapy and serious gaming. Additionally, we assessed the potential utility of robotic device output parameters and game performance metrics as indirect indices of motor improvement.

Materials and methods

Subjects

The study cohort included eight stroke patients (5 women and 3 men) ranging in age from 39 to 60 years old who were right-handed according to the Edinburgh Handedness Inventory (13). They were recruited through the registries of stroke survivors who agreed to be contacted for stroke recovery studies that are maintained at Massachusetts General Hospital. Each had suffered an ischemic stroke affecting the left middle cerebral artery territory at least 6 months prior to recruitment. According to their medical records, they presented with acute unilateral loss of hand strength (Medical Research Council scale score <4, on 0-5 scale in which 5 is normal) that lasted for >48 h. They did not have hearing, vision, language, or cognitive deficits. Institutional review board approval of the study was granted by the Massachusetts General Hospital Human Research Committee (protocol number 2005P000570) and all participants provided written informed consent.

Data collection

We evaluated a total of 928 game rounds distributed over 386 training sessions. Patients were trained with a robotic hand rehabilitation system coupled to an interactive game for 45 min per day, 3 days per week, over a 10-week period. Each training session consisted of four 8-minute-long scenarios separated by 1- to 5-minute rest breaks. One session was conducted on each of three training days per week. Training was conducted at the patients' homes under supervision to ensure compliance. Motor performance was assessed prior to rehabilitation training (Pre), after finishing the treatment period (Post), and again 1 month after finishing the treatment in a follow-up assessment (FU) intended to probe the persistence of motor benefits over time. Motor performance was assessed via four clinical motor scales: The Fugl-Meyer scale, which we used to assess sensorimotor impairment (includes upper extremity, wrist, hand, coordination, sensation, passive joint motion, joint pain, and total scores) (14); the Action Research Arm Test (ARAT), which includes grasp, grip, pinch, gross movement, and total scores (15); the Modified Ashworth scale to assess spasticity of the elbow, wrist, fingers, and thumb (16); and the Box and Blocks test of gross manual dexterity (17).

Rehabilitation system

Patients underwent training with a newly re-designed robotic hand rehabilitation device called the Magnetic Resonance compatible Hand-Induced RObotic Device, version 3 (MR_CHIROD v3) (18-20). It was engineered to provide adjustable levels of force for handgrip exercising (Fig. 1). Continuous, grip-opposing, restoring force was provided by a low-friction glass cylinder/graphite piston assembly under the control of an electronic pneumatic pressure regulator with a portable air compressor that allowed the MR_CHIROD v3 to be used in a home environment. The MR_CHIROD v3 device was linked to a low-cost, Arduino-compatible microcontroller via a simple USB serial interface (57.6-kbps data transfer rate) thereby allowing pressure control, force/displacement acquisition, and interface with a laptop computer.

Figure 1 Magnetic resonance compatible Hand-Induced RObotic Device v3. Robotic hand rehabilitation system.

Game software

A serious game was developed in Microsoft Visual C++ for use on a Microsoft Surface (Microsoft Corp., Redmond, WA). The user plays the game via an avatar, namely a small green alien in a flying saucer that travels through a linear scrolling labyrinth (right to left scrolling). The speed of the saucer starts slow and increases linearly to a final level for each stage. The saucer flies through a scene, earning points for collision-free flight. The scene is populated with obstacles and rewards, collisions with which contribute negatively and positively to the user's score, respectively. During the game, the patient uses the MR_CHIROD handle to control the altitude of the saucer such that the handle position controls saucer height on the screen directly, with the fully open and fully closed positions corresponding to the lowest and highest screen positions. The goal of the game is to maximize one's overall score by avoiding obstacles and collecting rewards (Fig. 2). To increase user involvement, a soundtrack is looped and several sound effects are played upon collision with obstacles and rewards.

Figure 2 Screenshot view of the serious rehabilitative game showing the player's avatar (alien in a spaceship), three obstacles (columns with red tops), and a reward (red gem).

The game scenario was parametrized through an Extensible Markup Language (XML) script (‘stage script’) that specifies what the game engine loads in the form of sprites (graphic elements such as the avatar, obstacles, and rewards), the background picture, and game information. The stage script specifies obstacle and reward point values as well as all game components and mechanisms, including speed, sounds, and supportive visuals. Obstacles can be avoided with large, low accuracy motions to reach a safe position, whereas reward contact requires fine position control within a small range of motion. However, for a greater challenge in high-level play, users face more closely-spaced obstacles that leave narrow passages that require fine control to negotiate. The combination of obstacles and rewards allows the design of precise and complex trajectories for training gross motor and fine grip motions.

Prior to the commencement of each stage of the game, the prescribed pressure command is sent to the MR_CHIROD v3. When the game starts, the game engine polls the MR_CHIROD interface to retrieve the handle position in encoder counts and normalizes the position based on the range of motion set by end-stops on the device appropriate to each patient, and then updates the vertical and horizontal position coordinates of the avatar.

Game scores are calculated continuously based on the following formula:

where the instantaneous score S(t) is equal to the previous score S(t - 1) plus any reward collected, R(t), minus any penalty for hitting an obstacle, O(t), plus the constant g, which is the incremental score for each time interval. The scoring algorithm was tuned by adjusting R, O, and g in the stage script. The game was programmed to save each user's position trajectory, events at each time interval, score, handle pressure, and timestamp into a secondary XML script, which we used for offline analysis of raw and processed information. The stage script was individualized such that the handle force setting was set to ~75% of the user's maximal grip strength, thereby requiring our patients to exert effort and generate correspondingly large motor cortex activation, while not being so high as to impede completion of the training sessions.

Data analysis

For the purposes of the present study analyses, motor impairment was classified according to a three-level Fugl-Meyer upper extremity (FM UE) scale scheme (21), in which an FM UE score <20 was classified as severe, a score in the range of 20-40 was classified as moderate, and a score >40 was classified as mild. Improvement of at least 4.25 on the FM UE scale (22) and/or a smallest real difference of at least 5.5 blocks/min on the Box and Blocks test (23) were considered signs of clinically meaningful improvement in grip ability.

The mean force and the Pearson correlation coefficient between handle position and force (an indirect measure of grip control) were calculated for each round. These device metrics and the game metrics (final score, collisions, and rewards) for each of four scenarios were averaged for each session. Associations of device and game metrics with clinical motor scale scores were assessed with the Spearman ρ correlation coefficient after Bonferroni correction of multiple comparisons. Stepwise linear regression analysis was used to assess which metrics were significant predictors of motor performance, as indexed by FM UE, ARAT grip, and Box and Blocks scores. All statistical analyses were performed in SPSS version 23.0 (IBM, Inc.). Analyses with two-tailed P-values <0.05 were regarded as statistically significant.

Results

Log files from a total of 928 game rounds were processed offline. Of 389 training sessions, 6 early data sets were discarded due to equipment malfunction, affecting 2 of the 8 patients' results. All 8 patients had significant motor impairments before they began training (Table I). Specifically, according to our three-level FM UE scale-based classification scheme, two patients (#4, #5) had severe (score <20) impairment while the other four had moderate (score, 20-40) motor impairment prior to starting the training program. ARAT grip and Box and Blocks scale scores also indicated that patients #4 and #5 had severe motor impairment (Table I). Patient #5 presented with spasticity (Modified Ashworth scale scores: Elbow, 3; wrist, 4; fingers, 3; and thumb, 3). Patients #1 and #5 showed marginal progress during the rehabilitation period, as indicated by attending a clinically important difference for grip ability of ≥4.25 on the FM UE scale. According to the criterion of a smallest real difference of 5.5 blocks/min in the Box and Blocks test, patient #3 showed significant improvement.

Table I Demographic characteristics of and motor assessment scores for each patient.

Table I

Demographic characteristics of and motor assessment scores for each patient.

			FM UE			ARAT grip			Box and Blocks
Patient no.	Age, years	Gender	Pre	Post	FU	Pre	Post	FU	Pre	Post	FU
1	39	Female	27	31	32	12	12	12	43	45	47
2	64	Male	36	36	36	12	12	12	54	56	57
3	50	Male	36	36	36	12	12	12	69	82	92
4	59	Female	18	21	21	0	0	0	0	0	2
5	60	Female	15	21	17	3	6	6	0	0	0
6	46	Female	28	29	28	12	12	12	13	13	14
7	40	Male	32	32	32	12	12	12	46	44	45
8	33	Female	32	3	32	12	12	12	51	56	56

[i] Pre, before training; Post, at the conclusion of training; FU, at the follow-up test 1 month after completing the training; FM UE, Fugl-Meyer upper extremity; ARAT, Action Research Arm Test.

In most cases, the evolution of robotic device metrics (Fig. 3) and game metrics (Fig. 4) followed discernable patterns. For example, the mean forces exerted by subjects #2, #3, #7 and #8 were higher than those of the other patients and remained at similar levels throughout rehabilitation (Fig. 3), corresponding with the FM UE assessments of their motor functions. The same subjects had the highest force-position correlations. Subject #1 showed a discernable increase, nearly reaching the same correlation value by the end of training (Fig. 3). Regarding game performance, the subjects could be divided into three types: i) good performers (#1, #2 and #3), with relatively high scores, a large number of rewards, and few collisions, albeit with limited improvement; ii) patients with (at least initially) low scores who achieved marked improvement (#4, #5, #6, and #7); and iii) one poor performer (#8) whose scores started and remained low. The middle group patients tended to show rapid improvement initially followed by a gradual improvement such that their metrics approached the levels of the first group.

Figure 3 Robotic metrics. (A) Evolution of mean force and (B) position-force correlation during training for each subject. Numbers 1-8 correspond with patient numbers. a.u., arbitrary units.

Figure 4 Changes in game metrics over the training period, including (A) rewards, (B) collisions and (C) final scores. Numbers 1-8 correspond with patient numbers.

We observed strong correlations of robotic device metrics and game metrics with motor ability scale indices (Table II). Of the five device and game metrics included in our correlational analyses, the robotic device metric mean force had the strongest correlations with motor ability scale indices. Among the three game metrics, number of collisions had the strongest correlation with motor scale scores. Stepwise linear regression analysis indicated that device and game metrics were indeed predictive of motor scale scores. Specifically, independent predictors of FM UE score were mean force (standardized Beta=0.662; P<0.001) and collisions (standardized Beta=-0.273; P<0.039). Meanwhile, independent predictors of ARAT grip score were mean force (standardized Beta=0.336; P=0.049) and collisions (standardized Beta=-0.471; P=0.007). Finally, independent predictors of Box and Blocks scores were mean force (standardized Beta=0.585; P<0.001), collisions (standardized Beta=-0.289; P=0.007), and correlation (standardized Beta=0.223; P=0.033). In order of most to least predictable, Box and Blocks scale (R2=0.821), FM UE scale (R2=0.700), and ARAT grip scale (R2=0.499) scores all correlated very significantly with metrics (all 3, P<0.001).

Table II Analysis of robotic system and game metrics correlations with motor assessment scale scores.

Table II

Analysis of robotic system and game metrics correlations with motor assessment scale scores.

	FM UE		ARAT grip		Box and Blocks
Parameter	ρ	P-value	ρ	P-value	Ρ	P-value
Robotic system
Mean force	0.869	<0.001	0.677	<0.001	0.865	<0.001
Position-force correlation	0.651	<0.001	0.228	0.234	0.613	<0.001
Game
Rewards	0.528	0.002	0.330	0.070	0.488	0.005
Collisions	-0.642	<0.001	-0.585	0.001	-0.670	<0.001
Score	0.590	<0.001	0.602	<0.001	0.636	<0.001

[i] FM UE, Fugl-Meyer upper extremity; ARAT, Action Research Arm Test.

Discussion

In this study, we examined the utility of a hand rehabilitation system for stroke patients consisting of a serious game coupled to a magnetic resonance (MR)-compatible robotic device. Combining robotic technology with serious gaming has the potential to achieve good patient engagement, immersion, and motivation and thus to produce motor skill-relearning associated neuronal activation that can, in principle, promote neuroplasticity (24). We observed large game metric improvements in the early days of training, suggesting that more measurements should be monitored over the initial weeks of training to establish stronger conclusions. These preliminary results hint that finer-grained temporal analyses will show that such data would be helpful for building a motor ability improvement prediction model.

Very few studies (25,26) have attempted to assess the additional benefits of this combination of technologies. Patients #1 and #5 in our study showed marginal motor improvements on the FM UE scale (27,28). Compared with conventional therapy, these improvements appear to be similar to or greater than is achieved with conventional therapy (28). Nevertheless, the combination of robotics with serious gaming addresses known limitations (29) of conventional post-stroke rehabilitation including limited availability of specialized facilities, transportation challenges, patient non-compliance, and costs.

Previous attempts to predict motor outcomes have been hindered by methodological flaws and a lack of external validation (30,31). The inclusion of neuroimaging and neurophysiological indices in clinical outcome predictive models could improve their accuracy (30,31). These additional data become more important when clinical status is determined based on self-reported assessments, a common practice for chronic stroke patients who often follow prescribed, but unsupervised, rehabilitation protocols at home. Self-reported clinical monitoring becomes even less reliable when patients have cognitive deficits. The strong correlations of our device and game metrics with motor scale scores suggests that our hand rehabilitation system can provide indirect but objective assessments of motor improvement. Moreover, MR compatibility allows state of the art neuroimaging during game play to provide insights into underlying structural remodeling and reorganization processes of functional recovery in the brain during the performance of rehabilitation tasks.

Although gaming in post-stroke rehabilitation has been considered a promising therapeutic resource, there has been limited examination of clinical efficacy. Therapeutic outcomes were evaluated in only about half of 31 reviewed studies and no relationship was found between game score and clinical findings in the remaining studies (32). Even so, correlation between game score and clinical tests have the potential to guide the development of biomedical systems that aid in treatment and evaluation (32). In our game, the score element ‘number of collisions’ had stronger associations with clinical motor scale scores than ‘number of rewards’ or ‘total score’. Hence, our results suggest that game metrics could be used for clinical evaluation, but also that different weighting of score elements would be appropriate indicated.

Regarding gender, our sample was too small to allow any meaningful analysis of gender differences. It is evident that, in our sample, the three male patients started the rehabilitation period with generally better clinical scores, and thus less severe stroke-related impairment, than the five female patients; the highest FM UE pre-rehabilitation period score observed among the woman was the same as the lowest observed among the men. Both patients that showed marginal improvement, according to FM UE scores, were women. Thus, it may be that our protocol was particularly beneficial to patients with relatively more severe impairment.

Although the introduction of computer gaming into rehabilitation is quite new, robotics have been used extensively for motor performance assessments during rehabilitation (33). For example, the InMotion2 robot (Bionik Laboratories, Corp., Toronto, Canada), a commercial version of the MIT-Manus, provides automated estimates upper extremity clinical scores for chronic stroke patients based on multiple regression models of kinetic and kinematic macro-metrics (34). In agreement with this approach, we observed strong correlations between robotic device metrics and clinical scores. Importantly, our stepwise regression modeling demonstrated that the inclusion of gaming metrics increased the predictive power of clinical score models.

System advantages

Notably, the MR_CHIROD v3 system was designed with features that facilitate its use and dissemination. The system's USB serial connectivity and available WiFi compatibility allow it to be used with multiple commercial operating systems and game consoles. The master-side of the polling-based serial communication protocol can also be implemented in different programs. To enable ease of replication and wider use of the system, it has multiple 3D-printable and uncomplicated machinable components as well as low-cost electronic and pneumatic components.

The strong magnetic fields characteristic of the modern MR machines limit the range of actuators, sensors, and materials that can be used in an MR-compatible rehabilitation devices. Ferrous materials, electromagnetic actuators, and unshielded conductors maybe forbidden for use in MR scanners altogether for safety reasons or poorly suited due to imaging data distortion. The MR-compatibility of the MR_CHIROD v3 robotic device-made possible by plastic parts (3D-printed and machined), shielded sensors/cables, and MR-compatible ball bearings and other components-mean that patients brain activation can be analyzed during its use. Generally, upper-extremity rehabilitation devices are not made to MR-compatible (35); for exceptions see (36). In addition to being MR-compatible, the currently used MR_CHIROD version 3 (manuscript describing the design and testing is in preparation) is portable and suitable for standardized use across clinical and home environments (18). Whereas the high cost of fabricating our prior design of the MR_CHIROD device (version 2) was limiting (19), the version 3 redesign yielded fabrication cost reduction as well as overall simplification of the fabrication process.

Finally, it is our view that the combination of robotic technologies with serious gaming can be applied as a rehabilitation tool as well as in multiparametric modeling of motor performance and/or of the rehabilitation outcome. Whereas device metrics are strongly linked to motor skill rehabilitation per se, serious gaming metrics can reflect cognitive factors, such as attention, working memory, and decision-making processes. Although such cognitive factors are not typically regarded as relevant for understanding motor performance, cognitive abilities represent fundamental constraints on learning and execution of movements (37,38).

Study limitations

This study has a couple of notable limitations. Firstly, the small number of participants was the most important major limitation of this pilot study. Studies with larger cohorts are required to validate our preliminary results and allow clinical trials focused on new rehabilitation approaches. Secondly, the applicability of the MR_CHIROD v3 is limited by the simplicity of its grip functions and the lack of multiple degrees of freedom. Notwithstanding, the device's grip functions are critical for stroke patients given that, from a neurology point of view, grip rehabilitation is the most important goal for achieving independence in daily life, including the ability to reliability grasp and hold onto objects without dropping them. In this context, it is important to note that although robotic therapy has been shown to improve arm motor function after stroke (39-43), apart from a few reports (43-45), these efforts have not focused on the hand (46) as the presently examined system does.

Conclusion

The present results demonstrate that MR_CHIROD v3 device output parameters and linked serious game score elements can be used as indices of hand motor function during post-stroke hand rehabilitation training. These robot and game metrics showed especially marked improvements during the initial 2 weeks of rehabilitative training in the present study sample. Clinical motor scale scores were found to associate very strongly with these indices, with the best correlated metric being average force. Game score elements and device output parameters can be combined to build a more reliable predictive model of clinical motor performance and, potentially, replace commonly used unreliable self-assessment methods. Larger studies are needed to inform the development of models that can be used reliably for remote monitoring and patient prognosis applications.

Acknowledgements

We wish to thank Dr Bruce R. Rosen, M.D., Ph.D. (Director of the Athinoula A. Martinos Center for Biomedical Imaging), for his support in the implementation of the imaging studies.

Funding

This work was supported by a grant from the National Institute of Neurological Disorders and Stroke (grant no. 1R01NS105875-01A1) of the National Institutes of Health.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Authors' contributions

LGA analyzed the data and prepared the manuscript. GDN designed and created the computer game. MPO designed, manufactured and tested the robotic device and edited the manuscript. CP and SL acquired the raw data and assessed their authenticity. MAM provided data interpretation and manuscript editing. AAT designed and supervised the study. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Institutional review board approval of the study was granted by the Partners Human Research Committee (protocol no. 2005P000570) and all participants provided written informed consent.

Patient consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

References