Suitability of Kinect for measuring whole body movement patternsduring exergaming

Mike van Diest a,b,n, Jan Stegenga a, Heinrich J. Wörtche a, Klaas Postema c,Gijsbertus J. Verkerke c,d, Claudine J.C. Lamoth b

a INCAS3, Assen, The Netherlandsb Center for Human Movement Sciences, University of Groningen, University Medical Center Groningen, The Netherlandsc Center for Rehabilitation, University of Groningen, University Medical Center Groningen, The Netherlandsd Dept of Biomechanical Engineering, University of Twente, Enschede, The Netherlands

a r t i c l e i n f o

Article history:Accepted 16 July 2014

Keywords:ExergameBalance quantificationPrincipal Component AnalysisFall prevention

a b s t r a c t

Exergames provide a challenging opportunity for home-based training and evaluation of postural controlin the elderly population, but affordable sensor technology and algorithms for assessment of whole bodymovement patterns in the home environment are yet to be developed. The aim of the present study wasto evaluate the use of Kinect, a commonly available video game sensor, for capturing and analyzingwhole body movement patterns. Healthy adults (n¼20) played a weight shifting exergame under fivedifferent conditions with varying amplitudes and speed of sway movement, while 3D positions of tenbody segments were recorded in the frontal plane using Kinect and a Vicon 3D camera system. PrincipalComponent Analysis (PCA) was used to extract and compare movement patterns and the variance inindividual body segment positions explained by these patterns. Using the identified patterns, balanceoutcome measures based on spatiotemporal sway characteristics were computed. The results showedthat both Vicon and Kinect capture 490% variance of all body segment movements within three PCs.Kinect-derived movement patterns were found to explain variance in trunk movements accurately, yetexplained variance in hand and foot segments was underestimated and overestimated respectively by asmuch as 30%. Differences between both systems with respect to balance outcome measures range 0.3–64.3%. The results imply that Kinect provides the unique possibility of quantifying balance ability whileperforming complex tasks in an exergame environment.

& 2014 Elsevier Ltd. All rights reserved.

1. Introduction

Fall-related injuries account for high rates of permanentimmobility and mortality in older adults (Kannus et al., 2005;Rubenstein, 2006). One of the major predictors of falls is age-related deterioration of postural control (Delbaere et al., 2010).Postural control is the act of maintaining, achieving or restoring astate of balance during any activity (Pollock et al., 2000). Appro-priate control of posture and balance underlies functional skills,and is a pre-requisite for normal activities of daily living. Posturalcontrol is always embedded in goal-directed actions, thus requir-ing subtle adjustment of whole body movements (Huang andBrown, 2013). Exercise videogames (Exergames) requiring wholebody movements for gameplay provide a method for trainingpostural control in the home environment (Lamoth et al., 2011;

Van Diest et al., 2013). Although supervision and feedback duringtraining programs have shown to increase training programeffectiveness (Wu et al., 2010), many home based physical activityprograms are characterized by little or no ongoing evaluation ofactivities (Ashworth et al., 2005). Exergames however allow forcapturing whole body movements, thereby holding the potentialto evaluate user movement patterns during exergaming.

Today, state of the art marker-based 3D camera (MBC) systemsare abundantly used for recording and analysis of whole bodymovements for gain in motor function in a research setting (Zhouand Hu, 2008). MBC systems however, are too expensive and bulkyfor use in the home situation, which conversely requires use of low-cost sensors for capturing body movements. The Microsoft™

Kinects (Kinect) is a popular low-cost markerless motion captionsystem developed for the gaming industry. Kinect combines a videocamera and an infra-red depth sensor to generate a colored pointcloud consisting of about 300,000 colored dots per frame, fromwhich anatomical landmarks are computed (Khoshelham andElberink, 2012). Previous studies comparing Kinect with state ofart MBC systems showed that the position of anatomical landmarks

Contents lists available at ScienceDirect

journal homepage: www.elsevier.com/locate/jbiomechwww.JBiomech.com

Journal of Biomechanics

http://dx.doi.org/10.1016/j.jbiomech.2014.07.0170021-9290/& 2014 Elsevier Ltd. All rights reserved.

n Correspondence to: Dr. Nassaulaan 9 9401 HJ Assen The Netherlands.Tel.: þ31 592 860 000, fax: þ31 592 860 001.

E-mail address: mikevandiest@incas3.eu (M. van Diest).

Journal of Biomechanics 47 (2014) 2925–2932

from Kinect-generated point clouds is measured with high test-retest reliability; ICC differences between Kinect and an MBCsystem are r0.16 (Clark et al., 2012). Spatial and temporal accuracyhowever, is not equal for all anatomical landmarks, particularlyduring movement (Bonnechère et al., 2012; Clark et al., 2012, 2013).Deviations in body segment position up to 10 cm and joint range ofmotion up to 10% are reported, with largest inaccuracies found forlower body extremities (Bonnechère et al., 2013; Obdrzálek et al.,2012). When using Kinect motion capture data for analyzingmovement patterns during exergaming, Kinect's limited accuracycould affect patterns found.

The aim of the current study is to evaluate the suitability ofKinect for whole body motion capture with the objective ofmovement pattern analysis during exergaming. To this end weadopted principal component analysis (PCA) for comparing twoten-segment body models recorded with Kinect and an MBCsystem during exergaming. Whole body movement recordings ofboth systems were compared with respect to (1) the number ofprincipal components (PCs) that represent the global movement ofthe whole body movement data, (2) the variance in individualbody segment movements explained by the identified movementpatterns, and (3) outcome measures representing balance abilitywhich are computed from a signal constructed from the PCs. It isexpected that Kinect will display lower accuracy for identificationof movement patterns and body segment contributions, possiblyreflected in deviations in balance outcome measures.

2. Methods

2.1. Participants

Twenty young adults (11 women, 9 men; age 37.0716.6 years) participated inthis study. Inclusion criteria were 18–60 years of age and physically fit. Exclusioncriteria for the present study were: neurological, visual or musculoskeletalimpairments or use of medication that could affect postural control. All subjectsprovided written informed consent. The research was approved by the EthicalCommittee Human Movement Science, University Medical Center Groningen, inaccordance with the ethical standards of the declaration of Helsinki.

2.2. Procedure and instrumentation

Subjects were instructed to wear tight fitting dark clothing while playing acustom made weight-shifting exergame. The game challenged subjects to makesway movements in the frontal plane by moving the center of mass towards theedges of the base of support. Subjects stood in parallel stance within an 80�60 cmarea and were instructed to keep their feet on the ground during gameplay.Because previous studies reported that Kinect accuracy is not identical for all bodypostures and movements (Bonnechère et al., 2013, 2012), five different gameconditions were played: (1) self-selected sway speed and amplitude. (2) Gamespeed was increased by a factor 2. In the remaining three conditions subjects wereinstructed to (3) adopt maximum sway frequency at self-selected sway amplitude,(4) lift the leg situated contralateral to the sway direction, while performing thesway movement, and (5) maximally increase sway amplitude at self-selected swayfrequency. Each condition was played twice for one minute; the first trial wasconsidered a practice trial, the second was used for further analysis. Trials wererandomized, except for trial 1; this was always a ‘condition 1’ trial. The game wasinstalled on a Dell Latitude E4310 laptop (Dell, Round Rock, USA) and wascontrolled using Kinect (Microsoft Corp, Redmond, USA). A projector and a screen(3.55�2.60 m) positioned at a distance of 2 m were used to display the game. TheKinect was positioned 2 m in front of the subject at 60 cm height. 3D position dataof ten anatomical landmarks covering trunk and extremities were obtained usingKinect and OpenNI SDK v1.5.2.23 at an irregular sample frequency of about 30 Hz.Ten reflective markers were placed on the subject and 3D marker position datawere acquired at a sample frequency of about 170 Hz using a 12 camera Viconsystem, (Vicon V8, Oxford, UK) using Workstation 4.6 build 146 and D-flow v3.10.0(Motek Medical, Amsterdam, the Netherlands). Vicon and Kinect marker positionsare specified in Fig. 1.

2.3. Data analysis

Data were analyzed using Matlab 2013a (Mathworks, Natick, USA). Linearinterpolation of Kinect and Vicon ensured constant sampling at 30 Hz. Data

were scaled to unit variance to eliminate overall amplitude effects and synchro-nized for each trial by determining the phase lag between Kinect data and Vicondata at maximum cross co-variation. The phase lag between markers waspreserved. For all conditions and subjects the first ten sways were used for furtheranalysis.

To identify common features and deviations thereof of multivariate data setsPrincipal Component Analysis (PCA) is a particularly suitable method (Daffertshoferet al., 2004). In the current study PCA was used to identify and compare move-ment patterns in the whole body motion capture data recorded using Kinectand Vicon. Custom made Matlab software was used (Kaptein and Daffertshofer,2012). First, PCA was applied to the separate datasets of Vicon and Kinectfor each individual in each game condition. This allowed for comparingthe identified patterns by each system in terms of the number of PCs and associatedexplained variance representing the whole body movement pattern. There-after, PCA was applied to the concatenated data of Kinect and Vicon toevaluate the contribution of individual body segments for each system to theidentified PCs.

Because sway movements were made in horizontal direction, PCA was appliedto the horizontal component of the motion capture data. PCA was performedby first storing the recordings of the ten markers of each system in a system ofvectors Q

Q ¼qViconð1Þ

⋮qViconð10Þ

0B@

1CA ð1Þ

To eliminate possible interindividual and between-condition effects in move-ment patterns, PCA was performed for each subject and condition individually, thusyielding 20�5¼100 matrices Q with dimensions (m�n), where m and n representthe number of markers (10) and the number of frames covering the first ten swaymovements respectively. For Q the covariance matrix and its eigenvectors v!k andeigenvalues λk were computed. The identified PCs were sorted in descending orderof λk corresponding to the variance accounted for by each component. Note that λkmeasures the variance along the direction v!k and that ∑N

k ¼ 1λk ¼ 1, where N is thetotal number of PCs. An identical procedure was performed for Kinect data. Thetotal variance in the datasets explained by the main PCs was computed for eachsystem. The number of PCs that represented the common signal features in thetime series was determined by visual inspection of the eigenvalue spectra, usingdiscontinuities in the eigenvalue spectra as cut-off criterion.

Fig. 1. Point cloud data and the Kinect skeleton data as acquired from the Kinectdepth camera. Positions of Kinect anatomical landmarks used in the study areshown as white dots and Vicon reflective markers as diamonds. Vicon markerpositions included: acromion L/R (shouldertips), dorsal side of carpals L/R (wrist),T5, L1, lateral femoral condyle L/R (knees), and lateral metatarsophalangeal joint VL/R (feet).

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–29322926

For the combined Vicon and Kinect data analysis, the matrix Q containedrecordings of the ten markers of both systems

Q ¼

qViconð1Þ⋮

qViconð10ÞqKinectð1Þ

⋮qKinectð10Þ

0BBBBBBBBB@

1CCCCCCCCCA

PCA was performed for each subject and condition individually, thus yielding20�5¼100 matrices Q with dimensions (2m�n), here referred to as PCACombined.The PCs were computed and the time evolution along each PC was defined byprojection of the original data set onto the PC in question, explaining a percentageλk of the variance in the total dataset. Projections were defined as

ξk ¼ v!k UQ ð2Þ

The eigenvectors v!k or more specifically their coefficients v1, v2, vN, representthe individual Vicon and Kinect marker signals and provide information about thedata spread of each marker signal covered by each PC. The variance in each markerposition explained by each selected PC varck was given as percentages by

varckðmÞ ¼ 100 1�SSerrðmÞSSQ ðmÞ

� �ð3Þ

with m¼1,…,20, where m¼1,…,10 represent the ten Vicon markers andm¼11,…,20 represent the ten Kinect markers. SSQ ðmÞ and SSerrðmÞ are defined as

SSQ ðmÞ ¼ ∑n

j ¼ 1Q2

ðm;jÞ ð4Þ

SSerrðmÞ ¼ ∑n

j ¼ 1err2ðm;jÞ ð5Þ

where err is defined as

err¼Q�ð v!k U ξkÞ ð6Þ

2.4. Balance quantification

To enable quantification of balance ability based on the identified movementpatterns, a reconstruction q!ðMÞ

Kinect of the coherent Kinect signal was composedusing the projections along the main PCs of the ten segment body model of Kinect

q!ðMÞKinect ¼ ∑

M

k ¼ 1ξk Uλk

!∑M

k ¼ 1λk

,ð7Þ

where M represents the number of PCs in the model. Similarly the coherent Viconsignal was represented by q!ðMÞ

Vicon .Recent studies on postural control have shown that measures based on the

characteristics of sway patterns, including Index of Harmonicity (IH) and RMSvalues are sensitive to detect differences in healthy and pathologic postural control(Kang et al., 2009; Lamoth and van Heuvelen, 2011; Riva et al., 2013). Therefore, inthe present study these measures were computed from q!ðMÞ

Vicon and q!ðMÞKinect . IH was

defined as the quotient of the power spectral density (PSD) of the fundamentalfrequency and the cumulative sum of PSD of the fundamental frequency and thefirst five harmonics (Lamoth et al., 2002). Additional outcome measures based onexpected reflections of age-related changes in postural control on gameplay wereintroduced. Mean Sway Amplitude (MSA) and the standard deviation of swayamplitudes (stdMSA) were computed as well as Dominant Sway Frequency (DSF),defined as the frequency where the power of q!ðMÞ

Vicon and q!ðMÞKinect is maximal.

3. Results

3.1. Variance in whole body movement data explained by movementpatterns identified

PCA performed on Kinect and Vicon datasets separatelyshowed that three PCs representing the common signal featuresin the data covered 90–95% of the total variance within each of thedatasets. Differences in total variance explained between the twosystems were on average 1.2% and highest in the leg liftedcondition: 3.3% (Table 1). A combination of Vicon and Kinect datain PCACombined, shows that the first three components account for87–91% of the variance (Table 1).

3.2. Variance in individual marker movements explained bymovement patterns in concatenated ViconþKinect data set

A more detailed analysis of the movement patterns derivedfrom PCACombined and the variance in the individual Vicon andKinect marker signals explained by these patterns showed thatboth systems are able to identify differences in variance in markerpositions explained by individual PCs. Fig. 2 shows that the first PCprimarily reflects the sway movement during the weight-shiftingtask. This PC covers most of the variance in all marker movements,as represented by the eigenvectors, except for the foot markers.The second and third PC not only contain specific frequencycomponents, but also show drift and noise, which may reflectslower processes (like moving the base of support) and randomfluctuations, which are reflected in a power spectrum showingmultiple frequencies with high spectral power. In contrast, handand foot markers of both systems add to a higher extent to thesecond and third PC (Table 2). Figs. 3 and 4 show that the variancein body segment position explained by the three component PCAmodel, as quantified by varck, is not equal for all individual bodysegments; varck ranges from 63% to 98%. Differences wereobserved between contributions of trunk and extremity markers;trunk markers contribute 92–98% varck, whereas hand and footmarkers contribute 63–93% varck.

There were several differences observed between systems inthe variance in body segment positions explained by each PC.Fig. 4 shows that trunk markers recorded by Vicon and Kinectcontribute equally to PCACombined, whereas the extremities showdifferences in contribution to these PCs. Kinect showed relativelyhigh varck values for the feet and low varck values for the hands.Further analysis of the eigenvectors showed that the variance inindividual marker positions explained by individual PCs showsdifferences ranging from 0% to 30% varck. Different game condi-tions had only minor effects on contributions of markers toPCACombined.

3.3. Balance quantification

Fig. 5 shows that differences in outcome measures computedfrom q!ðMÞ

Vicon and q!ðMÞKinect are not consistent for all measures; DSF

showed minor differences between both devices on all gameconditions ranging 0.3–1.5% while differences between deviceson MSA and RMS ranged 3.9–16.0%. Values of DSF, MSA and RMSwere not consistently higher or lower for either device, contrary toIH, which was consistently lower for Kinect by 4.6–8.4%. Values forstdMSA showed relatively large differences between devices ran-ging 27.0–64.3%, where Kinect consistently reported higher values.Differences between devices were largest in the ‘increased swayspeed’ and ‘leg lifted’ conditions.

Table 1Percentage of variance of the coherent signal explained (VE) by the three main PCsaveraged between subjects (7std) for PCA performed on Vicon and Kinect dataseparately and combined, indicated by the columns ‘Vicon’, ‘Kinect’ andViconþKinect respectively. The column ‘Game condition’ represents the fivedifferent game conditions.

Game condition Vicon(%VE7std)

Kinect(%VE7std)

ViconþKinect(%VE7std)

Neutral 94.973.3 94.972.5 90.473.3Incr. game speed 94.773.6 93.973.0 90.173.2Incr. sway freq 91.874.8 93.573.3 87.075.4Contralat. leg lifted 94.274.5 90.973.5 87.873.6Incr. sway amp. 94.773.5 94.672.2 90.573.4

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–2932 2927

4. Discussion

The aim of the current study was to evaluate the suitability ofKinect for analysis of movement patterns during exergaming.Whole body movement recordings of Vicon and Kinect systemswere compared with respect to (1) the whole body movementdata variance explained by the identified movement patterns,(2) the variance in individual marker positions explained by thesepatterns and (3) outcome measures representing balance ability ascomputed from signals constructed using the identified movementpatterns.

Results showed that 490% of variance within all Kinect andVicon signals was covered by three PCs, indicating that relevantsignal features were identified accurately using Kinect motioncapture data. Eigenvector analysis showed differences betweenboth devices with respect to the variance in body segmentpositions explained by the PCs in all game conditions; Kinectunderestimated hand marker variance explained and overesti-mated foot marker variance explained by as much as 30%.Comparison of outcome measures showed that Kinect under-estimates IH by 5–8%, overestimates stdMSA by 27–64% and showsdeviations up to 16% for MSA, RMS and DSF.

The findings show that relevant movement patterns can beidentified using Kinect, even though markers are measured withvarying accuracy, resulting in deviations in variance in bodysegments explained by movement patterns, especially for extre-mities. This observation agrees with an earlier study, that reportedthat range of motion (ROM) of extremities measured using Kinect

showed lower agreement with Vicon measurements than ROM ofshoulders (Bonnechère et al., 2013). Explanations for the lowerKinect accuracy can be found in the Kinect resolution (640�480for depth and RGB camera), which is relatively low compared toMBC systems (up to 4704�3456) (Vicon Motion Systems Ltd.,2013) and the low and irregular sample frequency, which fluc-tuates between 25 and 30 Hz (Clark et al., 2013; Menna et al.,2011). Finally the algorithms used for 3D skeleton trackingimplemented in available middleware are not optimized for usein movement pattern analysis, thereby providing suboptimal 3Dskeleton information for the current purpose (Shotton et al., 2011).

The inter-individual differences in marker position varianceexplained by patterns identified, reflected in standard deviations,were higher for hands, knees and feet than for shoulders and backmarkers as measured by Vicon. Although the accuracy of Kinectwas lower for measuring extremities, the same trends wereobserved for Kinect data, indicating an absence of interactioneffects. Kinect thus shows consistent performance in all gameconditions. Interestingly the explained variance in the feet asmeasured by Kinect was higher than when measured using Vicon.Because signals were normalized prior to PCA to correct foramplitude effects and subjects were instructed to keep their feeton the ground while making sway movement, the feet as mea-sured with Kinect apparently were tracked less accurately thanhands and knees. Moreover, they moved synchronously to theother segments since they contribute to the same pattern. This isin agreement with Clark et al., who found that step and stridetime, requiring accurate identification of anatomical landmarks,

Fig. 2. Projections ξk (left panel), eigenvectors (middle panel) and the corresponding Power spectrum P[ξk] (right panel) of a representative example of a principalcomponent analysis where Vicon and Kinect data are combined (PCACombined). The eigenvectors are sorted 1:20, where 1:10 (above the dashed line) represent Vicon markersleft shoulder, right shoulder, T5, left hand, right hand, L1, left knee, right knee, left foot, right foot and 11:20 (below the dashed line) the corresponding Kinect markers. Handsare shown in red, feet in green. For this particular example eigenvalues λk of the main PCs are 82.4%, 7.6%, and 3.4%. Projection ξ1 oscillates with the sway frequency, as isshown in the third column. All eigenvectors contribute to this dominant PC except for the feet, recorded with Vicon. Projection ξ2 is less regular, holds more frequencies withhigh spectral power and shows higher contributions of hands and feet markers.

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–29322928

Table 2varck the variance in the shoulders, back, hands, knees and feet marker positions explained by the first three components shown for all five conditions7standard deviation for both Vicon and Kinect, indicated with ‘V’ and ‘K’respectively. Contributions of hands and feet to PC 2 and 3 showed relatively large differences between systems and are indicated by a square border.

Condition Mode Shoulders(%) ± std

Back(%)± std

Hands(%)± std

Knees(%)± std

Feet(%)± std

V K V K V K V K V K

Neutral1 95 ± 3.0 91 ± 15.8 89 ± 15.1 96 ± 3.2 79 ± 26.1 55 ± 27.4 77 ± 24.4 57 ± 33.4 33 ± 25.2 43 ± 33.72 1 ± 2.8 4 ± 9.6 5 ± 11.6 2 ± 2.3 10 ± 19.8 11 ± 17.7 12 ± 20.3 29 ± 31.4 18 ± 24.5 39 ± 33.83 1 ± 1.0 1 ± 2.5 1 ± 2.8 0 ± 0.8 4 ± 7.4 9 ± 14.7 3 ± 7.3 5 ± 11.9 24 ± 26.9 8 ± 14.81+2+3 98 ± 1.7 97 ± 4.3 94 ± 6.6 98 ± 1.6 93 ± 6.4 74 ± 21.2 92 ± 8.1 92 ± 6.6 76 ± 17.0 91 ± 5.5

Increased game Speed

1 95 ± 3.0 95 ± 2.4 91 ± 9.6 96 ± 2.6 81 ± 19.9 47 ± 24.9 71 ± 30.3 64 ± 29.9 30 ± 26.9 45 ± 31.92 1 ± 2.3 1 ± 2.2 2 ± 2.5 1 ± 1.7 4 ± 5.4 8 ± 9.0 9 ± 16.3 24 ± 29.0 29 ± 29.5 38 ± 33.13 1 ± 0.9 1 ± 1.1 2 ± 6.7 1 ± 1.0 6 ± 12.7 11 ± 17.1 8 ± 15.3 5 ± 9.1 21 ± 30.2 8 ± 14.01+2+3 97 ± 1.3 97 ± 1.6 95 ± 6.3 98 ± 1.9 91 ± 6.8 66 ± 22.3 89 ± 12.2 93 ± 4.1 81 ± 13.8 91 ± 6.7

Increased Sway Frequency

1 91 ± 10.7 92 ± 8.9 76 ± 31.3 92 ± 10.8 62 ± 33.5 42 ± 24.5 59 ± 36.4 58 ± 33.6 22 ± 21.6 46 ± 32.42 2 ± 4.6 4 ± 6.7 10 ± 21.6 4 ± 9.2 16 ± 23.7 14 ± 20.2 20 ± 27.2 23 ± 28.7 25 ± 27.9 31 ± 29.53 2 ± 2.5 1 ± 2.1 5 ± 12.4 1 ± 1.5 6 ± 13.3 13 ± 17.8 10 ± 19.4 10 ± 15.8 19 ± 24.3 13 ± 16.61+2+3 95 ± 7.4 97 ± 1.6 92 ± 10.6 97 ± 2.3 85 ± 12.5 69 ± 16.2 88 ± 13.0 91 ± 6.3 66 ± 24.5 90 ± 7.4

Contralat. Leg lifted

1 90 ± 12.8 93 ± 6 88 ± 18.7 94 ± 4.4 74 ± 29.0 37 ± 28.2 67 ± 28.9 46 ± 30.8 30 ± 21.0 32 ± 25.42 6 ± 10.6 4 ± 5 5 ± 11.7 2 ± 3.3 12 ± 20.4 15 ± 21.3 12 ± 19.1 32 ± 29.1 18 ± 18.6 37 ± 29.63 1 ± 1.5 1 ± 2 1 ± 3.0 1 ± 1.1 3 ± 5.9 10 ± 14.7 10 ± 14.8 11 ± 15.6 27 ± 20.0 17 ± 20.91+2+3 97 ± 2.8 98 ± 2 94 ± 9.6 98 ± 2.3 88 ± 9.5 63 ± 23.1 89 ± 8.1 89 ± 8.1 76 ± 16.8 85 ± 14.0

Increased Sway Amplitude

1 95 ± 2.1 96 ± 3 91 ± 12.5 97 ± 3.0 76 ± 27.2 49 ± 27.3 77 ± 20.1 76 ± 22.6 19 ± 20.3 57 ± 23.02 1 ± 1.7 1 ± 2 2 ± 2.4 1 ± 1.5 8 ± 11.8 11 ± 15.7 9 ± 14.0 13 ± 21.2 45 ± 32.3 20 ± 20.63 1 ± 0.7 1 ± 2 2 ± 7.4 1 ± 1.8 6 ± 12.1 12 ± 18.7 6 ± 8.0 4 ± 4.8 17 ± 21.4 12 ± 16.41+2+3 98 ± 2.0 98 ± 1 95 ± 7.0 98 ± 1.1 89 ± 13.1 72 ± 14.9 91 ± 7.2 93 ± 4.2 81 ± 14.9 89 ± 7.4

M.van

Diest

etal./

Journalof

Biomechanics

47(2014)

2925–2932

2929

showed only modest and poor validity respectively (Clark et al.,2013).

Differences between Vicon and Kinect with respect to balanceoutcome measures ranged from 0.3% to 64.3% and showed largebetween-outcome measure differences. Especially stdMSA showedlarge differences between systems, possibly because this measurerequires high spatial resolution for accurate computation.‘Increased sway frequency’ and ‘leg lifted’ conditions showed thelargest differences in outcome measures, indicating that para-meters that rely on faster movements and/or on movement ofextremities are better avoided when using the Kinect. Standarddeviations of the outcome measures were relatively large, yetsimilar for both systems under different conditions, thus

indicating high interindividual variability rather than low relia-bility of Kinect measurements. This emphasizes the need foradapting exergames to the individual player and implies thatone should focus on change in parameters, rather than theabsolute value.

There are several limitations in this study. First, the bodymarkers tracked by the devices were not located at exactly thesame spots; the Kinect seems to measure the foot segmentsslightly above the feet and many of the anatomical landmarksrecorded by Kinect are inside the body where no reflectivemarkers can be placed. This limitation could have affected thepercentage of variance of markers explained by patterns identified.Conversely it could be considered part of the results as it providesinsight in the effect of defining markers inside the body onmovement patterns identified. Second, only healthy young adultswere used in the study, while the exergame was developed forelderly. For the purpose of this study however, adults aged 18–60years were considered more suitable because larger variations inmovement speed and amplitude were expected than in the elderlypopulation, thereby exploring limitations of Kinect more broadly.

The findings of the present study have implications for thedesign of exergames for home based balance training in elderlyand for the development of algorithms for quantification ofbalance ability during exergaming. Accurate extraction of move-ment patterns from Kinect motion capture data unlocks theunique potential of exergames to quantify balance ability whileperforming complex tasks, thereby assessing balance ability in anenvironment akin to daily life, in which balance control is alwaystask embedded. Additionally quantification of balance abilityallows for adapting exergames to the individual capacities of theuser, thereby increasing training efficiency. To optimally benefitfrom adjusting exergames to the individual user during gameplay,outcome measures should be computed online, that is duringexergaming, rather than off-line as was done in the current study.Recent developments in sensor technology, machine learning andcheap processing power do provide the necessary tools for this.The development of algorithms for balance quantification shouldfocus on finding parameters that are sensitive to changes inbalance ability, thereby enabling (tele)monitoring of progressionor deterioration of balance ability. Exergames could thus in thenear future already serve as an early warning system for increasedfall risk. To determine which features define a good balance, recentadvances in machine learning with respect to data driven feature

Fig. 3. Typical example of marker trajectories of Kinect anatomical landmarks (leftstick figure) and Vicon reflective markers (right figure) of the same subject andtrial. Note that the trajectories of the lower extremities show differences inamplitude and direction between both systems and that the trajectories of handmarkers are more irregular and less smooth in Kinect.

Fig. 4. Variance captured (varc) on the first three PCs by the individual markers of the combined (Vicon and Kinect) PCA model. Black and gray bars indicate individual Viconand Kinect markers respectively. Error bars represent interindividual variability (standard deviation).

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–29322930

extraction and analysis of the time-varying structure of bodysegment movements raise high expectations.

5. Conclusion

Kinect identifies movement patterns in whole body motioncapture data with high accuracy yet the variance in hand and footsegment movements that is explained by the identified patterns isunderestimated and overestimated respectively. Comparison ofbalance outcome measures computed using Vicon and Kinectmotion capture data showed high inter-outcome measure varia-tion. These findings contribute to the development of sensortechnology and algorithms for quantification of balance abilityfor exergames for home based balance training for elderly.

Conflict of interest

The authors declare that they have no competing interests.

Acknowledgments

This work was supported by SPRINT Research Center, theProvince of Drenthe, the Municipality of Assen, the European Fundfor Regional Development and the Dutch Ministry of EconomicAffairs, Peaks in the Delta.

References

Ashworth, N., Chad, K., Harrison, E., Reeder, B., Marshall, S., 2005. Home versuscenter based physical activity programs in older adults. Cochrane DatabaseSyst. Rev., CD004017, http://dxdoi.org/10.1002/14651858.CD004017.pub2.

Bonnechère, B., Jansen, B., Salvia, P., Bouzahouene, H., Omelina, L., Cornelis, J.,Rooze, M., Jan, S.V.S., 2012. What are the current limits of the KinectTM sensor?In: Proceedings of the 9th International Conference on Disability, Virtual Reality& Associated Technologies. Laval, France, pp. 287–294.

Bonnechère, B., Jansen, B., Salvia, P., Bouzahouene, H., Omelina, L., Moiseev, F.,Sholukha, V., Cornelis, J., Rooze, M., Jan, S.V.S., Van Sint Jan, S., 2013. Validityand reliability of the Kinect within functional assessment activities: compar-ison with standard stereophotogrammetry. Gait Posture 39, 593–598.

Clark, R. a, Pua, Y.-H., Fortin, K., Ritchie, C., Webster, K.E., Denehy, L., Bryant, A.L.,2012. Validity of the Microsoft Kinect for assessment of postural control. GaitPosture 36, 372–377.

Clark, R.A., Bower, K.J., Mentiplay, B.F., Paterson, K., Pua, Y.-H., 2013. Concurrentvalidity of the Microsoft Kinect for assessment of spatiotemporal gait variables.J. Biomech. 46, 2722–2725.

Daffertshofer, A., Lamoth, C.J.C., Meijer, O.G., Beek, P.J., 2004. PCA in studyingcoordination and variability: a tutorial. Clin. Biomech. 19, 415–428.

Delbaere, K., Close, J.C.T., Heim, J., Sachdev, P.S., Brodaty, H., Slavin, M.J., Kochan, N.A.,Lord, S.R., 2010. A multifactorial approach to understanding fall risk in olderpeople. J. Am. Geriatr. Soc. 58, 1679–1685.

Huang, M.-H., Brown, S.H., 2013. Age differences in the control of postural stabilityduring reaching tasks. Gait Posture 38, 837–842.

Kang, H.G., Costa, M.D., Priplata, A. a, Starobinets, O. V, Goldberger, A.L., Peng, C.-K.,Kiely, D.K., Cupples, L.A., Lipsitz, L.A., 2009. Frailty and the degradation ofcomplex balance dynamics during a dual-task protocol. J. Gerontol. A Biol. Sci.Med. Sci. 64, 1304–1311.

Kannus, P., Sievänen, H., Palvanen, M., Järvinen, T., Parkkari, J., 2005. Prevention offalls and consequent injuries in elderly people. Lancet 366, 1885–1893.

Kaptein, R.G., Daffertshofer, A., 2012. UPMOVE: Unraveling Patterns of HumanMovement (version 0.2a)[software].

Khoshelham, K., Elberink, S.O., 2012. Accuracy and resolution of Kinect depth datafor indoor mapping applications. Sensors 12, 1437–1454.

Lamoth, C.J.C., Beek, P.J., Meijer, O.G., 2002. Pelvis-thorax coordination in thetransverse plane during gait. Gait Posture 16, 101–114.

Lamoth, C.J.C., Caljouw, S.R., Postema, K., 2011. Active video gaming to improvebalance in the elderly. Stud. Health Technol. Inform. 167, 159–164.

Lamoth, C.J.C., van Heuvelen, M.J.G., 2011. Sports activities are reflected in the localstability and regularity of body sway: older ice-skaters have better posturalcontrol than inactive elderly. Gait Posture 35, 489–493.

Menna, F., Remondino, F., Battisti, R., Nocerino, E., 2011. Geometric investigation of agaming active device. In: Proceedings of SPIE – The International Society forOptical Engineering. Munich, Germany, p. 8085.

Obdrzálek, S., Kurillo, G., Ofli, F., Bajcsy, R., Seto, E., Jimison, H., Pavel, M., 2012.Accuracy and robustness of Kinect pose estimation in the context of coaching ofelderly population. In: Proceedings of Annual Interantional Conference on IEEEEngineering in Medicine and Biology Society San Diego, pp. 1188–93.

Pollock, A.S., Durward, B.R., Rowe, P.J., Paul, J.P., 2000. What is balance? Clin.Rehabil. 14, 402–406.

Riva, F., Toebes, M.J.P., Pijnappels, M., Stagni, R., van Dieën, J.H., 2013. Estimating fallrisk with inertial sensors using gait stability measures that do not requirestep detection. Gait Posture 38, 170–174.

Rubenstein, L.Z., 2006. Falls in older people: epidemiology, risk factors andstrategies for prevention. Age Ageing 35 (Suppl. 2), Sii37–Sii41.

Fig. 5. Balance outcome measures computed from Vicon and Kinect PCA representations based on the three main components. Vicon and Kinect are indicated by black andgray bars respectively. Error bars represent interindividual variability (standard deviation). Outcomes shown in the upper panels do not reflect actual sway size due tonormalization of signals prior to PCA.

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–2932 2931

Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A.,Blake, A., 2011. Real-time human pose recognition in parts from single depthimages. In: Proceedings of 24th IEEE Conference on Computer Vision andPattern Recognition (CVPR). Colorado Springs, pp. 1297–1304.

Van Diest, M., Lamoth, C.J.C., Stegenga, J., Verkerke, G.J., Postema, K., 2013.Exergaming for balance training of elderly: state of the art and futuredevelopments. J. Neuroeng. Rehabil. 10, 101.

Vicon Motion Systems Ltd., 2013. Vicon Motion Systems Ltd – T-series. URL: ⟨http://www.vicon.com/System/TSeries⟩ (accessed 11.04.13.).

Wu, G., Keyes, L., Callas, P., Ren, X., Bookchin, B., 2010. Comparison of telecommu-nication, community, and home-based Tai Chi exercise programs on complianceand effectiveness in elders at risk for falls. Arch. Phys. Med. Rehabil. 91, 849–856.

Zhou, H., Hu, H., 2008. Human motion tracking for rehabilitation—a survey. Biomed.Signal Process. Control 3, 1–18.

M. van Diest et al. / Journal of Biomechanics 47 (2014) 2925–29322932