Research ArticleA Novel Human-Like Control Framework for Mobile MedicalService Robot

Xin Zhang 1 Jiehao Li 2 Wen Qi 2 Xuanyi Zhou 2 Yingbai Hu 3 Hao Quan 2

and Zhen Wang 2

1Soochow University Suzhou 215000 China2Department of Electronics Information and Bioengineering Politecnico di Milano Milano 20133 Italy3Department of Informatics Technical University of Munich Munich 85748 Germany

Correspondence should be addressed to Wen Qi wenqipolimiit

Received 17 June 2020 Revised 6 August 2020 Accepted 3 October 2020 Published 27 October 2020

Academic Editor Yanan Li

Copyright copy 2020 Xin Zhang et al -is is an open access article distributed under the Creative Commons Attribution Licensewhich permits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Recently as a highly infectious disease of novel coronavirus (COVID-19) has swept the globe more andmore patients need to be isolatedin the rooms of the hospitals so how to deliver themeals or drugs to these infectious patients is the urgentwork It is a reliable and effectivemethod to transportmedical supplies ormeals to patients using robots but how to teach the robot to the destination and to enter the doorlike a human is an exciting task In this paper a novel human-like control framework for the mobile medical service robot is consideredwhere a Kinect sensor is used to manage human activity recognition to generate a designed teaching trajectory Meanwhile the learningtechnique of dynamicmovement primitives (DMP)with theGaussianmixturemodel (GMM) is applied to transfer the skill fromhumansto robots A neural-based model predictive tracking controller is implemented to follow the teaching trajectory Finally some dem-onstrations are carried out in a hospital room to illustrate the superiority and effectiveness of the developed framework

1 Introduction

In the past few months a novel coronavirus has resulted inan ongoing outbreak of viral pneumonia in the world [1 2]More than 98000 people are known to be infected and over3400 deaths have been reported [3] -e symptoms of thesepatients include high temperature cough shortness ofbreath and headache and it is a highly infectious disease [4]More and more patients need to be isolated in independentrooms of the hospitals How to transport meals and med-icines to patients and reduce the infection for medical staffsimultaneously is a hot issue It is a safe and useful method totransport medical supplies or meals to patients using robots-erefore this paper focuses on how to control the robot tothe destination and to enter the door like a human at thesame time

Human-robot skill transfer is currently one of the sig-nificant topics in human-assisted systems Under differentexternal environmental conditions especially in the hospitalroom the main challenge for assisted medical rescue robots

is how to transport the medical supplies [5] safely -e robotsystem acquires the learning ability through the cognitiveknowledge transmitted by humans [6] At the same time bylearning the approach of human-in-the-loop the perfor-mance of the robot is improved making the system moreintelligent [7 8]

Human-robot interaction (HRI) is dedicated to thedevelopment of more intelligent and anthropomorphicrobots which is a subregion of human-computer interactionthat researches the interaction between man and robot[9 10] In some hazardous areas to minimize staff partic-ipation robots are required to perform operations HRItechnology can be used to perform remote operations onrobots efficiently [11]-erefore not only is it widely used inthe research of robotic systems but it also plays an essentialrole in the implementation of robotic systems Furthermorein some particular activities human-computer interactionplays an important role For example remote operation ofhuman-machine interaction for medical robots is the safestmethod in medical processes allowing the user to interact

HindawiComplexityVolume 2020 Article ID 2905841 11 pageshttpsdoiorg10115520202905841

with the robot through tactile signals for coordinatedfeedback [12] In order to improve surgeon performance anexperimental approach to characterize the human-robot-assisted surgery system is discussed In vision-based roboticcontrol the visual impedance scheme was used to imple-ment dynamic control at the activity level To complete theintegration of the vision servosystem and the conventionalservosystem Su et al [13] proposed the visual impedancescheme in the control scheme namely the characteristics ofthe image were applied to the impedance equation How-ever the flaw is that this research is still limited in terms of aself-adaptive decision Besides two themes are widely usedin HRI [14] one of which was the human-computer in-teraction interface which was represented by the operationof the keyboard and the other was the human-machineinteraction represented by a touch screen operation-erefore this document focuses primarily on human-robotinteraction with demonstration teaching

At the same time another advanced technology forhuman-machine operation is the skill transmission throughdemonstration teaching [15 16] and during this process themotion control strategies and the generalized output [17]were learned to transfer the motor skills to the robot by themovement of the demonstrator Behavior perception be-havior representation and behavior reproduction are threeprocesses that are imitating the process of learning Somespecial feature methods can be used to program the learningprocess such as Dynamic Motion Primitives (DMP) andHidden Markov Models (HMM) [18 19] -e stochasticmodels like the Gaussian mixture model (GMM) havesome powerful capability to code and process noise so thatthey have the ability to handle high-dimensional problemsmore effectively -e trajectory-level representation that ison the basis of the probability model uses the characteristicsof the stochastic model to model the motion trajectorythereby solving the problem more efficiently Motion tra-jectory reproduction and motion control belong to thecategory of behavior reproduction where trajectory re-production is a process of transmitting coded data It ismainly transmitted for some techniques of regression suchas Gaussian Process Regression (GPR) and Gaussian HybridRegression (GMR) and the feedback variable is a playbackbehavior learned from the presenter In other words it is ageneralized output that maps to robot motion control formotion reproduction [20ndash22]

-e most widely used technologies for scientific researchwidely used in statistical models of human behavior aremachine learning (ML) and deep learning (DL) technolo-gies In the previous work in order to compare the rec-ognition rates for identifying human activities differentcombined sensors were adopted and a deep convolutionalneural network (DCNN) was applied to the HAR system[23] In addition the ML method is used to enhance HARrsquosadaptive identification and real-time monitoring system toovercome time-consuming strategies -ese classifiers havebeen proven to recognize more human actions in dynamicsituations thereby improving accuracy enhancing robust-ness and achieving time-saving effects [13 24] Moreover ahybrid hierarchical classification algorithm combining DL

and the method that is based on the threshold is proposed todifferentiate complex events in order to calculate quicklyAlthough our previous research has proposed many effectiveframeworks most acceptable results have been achievedwith limited assumptions and conditions that cannot meetthe complex environment in the medical room

In this article a novel human-like control framework formobile medical service robots is presented where Kinectsensors are used to achieve human activity recognition togenerate a designed teaching trajectory At the same timethe learning technique of dynamic movement primitives(DMP) with the Gaussian mixture model (GMM) is appliedto transfer the skill from human to machine In addition aneural-enhanced model predictive tracking controller isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to accountfor the effectiveness and superiority of the framework -emain contribution of this paper is as described below

(1) In order to control the mobile service robot totransport medical supplies or meals to the patientwho suffers from the new coronavirus a novel hu-man-like control framework is discussed

(2) Kinect sensors technology is applied to the skilltransfer of the mobile medical service robot forcollecting the movement points and the method ofDMP with GMM is used to congress the points

(3) To efficiently track the teaching trajectory underuncertain disturbances a neural network enhancedpredictive tracking control scheme is presentedAlso some demonstrations are carried out to illus-trate the developed structure

-e structure of this essay is as described below Section2 describes the overview of human-like control Somedemonstrations are discussed in Section 3 Finally theconclusion and future point are summarized in Section 4

2 Methodology

21 Human Activity Recognition Using Kinect SensorsHuman activity recognition technology can be applied tofollow the position of an operator using a Kinect device Asshown in Figure 1 the operator selects the depth message onthe Kinect sensor and the color vision is collected in theKinect depth image [25] To effectively construct themovement information it is combined with the color imageand the depth image in such a way that the origin is locatedat the center of the depth camera -us we assume that thecoordination system in camera space follows a right-handconvention [26]

In this case M(k) (x(k) y(k) z(k)) represents thethree coordinates of the sequence of frame and we defineD(i j) and ϱ(i j) as the depth image points and color imagerespectively -en we can obtain the Bayes rule to evaluatethe probability of M(t|ϱ)

M(t|ϱ) (M(t|ϱ)M(t))

M(ϱ) (1)

2 Complexity

where M(ϱ) and M(ϱ|t) denote the exercise dataset andprior probabilities of skin color respectively

It is on account of self-occlusion or lacking of jointinformation at some stages that we need to incorporateother functions that can offer data about human shape soas to upgrade the precision of the classifier [6 27] Weadopt orthogonal Cartesian planes on the depth map toobtain the positive 2D image and the profile obtained-en by converting Cartesian coordinates to polar co-ordinates the silhouette of a person can be efficientlyprocessed

Ri

xi minus xj1113872 11138732

minus yi minus yj1113872 11138732

1113970

θi tanminus 1yi minus yj

xi minus xj

(2)

where (xi yi) and (Ri θi) represent the coordinates of theoutline of the human body and Radius and angle in polarcoordinates respectively Besides (xj yj) is the centercoordinate of the human contour -e overall order of eachactivity with front and side views is averaged and the av-erage Smean from the initial frame to the final frame is definedas follows

Smean 1T

1113944

T

t1I(x y z t) (3)

22 Trajectory Generation via DMP with GMM After theKinect device has collected the path information teaching byhuman demonstration the mobile robot needs to learn thecreated trajectory [28]-e teaching trajectory is determinedby dynamic movement primitive technology (DMP) andthen rebuilt by the Gaussian mixture model (GMM) togeneralize the movement trajectory

Ψ Φl( 1113857 1113944

ξ

ξ1ΨξΨ Φl|ξ( 1113857 (4)

where Ψ(ξ) is the prior probability Ψ(Φlξ) is the condi-tional probability distribution which follows the Gaussiandistribution and ξ is the number of Gaussian modeldistribution

-erefore by using a Gaussian mixture model the entireteaching dataset can be expressed as follows

Ψ Φl|ξ( 1113857 N Φlψξ 1113944ξ

⎛⎝ ⎞⎠

1

(2π)E

1113936ξ1113868111386811138681113868

1113868111386811138681113868

1113969 eminus05 Φlminusψξ( )

T1113936

minus 1ξ Φlminusψξ( )

(5)

where E is the dimension of the GMR and determined byπξ ψξ 1113936ξ1113966 1113967

-e Gaussian distribution can be addressed as

Φfξ|Φsξ sim N ψfξprime 1113944prime

fξ

⎛⎝ ⎞⎠

ψfξprime ψfξ + 1113944fsξ

1113944

minus1

sξΦsξ minus ψsξ1113872 1113873

(6)

where ψξ ψfξ ψsξ1113966 1113967 and 1113936ξ 1113936sfξ1113936fsξ1113966 1113967-erefore the average ψf

prime and variance 1113936fprime of GMR of

the number of ξ Gaussian components can be evaluated as

ψfprime 1113944

M

ξ1ηξψfξprime 1113944

prime

f

1113944M

k1η2ξ 1113944prime

fξ ζξ

G Φs|ξ( 1113857

1113936Mξ1 G Φs|i( 1113857

(7)

where ψfprime is the estimation variable and Φs is the corre-

sponding space parameter (ΦfprimeΦS) is the generalized

Feature detection

3D joints detection 3D joints normalization

Kinect sensor

Post analysis

Posture classification Posture detection

Posture classification

Activity recognition

Human moving point (m)

5

45

4

35

3

25

20ndash02

ndash04ndash06

ndash08ndash1

ndash12ndash14

ndash2ndash15ndash1ndash05

0051

Figure 1 -e overview of human activity recognition using Kinect sensors

Complexity 3

points which produces a smooth movement trajectoryunder the covariance constraint 1113936f

prime-e GMM model including a multidimensional prob-

ability density function is composed of multiple Gaussianprobability density functions -e Gaussian model is onlyrelated to two parameters the mean and variance As we allknow different learning mechanisms can directly affect theaccuracy convergence and stability of the model Assumethat an M-order GMM is weighted and summed with M

Gaussian probability density functions

P(X|λ) 1113944M

n1ΦnQn(X) n 1 2 M (8)

where X denotes D dimensional random vector and M is onbehalf of the order of the model while Phin represents theweight of each Gaussian component satisfyingsumM

n1 Phin 1 Furthermore mathcalQn(X) is eachGaussian component which is a Gaussian probabilitydensity function of D dimension and can be expressed asfollows

Qn(X) 1

(2π)(D2)

1113936n

11138681113868111386811138681113868111386811138681113868(12)

1113969

lowast exp minus05 x minus μn( 1113857T

1113944

M

n1X minus μn( 1113857

⎧⎨

⎩

⎫⎬

⎭

(9)

where mun is on behalf of the mean vector and sumn denotesthe covariance matrix -en GMM can be expressed by thethree parameters of mean vector covariance matrix andmixed weight -erefore the GMM can be described as

λ μn 1113944n

Φn n 1 2 M⎧⎨

⎩ (10)

23 Neural Approximation In order to efficiently transferthe trajectory teaching by human demonstration it isnecessary to control the uncertain disturbance in the tra-jectory tracking process for the mobile robot [29ndash31] Toovercome the hidden safety hazards in scooter operation[32ndash34] a control scheme based on RBFNN was imple-mented for the elderly walker system -is scheme hascertain disturbances and unknown dynamic characteristicsDesign a constant smooth function G(K) Rq⟶ R toconnect the approximation capability where the RBFNNcontrol scheme is applied for evaluating the dynamics ofuncertainty such as load friction and mechanism structure[35ndash37]

Gnn Kin( 1113857 JTΘ Kin( 1113857 (11)

where Kin isin Ω sub Rq represents the input of RBFNNΘ(Kin) and Θi(Kin) are the activation function dependingon Gaussian function respectively andJ [ξ1 ξ2 ξm] isin Rm represents the weight in the hid-den layer

Θi Kin( 1113857 expminus Kin minus u

Ti1113872 1113873 Kin minus ui( 1113857

η2i⎡⎢⎣ ⎤⎥⎦ (12)

where i 1 2 m ui [ui1 ui2 uiq]T isin Rq and ηi isthe variance

-en Θ(Kin) can be defined as

Θ Kin( 1113857

leϖ (13)

where ϖ is a positive constant-en we have

Gnn Kin( 1113857 JlowastTΘ Kin( 1113857 + ε (14)

where Jlowast is the desired weight subjected to ΦKinsub Rq and

εle τHence we have

Jlowast

argminKinisinRq sup Gnn Kin( 1113857 minus JTΘ Kin( 1113857

11138681113868111386811138681113868

111386811138681113868111386811138681113882 1113883 (15)

whereΘ(Kin) denotes the activation function depending onthe Gaussian function

24 Neural-Based Model Predictive Tracking ControlHuman motion points are collected by the Kinect sensorand then the generated trajectory can be obtained with themethod of DMP and GMM [18 38 39] Finally the next taskof the mobile robot is to follow the teaching trajectory [40]

Figure 2 exhibits the kinematic model of the mobilemedical service robot where (xr yr) and (xf yf) denotethe coordinate of the rear axis and front axis respectively P

is the circle center and R denotes the steering radius L andM represent the wheel trajectory vr and vf denote the rearspeed and front speed respectively δf and φ denote thesteering angle and yaw angle respectively

-e trajectory tracking control of the mobile scroll wheelsystem can be represented as

_xr

_yr

φ

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

cosφ

sinφ

0

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦vr +

0

0

1

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦ω (16)

where ξs [xr yrφ]T is the system state and uS [vrω]T isthe control state

Hence the dynamic model of the mobile robot can beaddressed as

mxr

myr

Φc + Fa1 cos δf + Fa2 cos δf + Fa3 + Fa4

myr

minusmxr

Φc + Fb1 cos δf + Fb2 cos δf + Fb3 + Fb4

Izφ

A Fb1 cos δf + Fb2 cos δf1113872 1113873

minus B Fb3 + Fb4( 1113857M minusFa1 cos δf + Fa2 cos δf minus Fa3 + Fa41113872 1113873

(17)

where Fa1 Fa2 Fa3 and Fa4 are the wheel force of left frontright front left rear and right front respectivelyΦc denotesthe center yaw velocity and IZ represents the rotationalinertia

4 Complexity

F b3 F a3

F a4Fb4

F b2

M

B

Y

O X

L

G

(Xr Yr)

(Xf Yf)

A

F b1

Fa1

F a2ω c

δf

δf

Figure 2 -e kinematic model and the dynamic model of the mobile medical service robot

Movement pointsequations (1)ndash(3)

Trajectory generationDMP with GMM

equations (4)ndash(10)

Trackingcontrollerequations(19)ndash(23)

Neuralapproximation

equations (11)ndash(15)

Uncertain physicalinteraction

Service robot

xd yd φd xe ye φe

xr yr φ

Fd

Figure 3 Block diagram of neural fuzzy-based tracking control

Kinect sensors

Service robot

Medical room

Control center

SigamaSurgical robot KUKATracking trajectory

(service robot)

Tracking trajectory(human)

Figure 4 -e overview scenario of the medical room to transport the meals for patients

Complexity 5

201510

50

ndash5ndash10ndash15

1 09 08 07 06 05 04 03 02 01 0

(a)

1

08

06

04

02

01 09 08 07 06 05 04 03 02 01 0

(b)

3

2

1

0 321

Y

X

Generated trajectoryTeaching trajectory

(c)

Figure 5 -e regression result of teaching by demonstration using DMP with GMM (demonstration 1) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

10

5

0

ndash5

1 08 06 04 02 0

(a)

1

08

06

04

02

01 08 06 04 02 0

(b)

12

9

6

3

0 3 6 9

Y

X

Obstacle

(c)

Figure 6 -e regression result of teaching by demonstration using DMP with GMM (demonstration 2) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

6 Complexity

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

where M(ϱ) and M(ϱ|t) denote the exercise dataset andprior probabilities of skin color respectively

It is on account of self-occlusion or lacking of jointinformation at some stages that we need to incorporateother functions that can offer data about human shape soas to upgrade the precision of the classifier [6 27] Weadopt orthogonal Cartesian planes on the depth map toobtain the positive 2D image and the profile obtained-en by converting Cartesian coordinates to polar co-ordinates the silhouette of a person can be efficientlyprocessed

Ri

xi minus xj1113872 11138732

minus yi minus yj1113872 11138732

1113970

θi tanminus 1yi minus yj

xi minus xj

(2)

where (xi yi) and (Ri θi) represent the coordinates of theoutline of the human body and Radius and angle in polarcoordinates respectively Besides (xj yj) is the centercoordinate of the human contour -e overall order of eachactivity with front and side views is averaged and the av-erage Smean from the initial frame to the final frame is definedas follows

Smean 1T

1113944

T

t1I(x y z t) (3)

22 Trajectory Generation via DMP with GMM After theKinect device has collected the path information teaching byhuman demonstration the mobile robot needs to learn thecreated trajectory [28]-e teaching trajectory is determinedby dynamic movement primitive technology (DMP) andthen rebuilt by the Gaussian mixture model (GMM) togeneralize the movement trajectory

Ψ Φl( 1113857 1113944

ξ

ξ1ΨξΨ Φl|ξ( 1113857 (4)

where Ψ(ξ) is the prior probability Ψ(Φlξ) is the condi-tional probability distribution which follows the Gaussiandistribution and ξ is the number of Gaussian modeldistribution

-erefore by using a Gaussian mixture model the entireteaching dataset can be expressed as follows

Ψ Φl|ξ( 1113857 N Φlψξ 1113944ξ

⎛⎝ ⎞⎠

1

(2π)E

1113936ξ1113868111386811138681113868

1113868111386811138681113868

1113969 eminus05 Φlminusψξ( )

T1113936

minus 1ξ Φlminusψξ( )

(5)

where E is the dimension of the GMR and determined byπξ ψξ 1113936ξ1113966 1113967

-e Gaussian distribution can be addressed as

Φfξ|Φsξ sim N ψfξprime 1113944prime

fξ

⎛⎝ ⎞⎠

ψfξprime ψfξ + 1113944fsξ

1113944

minus1

sξΦsξ minus ψsξ1113872 1113873

(6)

where ψξ ψfξ ψsξ1113966 1113967 and 1113936ξ 1113936sfξ1113936fsξ1113966 1113967-erefore the average ψf

prime and variance 1113936fprime of GMR of

the number of ξ Gaussian components can be evaluated as

ψfprime 1113944

M

ξ1ηξψfξprime 1113944

prime

f

1113944M

k1η2ξ 1113944prime

fξ ζξ

G Φs|ξ( 1113857

1113936Mξ1 G Φs|i( 1113857

(7)

where ψfprime is the estimation variable and Φs is the corre-

sponding space parameter (ΦfprimeΦS) is the generalized

Feature detection

3D joints detection 3D joints normalization

Kinect sensor

Post analysis

Posture classification Posture detection

Posture classification

Activity recognition

Human moving point (m)

5

45

4

35

3

25

20ndash02

ndash04ndash06

ndash08ndash1

ndash12ndash14

ndash2ndash15ndash1ndash05

0051

Figure 1 -e overview of human activity recognition using Kinect sensors

Complexity 3

points which produces a smooth movement trajectoryunder the covariance constraint 1113936f

prime-e GMM model including a multidimensional prob-

ability density function is composed of multiple Gaussianprobability density functions -e Gaussian model is onlyrelated to two parameters the mean and variance As we allknow different learning mechanisms can directly affect theaccuracy convergence and stability of the model Assumethat an M-order GMM is weighted and summed with M

Gaussian probability density functions

P(X|λ) 1113944M

n1ΦnQn(X) n 1 2 M (8)

where X denotes D dimensional random vector and M is onbehalf of the order of the model while Phin represents theweight of each Gaussian component satisfyingsumM

n1 Phin 1 Furthermore mathcalQn(X) is eachGaussian component which is a Gaussian probabilitydensity function of D dimension and can be expressed asfollows

Qn(X) 1

(2π)(D2)

1113936n

11138681113868111386811138681113868111386811138681113868(12)

1113969

lowast exp minus05 x minus μn( 1113857T

1113944

M

n1X minus μn( 1113857

⎧⎨

⎩

⎫⎬

⎭

(9)

where mun is on behalf of the mean vector and sumn denotesthe covariance matrix -en GMM can be expressed by thethree parameters of mean vector covariance matrix andmixed weight -erefore the GMM can be described as

λ μn 1113944n

Φn n 1 2 M⎧⎨

⎩ (10)

23 Neural Approximation In order to efficiently transferthe trajectory teaching by human demonstration it isnecessary to control the uncertain disturbance in the tra-jectory tracking process for the mobile robot [29ndash31] Toovercome the hidden safety hazards in scooter operation[32ndash34] a control scheme based on RBFNN was imple-mented for the elderly walker system -is scheme hascertain disturbances and unknown dynamic characteristicsDesign a constant smooth function G(K) Rq⟶ R toconnect the approximation capability where the RBFNNcontrol scheme is applied for evaluating the dynamics ofuncertainty such as load friction and mechanism structure[35ndash37]

Gnn Kin( 1113857 JTΘ Kin( 1113857 (11)

where Kin isin Ω sub Rq represents the input of RBFNNΘ(Kin) and Θi(Kin) are the activation function dependingon Gaussian function respectively andJ [ξ1 ξ2 ξm] isin Rm represents the weight in the hid-den layer

Θi Kin( 1113857 expminus Kin minus u

Ti1113872 1113873 Kin minus ui( 1113857

η2i⎡⎢⎣ ⎤⎥⎦ (12)

where i 1 2 m ui [ui1 ui2 uiq]T isin Rq and ηi isthe variance

-en Θ(Kin) can be defined as

Θ Kin( 1113857

leϖ (13)

where ϖ is a positive constant-en we have

Gnn Kin( 1113857 JlowastTΘ Kin( 1113857 + ε (14)

where Jlowast is the desired weight subjected to ΦKinsub Rq and

εle τHence we have

Jlowast

argminKinisinRq sup Gnn Kin( 1113857 minus JTΘ Kin( 1113857

11138681113868111386811138681113868

111386811138681113868111386811138681113882 1113883 (15)

whereΘ(Kin) denotes the activation function depending onthe Gaussian function

24 Neural-Based Model Predictive Tracking ControlHuman motion points are collected by the Kinect sensorand then the generated trajectory can be obtained with themethod of DMP and GMM [18 38 39] Finally the next taskof the mobile robot is to follow the teaching trajectory [40]

Figure 2 exhibits the kinematic model of the mobilemedical service robot where (xr yr) and (xf yf) denotethe coordinate of the rear axis and front axis respectively P

is the circle center and R denotes the steering radius L andM represent the wheel trajectory vr and vf denote the rearspeed and front speed respectively δf and φ denote thesteering angle and yaw angle respectively

-e trajectory tracking control of the mobile scroll wheelsystem can be represented as

_xr

_yr

φ

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

cosφ

sinφ

0

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦vr +

0

0

1

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦ω (16)

where ξs [xr yrφ]T is the system state and uS [vrω]T isthe control state

Hence the dynamic model of the mobile robot can beaddressed as

mxr

myr

Φc + Fa1 cos δf + Fa2 cos δf + Fa3 + Fa4

myr

minusmxr

Φc + Fb1 cos δf + Fb2 cos δf + Fb3 + Fb4

Izφ

A Fb1 cos δf + Fb2 cos δf1113872 1113873

minus B Fb3 + Fb4( 1113857M minusFa1 cos δf + Fa2 cos δf minus Fa3 + Fa41113872 1113873

(17)

where Fa1 Fa2 Fa3 and Fa4 are the wheel force of left frontright front left rear and right front respectivelyΦc denotesthe center yaw velocity and IZ represents the rotationalinertia

4 Complexity

F b3 F a3

F a4Fb4

F b2

M

B

Y

O X

L

G

(Xr Yr)

(Xf Yf)

A

F b1

Fa1

F a2ω c

δf

δf

Figure 2 -e kinematic model and the dynamic model of the mobile medical service robot

Movement pointsequations (1)ndash(3)

Trajectory generationDMP with GMM

equations (4)ndash(10)

Trackingcontrollerequations(19)ndash(23)

Neuralapproximation

equations (11)ndash(15)

Uncertain physicalinteraction

Service robot

xd yd φd xe ye φe

xr yr φ

Fd

Figure 3 Block diagram of neural fuzzy-based tracking control

Kinect sensors

Service robot

Medical room

Control center

SigamaSurgical robot KUKATracking trajectory

(service robot)

Tracking trajectory(human)

Figure 4 -e overview scenario of the medical room to transport the meals for patients

Complexity 5

201510

50

ndash5ndash10ndash15

1 09 08 07 06 05 04 03 02 01 0

(a)

1

08

06

04

02

01 09 08 07 06 05 04 03 02 01 0

(b)

3

2

1

0 321

Y

X

Generated trajectoryTeaching trajectory

(c)

Figure 5 -e regression result of teaching by demonstration using DMP with GMM (demonstration 1) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

10

5

0

ndash5

1 08 06 04 02 0

(a)

1

08

06

04

02

01 08 06 04 02 0

(b)

12

9

6

3

0 3 6 9

Y

X

Obstacle

(c)

Figure 6 -e regression result of teaching by demonstration using DMP with GMM (demonstration 2) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

6 Complexity

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

points which produces a smooth movement trajectoryunder the covariance constraint 1113936f

prime-e GMM model including a multidimensional prob-

ability density function is composed of multiple Gaussianprobability density functions -e Gaussian model is onlyrelated to two parameters the mean and variance As we allknow different learning mechanisms can directly affect theaccuracy convergence and stability of the model Assumethat an M-order GMM is weighted and summed with M

Gaussian probability density functions

P(X|λ) 1113944M

n1ΦnQn(X) n 1 2 M (8)

where X denotes D dimensional random vector and M is onbehalf of the order of the model while Phin represents theweight of each Gaussian component satisfyingsumM

n1 Phin 1 Furthermore mathcalQn(X) is eachGaussian component which is a Gaussian probabilitydensity function of D dimension and can be expressed asfollows

Qn(X) 1

(2π)(D2)

1113936n

11138681113868111386811138681113868111386811138681113868(12)

1113969

lowast exp minus05 x minus μn( 1113857T

1113944

M

n1X minus μn( 1113857

⎧⎨

⎩

⎫⎬

⎭

(9)

where mun is on behalf of the mean vector and sumn denotesthe covariance matrix -en GMM can be expressed by thethree parameters of mean vector covariance matrix andmixed weight -erefore the GMM can be described as

λ μn 1113944n

Φn n 1 2 M⎧⎨

⎩ (10)

23 Neural Approximation In order to efficiently transferthe trajectory teaching by human demonstration it isnecessary to control the uncertain disturbance in the tra-jectory tracking process for the mobile robot [29ndash31] Toovercome the hidden safety hazards in scooter operation[32ndash34] a control scheme based on RBFNN was imple-mented for the elderly walker system -is scheme hascertain disturbances and unknown dynamic characteristicsDesign a constant smooth function G(K) Rq⟶ R toconnect the approximation capability where the RBFNNcontrol scheme is applied for evaluating the dynamics ofuncertainty such as load friction and mechanism structure[35ndash37]

Gnn Kin( 1113857 JTΘ Kin( 1113857 (11)

where Kin isin Ω sub Rq represents the input of RBFNNΘ(Kin) and Θi(Kin) are the activation function dependingon Gaussian function respectively andJ [ξ1 ξ2 ξm] isin Rm represents the weight in the hid-den layer

Θi Kin( 1113857 expminus Kin minus u

Ti1113872 1113873 Kin minus ui( 1113857

η2i⎡⎢⎣ ⎤⎥⎦ (12)

where i 1 2 m ui [ui1 ui2 uiq]T isin Rq and ηi isthe variance

-en Θ(Kin) can be defined as

Θ Kin( 1113857

leϖ (13)

where ϖ is a positive constant-en we have

Gnn Kin( 1113857 JlowastTΘ Kin( 1113857 + ε (14)

where Jlowast is the desired weight subjected to ΦKinsub Rq and

εle τHence we have

Jlowast

argminKinisinRq sup Gnn Kin( 1113857 minus JTΘ Kin( 1113857

11138681113868111386811138681113868

111386811138681113868111386811138681113882 1113883 (15)

whereΘ(Kin) denotes the activation function depending onthe Gaussian function

24 Neural-Based Model Predictive Tracking ControlHuman motion points are collected by the Kinect sensorand then the generated trajectory can be obtained with themethod of DMP and GMM [18 38 39] Finally the next taskof the mobile robot is to follow the teaching trajectory [40]

Figure 2 exhibits the kinematic model of the mobilemedical service robot where (xr yr) and (xf yf) denotethe coordinate of the rear axis and front axis respectively P

is the circle center and R denotes the steering radius L andM represent the wheel trajectory vr and vf denote the rearspeed and front speed respectively δf and φ denote thesteering angle and yaw angle respectively

-e trajectory tracking control of the mobile scroll wheelsystem can be represented as

_xr

_yr

φ

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

cosφ

sinφ

0

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦vr +

0

0

1

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦ω (16)

where ξs [xr yrφ]T is the system state and uS [vrω]T isthe control state

Hence the dynamic model of the mobile robot can beaddressed as

mxr

myr

Φc + Fa1 cos δf + Fa2 cos δf + Fa3 + Fa4

myr

minusmxr

Φc + Fb1 cos δf + Fb2 cos δf + Fb3 + Fb4

Izφ

A Fb1 cos δf + Fb2 cos δf1113872 1113873

minus B Fb3 + Fb4( 1113857M minusFa1 cos δf + Fa2 cos δf minus Fa3 + Fa41113872 1113873

(17)

where Fa1 Fa2 Fa3 and Fa4 are the wheel force of left frontright front left rear and right front respectivelyΦc denotesthe center yaw velocity and IZ represents the rotationalinertia

4 Complexity

F b3 F a3

F a4Fb4

F b2

M

B

Y

O X

L

G

(Xr Yr)

(Xf Yf)

A

F b1

Fa1

F a2ω c

δf

δf

Figure 2 -e kinematic model and the dynamic model of the mobile medical service robot

Movement pointsequations (1)ndash(3)

Trajectory generationDMP with GMM

equations (4)ndash(10)

Trackingcontrollerequations(19)ndash(23)

Neuralapproximation

equations (11)ndash(15)

Uncertain physicalinteraction

Service robot

xd yd φd xe ye φe

xr yr φ

Fd

Figure 3 Block diagram of neural fuzzy-based tracking control

Kinect sensors

Service robot

Medical room

Control center

SigamaSurgical robot KUKATracking trajectory

(service robot)

Tracking trajectory(human)

Figure 4 -e overview scenario of the medical room to transport the meals for patients

Complexity 5

201510

50

ndash5ndash10ndash15

1 09 08 07 06 05 04 03 02 01 0

(a)

1

08

06

04

02

01 09 08 07 06 05 04 03 02 01 0

(b)

3

2

1

0 321

Y

X

Generated trajectoryTeaching trajectory

(c)

Figure 5 -e regression result of teaching by demonstration using DMP with GMM (demonstration 1) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

10

5

0

ndash5

1 08 06 04 02 0

(a)

1

08

06

04

02

01 08 06 04 02 0

(b)

12

9

6

3

0 3 6 9

Y

X

Obstacle

(c)

Figure 6 -e regression result of teaching by demonstration using DMP with GMM (demonstration 2) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

6 Complexity

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

F b3 F a3

F a4Fb4

F b2

M

B

Y

O X

L

G

(Xr Yr)

(Xf Yf)

A

F b1

Fa1

F a2ω c

δf

δf

Figure 2 -e kinematic model and the dynamic model of the mobile medical service robot

Movement pointsequations (1)ndash(3)

Trajectory generationDMP with GMM

equations (4)ndash(10)

Trackingcontrollerequations(19)ndash(23)

Neuralapproximation

equations (11)ndash(15)

Uncertain physicalinteraction

Service robot

xd yd φd xe ye φe

xr yr φ

Fd

Figure 3 Block diagram of neural fuzzy-based tracking control

Kinect sensors

Service robot

Medical room

Control center

SigamaSurgical robot KUKATracking trajectory

(service robot)

Tracking trajectory(human)

Figure 4 -e overview scenario of the medical room to transport the meals for patients

Complexity 5

201510

50

ndash5ndash10ndash15

1 09 08 07 06 05 04 03 02 01 0

(a)

1

08

06

04

02

01 09 08 07 06 05 04 03 02 01 0

(b)

3

2

1

0 321

Y

X

Generated trajectoryTeaching trajectory

(c)

Figure 5 -e regression result of teaching by demonstration using DMP with GMM (demonstration 1) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

10

5

0

ndash5

1 08 06 04 02 0

(a)

1

08

06

04

02

01 08 06 04 02 0

(b)

12

9

6

3

0 3 6 9

Y

X

Obstacle

(c)

Figure 6 -e regression result of teaching by demonstration using DMP with GMM (demonstration 2) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

6 Complexity

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

201510

50

ndash5ndash10ndash15

1 09 08 07 06 05 04 03 02 01 0

(a)

1

08

06

04

02

01 09 08 07 06 05 04 03 02 01 0

(b)

3

2

1

0 321

Y

X

Generated trajectoryTeaching trajectory

(c)

Figure 5 -e regression result of teaching by demonstration using DMP with GMM (demonstration 1) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

10

5

0

ndash5

1 08 06 04 02 0

(a)

1

08

06

04

02

01 08 06 04 02 0

(b)

12

9

6

3

0 3 6 9

Y

X

Obstacle

(c)

Figure 6 -e regression result of teaching by demonstration using DMP with GMM (demonstration 2) (a) DMP to encode the trajectorypoints (b) Gaussian components of GMM and (c) regression results

6 Complexity

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

Besides we assume the following condition to evaluatethe lateral force of the robot tire [14]

Fb1 ψδFΓδF

Fb2 ψδBΓδB

ψδF β +MΦr

vx

minus δf

ψδB β +MΦr

vx

(18)

where ψδF and ψδB are tire cornering angle ΓδF and ΓδB

denote cornering stiffness and β is the slip angle-e tracking error can be addressed as

_xe _xr minus _xd( 1113857 minusvd sinφd xr minus xd( 1113857 + cosφd v minus vd( 1113857

_ye _yr minus _yd( 1113857 vd cosφd yr minus yd( 1113857 + sinφd v minus vd( 1113857

φ

e φ

minus φ

d( 1113857 vd

L cos2δd

δ minus δd( 1113857 +tan δd

Lv minus vd( 1113857

(19)

-en we discretize the error function as

004002

0ndash002ndash004

0 20 40 60 80 100 120 140 160 0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 1600 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

0 20 40 60 80 100 120 140 160

Y err

or (m

)004002

0ndash002ndash004

X err

or (m

)

Y pos

ition

(m)

X pos

ition

(m)

V rol

lato

r (m

)

3

2

1

0

15

1

05

0

002001

0ndash001ndash002

002001

0ndash001ndash002

Time (s)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)

315

0ndash15

3

4

3

2

2

1

1

0

0

ndash1

ndash1

ndash2

ndash2ndash3

ndash3

Y (m

)

X (m)

ActualDesired

ActualDesired

Mobile rollatorTeaching trajectory

Figure 7 Teaching results of demonstration 1 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

Complexity 7

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

1113957X(n + 1) Hnt1113957X(n) + Knt1113957u(n) (20)

subjected to Hnt

1 0 minusvdT sinφd

0 1 vdT cosφd

0 0 1

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦Knt

T cosφd 0T sinφd 0

(tan δdL)T (vdL cos2δd)T

⎡⎢⎢⎢⎢⎢⎣⎤⎥⎥⎥⎥⎥⎦ and T is the sampling time

In order to reliably and smoothly grasp the desiredtrajectory state errors and control parameters must beconstrained

V(n) 1113944N

l1

1113957XT(n + l|n)Z(n + l) + 1113957u

T(n + l minus 1)F1113957u(n + l minus 1)

(21)

where Z and F are weighting factors NP is the predictionhorizon and Ne is the control horizon -en the actualcontrol variable can be determined as

u(t) u(t minus 1) + Δulowastt (22)

It is on account of considering the safety and stability of therobot [41 42] that it is of necessity to restrict the control limitand control increment Combined with the mobile robotsystem the control constraint can be presented as follows

minus10

minus451113890 1113891le ule

10

451113890 1113891

minus01

minus021113890 1113891leΔUle

01

021113890 1113891

(23)

Based on the overall control scheme the framework ofneural approximation for tracking control using DMP withGMM is shown in Figure 3 -e Kinect sensor detects thehuman movement points and then generates the teachingtrajectory using the technology of DMP with GMM -enthe neural-based model predictive tracking controller iscarried out to realize the path following

3 Results and Discussions

In this section the overview scenario of the medical room totransport the meals for patients is presented in Figure 4

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

0 20 40 60 80 100 120 140

0 20 40 60 80 100 120 140

Tracking trajectoryTeaching trajectory

Mobile rollatorTeaching trajectory

Obstacle

X pos

ition

(m)

X err

or (m

)

Y err

or (m

)

Pitc

h an

gle (

deg)Ro

ll an

gle (

deg)V r

olla

tor (

m)

Time (s)

12

6

0

Y pos

ition

(m) 12

6

0

006

003

0

ndash003

004002

0ndash002

004

0020

ndash002

ndash004

04

02

0

ndash02

003

0

ndash003

12

12

10

10

8

8

6

6

4

4

2

2

0

0ndash2

ndash2

Y (m

)

X (m)

Figure 8 Teaching results of demonstration 2 in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance

8 Complexity

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

-ere are two Kinect sensors (XBOX 360) used in thisdemonstration-e surgical medical robot (LWR4+ KUKAGermany) is used to feed the meals to patients where thehaptic manipulator (SIGMA 7 Force Dimension Switzer-land) is applied to control the KUKA arm remotely -emain purpose of this demonstration is that the developedmobile medical service robot can safely transport the mealsor medicines to the medical bed like a human withoutcollisions

-e Kinect sensor can detect human activity points andgenerate a teaching trajectory based on the method of DMPand GMM -en the mobile robot can follow the teachingtrajectory via human demonstration -e result of thelearning method including the DMP GMM and the re-gression result of teaching trajectory is displayed in Fig-ures 5 and 6 It is noted that there are two demonstrationsconsidered in this section which aims to evaluate theproposed framework for mobile medical service robot inskill transfer via teaching by demonstration

Figure 7 exhibits teaching results of demonstration 1 inx-position y-position x-error y-error robot tracking ve-locity roll angle pitch angle and tracking performance Itcan be concluded that the mobile medical service robot canfollow the teaching trajectory collected by Kinect sensors-e y-position error and x-position error can be constrainedin a reasonable range within plusmn003 meters indicating thatthe mobile robot can avoid the medical devices and sur-geons On the other hand because of the neural-basedpredictive tracking controller the velocity response of themobile robot under uncertain disturbance is smooth Inparticular the roll angle and pitch angle can maintain astable range

In addition to further illustrate the improvement of skilltransfer scheme using multisensors fusion technologydemonstration 2 to avoid obstacles such as medical devicesand medical staff is carried out Figure 8 displays theteaching performance in x-position y-position x-error y-error robot tracking velocity roll angle pitch angle andtracking performance From the tracking performance of x-position and y-position the mobile medical service robotcan efficiently follow the teaching trajectory and avoidobstacles -e x-position error and y-position error also canbe maintained at a high accuracy which is within plusmn006meters in x-position and plusmn003 meters in y-position At thesame time the neural-based predictive controller can con-strain the mobile robot body and the pitch angle and rollangle are within plusmn002 degrees and plusmn003 degreesrespectively

4 Conclusion

In this paper a novel human-like control framework isimplemented to control a mobile service robot using aKinect sensor and DMP with GMM It aims to bridge thehuman activity recognition techniques and assist the mobilemedical service robot and allows the robot to cooperate withthe medical staff -e Kinect sensor is used to detect humanactivities to generate a set of movement points and then theteaching method including dynamic movement primitives

with the Gaussian mixture model can generate the desiredtrajectory To achieve stable tracking a model predictivetracking control scheme based on neural networks isimplemented to follow the teaching trajectory Finally somedemonstrations are carried out in a medical room to validatethe effectiveness and superiority of the developedframework

Human-machine collaborative control based on theInternet of -ings (IoT) is the future research direction Inour lasted work [43] we have successfully used IoT tech-nology to exploit the best action in human-robot interactionfor the surgical KUKA robot Instead of utilizing compliantswivel motion HTC VIVE PRO controllers used as theInternet of -ings technology are adopted to detect thecollision and a virtual force is applied on the elbow of therobot enabling a smooth rotation for human-robot inter-action Future work combined with the IoT technology andmultisensors the concept of the intelligent medical roomwill be considered to strengthen the human-robotcooperation

Data Availability

No data were used to support this study

Conflicts of Interest

-e authors declare that they have no conflicts of interest

Acknowledgments

-is work was supported by the National Key Research andDevelopment Program of China under Grant2019YFC1511401 and the National Natural Science Foun-dation of China under Grant 61103157

References

[1] W-J Guan Z-Y Ni Y Hu et al ldquoClinical characteristics ofcoronavirus disease 2019 in Chinardquo New England Journal ofMedicine vol 382 no 18 2020

[2] F Pan T Ye P Sun et al ldquoTime course of lung changes onchest ct during recovery from 2019 novel coronavirus (covid-19) pneumoniardquo Radiology vol 295 no 3 2020

[3] H Chen J Guo C Wang et al ldquoClinical characteristics andintrauterine vertical transmission potential of covid-19 in-fection in nine pregnant women a retrospective review ofmedical recordsrdquoCe Lancet vol 395 no 10226 pp 809ndash8152020

[4] Y Bai L Yao T Wei et al ldquoPresumed asymptomatic carriertransmission of covid-19rdquo Journal of the American MedicalAssociation vol 323 no 14 pp 1406-1407 2020

[5] H Su C Yang G Ferrigno and E De Momi ldquoImprovedhuman-robot collaborative control of redundant robot forteleoperated minimally invasive surgeryrdquo IEEE Robotics andAutomation Letters vol 4 no 2 pp 1447ndash1453 2019

[6] Z Li B Huang Z Ye M Deng and C Yang ldquoPhysicalhuman-robot interaction of a robotic exoskeleton by ad-mittance controlrdquo IEEE Transactions on Industrial Electronicsvol 65 no 12 pp 9614ndash9624 2018

[7] T Klamt M Kamedula H Karaoguz et al ldquoFlexible disasterresponse of tomorrow final presentation and evaluation of

Complexity 9

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

the centauro systemrdquo IEEE Robotics amp AutomationMagazinevol 26 no 4 pp 59ndash72 2019

[8] J Li J Wang H Peng L Zhang Y Hu and H Su ldquoNeuralfuzzy approximation enhanced autonomous tracking controlof the wheel-legged robot under uncertain physical interac-tionrdquo Neurocomputing vol 410 pp 342ndash353 2020

[9] M Deng Z Li Y Kang C P Chen and X Chu ldquoA learning-based hierarchical control scheme for an exoskeleton robot inhuman-robot cooperative manipulationrdquo IEEE Transactionson Cybernetics vol 50 no 1 pp 112ndash125 2018

[10] X Wu Z Li Z Kan and H Gao ldquoReference trajectoryreshaping optimization and control of robotic exoskeletonsfor human-robot co-manipulationrdquo IEEE Transactions onCybernetics vol 50 no 8 pp 3740ndash3751 2019

[11] T Klamt M Schwarz C Lenz et al ldquoRemote mobile ma-nipulation with the centauro robot full-body telepresence andautonomous operator assistancerdquo Journal of Field Roboticsvol 37 no 5 pp 889ndash919 2019

[12] Z Li F Chen A Bicchi Y Sun and T Fukuda ldquoGuesteditorial neuro-robotics systems sensing cognition learningand controlrdquo IEEE Transactions on Cognitive and Develop-mental Systems vol 11 no 2 pp 145ndash147 2019

[13] H Su W Qi C Yang A Aliverti G Ferrigno andE De Momi ldquoDeep neural network approach in human-likeredundancy optimization for anthropomorphic manipula-torsrdquo IEEE Access vol 7 pp 124207ndash124216 2019

[14] Z G Li Z Ren K Zhao C Deng and Y Feng ldquoHuman-cooperative control design of a walking exoskeleton for bodyweight supportrdquo IEEE Transactions on Industrial Informaticsvol 16 no 5 pp 2985ndash2996 2019

[15] Y Hu X Wu P Geng and Z Li ldquoEvolution strategieslearning with variable impedance control for grasping underuncertaintyrdquo IEEE Transactions on Industrial Electronicsvol 66 no 10 pp 7788ndash7799 2018

[16] XWu and Z Li ldquoCooperative manipulation of wearable dual-arm exoskeletons using force communication between part-nersrdquo IEEE Transactions on Industrial Electronics vol 67no 8 pp 6629ndash6638 2019

[17] H Su C Yang H Mdeihly A Rizzo G Ferrigno andE De Momi ldquoNeural network enhanced robot tool identi-fication and calibration for bilateral teleoperationrdquo IEEEAccess vol 7 pp 122041ndash122051 2019

[18] Z Cao Y Niu and H R Karimi ldquoSliding mode control ofautomotive electronic valve system under weighted try-once-discard protocolrdquo Information Sciences vol 515 pp 324ndash3402020

[19] X Zhao X Wang L Ma and G Zong ldquoFuzzy-approximation-based asymptotic tracking control for a class of uncertainswitched nonlinear systemsrdquo IEEE Transactions on Fuzzy Sys-tems vol 28 no 4 pp 632ndash644 2019

[20] J Li J Wang S Wang et al ldquoParallel structure of six wheel-legged robot trajectory tracking control with heavy payloadunder uncertain physical interactionrdquo Assembly Automationvol 40 no 5 pp 675ndash687 2020

[21] H Su S E Ovur X Zhou W Qi G Ferrigno andE De Momi ldquoDepth vision guided hand gesture recognitionusing electromyographic signalsrdquo Advanced Robotics vol 34no 15 pp 985ndash997 2020

[22] H Su Y Schmirander S E Valderrama et al ldquoAsymmetricbimanual control of dual-arm serial manipulator for robot-assisted minimally invasive surgeriesrdquo Sensors and Materialsvol 32 no 4 p 1223 2020

[23] W Qi H Su C Yang G Ferrigno E De Momi andA Aliverti ldquoA fast and robust deep convolutional neural

networks for complex human activity recognition usingsmartphonerdquo Sensors vol 19 no 17 p 3731 2019

[24] W He T Meng X He and C Sun ldquoIterative learning controlfor a flapping wing micro aerial vehicle under distributeddisturbancesrdquo IEEE Transactions on Cybernetics vol 49 no 4pp 1524ndash1535 2018

[25] Z Li B Huang A Ajoudani C Yang C-Y Su and A BicchildquoAsymmetric bimanual control of dual-arm exoskeletons forhuman-cooperative manipulationsrdquo IEEE Transactions onRobotics vol 34 no 1 pp 264ndash271 2017

[26] Y Hu Z Li G Li P Yuan C Yang and R Song ldquoDevel-opment of sensory-motor fusion-based manipulation andgrasping control for a robotic hand-eye systemrdquo IEEETransactions on Systems Man and Cybernetics Systemsvol 47 no 7 pp 1169ndash1180 2016

[27] Z Liu H R Karimi and J Yu ldquoPassivity-based robust slidingmode synthesis for uncertain delayed stochastic systems viastate observerrdquo Automatica vol 111 Article ID 108596 2020

[28] Q Wei Z Li K Zhao Y Kang and C-Y Su ldquoSynergy-basedcontrol of assistive lower-limb exoskeletons by skill transferrdquoIEEEASME Transactions on Mechatronics vol 25 no 2pp 705ndash715 2019

[29] H Peng J Wang W Shen and D Shi ldquoCooperative attitudecontrol for a wheel-legged robotrdquo Peer-to-Peer Networkingand Applications vol 12 no 6 pp 1741ndash1752 2019

[30] Z Li J Li S Zhao Y Yuan Y Kang and C P ChenldquoAdaptive neural control of a kinematically redundant exo-skeleton robot using brain-machine interfacesrdquo IEEETransactions on Neural Networks and Learning Systemsvol 30 no 12 pp 3558ndash3571 2018

[31] W He and Y Dong ldquoAdaptive fuzzy neural network controlfor a constrained robot using impedance learningrdquo IEEETransactions on Neural Networks and Learning Systemsvol 29 pp 1174ndash1186 2017

[32] X Zhang J Li S E Ovur et al ldquoNovel design and adaptivefuzzy control of a lower-limb elderly rehabilitationrdquo Elec-tronics vol 9 no 2 p 343 2020

[33] L Zhang Z Li and C Yang ldquoAdaptive neural network basedvariable stiffness control of uncertain robotic systems usingdisturbance observerrdquo IEEE Transactions on IndustrialElectronics vol 64 no 3 pp 2236ndash2245 2016

[34] Z Li C Xu Q Wei C Shi and C-Y Su ldquoHuman-inspiredcontrol of dual-arm exoskeleton robots with force and im-pedance adaptationrdquo IEEE Transactions on Systems Man andCybernetics Systems pp 1ndash10 2018

[35] H Su N Enayati L Vantadori A Spinoglio G Ferrigno andE De Momi ldquoOnline human-like redundancy optimizationfor tele-operated anthropomorphic manipulatorsrdquo Interna-tional Journal of Advanced Robotic Systems vol 15 2018

[36] Z Wu H R Karimi and C Dang ldquoA deterministic annealingneural network algorithm for the minimum concave costtransportation problemrdquo IEEE Transactions on Neural Net-works and Learning Systems vol 24 no 7 pp 699ndash708 2019

[37] J Sandoval H Su P Vieyres G Poisson G Ferrigno andE De Momi ldquoCollaborative framework for robot-assistedminimally invasive surgery using a 7-DoF anthropomorphicrobotrdquo Robotics and Autonomous Systems vol 106 pp 95ndash106 2018

[38] J Gong Y Jiang andW XuModel Predictive Control for Self-Driving Vehicles Beijing Institute of Technology Press Bei-jing China 2014

[39] H Ren H R Karimi R Lu and Y Wu ldquoSynchronization ofnetwork systems via aperiodic sampled-data control withconstant delay and application to unmanned ground

10 Complexity

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11

vehiclesrdquo IEEE Transactions on Industrial Electronics vol 67no 6 pp 4980ndash4990 2019

[40] B Xiao X Yang H R Karimi and J Qiu ldquoAsymptotictracking control for a more representative class of uncertainnonlinear systems with mismatched uncertaintiesrdquo IEEETransactions on Industrial Electronics vol 66 no 12pp 9417ndash9427 2019

[41] Z Li C Yang C-Y Su J Deng andW Zhang ldquoVision-basedmodel predictive control for steering of a nonholonomicmobile robotrdquo IEEE Transactions on Control Systems Tech-nology vol 24 no 2 pp 553ndash564 2015

[42] H Peng J Wang W Shen D Shi and Y Huang ldquoCom-pound control for energy management of the hybrid ultra-capacitor-battery electric drive systemsrdquo Energy vol 175pp 309ndash319 2019

[43] H Su S Ertug Ovur Z Li et al ldquoInternet of things (IoT)-based collaborative control of a redundant manipulator forteleoperated minimally invasive surgeriesrdquo in Proceedings ofthe 2020 IEEE International Conference on Robotics andAutomation (ICRA) Paris France September 2020

Complexity 11