Human Behavior Modeling in a Cursor Tracking Game · Human Behavior Modeling in a Cursor Tracking...
Transcript of Human Behavior Modeling in a Cursor Tracking Game · Human Behavior Modeling in a Cursor Tracking...
Human Behavior Modeling in a Cursor Tracking Game
A Thesis Presented
by
Yijie Lu
to
The Department of Mechanical and Industrial Engineering
in partial fulfillment of the requirements
for the degree of
Master of Science
in
Mechanical Engineering
Northeastern University
Boston, Massachusetts
August 2019
ACKNOWLEDGMENTS
I would like to thank my thesis advisor Prof. Rifat Sipahi for his guidance and persistent
help on my research, which greatly helped me to finish this thesis.
I would also like to thank my thesis reader, Prof. Tansel Yucelen, who provided me
valuable advice on improvements to my work.
In addition, I would like to express my deepest appreciation to my parents. I am grateful
for their generous support for me pursuing the Master degree.
At last, I am also thankful for my pets: Fatty, Boss, and PangPang. Their companionship
helped me overcome the obstacles of the study and also brought precious joys to my graduate student
life.
i
TABLE OF CONTENTS
LIST OF FIGURES iv
LIST OF TABLES vi
NOMENCLATURE vii
ABSTRACT viii
1 Introduction 11.1 Human Behavior Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Model Reference Adaptive System . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2.1 System Equations [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 Reference Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Experiments with Human Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.1 Reaction Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.3.2 Control Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.3.3 Collected Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Extended State Observer 122.1 Active Disturbance Rejection Control [2] . . . . . . . . . . . . . . . . . . . . . . 122.2 Formulations of ESO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3 Implementation of ESO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.3.1 Second-Order States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.2 Third-Order System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3.3 Other Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3 Modeling and Validation 233.1 Fit Percentage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.2 Linear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.2.1 Linear Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243.2.2 Validation of the Linear Model . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3 Nonlinear Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.1 Nonlinear Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.3.2 Validation of the Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . 29
ii
4 Prediction of Human Behavior 324.1 Direct Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 Indirect Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.2.1 Data Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.2.2 Prediction of the Linear Model . . . . . . . . . . . . . . . . . . . . . . . . 394.2.3 Prediction of the Nonlinear Model . . . . . . . . . . . . . . . . . . . . . . 42
5 Conclusion 455.1 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
References 48
iii
LIST OF FIGURES
1.1 Human-in-the-loop MRAC System [3] . . . . . . . . . . . . . . . . . . . . . . . . 21.2 The Coordinate System Used in the Aircraft Motion Control (The Picture is Modi-
fied based on [4]) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.3 The Graphical User Interface of Human-involved MRAC Experiment . . . . . . . 91.4 Example of MRAC System Output with Human Subject’s Cursor Input . . . . . . 111.5 Example of MRAC System Output with Unfrozen Game Setting . . . . . . . . . . 11
2.1 The Architecture of Original ADRC System with ESO . . . . . . . . . . . . . . . 122.2 The Architecture of Human Dynamics which Applied with ESO [5] . . . . . . . . 142.3 Plot of Tracking Variable with respect to Time . . . . . . . . . . . . . . . . . . . . 162.4 Phase Plot of z3 with respect to z1 (Subject #9) . . . . . . . . . . . . . . . . . . . 172.5 Phase Plot of z3 with respect to z1 (All Subjects, Second-Order System) . . . . . . 182.6 Comparison between Actual Position x1 and Estimated Position z1 Obtained Based
on the Three-Order System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.7 Phase Plot of z3 with respect to z1 Obtained Based on the Three-Order System . . 202.8 Phase Plot of z3 with respect to z1 (All Subjects, Third-Order System) . . . . . . . 21
3.1 Block Diagram of Human Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . 243.2 Example of Linear Identification Results . . . . . . . . . . . . . . . . . . . . . . . 253.3 Block Diagram of the Structure of the Nonlinear ARX Model [6] . . . . . . . . . . 283.4 The Effect of Applied Filter to Nonlinear Identification Result . . . . . . . . . . . 29
4.1 Command Error Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.2 System Output and Output Error Comparison . . . . . . . . . . . . . . . . . . . . 344.3 Error Comparison between the Linear Model and the Nonlinear Model (Case 1) . . 354.4 Error Comparison between the Linear Model and the Nonlinear Model (Case 2) . . 364.5 Results of System Output with Two Different Input Provider Cases . . . . . . . . . 374.6 Comparison of Control Commands from Linear Identification (From left to right:
Case 1 and Case 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.7 Derivative of Commands and Its +1/-1 Plot (Case 1) . . . . . . . . . . . . . . . . . 394.8 Optimization Result for +1/-1 Plot of Derivatives . . . . . . . . . . . . . . . . . . 404.9 Indirect Prediction Results of Linear Models (Case 1 and 2) . . . . . . . . . . . . . 414.10 Indirect Prediction Results of Linear Models (Case 3 and 4) . . . . . . . . . . . . . 42
iv
4.11 Indirect Prediction Results of Nonlinear Models (Case 1 and 2) . . . . . . . . . . . 434.12 Indirect Prediction Results of Nonlinear Models (Case 3 and 4) . . . . . . . . . . . 43
v
LIST OF TABLES
1.1 Parameters Used in the Nonlinear Dynamical System [7] . . . . . . . . . . . . . . 61.2 Parameters Used in the Neal-Smith Pilot Model [7] . . . . . . . . . . . . . . . . . 71.3 Stability for Different Reference Model Design Methods [1] . . . . . . . . . . . . 8
3.1 Best Linear Fit Result for Each Subject . . . . . . . . . . . . . . . . . . . . . . . 263.2 Best Fit Result for 5, 10, 50 Poles and Zeros . . . . . . . . . . . . . . . . . . . . . 273.3 Best Fit Result for Each Subject as Model Provider . . . . . . . . . . . . . . . . . 303.4 Nonlinear Fit Results for a Given Delay in Timesteps . . . . . . . . . . . . . . . . 303.5 Each Subject’s Total Nonlinear Fit Numbers for a Given Delay in Timesteps . . . . 31
vi
NOMENCLATURE
MRAC Model Reference Adaptive Control
RMDP Rightmost Dominant Pole
RMNP Rightmost Non-Dominant Pole
LQR Linear-Quadratic Regulator
ADRC Active Disturbance Rejection Control
ESO Extended State Observer
vii
ABSTRACT
The growing demand for human-machine system applications makes it more frequent todeal with human-in-the-loop control problems. The Model Reference Adaptive Controller (MRAC)can adaptively handle uncertainties in dynamical systems and can be useful in human-in-the-loopsettings. But once a human becomes a command provider of a control system, human reaction delaycan make the system unstable.
In this thesis, through the experimental data of the human-in-the-loop MRAC system ap-proved by the IRB, the control behaviors of human subjects are investigated. By using the ExtendedState Observer proposed in the Active Disturbance Rejection Control system, this thesis finds that allthe human subjects in the study have a similar control pattern when participating in the experiment,which indicates that there is a potential generalized behavior model for human subjects.
Then, the human-in-the-loop MRAC experimental data were subjected to both linear andnonlinear identification through the System Identification Toolbox provided by Matlab. The resultsshow that only using the modeling tools provided by Matlab is not enough to directly match all thedata, therefore these models cannot be used to predict the control behavior of most of the subjects.However, these models can indirectly predict the human behavior in the experiment, that is, predictthe intention of human as to whether the human will control the cursor up or down in the near future.Throughout the test for different trials, the best predictive model is found to be the nonlinear model,which has an accurate prediction of human intentions in most cases. This promising result showsthat it is possible to utilize this model in the future to prevent the human subject’s excessive controlin the control loop.
viii
Chapter 1
Introduction
1.1 Human Behavior Modeling
In recent years, from the professional machine to the common automobile and aircraft,
a variety of concept systems that do not require real-time supervision and manipulation from the
human operator are emerging one after another. However, due to the immaturity of the current
technology, we still have a long way to go before a completely fully automatic era has arrived.
Therefore, engineers, for now, have to consider the effect of human-in-the-loop when designing an
interactive engineering system. Due to human’s natural imperfections such as the reaction delay,
knowledge level, and decision accuracy, both the performance and stability of the controlled dynam-
ical system will be impacted because of the involvement of the human operator. Therefore, how to
better integrate the control effort of human into the control system so that the human-in-the-loop
system can be more efficient is a common topic investigated by researchers in various fields [8–24].
One optimization method of the human-in-the-loop system is to treat the human operator’s
command input as a disturbance. The human input will be processed through the controller of the
human-in-the-loop system in real-time and then utilized to generate the actual system input [13, 25].
In real life situation, humans inevitably encounter some unpredictable incidents. Taking driving as
an example, if a driver witnesses a sudden deceleration of the front car, an unexpected lane changing
or an obstacle on the road, although the driver may think it is intuitive and easy to response, it is
difficult for the vehicle to predict exactly how the driver may respond even if the vehicle can sense
upcoming incidents. Therefore, another method to optimize the human-in-the-loop system is to
establish the behavior model of the human operator through a large amount of data, so that the
system can predict the operator’s intention in advance and that the corresponding control algorithm
1
CHAPTER 1. INTRODUCTION
can be employed according to the prediction [26–34].
It can be said that human models play a vital role in the reliability of human-in-the-loop
systems. Hence researchers are paying more attention to find suitable human behavior models when
designing human-in-the-loop systems. As early as the last century, to better understand the control
behaviors of pilots in flying, the researchers proposed a variety of simplified pilot models, such
as the Crossover and Precision pilot behavior model [35, 36], and also the Tustin pilot model in
[37]. However, such behavior models are more of a linear model, while the human model in the
real world should contain a certain level of nonlinearity. Therefore, these models tend to be used
to replace the real human only in the system design phase for the sake of convenience. To meet
with the actual control system requirements, researchers have proposed various behavior modeling
methods, including the modified linear model [38–44], fuzzy interface system (FIS) model [45–47],
and the model simulating human’s cognitive process [48, 49].
1.2 Model Reference Adaptive System
Human Dynamics
Reference Command
Reference Model
ControllerUncertainDynamical
System
−
ParameterAdjustment
Mech.
System Error
Outer Loop Inner Loop
Figure 1.1: Human-in-the-loop MRAC System [3]
Adaptive control can be divided into three types: open-loop adaptive control, direct adap-
tive control, and indirect adaptive control [50]. The Model Reference Adaptive Controller (MRAC)
system, here, is an direct adaptive control method. By comparing the output values of the reference
model and uncertain dynamical system or plant, the difference will be calculated, and then the corre-
sponding adjustment will feedback to the controller itself through closed-loop. Figure 1.1 shows the
2
CHAPTER 1. INTRODUCTION
block diagram of the human-in-the-loop MRAC system, given by [3]. The control system and ex-
perimental data obtained via simulations here inspired by this figure. Compared with the traditional
MRAC system, the input to the MRAC system is called the Outer Loop here, which represents the
human itself who is operating the system. Based on the fact that human fully understands the objec-
tives, through a series of observations, reflections and execution processes, etc., the human operator
can use a variety of methods to input control commands to the robotic system in the Inner Loop
of MRAC. In the Inner Loop, there is no difference from the traditional MRAC architecture. The
parameters in the controller will change in real-time according to the output of the system, so the
accuracy of the control results can be guaranteed to the utmost. Here, the mathematical modeling
of the various parts of the Inner Loop comes from [3].
1.2.1 System Equations [1]
The uncertain dynamical system in the MRAC system can be conveniently represented by
the equation shown below
xp(t) = Apxp(t) +BpΛu(t) +Bpδp(xp(t)), xp(0) = xp0 , (1.1)
where xp represents the accessible state vector in uncertain dynamical system. Both Ap and Bp are
assumed to be system matrices in known models, Ap ∈ Rnp×np , Bp ∈ Rnp×m. Λ ∈ Rm×m+ ∩Dm×m is an unknown control effectiveness matrix, u(t) ∈ Rm is the input to the system. δp is an
uncertainty function, here defined as
δp(xp) = WTp σp(xp), xp ∈ Rnp , (1.2)
where Wp ∈ Rs×m stands for a unknown weight matrix, σp : Rnp → Rs stands for a known basis
function, defined as
σp (xp) = [σp1 (xp) , σp2 (xp) , . . . , σps (xp)]T . (1.3)
Here, xc(t) ∈ Rnc is the integrator state and satisfies
xc(t) = Epxp(t)− c(t), xc(0) = xc0 , (1.4)
where Ep ∈ Rne×np is used to select a subset from xp(t), and c(t) is the command generated by
human dynamics in the outer loop. By combining Equation 1.1 and Equation 1.4 together, we have
x(t) = Ax(t) +BΛu(t) +BWTp σp (xp(t)) +Brc(t), x(0) = x0 (1.5)
3
CHAPTER 1. INTRODUCTION
where
A ,
Ap 0np×nc
Ep 0nc×nc
∈ Rn×n
B ,[BT
p , 0Tnc×m
]T ∈ Rn×m
Br ,[0Tnp×nc
,−Inc×nc
]T∈ Rn×nc
and the augmented state vector is given by x(t) , [xTp (t), xTc (t)]T ∈ Rn, and x0 , [xTp0 , xTc0 ]T ∈
Rn,n = Np + nc.
According to the feedback control law, the signal in the inner loop is
u(t) = un(t) + ua(t), (1.6)
where un(t) ∈ Rm is nominal control input, and ua(t) ∈ Rm is adaptive control input. Assume that
the nominal control input is
un(t) = −Kx(t), (1.7)
where K ∈ Rm×n, Ar , A−BK. Substituting, one obtains
x(t) = Arx(t) +Brc(t) +BΛ[ua(t) +WTσ(x(t))], (1.8)
where WT , [Λ−1WTp , (Λ
−1 − Im×m)K] ∈ R(s+n)×m is an unknown weight aggregated matrix.
σ is a known aggregated basis function and defined as
σT(x(t)) , [σTp (xp(t)) , xT(t)] ∈ Rs+n. (1.9)
Assume next that the adaptive control input is in the following form
ua(t) = −WT(t)σ(x(t)), (1.10)
where W (t) ∈ R(s+n)×m is an estimation of W and satisfies the following relationship
˙W (t) = γσ(x(t))eT(t)PB, W (0) = W0, (1.11)
where γ ∈ R+ is called the learning rate. System error is then given by
e(t) , x(t)− xr(t), (1.12)
where xr(t) ∈ Rn is the reference state vector and satisfies the reference system
xr(t) = Arxr(t) +Brc(t), xr(0) = xr0 . (1.13)
4
CHAPTER 1. INTRODUCTION
In addition, P ∈ Rn×n+ ∩ Sn×n is a solution to the Lyapunov equation
0 = ATr P + PAr +R, (1.14)
where R ∈ Rn×n+ ∩ Sn×n. For a given R, there is always a unique P ∈ Rn×n+ ∩ Sn×n.
In order to simulate the human-in-the-loop MRAC system in a real-world situation, here
Mr. Yanchao Wang [1] selects a well-developed Boeing 747 aircraft model from [7] as the uncertain
dynamical system. When the pilot performs a pitching motion by using the joystick, the pitch
angle of the aircraft will be subjected to change. To simplify the simulation, only the longitudinal
movement of the aircraft will be simulated.
Pitch Angle
Center of Gravity
Figure 1.2: The Coordinate System Used in the Aircraft Motion Control (The Picture is Modified
based on [4])
The mathematical model of the aircraft, in this case, can be found in [7], as shown below
for the equation of motion of the aircraft at a height of 40,000 ft and cruising at 0.8 Mach.
x = Apx(t) +Bpu(t) +WTσ(x(t)), x(0) = x0, (1.15)
where x(t) = [u,w, q, θ]′, u and w represent the velocities along the x-axis and y-axis respectively,
while q represents the angular velocities along y-axis. θ is the pitch angle in the Euler coordi-
nate system established by the aircraft fuselage, as shown in Figure 1.2. Here, W is the unknown
weighting matrix. σ(x(t)) = [1, u(t), w(t)]T is the known basis function.
Since the human subjects experiments were performed in Matlab on the PC platform, the
above continuous-time equation needs to be converted into the discrete-time form. By performing
5
CHAPTER 1. INTRODUCTION
the discrete-time conversion on Equation 1.8, the following equation can be easily obtained:
x[n] = x[n− 1] + (Arx[n− 1] +Brc[n]
+BΛ[(−WT[n] +WT)σ(x[n− 1])])∆t.(1.16)
According to the parameters given in [7], the parameters used in the simulation are shown in the
following table. Here simulation sample time was selected as ∆t = 0.005 seconds.
Table 1.1: Parameters Used in the Nonlinear Dynamical System [7]
Ap
−0.003 0.039 0 −0.322
−0.065 −0.319 7.74 0
0.0201 −0.101 −0.429 0
0 0 1 0
Bp
[0.01 −0.18 −1.16 0
]TW
[0.1 0.3 −0.3
]TΛ 1
γ 50
Ep
[0 0 0 1
]
1.2.2 Reference Model
It was shown in [1] that the output of a nonlinear MRAC closed-loop system will be
stable if the reference model meets some LMI conditions. Thus it can be much easier to design a
human-in-the-loop MRAC system by separating the problem into two parts. In the first part, the
control problem is designing a zero-delay dynamical system with uncertainties and nonlinearities.
The second part, however, relates to a linear reference model and human linear model with human
reaction delay. Therefore, the only challenging problem will be to stabilize the linear combination
model in part two.
Since there exist various K that can make Ar = A−BK Hurwitz matrix in the reference
model, which is necessary for a stable control system, then how to find the most appropriate value
of K among those is critical. In order to understand the impact of different K values on human
performance, a variety of methods can be used to design the reference model. In [1], a total of three
6
CHAPTER 1. INTRODUCTION
designing methods for reference model were taken into consideration, namely: Classical Linear-
Quadratic Regulator (LQR), Modified LQR, and Pole Placement.
Here, for the convenience of comparing the results, the Neal-Smith model was used to
represent the behavior of real pilot controlling the pitch angle of the aircraft. The Neal-Smith model
is given by [51]
H(s) = kpTps+ 1
Tzs+ 1e−τs (1.17)
where kp is the model gain. Tp is the lead time constant. Tz is the lag time constant, and τ is the
reaction delay of the pilot itself. These parameters are selected as shown in the below Table.
Table 1.2: Parameters Used in the Neal-Smith Pilot Model [7]
kp 4.5
Tp 1
Tz 7.4
τ 0.5
In brief, Classical LQR cannot choose the real part of the rightmost dominant pole (RMDP).
For Modified LQR, the RMDP cannot easily shift to left of the complex plane. In Pole Placement
method, it was shown that rightmost non-dominant pole (RMNP) can impact the peak time of the
stable reference model.
Since the focus of the experiment is to find the impact of time-domain step response and
the reference model poles positions on human dynamics performance, the pole placement method
is more suitable with the experimental requirements. Thus, the pole placement method was chosen
to design the reference model in this human-in-the-loop MRAC experiment.
By fixing the RMDP at -0.02 and setting the peak value to 1.7, the following pole sets can
be obtained and utilized in the experiment
P1 = −[0.02, 0.31, 2, 2.2, 2.3],
P2 = −[0.02, 0.35, 0.9, 1.3, 1.7],
P3 = −[0.02, 0.5, 0.6, 0.7, 0.8].
Although the peak values of these three pole sets are very close, the peak time is quite different.
The peak time for pole set 1 is 4.004 seconds, while pole set 2 and pole set 3 are 6.014 seconds and
8.176 seconds respectively. See details in [1].
7
CHAPTER 1. INTRODUCTION
Table 1.3: Stability for Different Reference Model Design Methods [1]
Methods RMDP RMNP Stability
Classical LQR
-0.0117 -0.2943 Stable
-0.0117 -0.1907 Stable
-0.0117 -0.1249 Stable
-0.0117 -0.1064 Stable
Modified LQR
-0.0118 -0.1741 Stable
-0.0121 -0.1147 Stable
-0.0123 -0.0642 Stable
-0.0125 -0.0203 Stable
+0.0187 -0.0126 Unstable
Pole Placement
-0.0126 -0.2139 Stable
-0.0126 -0.0817 Stable
+0.0076 -0.0126 Unstable
1.3 Experiments with Human Subjects
The human-in-the-loop experiment described herein with volunteers participated was led
by Mr. Yanchao Wang and approved by the Institutional Review Board (IRB) with IRB#: 18-04-
05, see [1]. In experiments, a total of 16 subjects were recruited, but only 14 of them actually
participated, 11 of whom were male and 3 were female. None of the recruited subjects encounter
health issues and learning disabilities as self-reported on the pre-experiment survey.
1.3.1 Reaction Time
It is obvious that the reaction time varies through different people. Although running
simulation game itself will not require obtaining the subjects’ reaction time, having the reaction
time of subjects has a great effect on the modeling analysis after experiments. The reaction time
test was performed through online website Human Benchmark [52]. In the test, the subject needs to
click the mouse whenever he or she saw the screen’s color was changed. Such a reaction time test
is merely a simple test and does not represent the actual situation. In the real situation, the aircraft
pilot’s tasks are much more complex, which usually cost more times to observe, decide, and execute.
8
CHAPTER 1. INTRODUCTION
However, the experiment is also a simple task, since the subject only needs to move the mouse to
achieve the goal (see next section for details of control game). Therefore, it can be assumed that
the subject in these two experiments has a similar reaction time. According to the obtained reaction
time test data, the average reaction time of all 14 subjects is 0.355 seconds.
1.3.2 Control Game
The human-in-the-loop MRAC system mentioned above shows that human dynamics or
the outer loop should have 2 separate input signal; one is the reference signal, while another one
is the output signal of uncertain dynamical system or plant. As for the output of human dynamics,
there should be only one command signal. Based on these fundamental requirements, an interactive
Matlab game with the following user interface can be designed, as seen in Figure 1.3. The main body
Visual cue
Human mouse input
Fixed setpoint
Output of nonlinear dynamics
Time
Po
siti
on
Red: The human starts the game at time = 3 sec
Figure 1.3: The Graphical User Interface of Human-involved MRAC Experiment
of this interactive game was developed by Prof. Tansel Yucelen (University of South Florida). The
modification including the reference model, graphical user interface, and experiment preparation
was all developed by Mr. Yanchao Wang, see [1].
The objective of this game is to make the blue line as close as possible to the green line
as well as maintain it until the end of the experiment. The human subject needs to use a mouse
to control the vertical slider on the left. Whenever the mouse drags the slider to a different value,
9
CHAPTER 1. INTRODUCTION
the blue line will then have a related movement, namely, move upward or downward. It is worth
mentioning that the slider bar does not control the blue line directly. There exists a nonlinear dy-
namics which represents the ”aircraft dynamics”. The blue line’s value here represents the aircraft’s
pitch angle, while the green line’s value represents the desired pitch angle which the aircraft needs
to reach (Here assuming the reference value of pitch angle is one). The slider bar is an abstracted
aircraft joystick.
During the game, the first three seconds are called the ”frozen time”, the subject will have
three seconds to get ready before the actual starting. Theoretically, the blue cursor will constantly
remain zero for the first three seconds until, when the blue cursor touches the red line, which is the
actual starting line. Then the subject was allowed to use the mouse to achieve the objective. How-
ever, due to the experiment setting problem, the first four subjects’ gaming set allows the subjects to
move the mouse cursor during the first three seconds. This problem was fixed for the subjects No.6
to No.16.
For each subject, three different pole sets for the reference model were utilize. Each was
played for 5 times. However, the order of pole sets in each game group is randomized to make
the subject behave as consistent as possible. Between each game group, there will be a one minute
break. It is expected that each subject’s entire control game playtime is about 12 minutes.
1.3.3 Collected Data
A total of 210 experimental results were obtained from 14 subjects. What can be found
from the data is that although subjects have different ways to handle the control task, not a single
subject can fully guarantee to stabilize the uncertain dynamical system with the help of the MRAC
controller. A sample of the result obtained in the experiments is shown in Figure 1.4, the green line
and blue curve have the same meaning mentioned in the last section, while the red wave represents
the actual input value through the slider. Although the red wave here is presented and shown directly,
only the green line and blue curve are seen by the subjects during the gaming process.
As described in the last section, there exist two initial conditions for 14 subjects’ experi-
ments. For the first four subjects’ experiments, although the slider bar was fixed, the mouse cursor
could still move to the slider bar and may inadvertently apply a small disturbance to the slider. As
can be seen from the collected data, the input commands for the inner loop given by these people
have some initial value in the first few seconds, as shown in Figure 1.5. Hence, in this thesis, the
first four subjects’ recording data are not used for analysis and modeling.
10
CHAPTER 1. INTRODUCTION
Figure 1.4: Example of MRAC System Output with Human Subject’s Cursor Input
Figure 1.5: Example of MRAC System Output with Unfrozen Game Setting
11
Chapter 2
Extended State Observer
2.1 Active Disturbance Rejection Control [2]
TransientProfile
Generator�1𝑏𝑏0
𝑣𝑣 𝑦𝑦𝑣𝑣1
−𝑣𝑣2
Nonlinear Weighted
Sum−
𝜃𝜃1
𝜃𝜃2𝑢𝑢0
−
𝑢𝑢Plant
𝑏𝑏0
Extended State
Observer
𝑧𝑧3𝑧𝑧2𝑧𝑧1
Figure 2.1: The Architecture of Original ADRC System with ESO
The Extended State Observer (ESO) was originally an important part of Han’s Adaptive
Disturbance Rejection Control theory [2], the ADRC framework is shown in Figure 2.1. In the fig-
ure, v is the desired value for y, while v1 and v2 are desired trajectory and its derivative respectively.
u is the input to plant after adjusting. Han’s ADRC control is a good alternative to PID control since
the input is adjusting based on the observed results from ESO. Besides, regardless of the form of
nonlinearities in the dynamical system, it can be estimated through ESO and then returned to the
controller for appropriate control optimization.
Although the human itself cannot be brought into any control method since the human
is not an engineering system, it is still possible to utilize ESO’s characteristics, in which the non-
linearity of the system can be estimated without knowing the explicit dynamical system model, to
12
CHAPTER 2. EXTENDED STATE OBSERVER
estimate the nonlinearities of human dynamics.
2.2 Formulations of ESO
Suppose there is a second-order system with the following equation:x1 = x2
x2 = F + bu
y = x1
(2.1)
where u is the input, while y is the output. F is an unknown function which could be a multivariable.
Assume F is an additional state variable, which is F = x3. Then introduce another denotation G,
let G = x3. The equation above can be rewrite as
x1 = x2
x2 = x3 + bu
x3 = G
y = x1
(2.2)
which is easy to build the following extended state observer. By denoting the xi with zi as an
estimation of the system states, the function can be further redescribed asz1 = z2 − β01ez2 = z3 + bu− β02h (e, 0.5, T )
z3 = −β03h (e, 0.25, T )
(2.3)
where β01, β02, and β03 are nothing but a group observer gains, which can be selected as wish. e
is the representation of the tracking error, where e = z1 − y. However, h, as shown below, is a
denotation of the nonlinear coefficient function proposed by Han (see details in [2]).
h(e, α, δ) =
eδ1−α , |e| ≤ δ,|e|α sign(e), |e| > δ.
(2.4)
Based on the equation above, the discrete-time formulation of the ESO can be easily
obtained: z1(n+ 1) = z1(n) + Tz2(n)− β01ez2(n+ 1) = z2(n) + T (z3(n) + bu(n))− β02h(e, 0.5, T )
z3(n+ 1) = z3(n)− β03h(e, 0.25, T ).
(2.5)
13
CHAPTER 2. EXTENDED STATE OBSERVER
The observer gains can be selected in many ways. For convenient, the observer gains here
are also selected based on Han’s method:
β01 = 1 β02 =1
2T 0.5β03 =
2
52T 1.2(2.6)
where T is the sampling period. In this thesis, the sampling period is set at 0.005 seconds.
2.3 Implementation of ESO
It is difficult to assert how many orders in human dynamical system. However, given the
fact that in this experiment, the subject observes the blue cursor’s position and then manipulates the
mouse through force, it can be inferred that human dynamics has at least two states, namely x1 and
x2. For the sake of simplicity, only second-order and third-order system were tested in this paper.
2.3.1 Second-Order States
ReferenceAngle
HumanDynamics
Extended State Observer
𝑥𝑥1𝑥𝑥2𝑥𝑥3 +
−
𝑒𝑒1𝑧𝑧1𝑧𝑧2𝑧𝑧3
−
+
𝑢𝑢PitchAngle
Figure 2.2: The Architecture of Human Dynamics which Applied with ESO [5]
Assuming the human dynamics with the second-order system has the architecture as Fig-
ure 2.2 shown, then the corresponding equations can be derived asx1(t) = x2(t)
x2(t) = F (x1(t), x2(t), t) + bu(t)
y(t) = x1(t)
(2.7)
where F is the unknown nonlinear human dynamics function, and u(t) is the input that subject
observers from the computer screen. The coefficient b is assumed to be one (b = 1), with appropriate
14
CHAPTER 2. EXTENDED STATE OBSERVER
units. As mentioned above, ESO’s formulation for the second-order system isz1(t) = z2(t)− β01ez2(t) = z3(t) + u(t)− β02h(e, 0.5, 005)
z3(t) = −β03h(e, 0.25, 0.005).
(2.8)
In this setting, since the actual mouse position x1 is known through the collected data, the
tracking performance can be measured through the difference between x1 and z1. An assumption is
made here that if the error e = x1 − z1 for the first state goes to zero, the tracking results for other
states are also reliable. Therefore, the reliability of z3 depends on steady-state error, and as long as
e→ 0, the unknown function F can be estimated from the state x3.
Figure 2.3a shows a comparison of actual position x1 and tracking position z1 with respect
to time t, and each game set has a trial selected. It can be seen from the figure that only a minor
difference exists in between the orange line (x1) and the blue line (z1). Besides this example figure,
the average error across the 10 subjects is 0.0123, while the maximum and minimum are 0.0333
and 0.0019 respectively. Hence it can be proved that ESO’s tracking is reliable for x1 as well as for
unknown function F .
Figure 2.3b are the time plots of z3, and each plot corresponds to Figure 2.3a. The un-
known function F as estimated by z3 is clearly varying in each trial. Based on the fact that many
variables exist in each trial, such as subjects might take a different strategy in each trial, or subjects
might take time to familiarize the game settings, these difference in F is not only conceivable but
also acceptable. However, only obtain a couple of time plots of z3 cannot prove that there exists
a generally linear or nonlinear model across all subjects. Another perspective of view should be
considered to analyze the data.
15
CHAPTER 2. EXTENDED STATE OBSERVER
(a) Estimated Position z1 Compare to Actual Position x1
(b) Estimated Result of Nonlinearity z3 with respect to Time t
Figure 2.3: Plot of Tracking Variable with respect to Time
16
CHAPTER 2. EXTENDED STATE OBSERVER
(a) Original Phase Plot (b) 3D View of Phase Plot with Respect to Time t
Figure 2.4: Phase Plot of z3 with respect to z1 (Subject #9)
Here, considering human subject’s control effort z1 has a relationship with z3, since the
human nonlinearity might have a contribution to each change of mouse position, hence it is nec-
essary to investigate how G = dFdt ≈ z3 varies with respect to z1. As shown in Figure 2.4a, data
from subject #9’s multiple trials are plotted in one figure for a better comparison. Although it is
somewhat difficult to distinguish each trial from the figure, it is still visible from the figure that the
trials are similar in shape. To make the figure clearer and easier to recognize, the original phase
plot is converted into 3D view, as shown in Figure 2.4b. From this figure, it can be clearly seen that
for the derivatives of z1 and z3, even it is in different time, the ring shape patterns always have a
same size for the same derivative value of z1. This feature proves that z3 must be a function of z1.
Although it is not showing here, the points on the figure always begin with the origin and then over
time rotates clockwise from either left or right of the origin to form an infinitely shaped graph on
the plot.
Figure 2.5 shows all (except #9) subjects’ phase plot. It needs to be noted that the first
trial of all game set was deleted from the plot since the human subject may not be familiar with
the game setting and produce much more noise. Although the first trials are deleted and only one
subject’s data were showed here, this similar pattern actually exists among all trials and all subjects.
This similar pattern proves that human subjects may have a similar strategy when trying to control
the mouse to stabilize the pitch angle around the desired value. In other words, this indeed suggests
that a generalized model could potentially express the control intention of a set of subjects.
17
CHAPTER 2. EXTENDED STATE OBSERVER
(a) Sub.6 (b) Sub.7 (c) Sub.8
(d) Sub.11 (e) Sub.12 (f) Sub.13
(g) Sub.14 (h) Sub.15 (i) Sub.16
Figure 2.5: Phase Plot of z3 with respect to z1 (All Subjects, Second-Order System)
18
CHAPTER 2. EXTENDED STATE OBSERVER
2.3.2 Third-Order System
Similar to the second-order system, for human dynamics with the third-order system, the
equations are
x1(t) = x2(t)
x2(t) = x3(t)
x3(t) = F (x1(t), x2(t), x3(t), t) + u(t)
y(t) = x1(t).
(2.9)
The corresponding ESO’s formulation isz1(t) = z2(t)− β01e(t)z2(t) = z3(t)− β02h(e(t), 0.5, 005)
z3(t) = z4(t) + u(t)− β03h(e(t), 0.25, 005)
z4(t) = −β04h(e(t), 0.125, 005).
(2.10)
It should be noted that β04 here has no specific formulation. Its value needs to be tested
and selected according to the tracking result. Here, for the sake of simplicity in implementation,
β04 was selected as same as β03.
Figure 2.6: Comparison between Actual Position x1 and Estimated Position z1 Obtained Based on
the Three-Order System
Figure 2.6 shows the tracking mouse position results obtained through the three-order
state-space system. The plots were using the same data as Figure 2.3a. It may not easy to identify
19
CHAPTER 2. EXTENDED STATE OBSERVER
the difference between Figure 2.3a and Figure 2.6. However, for the three-order system, the average,
maximum, and minimum error across the 10 subjects are 0.0150, 0.0454, and 0.0020 respectively.
The error of the tracking position becomes much larger for all trials. And it is also can be proven that
if increasing the value of β04, the error will also increase at the same time. However, if decreasing
the value of β04, the error will decrease to as same as second-order system.
Figure 2.7: Phase Plot of z3 with respect to z1 Obtained Based on the Three-Order System
Similar to the second-order system, Figure 2.7 shows the plot z3 varies with respect to z1
in third-order ESO system. It can be seen that in the third-order system, the pattern still exists but on
a larger scale. This is because of the value selection of β04. The scale will change as β04 changes,
but the shape of the pattern will always remain the same. Figure 2.8 is a similar phase plot to 2.5
except obtained based on the third-order system.
20
CHAPTER 2. EXTENDED STATE OBSERVER
(a) Sub.6 (b) Sub.7 (c) Sub.8
(d) Sub.11 (e) Sub.12 (f) Sub.13
(g) Sub.14 (h) Sub.15 (i) Sub.16
Figure 2.8: Phase Plot of z3 with respect to z1 (All Subjects, Third-Order System)
21
CHAPTER 2. EXTENDED STATE OBSERVER
2.3.3 Other Systems
Although only two systems’ results were presented here, other orders of systems were
also tested for the collected data. It can be concluded that with the orders of the system increased,
the scale of z3 will become larger, while z1 remains the same. Therefore, the pattern will always
be the ∞-like shape. However, one major problem for higher-order state-space system is that the
observer coefficients for ESO are harder to determine. As stated in the previous section, the error is
related to the coefficients.
22
Chapter 3
Modeling and Validation
Despite this human subjects experiment is based on a basic computer game, in which
the subject only needs to move the mouse in two directions, it can be seen from the collected
results that even for the same person with the same set of conditions, his or her performance will
be very different. However, in the previous chapter, it was shown that not only the same subject
can have the same control logic across all trials but also all subjects may have a similar control
strategy, such as subjects might be taking similar actions to prevent overshoot when the system
output is close to the reference line. No matter what it is, since human executes an action by taking
information and processing it, it can be assumed that the human brain is an advanced controller
which includes a linear part and a nonlinear part similar to engineered controllers. However, how to
obtain a mathematical model for such a controller is a very complicated problem.
Matlab provides a handful tool called “System Identification Toolbox”, which could easily
estimate a linear or nonlinear model from measured data. As shown in Figure 3.1, it is assumed that
human subject in the outer loop of MRAC system only receives a reference signal and a feedback
signal from uncertain dynamical system as inputs and outputs one command signal to the inner loop
after the reaction. Since collecting data from previous experiments includes all of these informa-
tion, it is possible to create a black-box model of human dynamics characteristics with the System
Identification Toolbox. However, it is unknown at this point whether the linear model or nonlinear
model could best fit human subjects. Therefore, it is necessary to check all data in order to obtain a
reasonable human model.
23
CHAPTER 3. MODELING AND VALIDATION
HumanDynamics(With Delay)
Reference InputControl Command
System Output
(Feedback from Uncertain Dynamical System)
Figure 3.1: Block Diagram of Human Dynamics
3.1 Fit Percentage
In order to compare and evaluate the human model obtained through the toolbox, a scoring
standard is needed. Here, the similarity between the output from a model and the output from its
estimated model is called the goodness of fit [53]. The goodness of fit is defined as follow
Fit = 1− ‖xo − x‖‖xo − xo‖
(3.1)
where xo is the measured data, and x is the estimated data. If the estimated data and the measured
data are the exact same, then the goodness of fit will be 1. However, if the estimated data and the
measured data are not similar, then the goodness of fit can reach as low as −∞.
3.2 Linear Identification
3.2.1 Linear Modeling
There are a couple of modifications needed on the data before using System Identifica-
tion Toolbox to create the linear model. The first one is choosing the input data for the Toolbox.
Although the architecture of MRAC indicates that the human subject should have two inputs, the
estimated transfer function in the linear model should be single-input/single-output. And it is con-
ceivable that the human subject will also comprehensively process the two inputs received on the
screen. It is assumed here that human subject unconsciously calculates the difference between the
green line and the blue curve, and uses this difference as the input signal to affect the subsequent
action. Hence, the real input data here is the difference obtained by subtracting the system output
from the reference.
Besides this, the input and output were also selected from three seconds to twenty seconds
of every single experiment, which is the actual playable time for human subjects. The delay effect
24
CHAPTER 3. MODELING AND VALIDATION
for human subjects was set to 0.355 seconds, which is the average reaction delay of all subjects. The
initial condition for all subjects was also set to be zero. A total of seven different transfer function
models were generated for comparison of data. By importing the experimental data to the toolbox
to generate the transfer function, each subject gets a total of 105 (15 experiments multiplied by 7
linear models) transfer functions. Then we input the experimental data to the transfer functions and
perform the simulation to get the estimated command in each game of each subject. This makes it
easy to compare the accuracy of each transfer function using the goodness of fit function, hence the
best transfer function can be found. It should be noted that the number of poles and zeros of the
seven transfer functions used in linear identification are 1, 2, 3, 4, 5, 10 and 50.
3.2.2 Validation of the Linear Model
(a) Identification Result: 0.858243 (b) Identification Result: -3.608000
Figure 3.2: Example of Linear Identification Results
In Figure 3.2, two linear identification examples are displayed. It is more consistent with
common sense that linear identification can have a better performance only when the original sub-
ject’s command does not change drastically. For each subject, Table 3.1 presents the best goodness
of fit result. It can be seen from the Table 3.1 that any kind of transfer function setting cannot
perform high-accuracy identification for all subjects and game sets.
As is shown in Table 3.1, most of the best results are with 10 poles and zeros in the transfer
function. However, due to the limitations of the identification setting, it is not clear that the 10 poles
and zeros are the best linear identification parameters. It only shows that larger the number of poles
and zeros, the better the result will be. But there still exists an upper limit. For example, the fit
25
CHAPTER 3. MODELING AND VALIDATION
Table 3.1: Best Linear Fit Result for Each Subject
Subject# Game Set# Poles and Zeros# Fit Result
6 2 10 0.858243
7 3 10 0.821355
8 3 10 0.827832
9 2 10 0.662257
11 2 10 0.713371
12 1 10 0.808103
13 3 5 0.732491
14 3 5 0.688763
15 2 10 0.778785
16 2 10 0.685117
results with 50 poles and zeros are very poor, as shown in the Table 3.2. Although some subject’s
best fit results still have very high reliability in 50 poles and zeros, most of the best fit results are far
below zero. This indicates that linear identification did not play any roles at all for most subjects.
In addition, in the case of 5 poles and zeros, the model fit results are more balanced, most of the
trials show relatively good fits, while the model fit results of 10 poles and zeros are very good for a
few trials, but others are much worse compared to 5 poles and zeros. However, this raises another
question: if keeping the transfer function exactly the same, even the parameters are not changed, can
these transfer function, which performs very well in some specific game set and subject situations,
also maintain the same good fit results when inputting the data of other trials?
Here, since the linear model must be obtained from any subject’s one of 15 trials by using
the toolbox, the trial that provides the linear model is referred to as the model provider; and the trial’s
data used as inputs to the linear model to obtain the estimated output are referred to as input provider.
So for the data obtained from the experiment, there will be a total of 150 model providers and 150
input providers. For each linear identification setting, there will be a total of 22500 results (150
model providers multiply 150 input providers). By testing all seven linear identification settings,
157,500 results were obtained. It can be shown that most linear models cannot provide convincing
results by using other input data, so it is impossible to find a specific one from these linear models
that can be used as a general model for all subjects. It can be said that linear identification has great
limitations since the linear identification can only provide some good results with the same model
26
CHAPTER 3. MODELING AND VALIDATION
Table 3.2: Best Fit Result for 5, 10, 50 Poles and Zeros
Subject#Game Set 1 Game Set 2 Game Set 3
5 10 50 5 10 50 5 10 50
6 0.607851 0.840400 0.707800 0.804027 0.858243 -0.145560 0.078609 -0.305280 -0.429790
7 0.455898 0.557423 -0.166580 0.632803 0.656694 0.026586 0.765277 0.821355 0.363655
8 0.654744 0.591755 -0.523110 0.630255 0.756752 -0.111200 0.648606 0.827832 0.165349
9 0.487156 0.542807 -0.174740 0.577812 0.662257 0.022285 0.622201 0.635379 -0.337100
11 0.302698 0.400903 0.088544 0.581586 0.713371 -0.139520 0.435145 0.162498 0.150495
12 0.598148 0.808103 0.048967 0.716204 0.756947 -0.290170 0.286177 0.482929 0.452977
13 0.581897 0.717434 -0.628730 0.648952 0.580645 -0.098760 0.732491 0.308928 0.375717
14 0.620784 0.651688 -0.008180 0.459837 0.322682 -0.006160 0.688763 0.597345 0.281751
15 0.609446 0.738331 -0.442210 0.673742 0.778785 -0.358170 0.723705 0.761559 0.100832
16 -0.017970 -0.140540 -0.206400 0.412845 0.685117 0.048715 0.492024 0.665914 -0.148340
provider and input provider.
3.3 Nonlinear Identification
3.3.1 Nonlinear Modeling
Matlab provides three nonlinear model structures: Nonlinear ARX models, Hammerstein-
Wiener models, Nonlinear grey-box model, see [54] for details. Hammerstein-Wiener models are
used for a system which has static nonlinearities. Nonlinear grey-box models are used for the
system which has known difference equations but without parameters. Nonlinear ARX models,
however, have the widest range of applications because the system will be modeled through dynam-
ics networks. Since the nonlinear identification case here has no known function or form to express,
therefore the only way to identify is utilizing the nonlinear ARX model.
In Matlab, the structure of a nonlinear ARX model is shown in Figure 3.3. The regressors
block and nonlinearity estimator block will act together as a model for simulating the subjects
decision process. The regressors for this experiment’s identification process are selected based on
the average reaction of subjects. Since the average reaction time is 0.355 seconds and the collected
data is in discrete time ∆t = 0.005 seconds, the regressors first considered are based on u(t− 71),
u(t − 72), y(t − 1), and y(t − 2), where u is the input to the subjects and y is the output of the
model. However, other possible delay timesteps from 0 to 500 are also been tested.
27
CHAPTER 3. MODELING AND VALIDATION
Regressors Output 𝑦𝑦Input 𝑢𝑢 Nonlinear
Function
LinearFunction
Nonlinear Estimator
Figure 3.3: Block Diagram of the Structure of the Nonlinear ARX Model [6]
For nonlinear estimator, its equation is given by [55]
F (x) = LT(x− r) + d+ g(Q(x− r)) (3.2)
where LT(x−r)+d represents linear function block. x is the vector of regressors, in our case here,
x = [u(t − 71), u(t − 72), y(t − 1), y(t − 2)], while r is the mean of the vector x. L is a linear
coefficient, and d is a scalar output offset. g(Q(x − r)) represents the nonlinear function block,
where g is the nonlinear function, while Q is a projection matrix. F (x) is the output of nonlinearity
estimator, and it is actually the same as y(t).
In Matlab, nonlinearity estimator block estimates the nonlinear model through many dif-
ferent estimator networks. Here, the wavelet network is selected to create the nonlinear model. The
wavelet network is described as the following function expansion [56]:
F (x) = (x− r)PL+ as1f (bs1 ((x− r)Q− cs1)) + . . .
+ asnsf (bsns ((x− r)Q− csns))
+ aw1 g (bw1 ((x− r)Q− cw1 )) + . . .
+ awnwg (bwnw ((x− r)Q− cwnw)) + d
(3.3)
where f(z) = e−0.5zzT
is the scaling function, and g(z) =(m− zzT
)e−0.5zz
Tis the wavelet
function, while z is a vector input to the function. P stands for a different projection matrix than Q.
as and bs are scaling coefficients, while cs is the scaling vector. Similarly, aw, and bw are wavelet
coefficients, while cw is the wavelet vector. Depending on the settings of the nonlineariry estimator,
the number of scaling and wavelet coefficients can reach tens or even hundreds. Here, the total
number of scaling and wavelet coefficients are selected as 100. The rest of the symbols have the
same meaning as (3.2).
28
CHAPTER 3. MODELING AND VALIDATION
3.3.2 Validation of the Nonlinear Model
Figure 3.4a shows an example of the estimated output (blue) from nonlinear identification
and the real output (grey) from a human subject, as well as the goodness of fit result with percentage
unit. It can be seen from the figure that the raw nonlinear model output contains a lot of oscillations.
Therefore, to reduce these high frequency oscillations in the output, an extra filter is needed to
process the output of the nonlinear model. Here, a discrete-time low-pass filter is introduced as
y[n] = y[n− 1] +(x[n− 1]− y[n− 1])∆t
T(3.4)
where T = 0.025s and ∆t = 0.005s.
(a) Before Filtering (b) After Filtering
Figure 3.4: The Effect of Applied Filter to Nonlinear Identification Result
Figure 3.4b is the identification result after filtering. It can be seen that after filtering, the oscillations
reduced and the goodness of fit results are also improved.
When estimating the nonlinear model of subjects’ decisions, it can be shown that the result
of the goodness of fit becomes poor compared to the linear identification, if the model provider and
the input provider are the same subject. Table 3.3 shows the best goodness of fit of nonlinear
identification for each subject as the model provider under the settings of 71 timesteps delay. Here
for the sake of convenience, the data of game set 1 are called as the 1st to the 5th trials in the order
of recording, and the data of game set 2 are called the 6th to 10th trials. The data of set 3 are called
the 11th to 15th trials. It can be seen that as a model provider, the best fit results in most cases are
not produced with itself as input provider. And the model, for example, comes from game set 1 does
29
CHAPTER 3. MODELING AND VALIDATION
Table 3.3: Best Fit Result for Each Subject as Model Provider
Model ProviderFit Result for
Model ProviderInput Provider
Fit Result for Input
Provider
Sub.6 9th trial −0.804400 Sub.8 9th trial 0.399306
Sub.7 4th trial −0.319510 Sub.7 15th trial 0.439101
Sub.8 9th trial −1.651120 Sub.14 11th trial 0.411243
Sub.9 6th trial −0.031270 Sub.7 15th trial 0.401610
Sub.11 1st trial 0.472376 Sub.11 1st trial 0.472376
Sub.12 8th trial −0.902050 Sub.8 9th trial 0.412803
Sub.13 15th trial −0.053850 Sub.7 15th trial 0.459194
Sub.14 13th trial 0.318348 Sub.14 11th trial 0.406755
Sub.15 1st trial −0.759880 Sub.7 15th trial 0.553569
Sub.16 10th trial −1.855330 Sub.11 4th trial 0.364914
not necessarily have a good estimation result only when the input also comes from the same game
set, but the input from other game sets can also simulate a good result.
This is a very interesting phenomenon since the estimation result shows that importing the
input data to another model rather than itself will get a better result. Recall the linear identification’s
result, which fits well when the model provider and input provider are the same. The results of
nonlinear identification indicate that the nonlinear model obtained through System Identification
Toolbox actually contains more common factors that exist among all subjects, rather than each
subjects’ unique factors.
Table 3.4: Nonlinear Fit Results for a Given Delay in Timesteps
DelaySubject #
6 7 8 9 11 12 13 14 15 16
0 0.459561 0.375461 0.414429 0.398204 0.218329 0.334941 0.429534 0.524615 0.392828 0.278882
25 0.376060 0.396565 0.398901 0.366676 0.275598 0.470890 0.400084 0.534543 0.404611 0.244987
50 0.430955 0.416826 0.382706 0.287123 0.265373 0.394275 0.485902 0.429032 0.439614 0.253626
60 0.489254 0.397344 0.383052 0.351220 0.244491 0.438561 0.385082 0.392779 0.512777 0.392096
71 0.399306 0.439101 0.411243 0.401610 0.472376 0.412803 0.459194 0.406755 0.553569 0.364914
80 0.477761 0.456510 0.376196 0.412458 0.297346 0.425203 0.495331 0.421498 0.481258 0.361516
100 0.395634 0.365415 0.411243 0.401610 0.321410 0.406848 0.494069 0.406755 0.403182 0.394753
125 0.474224 0.365600 0.371908 0.450870 0.343916 0.383740 0.483663 0.431198 0.459535 0.386639
150 0.418020 0.400183 0.407227 0.420004 0.287910 0.409706 0.470098 0.415430 0.301897 0.267269
175 0.650060 0.486649 0.444235 0.315393 0.277429 0.374445 0.437013 0.412787 0.276025 0.458324
200 0.376537 0.329295 0.401495 0.294265 0.315413 0.366868 0.471194 0.399183 0.417840 0.397248
300 0.285547 0.498537 0.326338 0.226003 0.190628 0.374089 0.339326 0.353936 0.206401 0.275470
500 0.251686 0.180851 0.253274 0.208480 0.155381 0.379296 0.149751 0.194417 0.130350 0.177291
30
CHAPTER 3. MODELING AND VALIDATION
Table 3.5: Each Subject’s Total Nonlinear Fit Numbers for a Given Delay in Timesteps
DelaySubject #
6 7 8 9 11 12 13 14 15 16
0 5 19 22 14 0 19 14 21 5 3
25 6 10 21 9 1 10 16 37 11 0
50 9 17 17 4 1 19 31 39 12 1
60 13 19 14 18 0 22 34 28 13 6
71 20 37 28 26 3 25 34 33 14 8
80 20 38 24 22 1 18 43 39 10 12
100 22 23 26 26 3 28 37 33 4 13
125 23 33 19 26 4 22 35 35 13 17
150 26 21 20 32 4 32 27 21 3 3
175 17 27 15 6 1 13 18 13 1 17
200 9 7 15 1 1 11 19 12 5 6
300 1 6 1 0 0 6 8 3 0 1
500 1 0 1 0 0 2 0 0 0 0
The first attempt of setting for nonlinear identification was using 71 samples of delay
as regressors. However, this setting shows that the goodness of fit results are not good enough
compared to linear identification. Since the delay setting was based on the average reaction time, it
is a question that whether if this poor fit could be solved by using other settings of delay timesteps.
Table 3.4 shows the highest goodness of fit results for different subjects with various of different
delay timesteps. The first two high of fit for each subject are highlighted. The highest fit result in
the table is 0.650060, which is 175 delay samples from 11th trial of subject 6 (same model provider
and input provider). It can be seen from these results that the overall estimation performance of
nonlinear identification is indeed much lower than linear identification. Table 3.5 shows the total
number of good fit results with different delay samples, and it includes both the same provider
situation and the different provider situation. Similary, the first two high of fit numbers for each
subject are also highlighted. The results in Table 3.5 indicates that nonlinear identification’s models
indeed reflected the real condition of human subjects. In other words, the peak of the total fits
numbers occur between approximately 71 and 150 delay samples for each subject suggest that when
subjects played the Matlab game, the actual reaction delay for subjects should also be around 71 to
150 samples, which is 0.355 to 0.750 seconds. However, due to the poor performance of nonlinear
identification through System Identification Toolbox, the fit results are still not a good indication
that these nonlinear models can be used reliably.
31
Chapter 4
Prediction of Human Behavior
4.1 Direct Prediction
Here, direct prediction means comparing both estimated human commands and estimated
uncertain dynamical system output directly with the original output from the experiment.
(a) Linear Model (b) Nonlinear Model
Figure 4.1: Command Error Comparison
Previous chapter showed that some of the linear models and nonlinear models have pretty
good fit results for subjects’ command under specific conditions. Figure 4.1 shows the error of
command from two models obtained through System Identification Toolbox. The model in Figure
4.1a is a linear model from the 8th trial of subject 6. The model in Figure 4.1b is a nonlinear model
32
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
from the 11th trial of subject 6. Both of them are two best-performing models in their respective
identification categories.
Figure 4.2a and 4.2b are the system output obtained by applying two models into the
human-in-the-loop MRAC system, and the corresponding error of output. The linear model’s system
output seems promising since the error is relatively low (less than 0.025). However, for the nonlinear
model, the system output becomes unstable. As shown in Figure 4.2b, the output value exceeds +5/-
1 and rapidly reached negative infinity after 14 seconds.
The results in Figure 4.2 are achieved by using the same model provider and input provider.
If we change the input provider to other trials or subjects, the goodness of fit results will be no doubt
affected. The errors in Figure 4.3 and 4.4 are the results obtained through different input providers.
It should be noted that the two input providers are randomly chosen, and the estimated commands
are then imported to the human-in-the-loop MRAC system to substitute into human subjects and
obtain the system output results. The errors in these two figures, especially command errors suggest
that the difference between the real and estimated subject’s command are quite large. The errors
can even reach 3 unit at some point, knowing that the reference and input difference at the start of
the game is only 1 unit, therefore 3 unit difference is a huge error. The system’s output errors also
indicate that when importing other inputs, identification models are difficult to match the results of
the system output obtained by the original command. Figure 4.5 presents the comparison of sys-
tem outputs. It can be seen that the estimated output either has an overshoot or becomes unstable
over time. Therefore, it is impossible to predict the output of the human-in-the-loop MRAC system
directly by applying linear or nonlinear models.
33
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Linear Model
(b) Nonlinear Model
Figure 4.2: System Output and Output Error Comparison
34
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Linear Model Result
(b) Nonlinear Model Result
Figure 4.3: Error Comparison between the Linear Model and the Nonlinear Model (Case 1)
35
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Linear Model
(b) Nonlinear Model
Figure 4.4: Error Comparison between the Linear Model and the Nonlinear Model (Case 2)
36
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
Figure 4.5: Results of System Output with Two Different Input Provider Cases
37
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
4.2 Indirect Prediction
4.2.1 Data Processing
In contrast to the direct prediction, the indirect prediction means estimated output from
models need to be processed before comparing with the original output from the experiment. Figure
4.5 in the last section indicates that direct prediction may not be the best way to generate the model
of human subjects. But from the figure, one interesting pattern can be observed that the estimated
outputs seem to have similar growth and decrease tendency compared to the original outputs. So
it is worthwhile to calculate the derivative of the estimated and subject’s real command, and then
compare them.
Figure 4.6: Comparison of Control Commands from Linear Identification (From left to right: Case
1 and Case 2)
As shown in Figure 4.6, the real and estimated control command from linear identification
are drawn in the same figure for the sake of comparison. Clearly from Figure 4.6 the estimated com-
mand and the actual command from the subject do have a nearly simultaneous increase and decrease
tendency. However, for the actual derivative of command, since the human subject was moving the
mouse in a highly nonlinear regime during the play process, the absolute value of derivatives can
be extremely large. Therefore, both the real and estimated derivatives here should be normalized:
when the derivative is positive, set its value to 1, and when the derivative is negative, set its value to
38
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
Figure 4.7: Derivative of Commands and Its +1/-1 Plot (Case 1)
-1. To simplify, here only one case is demonstrated. The derivatives of actual human commands and
its +1/-1 plot are shown in Figure 4.7. Due to the fact that the human subject motion was recorded
with some discontinuity during the simulation game, the corresponding derivatives are also affected
and became discontinuous. To further eliminate the discontinuous of +1/-1 plot of derivatives, an-
other optimization attempt should be made. Here, the basic idea is to try to minimize the oscillation
number of the derivative value between zero and positive or negative in a specific period of time
(such as the oscillation between 2s to 4s in the right figure). To do this, all the zero between this
specific period should be set to equal to its adjacent value. As shown in Figure 4.8, the thick blue
line is the +1/-1 plot of derivatives after processing, while the slim orange one is the original. It can
be seen that the thick blue line is basically the approximate outer contour of the orange one after
ignoring the small disturbance.
4.2.2 Prediction of the Linear Model
With the modified positive/negative derivative plot of subject’s command, the estimated
command can be added without further process and is easily compared with, as shown in Figure 4.9.
The meaning of the blue line is still the same, which represents the modified subject’s command
derivative, while the red dot alongside the blue line is the estimated command derivative from linear
identification. In the prediction result figure, if the segment of red dots and the blue-line segment
39
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
Figure 4.8: Optimization Result for +1/-1 Plot of Derivatives
are highly coincident, then it could be said that this model has a perfect estimation ability for the
derivative of the command. As for the example case in Figure 4.9a, although red-dot segments are
about a second earlier than the blue-line segment, red-dot segments still match the change of signs
in the blue-line segments, which is still a promising result for prediction.
Figure 4.9b shows the prediction result of Case 2. As it can seen from the figure, Case 2’s
subject commands are significantly different from Case 1. Case 1’s subject commands are almost
alternately varying through entire playing time, and after each change of derivative’s sign, the sign
will remain the same for a while (normally larger than 1 second). Meanwhile, for Case 2, the sign
only changes in the first half playing time, and also almost every change is instantaneous. Still, the
prediction result for case 2 is convincing since the estimated signs of derivative matches the actual
data in the first half.
To further investigate the results of case 1 and 2, another two input providers were used
to obtain the prediction and compare with case 1 and case 2, as shown in Figure 4.10. Also as
mentioned in the last section, case 1 is selecting 15th trial data of subject 7 as input provider, while
case 2 is selecting 2nd trial of subject 8 as input provider. Here case 3 is derived from the 15th
trial data of subject 8, while case 4 is derived from the 2nd trial data of subject 7. From Figure
40
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Case 1 (b) Case 2
Figure 4.9: Indirect Prediction Results of Linear Models (Case 1 and 2)
4.10 it can be seen that both prediction results in case 3 and 4 are still convincing enough. Besides,
the derivatives of actual control commands also share the same patterns in case 1 and 3, as well as
case 2 and 4. The first two have patterns of alternating sign change, and changes last for at least
a second, while the last two both have a much smaller time of duration of the derivative changes.
After more comparisons, it can be concluded that these similarities were caused by the difference
in the game set condition. Since the game set 1 is actually more sensible than the set 3, subjects
will have two control strategy in set 1, namely: either excessively and frequently moving the slider
bar during the entire playtime, or slowly moving the slider bar to a balanced position where system
output is close to the reference line and then no longer intervening. However, for the game set 3,
because the response is much slower, the subjects will put more effort to move the slider through
time. Also, due to the slow response of set 3, the control effort of subjects often leads to a continuous
overshooting, which is the cause of alternating commands, therefore it is difficult to reach the stable
states for dynamical system output at the end. As for prediction purposes, it can be seen that the
more alternating the subjects’ commands are, the higher the accuracy of the identification model’s
prediction.
41
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Case 3 (b) Case 4
Figure 4.10: Indirect Prediction Results of Linear Models (Case 3 and 4)
4.2.3 Prediction of the Nonlinear Model
Since the high fit result of the nonlinear model does not necessarily mean the best predic-
tion, after comparison, the model obtained from the 12th trial of Subject 7 under the setting of 80
delay samples is selected as the indirect prediction’s nonlinear model.
Similar to the steps in the previous section, the prediction results in Figure 4.11 are ob-
tained by calculating the derivative of the estimated commands through the nonlinear model. It can
be seen from Figure 4.11a that the derivatives of estimated commands not only match the patterns of
actual data but also change signs at the same time with the actual situation, which is a much better
result compared to the linear model. However, as seen in Figure 4.11b, for the cases which subjects
manage to achieve the stable system output, the accuracy of prediction can be poor.
Figure 4.12 shows the nonlinear model’s prediction result for case 3 and 4. Unsurpris-
ingly, both cases with alternating characteristics show better prediction results than using linear
model: estimated and actual’s sign of derivative not only has a great match for most of the positive
or negative value but also predicts the right time period for each sign’s change. It needs to be noted
that because of the initial condition setting for the estimation process, the actual starting point for
nonlinear estimation is beginning from 1 second later of the game start. Therefore all estimated
values are set to zero before 1 second.
42
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
(a) Case 1 (b) Case 2
Figure 4.11: Indirect Prediction Results of Nonlinear Models (Case 1 and 2)
(a) Case 3 (b) Case 4
Figure 4.12: Indirect Prediction Results of Nonlinear Models (Case 3 and 4)
43
CHAPTER 4. PREDICTION OF HUMAN BEHAVIOR
Although not all prediction results are shown, it is certain that this nonlinear model pre-
diction method can provide similar prediction results as in Figure 4.11 and 4.12, if it meets the
conditions of 1) dynamical system does not stabilize yet; 2) the sign changes of commands deriva-
tive are nearly alternating. Except for the accurate prediction of the nonlinear model, another major
pro of the nonlinear model is that it can easily be transformed for use in real-time prediction. Since
the nonlinear model is actually a function of complex finite number of coefficients and it only takes
regressors as inputs, the function can be added to the human-in-the-loop MRAC system and acquire
the human subject’s intention of action 70 to 100 samples in advance. Therefore, it is conceivable
that if there is a suitable algorithm that can combine other characterization data with this prediction
model and calculate the possibility of the subject’s excessive control behavior, then the additional
methods can be used to either remind the human subject to be careful of the excessive action or take
a rapid action to assist the subject to stabilize the dynamical system.
44
Chapter 5
Conclusion
5.1 Concluding Remarks
The main contribution of this thesis is a series of analysis of the data obtained from pre-
vious human-in-the-loop MRAC experiments. It can be found from the obtained data that subjects
have different ways to handle the control task, and not all subjects can guarantee to stabilize the
uncertain dynamical system within 20 seconds even with the help of the controller. Three sets of
poles setting used in the reference model of MRAC in the experiment make the dynamical system
response from more sensitive to less sensitive. Although the sensitive controller setting is more
likely to form a stable result, on the other hand, it will also cause the highest overshoot. Similarly,
although a slower controller setting is more difficult to control, if enough control effort is put into
it, the system output will gradually approach the stable status. Therefore, it can be seen from the
experimental data that only relying on the adjustment of the controller setting does not guarantee
that the system has the best control effect for the human-in-the-loop situation.
In Chapter 2, the idea proposed in the ADRC system was adopted. By assuming that the
subjects’ control behavior can be represented by a set of state-space equations, then the unknown
function F, which consists of both linearity and nonlinearity of human behavior, can be estimated
according to ESO. In this paper, both second-order and third-order state space are tested respectively,
and the corresponding estimated variables are obtained. By analyzing the estimated variables, it can
be found that there is a certain pattern between the derivative of the estimated value of the unknown
equation F , z3, and the derivative of estimated mouse position, z1. Since this pattern is ubiquitous
between all trials of all subjects, a generalized model could potentially express the control intention
of a set of subjects.
45
CHAPTER 5. CONCLUSION
In Chapter 3, a set of linear models and nonlinear models were obtained through the
System Identification Toolbox provided in Matlab, based on the collected data. Although linear
models have the highest fit results compared to nonlinear models, this can only prove that the linear
models will have a good match if the changes in the cursor’s position are relatively small. In
addition, the modeling results also show that using different modeling parameters have a great
impact on the fit results. Since the average reaction time among all subjects is measured as 0.355
seconds, this number is set as the delay parameter of the model in the modeling settings. However,
the existing pattern in both the highest fit results table and total fit numbers table in nonlinear
validation indicate that the reaction time of subjects in this experiment should be actually roughly
between 0.355 and 0.750 seconds.
Chapter 4 uses the best linear and nonlinear model obtained in Chapter 3 to perform
human behavior predictions on experimental data from other trials or subjects. The results show
that neither linear nor nonlinear model can directly predict human behavior since the estimated
value is far from the actual value. At the same time, the best performing nonlinear model in Chapter
3 could generate too many oscillations in direct prediction. Therefore, another nonlinear model
should be adopted and used for further prediction attempt. In the indirect prediction, it is found that
if we take the derivative of both actual and estimated cursor position and then convert them into
+1/-1 forms, the prediction result can be much more accurate. Both linear and nonlinear models
can better predict the future control behavior of subjects, while the prediction of nonlinear model is
more synchronous with the actual behavior change. Therefore, it is worthy of being selected as a
human behavior prediction model for future optimization of the human-in-the-loop MRAC system.
5.2 Future Work
Due to the limitation of time, this thesis mostly focused on the effort of the acquisition
and verification of the model. However, there is still a lot of work to be investigated, such as:
• To show that there exists a generalized model between all the subjects in the experiment, a
simplified state-space system is assumed, and the corresponding variables are estimated by
using ESO. Although the result of estimate proves that z3 should be at least a function of z1,
and it is obvious that z3 should also have other independent variables excepts z1. Since this
thesis did not explore other possible independent variables for z3, the subsequent work can
start with the ESO estimation results and further analyze the characteristics of z3.
46
CHAPTER 5. CONCLUSION
• For human behavior modeling, the model obtained in this article comes from the System
Identification Toolbox of Matlab. Although it has a good prediction result to some extent,
due to the wavelet function, the stability of the model is difficult to be calculated. Therefore,
these models are limited to predict the possibility of the control behavior of the subjects under
certain conditions. So other mature methods of obtaining the behavior models should also be
considered.
• In this thesis, only some promising prediction models are obtained and used to predict the
control behavior of the subjects, while no specific quantitative standard is given to measure
the accuracy of the prediction. Although it can be seen that the accuracy is good enough by
visual observation of the comparison, an accuracy score of the model should be provided,
if the model needs to be substituted into the actual control system, so that the system can
distinguish different states of estimated human behavior.
• Based on the prediction model obtained in this paper, a comprehensive algorithm can be
developed after the fusion of other characteristic features of the system, specifically for iden-
tifying the excess behavior of the subjects. According to the calculated possibility of the
future excessive behavior, controller switching may be adopted to prevent the incident so that
the uncertain dynamical system is as stable as possible.
47
REFERENCES
[1] Y. Wang, “Human performance on model reference adaptive control systems with human re-
action delay,” Master’s thesis, Northeastern University, 2018.
[2] J. Han, “From pid to active disturbance rejection control,” IEEE transactions on Industrial
Electronics, vol. 56, no. 3, pp. 900–906, 2009.
[3] T. Yucelen, Y. Yildiz, R. Sipahi, E. Yousefi, and N. Nguyen, “Stability limit of human-in-
the-loop model reference adaptive control architectures,” International Journal of Control, pp.
1–18, 2017.
[4] Federal Aviation Administration, “Illustration of pitch effects,” accessed: 2019-
08-05. [Online]. Available: https://lessonslearned.faa.gov/ChinaAirlines140/Pitch Effects
Illustration.jpg
[5] Y. Lu, R. Sipahi, Y. Wang, and T. Yucelen, “Observer-based design to mathematically model
human behavior in a cursor tracking game,” 2019, submitted.
[6] The MathWorks, Inc., “Structure of nonlinear arx models,” accessed: 2019-08-05. [Online].
Available: https://www.mathworks.com/help/ident/ug/nonlin nlarx.png
[7] A. E. Bryson, Control of spacecraft and aircraft. Princeton university press Princeton, New
Jersey, 1993.
[8] L. Xiong, G. Teng, Z. Yu, W. Zhang, and Y. Feng, “Novel stability control strategy for dis-
tributed drive electric vehicle based on driver operation intention,” International journal of
automotive technology, vol. 17, no. 4, pp. 651–663, 2016.
48
REFERENCES
[9] L. Feng, C. Wiltsche, L. Humphrey, and U. Topcu, “Synthesis of human-in-the-loop control
protocols for autonomous systems,” IEEE Transactions on Automation Science and Engineer-
ing, vol. 13, no. 2, pp. 450–462, 2016.
[10] P. Van Overloop, J. Maestre, A. D. Sadowska, E. F. Camacho, and B. De Schutter, “Human-
in-the-loop model predictive control of an irrigation canal [applications of control],” IEEE
Control Systems Magazine, vol. 35, no. 4, pp. 19–29, 2015.
[11] T. Wenqian, L. Zongyuan, Q. Xiangju, W. Yanyang, and S. Liguo, “Aircraft-pilot system mod-
eling and pilot control behavior research for airdrop task,” IFAC-PapersOnLine, vol. 49, no. 32,
pp. 177–182, 2016.
[12] O. Celik and S. Ertugrul, “Predictive human operator model to be utilized as a controller using
linear, neuro-fuzzy and fuzzy-arx modeling techniques,” Engineering Applications of Artificial
Intelligence, vol. 23, no. 4, pp. 595–603, 2010.
[13] J. Tang, Q. Zhao, and R. Yang, “Stability control for a walking-chair robot with human in the
loop,” International Journal of Advanced Robotic Systems, vol. 6, no. 1, p. 3, 2009.
[14] R. A. Hess, “Modeling human pilot adaptation to flight control anomalies and changing task
demands,” Journal of Guidance, Control, and Dynamics, vol. 38, no. 6, pp. 655–666, 2015.
[15] T. Hatanaka, N. Chopra, and M. Fujita, “Passivity-based bilateral human-swarm-interactions
for cooperative robotic networks and human passivity analysis,” in 54th IEEE Conference on
Decision and Control (CDC), Osaka, Japan, 2015.
[16] D. Lee and M. W. Spong, “Passive bilateral teleoperation with constant time delay,” IEEE
transactions on robotics, vol. 22, no. 2, pp. 269–281, 2006.
[17] M. Xia, A. Rahnama, S. Wang, and P. J. Antsaklis, “On guaranteeing passivity and perfor-
mance with a human controller,” in IEEE 23rd Mediterranean Conference on Control and
Automation (MED), Torremolinos, Spain, 2015.
[18] J. B. Witte, “An investigation relating longitudinal pilot-induced oscillation tendency rating to
describing function predictions for rate-limited actuators,” Air Force Institute of Technology,
Wright-Patterson Air Force Base, Ohio, Tech. Rep., 2004.
[19] A. J. Thurling, “Improving UAV handling qualities using time delay compensation,” Air Force
Institute of Technology, Wright-Patterson Air Force Base, Ohio, Tech. Rep., 2000.
49
REFERENCES
[20] E. Yousefi, Y. Yildiz, R. Sipahi, and T. Yucelen, “The effect of remote-signal feedback archi-
tectures on the stability of human-in-the-loop telerobotics in the presence of time-delays,” in
IEEE Conference on Control Technology and Applications (CCTA), Kohala Coast, Hawai’i,
2017.
[21] S. Horl, “Agent-based simulation of autonomous taxi services with dynamic demand re-
sponses,” Procedia Computer Science, vol. 109, pp. 899–904, 2017.
[22] R. E. Yagoda and M. D. Coovert, “How to work and play with robots: an approach to modeling
human–robot interaction,” Computers in Human Behavior, vol. 28, no. 1, pp. 60–68, 2012.
[23] G.-J. M. Kruijff, M. Janıcek, S. Keshavdas, B. Larochelle, H. Zender, N. J. Smets, T. Mioch,
M. A. Neerincx, J. Diggelen, F. Colas et al., “Experience in system design for human-robot
teaming in urban search and rescue,” in Field and Service Robotics. Springer, 2014, pp.
111–125.
[24] M. Saida, J. R. Medina, and S. Hirche, “Adaptive attitude design with risk-sensitive optimal
feedback control in physical human-robot interaction,” in 2012 IEEE RO-MAN: The 21st IEEE
International Symposium on Robot and Human Interactive Communication. IEEE, 2012, pp.
955–961.
[25] T. Yucelen, Y. Yildiz, R. Sipahi, E. Yousefi, and N. Nguyen, “Stability limit of human-in-the-
loop model reference adaptive control architectures,” International Journal of Control, vol. 91,
no. 10, pp. 2314–2331, 2018.
[26] V. A. Shia, Y. Gao, R. Vasudevan, K. D. Campbell, T. Lin, F. Borrelli, and R. Bajcsy, “Semiau-
tonomous vehicular control using driver modeling,” IEEE Transactions on Intelligent Trans-
portation Systems, vol. 15, no. 6, pp. 2696–2709, 2014.
[27] A. Efremov, M. Tjaglik, U. Tiumentzev, and T. Wenqian, “Pilot behavior modeling and its
application to manual control tasks,” IFAC-PapersOnLine, vol. 49, no. 32, pp. 159–164, 2016.
[28] G. Zhao and C. Wu, “Mathematical modeling of driver speed control with individual differ-
ences,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 43, no. 5, pp.
1091–1104, 2013.
50
REFERENCES
[29] M. Li, H. Cao, X. Song, Y. Huang, J. Wang, and Z. Huang, “Shared control driver assistance
system based on driving intention and situation assessment,” IEEE Transactions on Industrial
Informatics, vol. 14, no. 11, pp. 4982–4994, 2018.
[30] R. B. Warrier, “Inferring human intent in novice human-in-the-loop control tasks,” Ph.D. dis-
sertation, 2018.
[31] D. Lee, A. Hansen, and J. K. Hedrick, “Probabilistic inference of traffic participants’ lane
change intention for enhancing adaptive cruise control,” in 2017 IEEE Intelligent Vehicles
Symposium (IV). IEEE, 2017, pp. 855–860.
[32] R. Li, Y. Li, S. E. Li, E. Burdet, and B. Cheng, “Indirect shared control of highly au-
tomated vehicles for cooperative driving between driver and automation,” arXiv preprint
arXiv:1704.00866, 2017.
[33] L. Li, Z. Zhu, Y. Chen, K. He, X. Li, and X. Wang, “Engagement control of automated clutch
for vehicle launching considering the instantaneous changes of driver’s intention,” Journal of
Dynamic Systems, Measurement, and Control, vol. 139, no. 2, p. 021011, 2017.
[34] S. Bae, S. M. Han, and S. Moura, “Modeling & control of human actuated systems,” IFAC-
PapersOnLine, vol. 51, no. 34, pp. 40–46, 2019.
[35] D. T. McRuer and H. R. Jex, “A review of quasi-linear pilot models,” IEEE transactions on
human factors in electronics, no. 3, pp. 231–249, 1967.
[36] D. T. McRuer and E. S. Krendel, “Mathematical models of human pilot behavior,” Advisory
Group for Aerospace Research and Development, Paris, Tech. Rep., 1974.
[37] A. Tustin, “The nature of the operator’s response in manual control, and its implications for
controller design,” Journal of the Institution of Electrical Engineers-Part IIA: Automatic Reg-
ulators and Servo Mechanisms, vol. 94, no. 2, pp. 190–206, 1947.
[38] G. Markkula, “Modeling driver control behavior in both routine and near-accident driving,”
in Proceedings of the human factors and ergonomics society annual meeting, vol. 58, no. 1.
SAGE Publications Sage CA: Los Angeles, CA, 2014, pp. 879–883.
[39] R. A. Hess, “A model for pilot control behavior in analyzing potential loss-of-control events,”
Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engi-
neering, vol. 228, no. 10, pp. 1845–1856, 2014.
51
REFERENCES
[40] ——, “Simplified approach for modelling pilot pursuit control behaviour in multi-loop flight
control tasks,” Proceedings of the Institution of Mechanical Engineers, Part G: Journal of
Aerospace Engineering, vol. 220, no. 2, pp. 85–102, 2006.
[41] A. J. Pick and D. J. Cole, “A mathematical model of driver steering control including neuro-
muscular dynamics,” Journal of Dynamic Systems, Measurement, and Control, vol. 130, no. 3,
p. 031004, 2008.
[42] T. Qu, H. Chen, D. Cao, H. Guo, and B. Gao, “Switching-based stochastic model predictive
control approach for modeling driver steering skill,” IEEE Transactions on Intelligent Trans-
portation Systems, vol. 16, no. 1, pp. 365–375, 2014.
[43] R. Mori and S. Suzuki, “Modeling of pilot landing approach control using stochastic switched
linear regression model,” Journal of Aircraft, vol. 47, no. 5, pp. 1554–1558, 2010.
[44] R. Hess and A. Modjtahedzadeh, “A control theoretic model of driver steering behavior,” IEEE
Control Systems Magazine, vol. 10, no. 5, pp. 3–8, 1990.
[45] Y. Zhang, H. Li, and R. Huang, “Investigation into pilot control model base on anfis for
teleoperation rendezvous and docking,” in 2016 IEEE Advanced Information Management,
Communicates, Electronic and Automation Control Conference (IMCEC). IEEE, 2016, pp.
1921–1926.
[46] C. Chen, W.-Q. Tan, X.-J. Qu, and H.-X. Li, “A fuzzy human pilot model of longitudinal
control for a carrier landing task,” IEEE Transactions on Aerospace and Electronic Systems,
vol. 54, no. 1, pp. 453–466, 2017.
[47] Y. Tan, H. Shen, M. Huang, and J. Mao, “Driver directional control using two-point preview
and fuzzy decision,” Journal of Applied Mathematics and Mechanics, vol. 80, no. 6, pp. 459–
465, 2016.
[48] L. Bi, J. Shang, G. Gan, and Y. Liu, “Modeling driver vehicle control in a cognitive archi-
tecture,” in 2011 Third Pacific-Asia Conference on Circuits, Communications and System
(PACCS). IEEE, 2011, pp. 1–4.
[49] R. Mori, “Pilot control modeling with stochastic periodical discrete movement,” IEEE Trans-
actions on Aerospace and Electronic Systems, vol. 54, no. 2, pp. 912–922, 2017.
52
REFERENCES
[50] I. D. Landau, R. Lozano, M. M’Saad, and A. Karimi, Adaptive control: algorithms, analysis
and applications. Springer Science & Business Media, 2011.
[51] D. Schmidt and B. Bacon, “An optimal control approach to pilot/vehicle analysis and the neal-
smith criteria,” Journal of Guidance, Control, and Dynamics, vol. 6, no. 5, pp. 339–347, 1983.
[52] “Reaction Time Test - Human Benchmark,” https://www.humanbenchmark.com/tests/
reactiontime, [Online; accessed 10-June-2019].
[53] The MathWorks, Inc., “goodnessOfFit,” https://www.mathworks.com/help/ident/ref/
goodnessoffit.html, [Online; accessed 09-Aug-2019].
[54] ——, “Nonlinear Model Identification,” https://www.mathworks.com/help/ident/
nonlinear-model-identification.html?s tid=CRUX lftnav, [Online; accessed 09-Aug-2019].
[55] ——, “What are Nonlinear ARX Models?” https://www.mathworks.com/help/ident/ug/
what-are-nonlinear-arx-models.html, [Online; accessed 09-Aug-2019].
[56] ——, “wavenet,” https://www.mathworks.com/help/ident/ref/wavenet.html, [Online; accessed
09-Aug-2019].
53