AdaptiveLab Talk1
-
Upload
noha-elprince -
Category
Documents
-
view
261 -
download
0
Transcript of AdaptiveLab Talk1
![Page 1: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/1.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
A Model-Based Feedback-Control Approach toBehaviour Modification ThroughReward-Induced Attitude Change
J.Ni, D. Kulic, and D. Davison
presented by: Noha El-Prince
April 16, 2013
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 2: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/2.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
1 Outline
2 Problem Definition
3 System ModelOverall ModelTheory of Planned BehaviorCognitive DissonanceTheory of Overjustification
4 Controller DesignAssumptions and Initial ConditionsController Design: Stage1Controller Design: Stage2
5 Simulation Results
6 Conclusion
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 3: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/3.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Problem Definition
Trying to change the behavior of a person to a desiredbehavior.
The person may have either a negative/positive attitudetowards the desired behavior.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 4: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/4.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Methodology
Model the internal cognitive psychological state of aperson.Design a controller based on the cognitive model.Goal: Tracking desired behavior via a sequence ofrewards.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 5: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/5.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overall System Model
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 6: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/6.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Theory of Planned Behavior
Aout[k] = Aout[k − 1] + ∆Aout[k − 1], (1)
∆Aout[k] = ∆ACDout [k] + ∆AOJ
out[k], (2)
Arew[k] = r1Arew[k − 1] + µ1(1− r1)R[k − 1], (3)
BI[k] = Aout[k] +Arew[k], (4)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 7: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/7.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Theory of Planned Behavior
B[k] =
Bd[k] if BI[k] ≥ Bd[k] and Aout[k] ≤ Bd[k]
Aout[k] if (BI[k] < Bd[k] and Aout[k] ≥ 0)
or Aout[k] > Bd[k]
0 otherwise.(5)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 8: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/8.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 9: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/9.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 10: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/10.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 11: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/11.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 12: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/12.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Cognitive Dissonance Theory (Block A)
A person’s behavior is inconsistent with one of hisattitudes ⇒ dissonance pressure
A person trying to reduce dissonance pressure by changingattitude/behavior
In our case : Inconsistency arises in 2 situations:
� The child declines the reward vs. value money� The child accepts the reward vs. feeling bored
How to quanitify dissonance pressure ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 13: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/13.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying Dissonance Pressure
Dissonance = ”%” of inconsistent cognitive pairs
PCDraw [k] =
Bsgn[k] Mincon[k]
Mincon[k]+Mcon[k]if Mincon[k] +Mcon[k] > 0
0 otherwise.
(6)
Bsgn[k] =
{+1 if B[k] ≥ Bd[k] or Aout[k] ≥ 0−1 otherwise.
(7)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 14: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/14.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying Dissonance Pressure - cont.
M1incon[k] =
{|Arew[k]| if sgn(Arew [k]) 6= Brel[k]
0 otherwise,(8)
M2incon[k] =
{|Aout[k]| if sgn(Aout[k]) 6= Bsgn[k]
0 otherwise,(9)
M1con[k] =
{|Arew[k]| if sgn(Arew [k]) = Brel[k]
0 otherwise,(10)
M2con[k] =
{|Aout[k]| if sgn(Aout[k]) = Bsgn[k]
0 otherwise,(11)
Mincon[k] =2∑
i=1
Miincon[k], Mcon[k] =
2∑i=1
Micon[k], (12)
Brel[k] =
{+1 if B[k] ≥ Bd[k]−1 otherwise.
(13)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 15: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/15.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Special Case: Attitude Reversal
Aout[k] small, R[k] small, Bd[k] is high ⇒ Child declinesthe reward
To reduce Diss. pressure: increase Aout OR “give up”jogging ⇒ Aout[k] <<<
r[k] =
+1 if Bd[k]−BI[k] > αrevAout[k], Aout[k] ≥ 0,
K1PCD[k] > 2Aout[k], and Arew[k] > 0,
−1 otherwise.
(14)
PCD[k] =
{(1− r2)PCD
raw [k] if r[k − 1] = 1
r2PCD[k − 1] + (1− r2)PCD
raw [k] otherwise.(15)
PrawCD
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 16: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/16.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Quantifying ∆Aout
Assume the change in Aout[k] is proportional to dissonancepressure, with proportionality constant K1 > 0:
∆ACDout [k] =
{−K1P
CD[k] if r[k] = 1
+K1PCD[k] otherwise.
(16)
PCD
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 17: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/17.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory (Block B)
Overjustification Theory
when a reward is given to a person to do something thatshe/he already enjoys doing, such rewards arecounter-productive in that they reduce the intrinsic desire ofthe person towards that behavior.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 18: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/18.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory - cont.
Let Bt[k] = minimal attitude level to which theoverjustification effect can drive Aout[k].Assume Bt[k] is a constant fraction of Bd[k], i.e.,
Bt[k] = αBd·Bd[k], (17)
for some constant 0 < αBd< 1.
If Bt[k] > Aout[k] ⇒ overjustification pressure does notdecrease Aout, and the reverse is true i.e.
Arelout[k] = max{0, Aout[k]−Bt[k]}. (18)
where Arelout[k]: a relative attitude with respect to Bt[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 19: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/19.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Overjustification Theory - cont.
Then the raw and filtered overjustification pressures, and the resulting change in intrinsic attitude, arecomputed just as in our previous work, but using Arel
out instead of Aout, as follows:
POJraw [k] =
Arelout[k]Arew[k] if Arel
out[k] > 0 and Arew[k] > 0and B[k] ≥ Bd[k]
0 otherwise,
(19)
POJ
[k] = r3POJ
[k − 1] + (1− r3)POJraw[k], (20)
∆AOJout[k] =
{−K2P
OJ [k] if K2POJ [k] ≤ Arel
out[k]
−Arelout[k] otherwise.
(21)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 20: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/20.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 21: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/21.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 22: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/22.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 23: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/23.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 24: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/24.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 25: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/25.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 26: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/26.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Assumptions
Mother Knows varoius plant parameters(µ1, r1, r2, r3, αrev, αBd, k1, k2)andA
∗0.
The child do not know the value of B∗d .
Bd[k + 1] is assigned to the child by end of day k.
i.c: PCD[0] = POJ [0] = Arew[0] = 0, Aout = A∗0.
Reward is not given everyday: N= Settling time
If impulsive reward applied at time 0, a transient(1− rk−1
2 ) appears.
Approach: wait for the transient to settle before applyingthe next impulsive reward.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 27: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/27.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage1
BI[k + 1] ≥ Bd[k + 1].
⇓R[k] >>> enough to force B[k + 1] > 0. >>
⇓Bsgn[k + 1] = +1.
⇓PCDraw [k + 1] > 0. >>
⇓Goal: increase Aout from −ve to +ve.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 28: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/28.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage1- cont.
BI[k] = Aout[k] +Arew[k]
= A0 + µ1R[k] ≥ Bd[k + 1]
R[k] =Bd[k + 1] + |A0|
µ1(22)
The associated dissonance pressure is:
PCDraw [k + 1] =
Bsgn[k + 1] · |Aout[k + 1]
|Aout[k + 1]|+Arew[k + 1]=
|A0||A0|+ µ1R[k]
.
(23)
Maximizing (23) subject to (22) results in Bd[k + 1] = 0 andR[k] = |A0|/µ1. For improved robustness:
Bd[k + 1] = 2ε (24)
R[k] =2Bd[k + 1] + |Aout[k]|
µ1=
2ε+ |Aout[k]|µ1
(25)
(26)Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 29: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/29.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .
Avoid exciting the OVJ dynamics that makes Aout ⇓ .Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 30: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/30.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .Avoid exciting the OVJ dynamics that makes Aout ⇓ .
Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 31: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/31.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2
Goal: (0 ≤ Aout[k] ≤ B∗d) for k = 0, N, 2N, 3N, . . .
Use sequence of reward impulses, each impulse appliedevery N days.
Inorder to raise Aout[k], give the child R[k]<<< enoughto be :
Rejected by the child ⇒ PCD < 0⇒ Aout ⇑ .Avoid exciting the OVJ dynamics that makes Aout ⇓ .Avoid attitude reversal.
Q. What is the appropriate value of R[k] that guaranteeabove three conditions satisfied ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 32: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/32.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
To enforce the child to reject the reward R[k] force:
BI[k + 1] < Bd[k + 1]
Aout[k] +Arew[k] < Bd[k + 1]
Aout[k] + r1Arew[k − 1] + µ1(1− r1)R[k − 1] < Bd[k + 1]
R[k] <Bd[k + 1]− r1Arew[k]−Aout[k]
µ1(1− r1)
R[k] <Bd[k + 1]−Aout[k]
µ1(27)
Equation(27) gurantees child reject reward and OJ = 0.
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 33: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/33.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
Attitude reversal is avoided on day k+1 if R[k] is chosen s.t :
Bd[k]−BI[k] ≤ αrevAout[k], Aout[k] ≥ 0
Bd[k] +Aout[k]−Arew[k] ≤ αrevAout[k]
Aout[k] + r1Arew[k − 1] + µ1(1− r1)R[k − 1] ≤ Bd[k + 1]
R[k] ≥ Bd[k + 1]− (αrev + 1)Aout[k]
µ1(28)
Equation(28) gurantees avoidance of attitude reversal.Q. How to keep R[k] at a reasonable level ?
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 34: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/34.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
By introducing a controller tuning parameter β ∈ (0, 1), theaggressiveness of attitude increase can be adjusted:
Ad = βAout[k] + (1− β)(Aout[k] +K1(1− rN−12 )).
R[k] =Aout[k]
µ1
(K1(1− rN−1
2 )
Aout[k] +K1(1− rN−12 )−Ad
− 1
). (29)
To avoid driving the attitude higher than needed (i.e., beyondB∗
d), we add a saturator as follows:
Ad = min{B∗d , βAout[k] + (1− β)(Aout[k] +K1(1− rN−1
2 ))}.(30)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 35: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/35.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Controller Design: Stage2 - cont.
Get the value of Bd[k + 1] from the formulas of R[k] :
Bdmin[k] = Aout[k](K1(1− r2)N−1
Aout[k] +K1(1− r2)N−1 −Ad(31)
Bdmax[k] = Aout[k](K1(1− r2)N−1
Aout[k] +K1(1− r2)N−1 −Ad+ αrev
(32)
Bdmin[k] < Bd[k + 1] ≤ Bdmax[k]. (33)
Bd[k + 1] = γBdmin[k] + (1− γ)Bdmax[k]. (34)
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 36: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/36.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES NO
NO
NO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 37: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/37.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES
NONO
NO
NO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 38: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/38.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Simulation Results
0 5 10 15 20 25 30 350
50
100
150
Day number (k)
Behavio
r (m
ins)
Bd
*
B[k]
Bd[k]
Open−Loop Implementation
0 5 10 15 20 25 30 35
0
50
100
YESYES
YESYES
NO NO NO NO NO NONO
Day number (k)
Rew
ard
Offere
d (
$)
R[k]
0 5 10 15 20 25 30 35
−50
0
50
Day number (k)
Attitude (
min
s)
Aout
[k]
∆ Aout
CD[k]
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 39: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/39.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.
Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.
In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.
Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 40: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/40.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).
Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.
Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 41: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/41.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series
![Page 42: AdaptiveLab Talk1](https://reader034.fdocuments.net/reader034/viewer/2022052600/55896fafd8b42a0f078b466c/html5/thumbnails/42.jpg)
Adaptive LabTalk Series
Electrical andComputer
Engineering
Outline
ProblemDefinition
System Model
Overall Model
Theory ofPlannedBehavior
CognitiveDissonance
Theory ofOverjustification
ControllerDesign
Assumptionsand InitialConditions
ControllerDesign: Stage1
ControllerDesign: Stage2
SimulationResults
Conclusion
Conclusion and Future Work
A new model-based behavior-modification algorithm havebeen developed.
Pros:
No reward are required in the long term.Good transient behavior (i.e. no overshoot).Flexible timing of the control scheme.
Cons:
The approach requires good knowledge of the plantparameters.In case closed-loop implementation: A regularmeasurement of Aout is needed.Lacks experimental validation of the plant model.
Future work:
Online parameter estimation of plant parameters.Experimental validation of plant model
Electrical and Computer Engineering Adaptive Lab Talk Series