Emerging Applications of VR Electrical and Computer Engineering Dept.
Dept. of Computer Science, Dept of Engineering Systems and ...
Transcript of Dept. of Computer Science, Dept of Engineering Systems and ...
Prof. Madhur [email protected]
Assistant ProfessorDept. of Computer Science, Dept of Engineering Systems and EnvironmentUVA Link-Lab
PhD, MS: University of PennsylvaniaCo-Founder Flexergy.ai [NSF SBIR]
F1/10 Autonomous Racing Founder and Race DirectorDirector of Cavalier Autonomous Racing
Research: Artificial Intelligence, Robotics, Autonomous Systems, and Control Systems
Applications: Cyber-Physical Systems, Smart Cities, and Autonomous Vehicles
Link-Lab UVA Engineering | Madhur Behl
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 2
Smart CitiesSmart and
Connected HealthAutonomous
Systems
CS
SIE
ECE
Civil
ME
What do I do..Modeling, simulation, control, optimization, and implementation of Cyber-Physical System
Cyber-Physical Energy Systems
Internet of things Critical Infrastructures & Smart Cities
Safety of Autonomous Vehicles
Data Predictive Control: Interfacing machine learning suitable for predictive control.
DeepExplainations:Answering open-ended queries using procedural generation and interpretable models.
DeepRacing AI:Algorithms for operating autonomous cars at the limits of their control
Madhur Behl - [email protected] of modeling for CPS – Fall 2020
Artificial Intelligence for
Autonomous Racing
The Internet of Wasted Things Autonomous Electric Vehicle
Charging Robot
• Course logistics (5-7 mins)• Course introduction ( the interesting stuff !)
This lecture
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 5
• Timings: Video Lectures released on Tue & Thu,
• Location: Online Asynchronous (on Collab) – with live office hours
• Course website: https://linklab-uva.github.io/modeling_cps/
• All lectures notes/slides/assignments/will be posted on this website.
• Piazza: piazza.com/virginia/fall2020/cs6501003sys6581005/home
• Prerequisites (must have):• Familiarity with Matlab / Simulink.• Mathematical maturity (differential equations, matrix operations, some calculus, probability
distributions)
•Prerequisites (good to have):• Machine learning• Temporal logic• Model predictive control
Course Logistics
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 6
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 7
Teaching Assistants
Aron Harder
Computer Science, PhD
Office Hours: TBA
• No midterm, No final exam.
• The course has three modules (more on this later). Each module is equally weighted. • Energy CPS, Medical CPS, and Automotive CPS
• ~3-4 assignments in each module, comprising of:• Problem sets• Coding/implementation assignments
• 2 late homework submissions permitted – No questions asked.• Should not be late by more than 1 day. 25% grade lost for each additional late day.
• Abide by the UVA honor system. Absolutely no code/solution sharing !
Grading
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 8
• Timings: Monday 2-3pm, or by appointment.
• Location: On Zoom [Link and pwd will be released on piazza]
• Available by appointment outside of the listed hours:• To discuss course assignments or lectures.• Research opportunities.
• Live office hours will also be held by the TA.
Office Hours
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 9
• Watch the videos for the lectures
• Check Piazza and the Course website for announcements, and assignments.
• Ask questions on piazza and during the weekly office hours!• Ask, disagree, debate..
Your responsibilities.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 10
what is this course about ?
Principles of Modeling for Cyber-Physical Systems
Lets break it down..
1. What are Cyber-Physical Systems ?2. What do you really mean by modeling ?3. What principles am I going to learn about ?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 11
Cyber-Physical Systems
Deeply integrating
computation, communication, and control
into physical systems
Physical = some tangible, physical device or system + environmentCyber = computational + communication
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 12
Application domains: Transportation
Faster, safer, more energy-efficient air
travel
Improved use of airspace
Autonomous unmanned drones
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 13
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 14
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 15
Application domains: Transportation
Safety, security, and control of autonomous cars
• Connected vehicles.• Autonomous
fleets/ride sharing.• Traffic management
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 16
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 17
Application domains: Energy
• Smart buildings.• Energy-efficient
operation.
• Smart homes.• EV charging/solar
rooftops
• Reliable and resilient electricity grid.
• Micro grids.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 18
Application domains: Healthcare + Biomedical
• Electronic patient record management.
• In home healthcare delivery.
Health and well being monitoring devices.
Safety, and security of medical devices and heath management systems.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 19
Application domains: Critical Infrastructure
• Water & waste management.• Storm-water/flood control.
Structural health monitoring
Utility infrastructure- Gas, Electricity, Steam.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 20
Application domains: … and many more
Agriculture Manufacturing Industrial Control
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 21
Characteristics of CPS
• Pervasive computation, sensing and control
• Networked at multi- and extreme scales
• Dynamically reorganizing/ reconfiguring
• High degrees of automation
• Dependable operation with potential requirements for high assurance of reliability, safety, security and usability
• With / without human in-the-loop
• Conventional and unconventional substrates / platforms
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 22
Closing the loop
Plant/System
Controller
ActuationSensingCyber
Physical
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 23
Human in the loop
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 24
Cyber-Physical Systems
Deeply integrating
computation, communication, control, and humans
into physical systems
Physical = some tangible, physical device or system + environmentCyber = computational + communicational
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 25
Cyber-Physical Systems - Goals
Transform how we interact with the physical world
Fusion of physical and computational sciences
Produce significant impact on society
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 26
Why is CPS hard ?
package org.apache.tomcat.session;
import org.apache.tomcat.core.*;
import org.apache.tomcat.util.StringManager;
import java.io.*;
import java.net.*;
import java.util.*;
import javax.servlet.*;
import javax.servlet.http.*;
/**
* Core implementation of a server session
*
* @author James Duncan Davidson [[email protected]]
* @author James Todd [[email protected]]
*/
public class ServerSession {
private StringManager sm =
StringManager.getManager("org.apache.tomcat.session");
private Hashtable values = new Hashtable();
private Hashtable appSessions = new Hashtable();
private String id;
private long creationTime = System.currentTimeMillis();;
private long thisAccessTime = creationTime;
private long lastAccessed = creationTime;
private int inactiveInterval = -1;
ServerSession(String id) {
this.id = id;
}
public String getId() {
return id;
}
public long getCreationTime() {
return creationTime;
}
public long getLastAccessedTime() {
return lastAccessed;
}
public ApplicationSession getApplicationSession(Context context,
boolean create) {
ApplicationSession appSession =
(ApplicationSession)appSessions.get(context);
if (appSession == null && create) {
// XXX
// sync to ensure valid?
appSession = new ApplicationSession(id, this, context);
appSessions.put(context, appSession);
}
// XXX
// make sure that we haven't gone over the end of our
// inactive interval -- if so, invalidate and create
// a new appSession
return appSession;
}
void removeApplicationSession(Context context) {
appSessions.remove(context);
}
/**
* Called by context when request comes in so that accesses and
* inactivities can be dealt with accordingly.
*/
void accessed() {
// set last accessed to thisAccessTime as it will be left over
// from the previous access
lastAccessed = thisAccessTime;
thisAccessTime = System.currentTimeMillis();
}
void validate()
Computing Control Systems
Crosses Interdisciplinary Boundaries
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 27
Why is CPS hard ?
• Disciplinary boundaries need to be realigned• New fundamentals need to be created• New technologies and tools need to be developed• Education need to be restructured
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 28
Why is CPS hard ?
• Disciplinary boundaries need to be realigned• New fundamentals need to be created• New technologies and tools need to be developed• Education needs to be restructured
Hmmm, I wonder if there is a course which is trying to achieve this?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 29
what is this course about ?
Principles of Modeling for Cyber-Physical Systems
Lets break it down..
1. What are Cyber-Physical Systems ?2. What do you really mean by modeling ?3. What principles am I going to learn about ?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 30
Modeling types: Physical modeling
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 31
Modeling types: Functional/compositional modeling
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 32
Do you know of any tools for functional modeling ?
Modeling types: Mathematical modeling
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 33
Modeling types: Computational modeling
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 34
Verification
Validation
A word about mathematical models
Physical SystemReality
ObservationsMathematical
models
Observationalerrors Modeling
errors
Computational/Simulation
models
Discretizationerrors
Control &implementation
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 35
“Essentially all models are wrong, but some are useful”
- George E.P. Box (statistician)
….this course is about building useful models.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 36
This course: Three CPS domains
Energy CPS Medical CPS Automotive CPS
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 37
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 38
Smart CitiesSmart and
Connected HealthAutonomous
Systems
CS
SIE
ECE
Civil
ME
Energy CPS Medical CPS Automotive CPS
1/Ugw
1/Ugi
1/Ugo
Cgo
Cgi
Tgo
Tg
Tgi
Tz
Q.rad,e
1/Uei1/Uew1/Ueo
Ceo Cei
Ta
Teo Tei
Q.sol,e
Q.solt/2
1/Uci
1/Ucw
1/Uco
Cci
Cci
Ta
Q.sol,c
Q.rad,c
Ta
Q.conv + Q
.sens
1/Uii 1/Uiw 1/Uio
Cii Cio
Tci
Tco
1/Uwin Tii Tio
Q.solt/2
Q.rad,g
[External
Walls]
Ti
[Ceiling]
[Floor]
[Internal
Walls]
[Windows]
White-boxState-space
TimedAutomata
Data-driven
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 39
Energy CPS Medical CPS Automotive CPS
EnergyPlus
Matlab
UPPAAL
Simulink
CARLA
Matlab/PythonPrinciples of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 40
Energy CPS ModulePrinciples of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 41
42Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
Price Volatility: Summer peak
20th, July 2015
Nominal price: $25/MWh Peak Price: $800/MWh
32x!
43Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
44
“All kilowatts are not created equally”
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 45
Price Volatility: Winter peak
24th, January 2014
Nominal price: $31.21/MWh Peak Price: $2,680.21/MWh
86x!
46Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
Price volatility is the new normal
PJM (ISO) Locational Marginal Prices (LMPs) example
47Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
At large scales: University Campus
72 MW
Peak (UCAP)
187 Buildings
300,000SCADA Tags
4 MillionGallons of chilled water (@42F)
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 48
~$28MAnnual Electricity Bill
In 2011
Peak > UCAP
30 min
$720,000Penalty for 30 minutes
Economic incentives for model based control
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 49
Why focus on Buildings ?
Portion of global energy use
Portion of electricity consumption in the United States
Portion of global total CO2 emissions
40%
70%
1/3Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 50
Model-Based control for buildings
Traditional rule-based building control
• Sequence of operations or planned steps.• Pre-defined rules set by building
engineers.• Purely reactive.• Equipment-level controllers (PID) ensure
reference tracking.
Start Chiller #3 at 4:00 am
Start Chiller #1
Chiller #3 at 90%
load
Took 23 min yesterday
Took 42 min today(but price increased !)
Model-based building control
• Model how the building will response to disturbances (weather, occupants etc.)
• Predictive• Control design:
• Energy-efficiency• Demand flexibility• Fault handling
• Okay to use equipment level PID control
Model Predictive Control (MPC)51Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
The control problem in buildings
52
Integrated control of:
• Heating• Cooling• Ventilation• Lighting• Blinds
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
State-space ‘RC’ thermal modeling
1/Ugw
1/Ugi
1/Ugo
Cgo
Cgi
Tgo
Tg
Tgi
Tz
Q.rad,e
1/Uei1/Uew1/Ueo
Ceo Cei
Ta
Teo Tei
Q.sol,e
Q.solt/2
1/Uci
1/Ucw
1/Uco
Cci
Cci
Ta
Q.sol,c
Q.rad,c
Ta
Q.conv + Q
.sens
1/Uii 1/Uiw 1/Uio
Cii Cio
Tci
Tco
1/Uwin Tii Tio
Q.solt/2
Q.rad,g
[External
Walls]
Ti
[Ceiling]
[Floor]
[Internal
Walls]
[Windows]
Wall Dynamics
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 53
Whole building energy simulation
Using EnergyPlus.
Generating input-output data
Modeling a building in MATLAB
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 54
Non-linear parameter estimation and model validation
Model Accuracy for Training data
RMSE: 0.062 °CR2: 0.983
Model Accuracy for Test Data
RMSE: 0.091 °CR2: 0.948
Baseline
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 55
Predictive control
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 56
Medical CPS ModulePrinciples of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 57
Software related vehicle recalls
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 58
• Over 600,000 cardiac medical devices recalled from 1990-2000
– 40% of which were due to software issues
• 2008-12: 15% of all the medical device recalls due to software
Implantable Pacemaker Implantable Cardioverter-Defibrillator (ICD)59Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
• Muscle contraction triggered by electrical signals
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 60
• Slow generation and conduction of electrical signals
• Slow heart rate
• Symptom: fainting, dizziness
• Could lead to heart attack
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 61
• Two leads in heart chambers
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 62
• Two leads in heart chambers
• Deliver electrical signals when heart rate is low
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 63
• Two leads in heart chambers
• Deliver electrical signals when heart rate is low
• Device malfunction may result in injury or death
• Flawed devices are recalled
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 64
Pacemaker Autonomous device with minimum human interactionLimited diagnostic/therapy capabilityIts safety must be evaluated within its environment
The physical plant:Complex dynamics of the heartInteraction between the heart and the bodyDomain knowledge
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 65
Pacemaker
Scenario 1
Scenario 2
Scenario 3
66Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 67
Refractory
Time
Vo
ut
Rest ERP RRP Rest
Refractory
Time
Vo
ut
Rest ERP RRP Rest
node
path
node
Principles of modeling for CPS – Fall 2020 68Madhur Behl - [email protected]
69Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected]
70
Cellular Level
Node Automaton
• Divide refractory period into time periods
• Model refractory properties as timers using timed automata.
• These time periods can be measured during EP study
Madhur Behl - [email protected]
72
The UPPAAL model of the closed-loop system
73
Model checking – Counter example guided.
Safety property
Ventricular rate should not be equal or above the upper rate limit for more than 30 beats
𝐴[]𝑛𝑜𝑡 𝑚𝑜𝑛𝑖𝑡𝑜𝑟. 𝑒𝑟𝑟
Heart Pacemaker
Aget!
Vget!
AP!
VP!
monitor
Madhur Behl - [email protected]
Automotive CPS ModulePrinciples of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 74
Localization and Mapping
Scene Understanding
Trajectory Planning and Control
Human Interaction
Where am I ?
Where/who/what/why of everyone/everything else ?
Where should I go next ? How do I steer and accelerate ?
How do I convey my intent to the passenger and everyone else ?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 75
Localization and Mapping
Scene Understanding
Trajectory Planning and Control
Human Interaction
Where am I ?
Where/who/what/why of everyone/everything else ?
Where should I go next ? How do I steer and accelerate ?
How do I convey my intent to the passenger and everyone else ?
Dee
p L
earn
ing
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 76
Localization and Mapping
Scene Understanding
Trajectory Planning and Control
Human Interaction
Where am I ?
Where/who/what/why of everyone/everything else ?
Where should I go next ? How do I steer and accelerate ?
How do I convey my intent to the passenger and everyone else ?
End
-to
-en
d le
arn
able
Dee
p N
eura
l Net
wo
rks
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 77
Center camera CNN
Steeringcommand
Drive-by-wire interface
End-to-End Deep Learning for Self Driving Cars - NVIDIA
Video credit:DeepTesla
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 78
Machine intelligence is largely about training data.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 79
Image courtesy: Cognata
When’s a pedestrian not a pedestrian? When it’s a decal.
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 80
One car ? or Multiple cars ?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 81
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 82
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 83
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 84
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 85
There is a bus right next to you!!Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 86
Autonomous Driving: End-to-End
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 87
Autonomous Driving: End-to-End
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 88
Autonomous Driving: End-to-End
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 89
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 90
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 91
what is this course about ?
Principles of Modeling for Cyber-Physical Systems
Lets break it down..
1. What are Cyber-Physical Systems ?2. What do you really mean by modeling ?3. What principles am I going to learn about ?
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 92
• Modeling for predictive control.
• Parameter estimation. • Linear and non-linear
• Model checking
• Model validation
• Model selection• Model abstraction/reduced order modeling
• End-to-end learning
This course: Modeling principles
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 93
This course: Learning objectives
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 94
DomainExpert
(Physical System)
Modelingprinciples
Tools &implementation
This course: Learning objectives
Principles of modeling for CPS – Fall 2020 95
The future belongs to those who learn more
skills and combine them in creative ways.
Madhur Behl - [email protected]
This course: Becoming a Cyber-Physicist
Parameter estimation
Med CPS
Automotive CPS
First principles modeling
Model checking
Timed automataData-driven modeling
State-SpaceModeling
Energy CPS
EnergyPlusUPPAAL
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 96
Cyber-Physicist
Next lecture:
• How to predict the future..
• State-space modeling using first principles.• Mechanical, electrical, thermal systems• ODEs and elements of white box modeling
Principles of modeling for CPS – Fall 2020 Madhur Behl - [email protected] 97