Michael Bloem PhD Defense - Optimization and Analytics for Air Traffic Management
-
Upload
mbloem -
Category
Engineering
-
view
88 -
download
4
Transcript of Michael Bloem PhD Defense - Optimization and Analytics for Air Traffic Management
Optimization and Analytics
for Air Traffic Management
Enabling Decision-Support Tools
Michael Bloem
Stanford University NASA Ames Research Center
Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
2
Air Traffic Management is Important
3
Air Traffic Management is Important
civil aviation
was responsible for
5.4% of US GDP
in 2012
[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]
3
Air Traffic Management is Important
civil aviation
was responsible for
5.4% of US GDP
in 2012
837 million passengers
carried by airlines
operating in US airspace
in 2012
[FAA’s 2014 "The Economic Impact of Civil Aviation on the US Economy"]
3
Air Traffic Management is Complex
4
Air Traffic Management is Complex
∼50,000 flights/day
5,000+ flights simultaneously
4
Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5
Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5
Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5
Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5
Air Traffic Management is
Accomplished Largely by
Human Decision Makers
5
Research Topics: ATM Decisions
6
Research Topics: ATM Decisions
1 Area supervisors:area configuration selection
– Motivation: prescriptive decision model
6
Research Topics: ATM Decisions
1 Area supervisors:area configuration selection
– Motivation: prescriptive decision model
2 Operations managers:assignment of flights to slots
– Motivation: insight into airline delay costs
6
Research Topics: ATM Decisions
1 Area supervisors:area configuration selection
– Motivation: prescriptive decision model
2 Operations managers:assignment of flights to slots
– Motivation: insight into airline delay costs
3 Flow managers:Ground Delay Program implementation
– Motivation: predictive capability and insights
6
Research Objective
decision-
support
tool
7
Research Objective
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
7
Research Approach
expert input
& feedback
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8
Research Approach
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8
Research Approach
fast-time
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8
Research Approach
fast-time
simulations
human-in-
the-loop
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
8
Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
9
En-Route Air Traffic Control Centers
10
En-Route Air Traffic Control Centers
10
Cleveland Center
11
Cleveland Center Area 4
4645
12
Cleveland Center Area 4
47
48
49
4546
36,000
31,000
24,000Longitude
Latitude
Altitude
13
Sample Area of Specialization Resources
Sectors
47
48
49
4546 4546
4749
48
14
Sample Area of Specialization Resources
Sectors
Controllers
available to fill
operating
positions
47
48
49
4546 4546
4749
48
09:00–09:30 :
09:30–11:00 :
14
Sample Area of Specialization Resources
Sectors
Controllers
available to fill
operating
positions
Workstations
47
48
49
4546 4546
4749
48
09:00–09:30 :
09:30–11:00 :
48
14
Modeling Decisions and Developing Algorithms
fast-time
simulations
human-in-
the-loop
simulations
expert input
& feedback
operational
decision
data
solution
algorithm
decision-
support
tool
decision
model
constraints
objective
15
Modeling Decisions and Developing Algorithms
fast-time
simulations
15
Sample Sectors, Configuration, and Traffic
sample sectors:
16
Sample Sectors, Configuration, and Traffic
sample sectors: at time step k:
Ck
16
Sample Sectors, Configuration, and Traffic
Tk
sample sectors: at time step k:
Ck
16
Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K∑
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
17
Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K∑
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
gk: single-time step advisory cost
17
Decision Model
Configuration Schedule Advisory Problem (CSA)
minimize
K∑
k=1
gk(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
gk: single-time step advisory cost
Ck: set of valid configurations at k
17
Sample Scenario
0timestep1 2 3 4 5
18
Sample Scenario
0timestep1 2 3 4 5
18
Sample Scenario
0timestep1 2 3 4 5
18
Sample Scenario
0timestep1 2 3 4 5
18
Sample Scenario
0timestep1 2 3 4 5
18
Sample Scenario
0timestep1 2 3 4 5
18
Configuration Constraints
0timestep1 2 3 4 5
19
Configuration Constraints
0timestep1 2 3 4 5
19
Configuration Constraints
0timestep1 2 3 4 5
C2
19
Possible Algorithm Advisory
0timestep1 2 3 4 5
20
Possible Algorithm Advisories
0timestep1 2 3 4 5
21
Possible Algorithm Advisories
0timestep1 2 3 4 5
25 = 32 possibilities
21
Advisory Cost: Static Cost
gSk(Ck, Tk)
20% 40% 60% 80% 100% 120%0
2
4
6
8
10
Open Sector Load[aircraft count/Monitor Alert Parameter]
Two−operating positionstatic costgS
k(Ck, Tk)
22
Advisory Cost: Static Cost
gSk(Ck, Tk)
20% 40% 60% 80% 100% 120%0
2
4
6
8
10
Open Sector Load[aircraft count/Monitor Alert Parameter]
Two−operating positionstatic cost
One−operating positionstatic cost
gSk(Ck, Tk)
22
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
23
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
sector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations
23
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
open sector gaining
sector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations
2nd operating position
23
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
23
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
single-time step advisory cost:
gk(Ck−1, Tk−1, Ck, Tk) = gSk(Ck, Tk)+β
RgRk(Ck−1, Tk−1, Ck, Tk)
23
Advisory Cost: Reconfiguration Cost
gRk(Ck−1, Tk−1,Ck, Tk)
open sector gaining
open sector losingsector & flights moving
(1)
(1)
(2)
(1)
(2)
(2)
(1)
Ck−1, Tk−1 Ck, Tk
between workstations 2nd operating position
2nd operating position
single-time step advisory cost:
gk(Ck−1, Tk−1, Ck, Tk) = gSk(Ck, Tk)+β
RgRk(Ck−1, Tk−1, Ck, Tk)
operational decision data & fast-time simulations
leveraged to select βR
23
Minimum-Cost Advisory
0timestep1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
24
Minimum-Cost Advisory
0timestep1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
gSk(C1, T1)
24
Minimum-Cost Advisory
0timestep1 2 3 4 5
0
00 3
0 1
0 0
8
5
0
gRk(C3, T3, C4, T4)
gSk(C1, T1)
24
Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize
K∑
k=1
gSk(Ck, Tk) + βRgR
k(Ck−1, Tk−1, Ck, Tk)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
25
Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize g(C,T)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
25
Decision Model and Solution Algorithm
Configuration Schedule Advisory Problem (CSA)
minimize g(C,T)
subject to Ck ∈ Ck, k = 0,1,2, . . . , K
Shortest path problem → use A∗ algorithm
25
Decision Model Relative to Previous Research:
Objectives and Constraints
AdvisoryCharacteristic
matchconfigurations to
traffic
few disruptivereconfigurations
number of opensectors
number ofoperatingpositions
26
Decision Model Relative to Previous Research:
Objectives and Constraints
AdvisoryCharacteristic
Cano et al.(2007)
MB et al.(2008–2009)
Tien et al.(2010–2012)
matchconfigurations to
traffic
constraintand
optimizeconstraint constraint
few disruptivereconfigurations
constraintnot
consideredconstraint
number of opensectors
minimize minimizenot
considered
number ofoperatingpositions
notmodeled
not modeled minimize
26
Decision Model Relative to Previous Research:
Objectives and Constraints
AdvisoryCharacteristic
Cano et al.(2007)
MB et al.(2008–2009)
Tien et al.(2010–2012)
matchconfigurations to
traffic
constraintand
optimizeconstraint constraint
few disruptivereconfigurations
constraintnot
consideredconstraint
number of opensectors
minimize minimizenot
considered
number ofoperatingpositions
notmodeled
not modeled minimize
26
Decision Model Relative to Previous Research:
Objectives and Constraints
AdvisoryCharacteristic
Cano et al.(2007)
MB et al.(2008–2009)
Tien et al.(2010–2012)
DecisionModel
matchconfigurations to
traffic
constraintand
optimizeconstraint constraint optimize
few disruptivereconfigurations
constraintnot
consideredconstraint optimize
number of opensectors
minimize minimizenot
consideredconstraint
number ofoperatingpositions
notmodeled
not modeled minimize constraint
26
Decision Model Relative to Previous Research:
Objectives and Constraints
AdvisoryCharacteristic
Cano et al.(2007)
MB et al.(2008–2009)
Tien et al.(2010–2012)
DecisionModel
matchconfigurations to
traffic
constraintand
optimizeconstraint constraint optimize
few disruptivereconfigurations
constraintnot
consideredconstraint optimize
number of opensectors
minimize minimizenot
consideredconstraint
number ofoperatingpositions
notmodeled
not modeled minimize constraint
26
Decision Model vs. Operational Decisions:
Problem Instances for Fast-Time Simulations
47
48
49
4546 4546
4749
48
• traffic and configurations from
231 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• five-minute time steps
27
Decision Model vs. Operational Decisions:
Few Disruptive Reconfigurations
3 6 9 12 150
500
1000
1500
2000
2500
Number ofOpen Sector
Instances
Duration [hours]
operational
model
28
Decision Model vs. Operational Decisions:
Match Configurations to Traffic
20% 40% 60% 80% 100% 120%0
2
4
6
8
10
Open Sector Load[aircraft count/Monitor Alert Parameter]
Low HighJust Right
gSk(Ck, Tk)
29
Decision Model vs. Operational Decisions:
Match Configurations to Traffic
Low Just Right High0
20
40
60
80
100
Percent ofOpen Sector-
Minutes
operational
model
30
Modeling Decisions and Developing Algorithms
fast-time
simulations
31
Modeling Decisions and Developing Algorithms
fast-time
simulations
31
Experts’ Feedback and Suggestion
Limitations of decision model
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
• other advisories: not too sub-optimal
32
Experts’ Feedback and Suggestion
Limitations of decision model
1 difficult to quantify safe and efficient operations
2 does not model individual human controllers
Suggestions for solution algorithm
• return ∼ 3 good advisories
• one advisory: optimal for decision model
• other advisories: not too sub-optimal
• distinct advisories
32
Advisory Difference Metric
Φ(C,C′)
Φ(C,C′) =
K∑
k=1
ϕ(Ck, C′k)
33
Advisory Difference Metric
Φ(C,C′)
Φ(C,C′) =
K∑
k=1
ϕ(Ck, C′k)
Configuration difference metric
ϕ(Ck , C′k) =
¨
1 if Ck and C′kcombine sectors differently
0 else
33
Multiple-Advisories Problem Statement
M ϵ-Optimal d-Distinct Configuration Schedule
Advisories Problem (M-ϵ-d-CSAs)
minimize
M∑
m=1
g(Cm, T)
subject to |CM| =M
Cmk∈ Ck k = 0,1,2, . . . , K, m = 1,2, . . . ,M
C1 ∈ C⋆
CSA( = optimal advisories for CSA)
g(Cm, T)− g(C1, T)
g(C1, T)≤ ϵ m = 2,3, . . . ,M
Φ(Cm, Cm′
) ≥ d ∀m,m′ where m 6=m′
34
Multiple-Advisories Problem Statement
M ϵ-Optimal d-Distinct Configuration Schedule
Advisories Problem (M-ϵ-d-CSAs)
minimize
M∑
m=1
g(Cm, T)
subject to |CM| =M
Cmk∈ Ck k = 0,1,2, . . . , K, m = 1,2, . . . ,M
C1 ∈ C⋆
CSA( = optimal advisories for CSA)
g(Cm, T)− g(C1, T)
g(C1, T)≤ ϵ m = 2,3, . . . ,M
Φ(Cm, Cm′
) ≥ d ∀m,m′ where m 6=m′
M-ϵ-d-CSAs is NP-complete
34
Algorithms for M-ϵ-d-CSAs
35
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
35
Sequential Distinct A∗ (SDA∗)
Finding advisory C1:
C, T C1shortest path
algorithm
36
Sequential Distinct A∗ (SDA∗)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
constrained
shortest path
algorithm
36
Sequential Distinct A∗ (SDA∗)
Finding advisory Cm:
C, T Cm
{C1, C2, . . . , Cm−1}
constrained
shortest path
algorithm
constrained shortest path problem is NP-complete
36
Constrained Shortest Path Algorithm:
Forward Distinct A∗ (FDA∗)
For some λ ≥ 0, find C̃2 that minimizes Lagrangian
L(C2, λ) = g(C2, T)︸ ︷︷ ︸
advisorycost
+λ�
d− Φ(C1, C2)�
︸ ︷︷ ︸
similaritycost
37
Constrained Shortest Path Algorithm:
Forward Distinct A∗ (FDA∗)
For some λ ≥ 0, find C̃2 that minimizes Lagrangian
L(C2, λ) = g(C2, T)︸ ︷︷ ︸
advisorycost
+λ�
d− Φ(C1, C2)�
︸ ︷︷ ︸
similaritycost
Another shortest path problem → use A∗
37
FDA∗ and Duality
Proposition
If M = 2, ϵ =∞, and |C⋆CSA| = 1,
then FDA∗ implements the dual objective:
h(λ) =minimizeC2∈C
L(C2, λ)
38
FDA∗ and Duality
Proposition
If M = 2, ϵ =∞, and |C⋆CSA| = 1,
then FDA∗ implements the dual objective:
h(λ) =minimizeC2∈C
L(C2, λ)
Corollary: If M = 2, ϵ =∞, |C⋆CSA| = 1,
λ = λ⋆, and strong duality holds,
then C̃2 satisfies a necessary condition for C2⋆
(C̃2 = second advisory returned by FDA∗)
38
Sequential Distinct A∗ (SDA∗)
Finding advisory Cm:
C,T Cm
{C1,C2, . . . ,Cm−1}
constrained
shortest path
algorithm
constrained shortest path problem is NP-complete
39
Sequential Distinct A∗ (SDA∗)
Finding advisory Cm:
C,T Cm
{C1,C2, . . . ,Cm−1}
FDA∗
λ
39
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
40
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
40
Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0timestep1 2 3 4 5
41
Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0timestep1 2 3 4 5
41
Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0timestep1 2 3 4 5
41
Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
0timestep1 2 3 4 5
41
Theoretical Guarantees for FBVISAS:
the Good News
42
Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
42
Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
Reconfiguration cost-dominated M-ϵ-d-CSAs instance:
reconfiguring never decreases the advisory cost
42
Theoretical Guarantees for FBVISAS:
the Good News
Simple M-ϵ-d-CSAs instance: M = 2, ϵ =∞, |C⋆CSA| = 1,
C1 = C2 = · · · = CK, and C0 ∈ C1
Reconfiguration cost-dominated M-ϵ-d-CSAs instance:
reconfiguring never decreases the advisory cost
Proposition
If one exists, FBVISAS finds an optimal C2⋆ for
simple reconfiguration cost-dominated instances
42
Theoretical Guarantees for FBVISAS:
the Bad News
Static cost-dominated M-ϵ-d-CSAs instance: βR = 0
43
Theoretical Guarantees for FBVISAS:
the Bad News
Static cost-dominated M-ϵ-d-CSAs instance: βR = 0
Proposition
If d > 1, FBVISAS does not return any C2 for
simple static cost-dominated instances
43
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
44
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
3 Lowest-Cost Paths (LCP)
• finds optimal solution to relaxation• many efficient algorithms
44
Algorithms for M-ϵ-d-CSAs
1 Sequential Distinct A∗ with Shortcuts (SDA∗-SC)
• novel algorithm based on A∗
2 Forward Backward Value Iterationwith Sequential Advisory Search (FBVISAS)
• novel algorithm based on value iteration
3 Lowest-Cost Paths (LCP)
• finds optimal solution to relaxation• many efficient algorithms
4 Value Iteration Fraction Optimalwith Exhaustive Advisory Search
• finds optimal solution• not computationally efficient
44
Modeling Decisions and Developing Algorithms
fast-time
simulations
45
Modeling Decisions and Developing Algorithms
fast-time
simulations
45
Evaluation of Algorithms on Small Instances
47
48
49
4546 4546
4749
48
• traffic and configurations from
two days in December 2011
• nine two-hour instances per day
• 18 total instances
• five-minute time steps
• C requires same number of open
sectors as were used operationally
• request two advisories (M = 2)
• ϵ = 0.2 constraint on cost of C2
• d requires 30 minutes of different
airspace configurations
46
Small Instances: Properties of C2
SDA∗-SC FBVISAS LCP0
20
40
60
80
100
Percent ofInstances
optimal C2⋆
47
Small Instances: Properties of C2
SDA∗-SC FBVISAS LCP0
20
40
60
80
100
Percent ofInstances
optimal C2⋆
feasible C2
47
Evaluation of Algorithms on Realistic Instances
47
48
49
4546 4546
4749
48
• traffic and configurations from
231 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• 4158 total instances
• five-minute time steps
• C only requires same initial
configuration as was used
operationally
• request three advisories (M = 3)
• ϵ = 0.5 constraint on cost
• d requires 30 minutes of different
airspace configurations
48
Realistic Instances: Advisory Costs
0.7 0.8 0.9 1.1 1.2 1.30
20
40
60
80
100
0.99 1.01
Percent ofInstances
Cost Ratio�g(C2 ,T)+g(C3 ,T) for SDA∗-SC
g(C2 ,T)+g(C3 ,T) for FBVISAS
�
SDA∗-SCworse
SDA∗-SCbetter
mean = 1.02
49
Realistic Instances: Computation Times
(on a Workstation)
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Computation
Times
[seconds]
SDA∗-SC FBVISAS
50
Modeling Decisions and Developing Algorithms
fast-time
simulations
51
Modeling Decisions and Developing Algorithms
51
Operational Airspace Sectorization
Integrated System (OASIS)
52
OASIS Screenshot: Multiple Advisories
53
Human-in-the-Loop Simulations Set Up
• eight retired FAA
personnel
• four simulated
scenarios
54
Human-in-the-Loop Simulations Set Up
• eight retired FAA
personnel
• four simulated
scenarios
Three experimental conditions in which users
1 generate configuration schedule
2 select from among algorithm-generated advisories
3 select from among algorithm-generated advisories
and then modify
54
Human-in-the-Loop Simulations Results
[Lee et al., 2013]
55
Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
55
Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
Providing multiple advisories added value
• in more than 60% of instances, users selected
second or third advisory
• when asked how many advisories they wanted the
tool to provide, users requested an average of 2.8
55
Human-in-the-Loop Simulations Results
[Lee et al., 2013]
Advisories enable safe and efficient operations
• average acceptability of selected
algorithm-generated advisories: > 4 out of 5
• user modifications were minor and led to
no significant improvement in acceptability
Providing multiple advisories added value
• in more than 60% of instances, users selected
second or third advisory
• when asked how many advisories they wanted the
tool to provide, users requested an average of 2.8
Computation times were rated as highly acceptable
55
Modeling Decisions and Developing Algorithms
56
Outline
• Air traffic management background
• Research topics
• Research objective and approach
• Case study:
decision-support tool for area supervisors
• Summary of contributions
57
Summary of Contributions
58
Summary of Contributions
1 Area supervisors:area configuration selection
– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths
– implemented algorithm in decision-support tool
58
Summary of Contributions
1 Area supervisors:area configuration selection
– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths
– implemented algorithm in decision-support tool
2 Operations managers:assignment of flights to slots
– designed maximum-likelihood-based algorithm that– generated insights from operational decision data
58
Summary of Contributions
1 Area supervisors:area configuration selection
– developed prescriptive decision model– designed efficient algorithms to findlow-cost and distinct paths
– implemented algorithm in decision-support tool
2 Operations managers:assignment of flights to slots
– designed maximum-likelihood-based algorithm that– generated insights from operational decision data
3 Flow managers:Ground Delay Program implementation
– deployed supervised learning and inversereinforcement learning models that
– produced predictive capability and insights
58
Relevant peer-reviewed conference papers1 M. Bloem and P. Gupta, "Configuring Airspace Sectors with Approximate Dynamic
Programming," ICAS, September 2010.
2 M. Bloem and H. Huang, "Evaluating Delay Cost Functions with Airline Actions inAirspace Flow Programs," USA/Europe ATM R&D Seminar, June 2011.
3 M. Bloem and N. Bambos, "Coordinated Tactical Air Traffic and Airspace Management,"IEEE CDC, December 2011.
4 M. Bloem, M. Drew, C. F. Lai, and K. Bilimoria, "Advisory Algorithm for Scheduling OpenSectors, Operating Positions, and Workstations," AIAA ATIO, September 2012.
5 M. Bloem, H. Huang, and N. Bambos, "Approximating the Likelihood of Historical AirlineActions to Evaluate Airline Delay Cost Functions," IEEE CDC, December 2012.
6 M. Bloem and N. Bambos, "An Approach for Finding Multiple Area of SpecializationConfiguration Advisories," AIAA ATIO, August 2013.
7 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloningand Inverse Reinforcement Learning," AIAA ATIO, June 2014.
8 M. Bloem and N. Bambos, "Infinite Time Horizon Maximum Causal Entropy InverseReinforcement Learning," IEEE CDC, December 2014.
Relevant journal articles1 M. Bloem, P. Gupta, and P. Kopardekar, "Algorithms for Combining Airspace Sectors," Air
Traffic Control Quarterly, Vol. 17, No. 4, 2009.
2 M. Bloem, M. Drew, C. F. Lai, and K. D. Bilimoria, "Advisory Algorithm for Scheduling OpenSectors, Operating Positions, and Workstations," AIAA Journal of Guidance, Control, andDynamics, Vol. 37, No. 4, July–August 2014.
3 M. Bloem and N. Bambos, "Air Traffic Control Configuration Advisories fromNear-Optimal Distinct Paths," AIAA Journal of Aerospace Information Systems, Vol. 11,No. 11, November 2014.
4 M. Bloem and N. Bambos, "Ground Delay Program Analytics with Behavioral Cloningand Inverse Reinforcement Learning," AIAA Journal of Aerospace Information Systems,accepted on 21 December 2014, available online on 03 March 2015.
5 M. Bloem and N. Bambos, "Stochastic Models of Ground Delay ProgramImplementation for Prediction, Simulation, and Insight," Journal of AerospaceOperations, invited submission to special issue, expected publication in late 2015.
59
Acknowledgments
• Stanford
• NASA
• Family & friends
60
Questions?
Backup Slides
62
Why Improve Area Configuration Planning?
1 more efficient flights
2 fewer disruptions to area supervisors and
traffic managers
3 better controller staff management
4 prepare for increased flexibility in future operations
→ more complex planning
63
Modeling Decisions and Developing Algorithms
fast-time
simulations
64
Operational Decision Data Set and
Fast-Time Simulation Setup
47
48
49
4546 4546
4749
48
• traffic and configurations from
230 days in 2011 and 2012
• 6 am to midnight local time
• rolling horizon: implement first
hour of two-hour advisories
• five-minute time steps
• only airspace configurations
• constraint: only use typical
configurations
65
Static Cost versus Reconfiguration Cost
700 800 900 1000 11000
50
100
150
200
Total
Reconfiguration
Cost
Total Static Cost
operational
βR = 0.5
βR = 15
ratio(βR) =total reconfiguration cost(βR)
total static cost(βR)
error(βR) =��ratio(operational)− ratio(βR)
��
66
Selecting βR
0 1.75 5.0 10 155
10
15
20
25
Total Error
βR
set βR = 1.75
67
Uncertainty in Traffic Predictions:
Why Not Incorporated?
• Experts understand prediction errors
• Avoid inconsistency with other deterministic tools
68
Uncertainty in Traffic Predictions:
Uncertain Future Aircraft Counts
0 5 10 15 200
0.05
0.1
0.15
0.2
0.25
Aircraft Count
Probability
prediction
distribution
69
Uncertainty in Traffic Predictions:
Impact on Advisory Costs
Advisories generated using predicted traffic with realistic errors
are typically 2.5%–12.5% costlier
than advisories generated with perfectly-predicted traffic
70
Uncertainty in Traffic Predictions:
Approximate Dynamic Programming Approach
• Finite time horizon Markov decision process (MDP)
– state: sector aircraft counts and predictions thereof– action: current configuration (not a schedule)– state transitions: based on prediction errors inoperational tool
– objective: similar to decision model (CSA)
• Rollouts algorithm performs 15% better than a
heuristic and within 2% of optimal
• Rollouts algorithm computes solutions fast enough
for real-time implementation
M. Bloem and P. Gupta, "Configuring Airspace Sectors withApproximate Dynamic Programming," ICAS, September 2010.
71
Uncertainty in Traffic Predictions:
Other Approaches
• improve predictions with a statistical model
• discount contribution to cost by predicted aircraft
• stochastic shortest path problem
• risk-averse advisory
• larger MDP formulation
• robust set of advisories
72
SDA∗ with Shortcuts (SDA∗-SC)
Objectives:
1 computation time ≤ that of A∗ for each advisory
→ use data from reverse A∗ for C1
to find "shortcuts"
2 no re-tuning of λ per instance or advisory
→ normalize advisory cost and similarity cost terms
73
Forward Backward Value Iteration
with Sequential Advisory Search (FBVISAS)
• Use value iteration to find minimum-cost advisory
through each Ck ∈ Ck, k = 1, . . . ,K
• Sort these advisories from low to high cost
• Initialize set of advisories to return: CM ← ∅
• Loop:
– select next advisory– if advisory is sufficiently different from otheradvisories in C
M, add it to CM
– if |CM| =M, stop
74
Multiple Advisories Problem is NP-Complete
For an arbitrary instance of the independent set
problem on G(V,E), construct M-ϵ-d-CSAs instance:
• M = required number of independent nodes
• gk() = 0
• K = 1 and C1 = V
• d = 1
• configuration difference metric:
ϕ(Ck,C′k) =
¨
1 if Ck and C′kindependent in G(V,E)
0 else
∃ solution to the independent set instance
⇔ ∃ solution to the M-ϵ-d-CSAs instance
75
Computational Complexity
of Multiple Solutions Problem Algorithms
Algorithm Complexity
VIFOEAS O(n3K)a
FBVISAS O(n2K + nK log(nK + 1))
SDA∗-SC O(Mn2K log(nK +1))
Eppstein lowest-cost paths O(n2K + nK log(nK + 1) +M)
Suurballe lowest-cost node-disjoint paths O(Mn2K log(nK +1))b
a This does not include the complexity of searching through�|Cϵ |
M
�advisory
subsets.b This assumes that Dijkstra’s algorithm is used as a subroutine for findingshortest paths.
76
Airline Delay Cost Model Evaluation
decision-
support
tool
77
Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
78
Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
default assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
78
Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
airline-selected assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
78
Airline Delay Cost Model Evaluation
Motivation: how costly is a flight delay?
Operational decision data:
airline-selected assignment
ABC1
10:00
ABC2
10:30
Slot #60
11:00
Slot #90
12:00
Contribution: novel algorithm enables determination of
delay cost model and cost noise parameters that
maximize an approximation of likelihood of data
78
Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
79
Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
79
Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
• determined delay cost model that maximizes
approximate likelihood of data for each airline
79
Airline Delay Cost Model Evaluation
• developed novel algorithm for finding cost noise
parameters that maximize an approximation of the
likelihood of minimum-cost matching problem
instance solutions, given a candidate cost model
• evaluated more than ten proposed airline delay
cost models using hundreds of slot swap decisions
• determined delay cost model that maximizes
approximate likelihood of data for each airline
• estimated cost noise parameters for each delay
cost model for each airline
79
Ground Delay Program (GDP)
Implementation Modeling
decision-
support
tool
80
Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
81
Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
Motivation: predict & understand GDP implementation
81
Ground Delay Program (GDP)
Implementation Modeling
Operational decision data:
arrival rate control arrivals
start end per
airport time time hour
SFO 9:00 am noon 30
Motivation: predict & understand GDP implementation
Contribution: produced predictive models and insights
by developing behavioral cloning and inverse
reinforcement learning models of GDP implementation
81
Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
82
Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
82
Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
• determined that neither class of model provides
much evidence that conditions beyond those in the
next two hours impact GDP implementation
82
Ground Delay Program (GDP)
Implementation Modeling
• evaluated behavioral cloning and inverse
reinforcement learning models of GDP
implementation at EWR and SFO using hundreds of
days of operational decision data
• demonstrated that a behavioral cloning model can
produce superior predictions of GDP
implementation
• determined that neither class of model provides
much evidence that conditions beyond those in the
next two hours impact GDP implementation
• estimated a reward function that provides insight
into metrics guiding GDP implementation
82