Space-time Correlation Analysis of Quality-of-Service at Major US Airports O’Hare International Airport and Minneapolis Airport: A case study

Loan Le, lle@gmu.edu George Donohue, gdonohue@gmu.eduChun-Hung Chen, cchen9@gmu.eduSystems Engineering and Operations Research Department, ST2, MSN 4A6George Mason University, 4400 University Drive, Fairfax, VA 22030Tel. 703-993-1670Fax. 703-993-1521

Submission date: Aug 1st, 2003

Words count: 7,478

Abstract:

In this paper we study and analyze the space-time correlation among different variables in the NAS such as capacity utilization, delay, and weather at hub airports. A Quality-of-Service (QoS) Index is formalized as a function of average of delays, variability of delays, and number of cancellations. A user-friendly graphic tool was first developed in C++/OpenGL to visualize geometrically and chronologically the large, complex data sets of ASPM and BTS. This tool can show how the QoS indexes evolve at major airports visually as time and weather change and help determine their relationships.

With this newly developed tool, we explore the ETMS dataset to study the space-time correlations of QoS Indexes for major airports in the NAS. We then focus on one particular case study involving ORD and MSP airports whose operational correlation can be easily identified by the tool. A time series analysis technique, autoregressive integrated moving average (ARIMA) model, is further utilized to quantify the correlation. The case study analyzes the data in the time period from September 2000 to March 2001, and consists of two scenarios: one representing the smallest arrival delays averaged over 30 days for flights inbound to MSP from ORD, the other representing the largest arrival delays. A high-order linear regression model is estimated with an impulse response function relating the output to the input. The analysis in the second scenario has shown parallel movement in the arrival delay of flights inbound to ORD versus those connecting ORD and MSP, delayed by a relatively stable time lag. Additionally, retrograde movement between delay and cancellation is also observed during winter 2000. The analysis also suggests that short-segment flights are most affected by delay propagation and respond faster to cancellation than longer segments. We are conducting extensive study on other eleven major US airports and will report the results in our final manuscript. This technique can also be utilized to evaluate or validate the output of NAS network simulation models that is of higher fidelity than looking at individual airport delays in isolation.

1. Introduction

1.1..........................................................................Background

In the United States after the deregulation in 1978, airlines have been building up hub and spoke systems to improve operation efficiency and competitiveness: connecting flights arrive and depart through a central hub airport in order to consolidate passengers traveling to smaller airports. Given this hub and spoke nature of airline service, delays at hub airports and other important non-hub nodes in the network (such as LaGuardia airport) can quickly proliferate throughout the entire aviation system, causing delays and ground holds across significant portions of the country.

Other than congestion-induced delays during peak periods, inclement weather condition most often causes reduction in airports’ capacity due to increased aircraft separations, and therefore triggers one of the FAA’s many flow control procedures. These range from airborne-holding to Mile-in-Trail restrictions applied to flying aircraft in heading the airports affected by adverse weather, from Ground Delay Programs where aircraft that are still on the ground but destined for the constrained airports are delayed until their Controlled Time of Departure (CTD), to Ground Stops in which all traffic is banned from departure for a period of specification.

Flight disruptions and irregular operations lead to many implications. Airlines have operational and customer service issues because aircraft are out of operational sequence, crews on delayed flights exceed the permitted duty time, and passengers miss connecting flights. In many cases, the airlines respond to the delay situation by canceling flights and accommodating passengers on alternative flights. This means that, although the reported delays increase significantly along with the traffic growth in recent years, the full impact of the cancellations and flight disruptions is understated in the delay and operational statistics. The impact is particularly burdensome for new entrant carriers that operate only a few flights at the airport. Because they have less flexibility, they offer fewer alternatives and some passengers are either accommodated on competitors' flights or on subsequent days.

1.2 Current approaches

Delay effects upon airlines’ economics have been addressed extensively in the literature [Kostiuk] [Shavell]. On the other hand, studies on propagating effects are given less consideration. In addition to assessing the cost implications to airlines, these researches also contribute to evaluating changes in the FAA’s flow control procedures.

Simulation has been used to quantitatively model the effects of changes in capacity and demand for US airports in terms of delay propagation [Schaefer]. This study made use of a deterministic model and therefore was dependent on the fidelity of this model. Propagating delay from a capacity-constrained airport was estimated for up to five subsequent connecting legs, which are linked using a connection-determination algorithm. There was no provision for cancellations, and none of the FAA’s flow control procedures were simulated.

Conducting statistical analysis is another approach to model empirically these down-line effects. [Beatty] used American Airlines schedule to model statistically this airline’s delay implication as a linear function of the length of the initial delay and the time of day it occurs. Again, the model assumed no cancellations.

1.3 Methodology

This paper presents some of the statistical analysis work that I am doing for the Air Transportation Systems Engineering Laboratory at George Mason University, which in turn is part of our model development and validation effort under contract with NASA Ames Research Center.

The main objective is to study the space-time relationship between US airports. More specifically, given some schedule-disruptive event at one airport such as adverse weather condition or traffic congestion, we are interested in knowing what the effects at connecting airports would be. A multi-leg flight, when delayed at one airport, is likely to propagate part of the delay to the next leg or even cause its cancellation.

The propagating effects are mainly the results of airlines’ decisions. This could be illustrated in the situations like Ground Delay Programs. A Ground Delay Program is one of the FAA’s Traffic Flow Management procedures to delay departures of flights inbound a weather-affected airport. In response to a Ground Delay Program, there are many possible reactions that an airline could have:

Do nothing but comply with the newly imposed time of departure

Cancel connecting flight(s) and maybe transfer passengers to another available flight

Keep the current schedule but divert flights to other capacity-unconstrained airports

Do nothing but comply with the newly imposed time of departure, and wait for others to cancel and shift up its flights

The particular city pair O’Hare International Airport (ORD) and Minneapolis Airport (MSP) is investigated as a case study. The analysis is narrowed down to looking at individual airlines that have scheduled flights serving this correspondence, since it could be reasonably assume that the response of the overall network is the aggregation of participating stakeholders reactions, among which airlines’ decisions play an important role. Observations for Northwest airline are presented in this report.

1.1..............................................................Technical approach

Given an average number of 25,000 flights and over 40 airlines operating each day, aggregate data is too complex and subject to noise. The approach of dealing with each individual airline and its flights and then aggregating all the airlines’ response together to form the network’s response has been chosen.

Existing databases on network performance store historical data for various measures: delays, cancellations, weather information, and safety incidents… Often, data come in aggregate values for every 15-minute interval called epoch. Hence, there are 96-epoch time series for any measure for any given day. This is also how I organized the data for the analysis. Dealing with flights of each individual airline required a fair amount of time and effort to query the databases and organize the values into corresponding time epochs.

2. Preliminary Visual Observation

I started the work by developing a visualization tool using C++/OpenGL in the hope of being able to observe some interesting patterns that could guide the subsequent statistical analysis later on. The prototype is aimed at visualizing airport performance metrics as time progresses.

This interface draws 32 US major airports at their respective geographic location in three dimensions. Any metric for any given day in the database from Jan 2000 to Aug 2001 can be selected and run

through the 96 time epochs. Values at each airport are log-transformed and rendered as a column with color changing from epoch to epoch: white means that airports operate under good weather condition and low traffic load, red refers to airports having bad weather condition, green shows airports operating with high traffic load and yellow is the combination of red and green conditions. Running one particular day gave insights into how the delay is propagated throughout the network and possible cause factors.

Another interface helps to look at the space-time airport correlation from another perspective. If the previous one is more focused on the cause factors and airport performance qualitative observation, the following allows to chose one or group of airports (i.e. hub airports) as controlling variables and another group of airports as dependent variables. Metric values are given for the controlling group and query is executed to search in the databases days and times that match these values. Corresponding values at dependent airports are then visualized for the following 4-hour window. Correlations if any are expected to show up in similar patterns of the metric distribution of dependent airports through many days.

In this particular case, ORD is chosen as independent variable, MSP, Los Angeles (LAX) and Detroit (DTW) are dependent variables. There seems to be a pattern in the delay distribution at MSP that repeats when ORD incurs large delay, and so does DTW. This observation seems supportive for more rigorous close-up look and further statistical analysis.

3. Case study

The city pair ORD-MSP was further investigated to see what would be the propagating effect caused by some schedule-disruptive event at ORD, which is a big hub for national as well as international flights, on the connecting flights to MSP.

3.1..................................................................Data preparation

A typical time series stores (aggregate) data of one metric of any given day. The values are computed for every 15-minute interval. For

this case study, the data span from March 2001 to May 2001 are used. The first 5 hours of the day are left out given insignificant traffic within this period (due to noise constraint), so there are 76 epochs left for any daily time series.

ORD is a big hub and therefore accommodates flights of many airlines, especially United Airlines (UA) and American Airlines (AA). In contrast, the data from the studied period show that the correspondence ORD-MSP is provided by three airlines, UA, AA and Northwest Airlines (NW). So for this case study, I was only focused on these three airlines.

Many input time series are considered for each of the three airlines at ORD and corresponding output time series at MSP involving only ORD-MSP correspondence:

Arrival Delay Number of cancellations Number of scheduled flights

3.2..........................................................Airline Market Shares

The two following plots show the airlines’ number of scheduled flights over the sampling period, in order to get an insight into how much each airline could contribute to the network response.

1 11 21 31 41 51 61 71 81 91

0

100

200

300

400

500

#Flights

Day

ORD inbound air line m arket shares (M arch to M ay 2001)

NW (mean=27)

AA (mean=303)

UA (mean=396)

1 11 21 31 41 51 61 71 81 91

0

5

10

15

#Flights

Day

ORD-M SP air line m arket shares (March to M ay 2001)

AA (mean=9.5)

UA (mean=13.2)

NW (mean=13.3)

Both plots show periodic pattern with peak and valley repeating every week, indicating that weekday traffic is higher than weekend traffic. Further more, both UA and AA make use of ORD as a big hub, with UA having more flights than AA. As for ORD-MSP correspondence, NW schedules more fights than the other two. Both plots also show that UA dropped slightly the number of scheduled flights around April, where as NW and AA have fairly stable schedules, so it would be better to first focus on AA and NW. Actually NW airline would be the best choice since AA has so many flights inbound ORD, and a much smaller number of connecting flights to MSP, therefore the propagating effect if any could be mild and very difficult to observe.

3.3. Airline performance trend

The following charts plot the number of early/on-time flights, the number of flights having positive delay less than 15 minutes, the number of flights having more than 15min delay but less than 30min and the number of flights late more than 30min against the number of cancellations for each day in the sampling period. So each day has 4 data points.

Looking at the y-axis shows that although NW has the most cancellations in some days, UA has the larget mean, and AA has the least. NW has a large variation and seems more aggressive in making cancellations, whereas UA and AA are more conservative and stable. The fact that AA is the most conservative – it has least data points at y=3 – may be due to its small number of connecting flights to MSP. The fact that NW is more subject to schedule disruption may be due to its practice of schedule connecting flights too tight? This would be explored in a subsequent section.

AA performance

#flights

#can

cella

tions

0 5 10 15

01

23

45

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

early or ontime flights

flights having delay<15min

flighs having delay<30min

flights having delay>=30min

Looking at the x-axis shows that NW has little delay compared to UA and AA given a value of cancellations. Does it mean that NW performs better for flown flights? The following boxplot depicts the distribution of scheduled flight time of three airlines for ORD-MSP flights. It can be seen that NW adds more padding time into its schedule, probable as a measure against its vulnerebility to schedule disruptive events.

If the previous charts are rather qualitative, the following one compares quantitatively the respective performance of the three airlines. The left column plots daily aggregate delay against cancellation, whereas the right plots daily average flight delay to account for the different number of flights. There are hence 92 data points for the sampling period. A striking feature observed from these plots is that as opposed to the intuitive thinking that when the number of cancellations goes up, delay will go down. One way to explain this could be the days when cancellations go up are oftentimes bad-weather days, during which flown flights are also affected.

3.4. Airport correlation and propagating effect

Since UA and NW have the most flights for ORD-MSP correspondence, it would make sense to focus on these two airlines first. The propagating effect depends in great part on how the airlines schedule their connecting flights. The next charts depict a typical day of UA schedule and a typical day of NW schedule. The sampled days are chosen arbitrarily.

ORD inbound flights are plotted on the left on rows corresponding to arrival time epoch. ORD-MSP flights are plotted by connected pairs of dots, which correspond to departure time from ORD and arrival time at MSP. As it can be observed from the charts, UA and NW schedule flights to from ORD to MSP on an hourly basis. NW flights depart at the first quarter of each hour (6am, 7am…), whereas UA flights depart mainly at the fourth quarter (6:45am, 7:45am…). Since the flight time is about 90min in average, i.e. 6 time epochs, correlation if any is expected to show up after 6 time lags plus some time lags due to passenger transit time or aircraft turn-around time.

Since UA has so many flights to ORD and only a much smaller number goes to ORD, any propagating effect could be mild and elusive. Therefore in the next section, NW is further investigated. Our goal is to determine the effect of NW delay/cancellation at ORD on NW delay/cancellation of ORD-MSP flights, and because these two metrics are highly related, it is required to consider both of them at the same time. I adopted to merge the two time series by a linear combination using a weight value for cancellation.

The following charts switch the axis to plot cancellation against daily aggregate arrival time for NW airline. The charts should not be interpreted as a function of delay to cancellation, but they rather depict the statistical performance trend of NW airline thought the sampling period. For instance, during days that NW had to cancel 2

25 20 15 10 5 0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

MSPORD#Flights

ORD-MSP

CorrespondenceArrivals to ORD

UA flight connection at ORD and MSP on March 12, 2001

flights inbound ORD, NW often incurred about 400 minutes of arrival delay. Since there is a large variation in arrival delay due to various stochastic factors, the regression fitting is applied to the median points to counter-balance the effect of many outliers.

The weights of cancellation at each airport are therefore determined by the slope of the fitting line. Fitting curves for UA and AA are depicted here for comparison purpose only. Actually, would it make more sense to plot cancellation against average arrival delay and then determine the cancellation weight from that, I am working on this and hopefully getting back with you to validate the results.

Following is the correlation result provided by SAS. It would be not appropriate to consider the whole sampling period since during good weather days there would be no propagating effect at all so no correlation. So the input here is a merged time series of cancellation and arrival delay of NW flights inbound ORD (using the weight of 222) concatenated over 10 days in which high delay was observed for NW ORD-MSP flights. ORD-MSP cancellation is weighted 87.Correlation of MSP and ORD

Variance of input = 3215.635Number of Observations 760

Crosscorrelations

Lag Covariance Correlation -1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1 -28 23.397331 0.01690 | .|. | -27 47.598510 0.03438 | .|* | -26 -70.820048 -.05115 | *|. | -25 -131.522 -.09499 | **|. | -24 64.195468 0.04637 | .|* | -23 -25.249406 -.01824 | .|. | -22 -50.714017 -.03663 | *|. | -21 -38.864154 -.02807 | *|. | -20 67.009252 0.04840 | .|* | -19 -21.129043 -.01526 | .|. | -18 -31.171285 -.02251 | .|. | -17 -109.473 -.07907 | **|. | -16 83.231233 0.06011 | .|* | -15 -56.867588 -.04107 | *|. | -14 67.094117 0.04846 | .|* | -13 -18.363308 -.01326 | .|. | -12 39.239186 0.02834 | .|* | -11 106.297 0.07677 | .|** | -10 35.475755 0.02562 | .|* | -9 -110.503 -.07981 | **|. | -8 55.924892 0.04039 | .|* | -7 295.414 0.21336 | .|**** | -6 42.685145 0.03083 | .|* | -5 -70.604466 -.05099 | *|. | -4 -20.800656 -.01502 | .|. | -3 362.427 0.26176 | .|***** | -2 -52.846982 -.03817 | *|. | -1 66.839723 0.04827 | .|* | 0 -1.744624 -.00126 | .|. | 1 298.299 0.21545 | .|**** | 2 148.571 0.10731 | .|** | 3 -63.506056 -.04587 | *|. | 4 -69.470571 -.05017 | *|. | 5 469.745 0.33927 | .|******* | 6 -20.438718 -.01476 | .|. | 7 -37.782594 -.02729 | *|. | 8 81.807016 0.05908 | .|* | 9 442.893 0.31988 | .|****** | 10 -40.371657 -.02916 | *|. | 11 -0.720204 -.00052 | .|. | 12 126.568 0.09141 | .|** | 13 229.126 0.16549 | .|*** | 14 -16.663215 -.01203 | .|. | 15 14.999253 0.01083 | .|. | 16 57.495442 0.04153 | .|* | 17 188.239 0.13596 | .|*** | 18 -87.260863 -.06302 | *|. | 19 -140.577 -.10153 | **|. | 20 190.764 0.13778 | .|*** | 21 80.675815 0.05827 | .|* | 22 -39.081943 -.02823 | *|. | 23 -138.575 -.10009 | **|. | 24 28.851541 0.02084 | .|. |

25 75.737204 0.05470 | .|* | 26 -51.042922 -.03687 | *|. | 27 -138.988 -.10038 | **|. |

... means two standard errors

The spikes at lag 5 and lag 9 are most interesting. Since average flight time from ORD to MSP is 90 minutes, i.e. 6 lags, spike at 9 makes more sense than the one at lag 5. That means NW airline usually schedules 45min for passenger transit time or aircraft turn-around time at ORD, and the positive correlation proves the propagating effect of any schedule disruptive event at ORD on MSP for the very next flight. Spikes at lag 13, 5, 1, -3 may be due to periodic scheduling practice of NW.

4. Conclusions

Understanding and mastering data mining techniques prove to be very useful in many applications. In my case, mining large databases to discover knowledge is the prerequisite for following modeling and simulation efforts. The most difficult thing of the mining process proved to be making data selection and sampling choices. Particularly for air transportation databases, the data is complex and inherently correlated. The complex nature of the system as well as space-time correlation make it difficult to decide which statistic models to use. My future would be try to quantify the correlation observed above, along with extending the analysis to 32 US major airports and 11 major airlines.

References:

Beatty, R., Hsu, R., Berry, L., Rome, J. Preliminary Evaluation of Flight Delay Propagation through an Airline Schedule, Proceedings of 2nd

USA/Europe ATM R&D Seminar, 1998.

Kostiuk, F. P., Gaier, E., Long, D., The Economic Impacts of Air Traffic Congestion, Air Traffic Control Quarterly, pp. 123-145, Volume 7, Number 2 1999.

Schaefer, L. and Millner, D. Flight Delay Propagation Analysis with the Detailed Policy Assessment Tool, Proceedings of the 2001 IEEE Systems, Man, and Cybernetics Conference.

Shavell, A. Z., Effects of Schedule Disruptions on the Economics of Airline Operations, Progress in Astronautics and Aeronautics, Volume 193, American Institute of Aeronautics and Astronautics, Inc., 2001.