Episode 3 E3-WP0-MWPM20091215-PPT-V0001-5 th ExCom Slides 1 Episode 3 - Ex-Com Final Report Tools &...

1

Episode 3

E3-WP0-MWPM20091215-PPT-V0001-5th ExCom Slides

Episode 3 - Ex-Com

Final Report

Tools & Validation

Dr. Ralph Leemüller

R&D Division - DFS

2

Episode 3

E3-WP0-MWPM20091215-PPT-V0001-5th ExCom SlidesBrussels, 13 & 14 Oct 2009 2

Overview

Validation Methodology Innovative Validation Techniques and Tools

Expert Groups Gaming Techniques Prototyping Sessions Modelling and Fast-time Simulations

Combined Techniques

Ep

isod

e 3

- C

AA

TS

II Fin

al D

isse

min

ati

on E

vent

3

Episode 3

E3-WP0-MWPM20091215-PPT-V0001-5th ExCom Slides3

Concept Validation Methodology

• Concept lifecycle (E-OCVM)• Concept lifecycle (E-OCVM)

Use of E-OCVM Episode 3 adressed Steps V1 – V3 All validations in accordance to methodology

Validation strategy Validation plans – Conduction of Exercise – Report

4

Episode 3


What is an Expert Group

Expert group working on a set of predetermined concepts in a structured manner Gain from group expertise

specific professional profiles using both the individual skills and the synergy of the group More complete vision; Larger number of factors; More reliable

The mission of an Expert Group in a validation project is providing: A reliable approach to real cases, even when assuming future situations; and Experience based judgement analysis for supporting the feasibility of the

developments to be undertaken.

5

Episode 3


Using Expert Groups

Pros Concepts development, refinement and clarification; Qualitative assessments; Prediction of potential issues regarding future situations; Support validation exercises design and development. Analysis of consistency of validation results.

Cons Difficulty to produce quantitative results; Limited availability of the experts; Biased vision; Trend to keep things the way they are.

Outputs: Operational details: refinement of DoDs and scenarios; Refinement of the approach of the validation exercises; Judgement of exercise results

6

Episode 3


ExpertGroup

6

Addressing the Concept Validation - Expert Groups

Concept clarification and operational feasibility of

processes

• Concept layered phases: Expert Groups are suitable for the initial validation stage

(concept definition and clarification) as well as to consider cross-functional issues, i.e. initial performance assessment and operational feasibility;

Pre-industrial development can be supported by experts.

• Concept layered phases: Expert Groups are suitable for the initial validation stage

(concept definition and clarification) as well as to consider cross-functional issues, i.e. initial performance assessment and operational feasibility;

Pre-industrial development can be supported by experts.

7

Episode 3


What is Gaming

Human in the loop activity applicable to simulate decision making processes;

Experts are ‘around a table’ playing different roles as they were living real processes (AOCs, Pilots, AMC, Network Manager,...);

Every role has a particular scope, objectives and strategies during the process;

Scenario running in real-time (or even slow time); Promoting non-routine thinking, thus enabling out-of-the-box

thinking.

8

Episode 3


Using Gaming

Pros Clarification of the concept; Qualitative assessment of processes feasibility: Detection of bottlenecks and shortcomings; Clarification on roles and responsibilities: Identification of the information flows among actors; Exploration on supporting tools requirements.

Cons Mostly qualitative assessment; quantitative results from SW-Games Without an adequate guidance the Gaming session may turn into an Expert Group; Results obtained during the game are dependant on the participants (expertise) and

platform limitations. Outputs:

Refinement of Operational details (Information flows, roles & responsibilities);

9

Episode 3


Supporting means to play

• Three different gaming approaches: Paper-based game; Web-based game; SW Platform-based:

- DARTIS; ACCES;- CHILL;- PROMAS.

• Three different gaming approaches: Paper-based game; Web-based game; SW Platform-based:

- DARTIS; ACCES;- CHILL;- PROMAS.

Process Description

What-if ToolSRM

Current Situation

The What-if tool analyses the future situation in terms of complexity, safety, workload and detects a demand-capacity imbalance.

Based on that, the tool proposes its better solution à Fixed Routes

The conclusion is that for offering the capacity demanded, a transition back to fixed route structure is needed while the unusual meteorological conditions remain.

1

3

2

MILITARY TRAINING AREA

MSP1Sector 1

Sector 3

Sector 4

Sector 5

Sector 6

1

2

6

3

7

8

10

9

1

Solution

Forbidden Area

Entry-exit points from sectors definedSRM What-if tool gives the directions the flows have to follow, but no which flights should follow which one

MSP1

Entry/Exit Points

Parametres: - Workload: - Complexity: - Safety: - DCB: OK

Entry and Exit points are defined, as well as the directions of the flows.SRM should communicate this information to the users implied, because they are who has to decide which flow follow which direction and how to accomodate the new RBTs according to the fixed routes and the new sectorisation, if needed.

10

Episode 3


Addressing the Concept Validation – Gaming

• Concept layered phases: All phases can be covered by gaming (Planning and Execution

Phases); Suitable for clarification on components of the concept better

than for global concept.

• Concept layered phases: All phases can be covered by gaming (Planning and Execution

Phases); Suitable for clarification on components of the concept better

than for global concept.

GamingExercise Paper-based Platform-based

Concept clarification and operational

feasibility of processes

11

Episode 3


What is prototyping session

Human in-the-loop experiments to build the ATMenvironment piece by piece and to address detailed issues (e.g. airspace, tools, roles).

The focus is on operability: Real-time technique Small-scale and/or part-task Iterative in nature Gradually building and refining Examine the options, moving towards a clear goal

12

Episode 3


Using Prototyping Session

Pros Flexibility Quick answers Suited to early concept assessment/maturation Reduced cost: small scale and simplified environment, to focus on specific issues

Cons Short-time scales are the major limitation

One week sessions Time between sessions

Training time Examinination of the concept with multiple conditions

Outputs: Assessment of operability and acceptability refinement/maturation of a defined set of concept elements Initial trends on some KPAs

13

Episode 3


Addressing the Concept Validation – Prototyping Session

• Concept layered phases: Execution phases is covered by prototyping; Suitable for feasibility on components of the concept better than

for global concept.

• Concept layered phases: Execution phases is covered by prototyping; Suitable for feasibility on components of the concept better than

for global concept.

Proto-typing

Operational feasibility

14

Episode 3


What is Modellingand Fast-time Simulation

Using models of ATM systems Use a representation of elements of the ATM concept Applied when it is impractical to recreate the experimental conditions

with human in the loop (large scope, long duration, wide area, …) Predict measures related to physical processes or to assess

feasibility of the processes.

15

Episode 3


Using Modelling and FTS

Pros Large scope in time or geographic area Short time of simulation run Reduced cost - targeted use of expert Reproducible Quantitative results on KPA’s (i.e. Capacity, Efficiency)

Cons Model agents may introduce simplifications Analysis and interpretation requires an experienced expert Level of maturity of concept influence the accuracy of output

Outputs: Assessment of quantitative results (KPA) refinement/maturation of a defined set of concept elements

16

Episode 3


Addressing the Concept Validation – Modelling and FTS

• Concept layered phases: All phases can be covered by gaming (Planning and Execution Phases). Macro model to extrapolate results on ECAC-wide applicability refinement/maturation of a defined set of concept elements Suitable for concept assessment on wide area and large

coverage.

• Concept layered phases: All phases can be covered by gaming (Planning and Execution Phases). Macro model to extrapolate results on ECAC-wide applicability refinement/maturation of a defined set of concept elements Suitable for concept assessment on wide area and large

coverage.

Model & FTS

Modelling Fast-Time

Concept clarification and Feasibility of large scope, long duration

17

Episode 3


Combining Techniques

Episode 3 E3-WP0-MWPM20091215-PPT-V0001-5 th ExCom Slides 1 Episode 3 - Ex-Com Final Report Tools &...

Documents

Transcript of Episode 3 E3-WP0-MWPM20091215-PPT-V0001-5 th ExCom Slides 1 Episode 3 - Ex-Com Final Report Tools &...