META Metrics Toolbox for quantifying complexity and adaptability of system designs
1
Main goal of META is to achieve a 5x improvement in system development speed over current practice. In order to do this metrics need to be used to synthesize and select the best possible system architecture: A key element is to choose system architectures and designs that achieve desired performance with minimum necessary structural and dynamic complexity as well as maximum adaptability. Structural Complexity is based on the type and number of system elements, their interconnections as well as the structural arrangement. Captures both component an interface complexity Key concept: Graph Energy Dynamic Complexity is based on the number and correlation amongst functional performance attributes of the system. Captures correlation and uncertainty Key concept: Shannon Entropy Organizational Complexity captures the number of staff required and rework Predicts META Program Cost and Schedule Key Concept: The Rework Cycle (System Dynamics) Adaptability – the ease with which changes can be made to a design in the future META Metrics Toolbox for quantifying complexity and adaptability of system desig Conclusion: META-speedup factor of 5 compared to current practice appears feasible, but will require multiple “mechanisms” to work together (C2M2L library coverage > 0.8, structural complexity reduction of 40% thanks to extensive architecture enumeration, 75% probability of early detection of design flaws, >2 layers of abstraction etc…). The current META metrics toolbox produced by the UTRC team contains structural, dynamic and Contact Dr. Olivier de Weck ([email protected] ) or Dr. Brian Murray ([email protected] ) for more information on this effort. Spectrum of System Metrics Structural Sample Results for GTF: - # of components: 73 - # of interfaces: 377 - Graph Energy: 124.8 - Structural Complexity: 717.7 Metrics Toolbox Organizational (Design Flow) Dynamical Structural DSM Sample Results for B777 EPS: - Schedule completion time: - Actual 61 m, META: 16 m - ECRs: 300, META: 100 - META Speedup Factor: 3.8 Structural Complexity drives the amount of Design and Integration Work required as well As the Engineering Change Generation Rate META System Dynamics Model Sample Results for 3-spool Turbofan: - Dynamic Complexity: 708.2 - Functional Interdependency E(B) = 34.5 - Shannon Entropy = 20.5 Other dynamic complexity metrics B – Matrix captures correlations amongst Functional attributes 19x19 Dynamic Complexity drives the amount of change and number of iterations required to achieve satisfactory performance in all Functional requirements Sensitivity Analysis of META outcomes Sample Results: dimensional_complexity 20 computational_complexity = 24.9869 nonlinear_complexity = 7.7852 timescale_complexity = 185.5963 It does not seem possible to compress META design evaluation into a single scalar metric. A vector of metrics is most useful, containing: - Structural Complexity (parts, interfaces, architecture, Graph Energy) - Dynamic Complexity (functional interdependence, uncertainty, Shannon Entropy) - Organizational Complexity ( number and productivity of staff, organizational assignments, Schedule, NRE) - Main Purpose of META Metrics META Metrics Takeaway
description
META Metrics Toolbox for quantifying complexity and adaptability of system designs. Main goal of META is to achieve a 5x improvement in system development speed over current practice. In order to do this metrics need to be used to synthesize and select the best possible system architecture: - PowerPoint PPT Presentation
Transcript of META Metrics Toolbox for quantifying complexity and adaptability of system designs
Main goal of META is to achieve a 5x improvement in system development speed over current practice. In order to do this metrics need to be used to synthesize and select the best possible system architecture:
A key element is to choose system architectures and designs that achieve desired performance with minimum necessary structural and dynamic complexity as well as maximum adaptability.
Structural Complexity is based on the type and number of system elements, their interconnections as well as the structural arrangement.
Captures both component an interface complexity Key concept: Graph Energy
Dynamic Complexity is based on the number and correlation amongst functional performance attributes of the system.
Captures correlation and uncertainty Key concept: Shannon Entropy
Organizational Complexity captures the number of staff required and rework Predicts META Program Cost and Schedule Key Concept: The Rework Cycle (System Dynamics)
Adaptability – the ease with which changes can be made to a design in the future
META Metrics Toolbox for quantifying complexity and adaptability of system designs
Conclusion: META-speedup factor of 5 compared to current practice appears feasible, but will require multiple “mechanisms” to work together (C2M2L library coverage > 0.8, structural complexity reduction of 40% thanks to extensive architecture enumeration, 75% probability of early detection of design flaws, >2 layers of abstraction etc…). The current META metrics toolbox produced by the UTRC team contains structural, dynamic and organizational metrics of complexity.
Contact Dr. Olivier de Weck ([email protected]) or Dr. Brian Murray ([email protected]) for more information on this effort.
Spectrum of System Metrics
Structural
Sample Results for GTF:-# of components: 73-# of interfaces: 377-Graph Energy: 124.8-Structural Complexity: 717.7
Metrics Toolbox
Organizational(Design Flow)
Dynamical
Structural DSM
Sample Results for B777 EPS:-Schedule completion time:- Actual 61 m, META: 16 m- ECRs: 300, META: 100- META Speedup Factor: 3.8
Structural Complexity drives the amount ofDesign and Integration Work required as wellAs the Engineering Change Generation Rate
META System Dynamics Model
Sample Results for 3-spool Turbofan:-Dynamic Complexity: 708.2- Functional Interdependency E(B) = 34.5- Shannon Entropy = 20.5
Other dynamic complexity metrics
B – Matrix captures correlations amongstFunctional attributes
19x19
Dynamic Complexity drives the amount ofchange and number of iterations required to achieve satisfactory performance in allFunctional requirements
Sensitivity Analysis of META outcomes
Sample Results:
dimensional_complexity 20
computational_complexity = 24.9869
nonlinear_complexity = 7.7852
timescale_complexity = 185.5963
It does not seem possible to compress META design evaluation into a single scalar metric.
A vector of metrics is most useful, containing:
-Structural Complexity (parts, interfaces, architecture, Graph Energy)
-Dynamic Complexity (functional interdependence, uncertainty, Shannon Entropy)
-Organizational Complexity ( number and productivity of staff, organizational assignments, Schedule, NRE)
-Main Purpose of META Metrics
1. Architecture Selection2. Outcome prediction
META Metrics Takeaway
