ECON 4292 Migration, Immigrant Adaptation, and Development Professor Michael J. Greenwood.
Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L =...
Transcript of Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L =...
![Page 1: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/1.jpg)
Capabilities and Challenges in CFD:
A Perspective from the DoD HPCMP
CREATETM-AV Kestrel Development
Team
David R. McDaniel, Robert H. Nichols, Scott A. Morton
HPCMP CREATETM-AV
Presented by Dr. David R. McDaniel
AIAA Aviation17, 5-9 June 2017
Distribution Statement A: Approved for public release; distribution is unlimited.
![Page 2: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/2.jpg)
2
Overview
Introduction
Kestrel overview
Snapshot of Kestrel production capabilities
Challenges and future directions
Summary
![Page 3: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/3.jpg)
Introduction
![Page 4: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/4.jpg)
4
DoD HPCMP CREATETM– AV Kestrel
Expanding Footprint of Kestrel Adoption• Over 500 active license holders (as of May 2017)• 21 Defense Orgs (Labs, Engineering and Test Centers) actively using Kestrel• All major manufacturers actively evaluating Kestrel• 5 Orgs affiliated with Other Federal Agencies using Kestrel to support US Gov’t Programs• Several select US Academic Institutions and the Service Academies using Kestrel to support DoD Programs
Kestrel is the fixed-wing product of the CREATETM-AV program
– Born from requirements gathered in 2007/8 to
address modeling & simulation deficiencies in
the DoD acquisition process
– Multi-mesh/multi-solver paradigm
• Unstructured near-body
• High order Cartesian off-body
• Adaptive Mesh Refinement
• Fast overset connectivity
– Full spectrum of aircraft type
• Fighter, Bomber, Tanker, Transport, UAV
– Full spectrum of flight conditions/missions
• Low-speed, transonic, supersonic
• Cruise, maneuver, take-off/land, refueling,
formation flight, store carriage and release, pilot
ejection, precision air-drop, and more…
Version 7
Key Technologies• Multi-mesh paradigm• Adaptive mesh refinement• High order Cartesian solver• Python-based common
software infrastructure• Generalized interface for
externally developed “plug-in” capability modules
CapabilitiesHigh fidelity coupled physics• Aerodynamics• Structural Dynamics• Propulsion• Flight Control Systems
![Page 5: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/5.jpg)
5
Introduction
Many cost/performance issues in DoD aircraft
acquisitions may be traced back to inadequate
modeling of multidisciplinary phenomena
– …or maybe the “operational application” of the physics
capabilities
Kestrel:
– Provide a production multidisciplinary capability for DoD
acquisition personnel
– Plan for the change “manage the chaos”
– Usability, robustness, efficiency, and accuracy are all
competing factors
![Page 6: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/6.jpg)
Kestrel Overview
![Page 7: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/7.jpg)
7
Kestrel Architecture
Kestrel User Interface (KUI/Carpenter)
– Pre-processing
Job setup and validation
Mesh manipulation
– Post-processing
Tracking file plotting and manipulation
Reduced-order model building
Kestrel Run Time Execution Software
– Common Scalable Infrastructure (CSI)
Unique event-driven infrastructure
Homogenous behavior in the infrastructure, physics capability in components
Data Warehouse – generic data definition and automatic language translation
– Modular Components
Elemental physics capabilities large degree of use case flexibility
Testable code units may be modified/replaced with confidence
Written in Python/C/C++/FORTRAN
![Page 8: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/8.jpg)
8
Job Setup and Input Validation
Tension between making the hard job easy to
set up and making the easy job hard to setup
Entity-driven setup
Automatic unit conversions and reference/freestream property calculations
Jobview:- Visual validation of complex job setup- Assembled body positions- Unit conversions/scaling- Position-dependent input locations- Boundary conditions
![Page 9: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/9.jpg)
9
Modular Components
Nothing prohibits use of derived or empirical models
KCFDUnstructured FVM 2nd order
RANS/DDES
SAMAirCartesian
FVM 3rd/5th order RANS/DDES
ModalSD2nd order modal structural solver
COFFEUnstructured
FEM high-orderRANS
Sierra/SDFEM structural solver
6DOFNewton/Euler
- or -Lagrange (AEDC)
Propulsion0D Engine
Models
SDK???
ROM-Based Aero
CG LoadsDistributed Loads
PrescribedMotionAnalytic or arbitrary rigid-body motion
FSI
MeshManager
OutputManager
MeshMove
TimeManager
![Page 10: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/10.jpg)
10
Parallelism
Hierarchy of MPI
communicators
Shared memory for
duplicate data
1 body per process
![Page 11: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/11.jpg)
11
Testing and Validation
Continuous testing model is vital to Kestrel process
Boldness and confidence to undertake substantial
changes to software
~3500 unit, ~250 integration, ~25 system tests each
night (~17k assertions)
Automatic Testing System executed every 2 weeks and
covers a large range of use cases and flow regimes
(~125 separate jobs)
![Page 12: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/12.jpg)
Kestrel Capabilities Snapshot
![Page 13: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/13.jpg)
13
Flow Solver Performance
ONERA M60.84 Mach5.06° AOABaseline Menterw/ wall functions
Holden Cylinder - Mach 16.01
![Page 14: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/14.jpg)
14
Flow Solver Performance
Transonic F-35 @ 14 deg AOA (AIAA 2015-0551)
F-16XL Unsteady Solution 20° AOASA-DDESM=0.242, 10k ft(AIAA 2015-2873)
- No special initializations- Performance and scalability on par
![Page 15: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/15.jpg)
15
F-160.6 Mach20k feet6 deg AOAF110-100 0D transientengine model
Propulsion Integration
30° PLA
85° PLA
C1 CompressorAEDC 16T8 blade rows333 blades
A-10 Inlet Distortion 21° AOA
Full annulus TF34 fan stagew/ static BC core flow
![Page 16: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/16.jpg)
16
Near-body/Off-body Solution Capability
Off-body Cartesian solver supports high order and
adaptive mesh refinement
Near-body unstructured solution coupled via overset
NASA CRM (DPW)Mach 0.85CL = 0.5
Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292)
![Page 17: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/17.jpg)
17
Multi-body Elastic EffectsNotional Sidewinder Release from Elastic F-16
Mach 0.9, Sea Level, SA+DDES
![Page 18: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/18.jpg)
18
Multi-body Elastic EffectsNotional Sidewinder Ejection from Elastic F-16
Mach 0.9, Sea Level, SA+DDES
![Page 19: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/19.jpg)
Challenges and Future Directions
![Page 20: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/20.jpg)
20
Challenges and Future Directions
Kestrel robust and maintainable simulation capability
must be balanced with accuracy requirements
Productionizing high-fidelity physics capabilities while…
– Minimizing code complexity (small code base)
– Adapting to future algorithm advancements
– Adapting to future hardware changes
– Supporting proprietary / custom applications
Mention of ongoing Kestrel development activities in the
context of these next topics should not be construed as
the ideal end-state solution
![Page 21: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/21.jpg)
21
Multiple Everything
Necessary to model multiple disciplines at multiple time
scales to capture target physics
Flow regimes of interest moving to opposite ends of the
speed spectrum
– UAV incompressible, highly-flexible
– Hypersonics transition, chemistry, heating
Unsteady, time-accurate simulations
– Example: Full-annulus multistage compressor
Billions of grid points required for 0.5% mass flow, 2% total property convergence
Time step restricted by rotation rate (on the order of 104 RPM)
Pressure waves must transit the distance between inflow/outflow several times
Throttle transients and aircraft maneuvers have time scales of seconds
Multiple gas species and chemistry effects
– Efficient perfect gas vs. multiple reacting species and inflow specifications
– Support for custom thermodynamics/chemistry models
![Page 22: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/22.jpg)
22
Mesh Generation and Refinement
Need for effective automatic meshing process for
general configurations
– 1st Geometry and Mesh Generation Workshop at AVIATION 2017
Solver-independent strand mesh approach
Standardized methods for determining when and
where adaptive mesh refinement should occur
Access to underlying geometry for constrained
surface mesh movement (CREATE Capstone SDK)
![Page 23: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/23.jpg)
23
High-Order (Unstructured) Solutions
Some use cases are out of reach of 2nd order solvers
– Scalability/memory restricts continued refinement of the mesh
– Numerical dissipation prevents needed level of solution convergence
High-order overset can be problematic
Mesh generation/visualization (tools AND training)
Kestrel/COFFE
– SU/PG FEM, strong convergence, path to high-order, adjoint consistent
P2 solution for NASA CRMVerification of order of accuracy
![Page 24: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/24.jpg)
24
(High Lift) Turbulence & Transition
Accurate solutions near/beyond stall are necessary
– New high-performance aircraft being designed to operate close to stall
– Turbomachinery blades typically operate near stall
Transition modeling is key in production environment
– Transport-equation-type models are a necessity
– Requirement in hypersonic flows
JAXA Standard Model, Mach 0.1723rd AIAA Hi-Lift Prediction Workshop (AVIATION 2017)
![Page 25: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/25.jpg)
25
Solution Parallelism
Shared memory compute architectures are dominating
– Knights Landing processors entering production in DoD HPCMP this summer
Coarse-grain, hybrid parallelization approaches critical to
future scalability
Kestrel shared memory requirements
– Low overhead with minimal code complexity (maximum portability)
– Compatible with persistent data accessible across multiple languages
Potential cache issues with large mesh partitions
Look at other parallelism avenues (time, discipline, etc.)
![Page 26: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/26.jpg)
26
Solution Parallelism
“Index distance to neighbor cell”
“Cel
l in
dex
in m
esh
par
titi
on
”
NACA 0012 on 32 processorsGhost elements packed at end
Blue = physical cellRed = partition boundary ghost cellGreen = physical boundary ghost cell
Fine-grain Partitioning Coarse-grain Partitioning
![Page 27: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/27.jpg)
27
Reduced-Order Modeling
Effective use of ROMs necessary for disruptive
impact to acquisition programs
Automated Maneuver Generationto Minimize Parameter Correlation
CFD Model
ROM Used For Integrated/Distributed AeroPredictions at Off-Design Conditions
𝑞 𝑥, 𝑡 = 𝑎𝑛 𝑡 𝜙𝑛(𝑥)
ROM Constructed Using On-Design Data
POD-Based (Distributed Loads):
Polyomial (Integrated Loads):
𝐶𝐿 = 𝑓(𝛼, 𝛽, 𝑝, 𝑞, 𝑟, … )
Morton and McDaniel, Thurs @ 1630 in Silver, 325-APA-41 (AIAA 2017-4237)
![Page 28: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/28.jpg)
28
Summary and Final Thoughts
Kestrel is a production-quality multidisciplinary
simulation tool for fixed-wing air vehicles targeting DoD
acquisition professionals
Kestrel development team must consider usability,
robustness, maintainability alongside accuracy
Three more challenges:
1. Which models/approaches/techniques do we invest in?
2. Symbology, coordinate systems, reference frames, etc.,
across different disciplines creates confusion
3. Lack of multidisciplinary validation data is debilitating
for adoption of multidisciplinary tools
![Page 29: Capabilities and Challenges in CFD: A Perspective from the ... · NASA CRM (DPW) Mach 0.85 C L = 0.5 Eymann & Nichols, Thurs @ 1400 in Tower Court B, 336-CFD-30 (AIAA 2017-4292) 17](https://reader035.fdocuments.net/reader035/viewer/2022071418/611637965758c90095634434/html5/thumbnails/29.jpg)
29
Material presented in this paper is a product of the CREATETM-AV element
of the Computational Research and Engineering for Acquisition Tools and
Environments (CREATE) Program sponsored by the U.S. Department of
Defense HPC Modernization Program Office.
The authors would like to thank the CREATE Management Team, the 96th
Test Wing RNCS/RNCE, and AEDC for their financial management and
facility support.
Acknowledgements