A Data-Driven Approach to Modeling and Validation of Advanced Thermal-Hydraulics Models

Nam Dinh, Yang Liu*, Chih-Wei Chang*Department of Nuclear Engineering

North Carolina State University, Raleigh, NC, USA* - PhD graduated

“Big Data in Nuclear Power Plants” Workshop Columbus, OH, December 11-12, 2018

Presenter: Dr. Jason Hou (NCSU)

Outline and references

• Data-driven Modeling and Validation– Yang Liu, Nam Dinh, Yohei Sato and Bojan Niceno, “Data-driven

modeling for boiling heat transfer: using deep neural networksand high-fidelity simulation results”, Applied Thermal Engineering, 144, pp.305-320, 2018

• Classification of Machine Learning – Chih-Wei Chang and Nam Dinh, “Classification of Machine

Learning Frameworks for Data-Driven Thermal Fluid Models”, International Journal of Thermal Science, 135, pp.559-579, 2019.

2

Data-driven modeling and validation

• The talk is focused on a “data-driven” modeling and validation of advanced models used for nuclear reactor thermal hydraulics simulation including both single-phase turbulent flow and multiphase flow with phase changes.

• It is motivated by the recognition that “scaling” and “validation data” present major obstacles in design, safety analysis, licensing, and, ultimately, commercialization of innovative nuclear power systems.

• A novel approach of “data-driven modeling” that brings together, – on one hand, high-fidelity numerical simulation and advanced validation

experiments to generate “big data”, and, – on the other hand, advanced techniques in data analytics and machine

learning to effectively utilize available multi-scale and field data to develop “data-driven” models that bridge the scale gap.

• The resulting approach has potential to shorten the time and efforts required for development and assessment of predictive capability.

3

Computer platform for thermal-hydraulic simulations

• System analysis – TRACE, RELAP5, CATHARE, …– GOTHIC

• Component analysis– Core subchannel analysis (CTF)– …– CFD (RANS)– MCFD (CMFD)

• High-fidelity simulation– DNS, LES– ITM (Interface-Tacking/Capturing Methods)

5

Two-fluid model based MCFD solver has been widely regarded as a promising tool for dealing with boiling scenario within complex systems

Two-fluid model: conservation equations

)𝜕𝜕(𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘𝜕𝜕𝑡𝑡

+ 𝛻𝛻 ⋅ 𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘𝐔𝐔𝑘𝑘 = 𝛤𝛤𝑘𝑘𝑘𝑘 − 𝛤𝛤𝑘𝑘𝑘𝑘

)𝜕𝜕(𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘𝐔𝐔𝑘𝑘𝜕𝜕𝑡𝑡

+ 𝛻𝛻 ⋅ 𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘𝐔𝐔𝑘𝑘𝐔𝐔𝑘𝑘 = −𝛼𝛼𝑘𝑘𝛻𝛻𝑝𝑝 + 𝛻𝛻 ⋅ )𝛼𝛼𝑘𝑘(𝜏𝜏𝑘𝑘 + 𝜏𝜏𝑘𝑘𝑡𝑡 + 𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘𝐠𝐠 + 𝛤𝛤𝑘𝑘𝑘𝑘𝐔𝐔𝑘𝑘 − 𝛤𝛤𝑘𝑘𝑘𝑘𝐔𝐔𝑘𝑘 + 𝐌𝐌𝑘𝑘𝑘𝑘

)𝜕𝜕(𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘ℎ𝑘𝑘𝜕𝜕𝑡𝑡

+ 𝛻𝛻 ⋅ 𝛼𝛼𝑘𝑘𝜌𝜌𝑘𝑘ℎ𝑘𝑘𝐔𝐔𝑘𝑘 = 𝛻𝛻 ⋅ 𝛼𝛼𝑘𝑘 𝜆𝜆𝑘𝑘𝛻𝛻𝑇𝑇𝑘𝑘 −𝜇𝜇𝑘𝑘Pr𝑘𝑘𝑡𝑡

𝛻𝛻ℎ𝑘𝑘 + 𝛼𝛼𝑘𝑘𝐷𝐷𝑝𝑝𝐷𝐷𝑡𝑡

+ 𝛤𝛤𝑘𝑘𝑘𝑘ℎ𝑘𝑘 − 𝛤𝛤𝑘𝑘𝑘𝑘ℎ𝑘𝑘 + 𝑞𝑞𝑘𝑘

Condensation&Evaporation

TurbulenceInterfacial

forces

Heat partitioning

Mass

Momentum

Energy

7

Disadvantage: requires closures to be introduced to make the conservation equations for individual phases solvable

Two-fluid model: closure relations

• Interfacial force

• Wall boiling

• Turbulence

• Bubble size

• Interfacial heat/mass transfer

8

Mechanistic modeling

• Mechanistic closure relations that aim to better describe the detailed underlying physics – Wall boiling

• Driven by the knowledge and experience of the researcher

• Limitations: the physics are not fully understood– Relies on artificial concepts– Has fixed functional form– Cannot exploit rich data resource

(Basu et al., 2005), (Yeoh et al., 2014), (Hoang et al., 2017), (Gilman and Baglietto,2017)

Shearing-off Random collisionSurface instabilityWake entrainment

(Hibiki and Ishii, 2003), (Sun et al., 2003) , (Brooks, 2014), (Kumar et al., 2017) … Credit: Prof. Brooks

9

Closure relations: wall boiling

( )(1 )wall Ev Qu Fc dry dry vaporq q q q K K q= + + − +

Controlled bynear wall voidfraction

Normal mode DNB mode

10

Mechanistic modeling framework

Knowledge Base

Application(Lessons Learned)

Assessment(Validation, UQ)

Simulation

Mechanistic Description

ExperimentsRelated Models

and Insights

Model Development

(Semi-empirical)

Traditional FrameworkTaking years to

decades

12

Needs and Opportunities

• Sub-grid-scale (SGS) physics models (or so-called closure relations) determine the accuracy of thermal-fluid modeling, and they are essential for simulation codes.

• "Big" data in thermal-hydraulics become available with advanced thermal and flow diagnostic methods such as infrared thermometry and PIV (Particle image velocimetry) techniques, and high-fidelity simulation (LES, DNS, ITM)

• Deep learning (DL) is a universal approximator [Hinton, 1989],

– Capturing multi-scale, multi-physics processes

– Discovering the underlying correlations behind the data to achieve the cost-effective closure development for

• new geometries, new coolants or system conditions (new designs). 13

Motivation

Progresses in other research fields can be introduced to help quantify and reduce uncertainty of MCFD solver

• New data sources– High-resolution experiments

• Infrared (IR) camera and particle image velocimetry (PIV)– High-fidelity simulations

• Direct numerical simulation (DNS) with interface tracking method (ITM)

• Statistical learning and machine learning– Reduced order modeling– Bayesian inference– Deep learning

14

Thermal fluid data

Thermal Fluid Data

Type QualitySource

Global data

Local data

Field data

Experiment Simulation Quantity Uncertainty

IET SET

S

D

M

15

Trendy 4-D (high spatio-temporal resolution)

Data and knowledge requirements

Theo

ry-b

ased

M

odel

s

Data-driven Models

Physics-informed Data-driven

Models

Kno

wle

dge

Req

uire

men

t

LowLow High

High

Data Requirement

Adopted after Karpatne, et al, 2017.

Physics-informed data-driven frameworks for model development16

Physics-informed data-driven modeling for nuclear applications

SystemSimulation

Data Science

Thermal Fluid

Models

17

• Artificial (Machine, Computational) Intelligence – Machine Learning

• Artificial Neural Networks (ANN / NN)– Deep Learning (Deep Neural Network, DNN)

• “Big Data”– Data Mining (Data Science, Predictive

Analytics)• Pattern Recognition, Clustering, Features

Selection, …

• Machine Learning:– Learning (Training) Strategies

• Supervised ML vs. Unsupervised Learning– Reinforcement Learning

Exploring New Approach and Capability

Data-driven modeling

• Focuses on the data about the physical process, aiming to find the connection between the input conditions of the process and the output QoIs of it

• Does not require the explicit understanding of the physical process

• Leverages rich data resources

• Non-parametric, universal approximator

• The data-driven modeling is possible today due to – the significantly increased data availability from high-resolution

experiments and high-fidelity simulations – the recent breakthrough in machine learning especially the deep

learning algorithm 18

Machine learning: construct a function that represents the unknown model that correlate inputs and targets

Data (inputs)X = {x1,…,xn}

MachineLearning

ML-based thermal fluid closuresML(X) ≈ Y

Data (targets)Y = {y1,…,yn}

Available, relevant, and adequately evaluated data

20

Deep neural networks

Deep neural networks exhibit good properties– Expressiveness

• DNN with proper weights set up can approximate any continuous functions (Barron, 1993)

– Optimization• DNN can be optimized and converges with

stochastic gradient descent (SGD)• Hyperparameters has strong influence on

DNN performance, requires trial-and-error along the training process

– Generalization• DNN usually has good performance in

predicting the case it has not been trained on

Forward-propagation of input features

21

Backward-propagation of loss function gradients

• Data produced by ITM– Number of grid: 224×224×320 = ~16

million cells for the computational domain

– Adaptive time step, (µs level)– Results sampled every 1ms

Yohei Sato, Bojan Niceno, Paul Scherrer Institute, Switzerland

Use of High Fidelity Simulation of Nucleate Boiling

23Pool boiling simulation setup

Case study: prediction of local boiling process using DNN

Data processing: high-fidelity simulation

• Detailed interface information and fluctuation of physical quantities is not only unnecessary but also incompatible with the averaged conservation equations for the two-fluid-model.

• High-fidelity data are averaged to be compatible with MCFD solver

𝑓𝑓 (𝒙𝒙, 𝑡𝑡)

=1𝜏𝜏

1𝑙𝑙3�𝑡𝑡−𝜏𝜏

𝑡𝑡�𝑥𝑥1− ⁄𝑙𝑙 2

𝑥𝑥1+ ⁄𝑙𝑙 2�𝑥𝑥2− ⁄𝑙𝑙 2

𝑥𝑥2+ ⁄𝑙𝑙 2�𝑥𝑥3− ⁄𝑙𝑙 2

𝑥𝑥3+ ⁄𝑙𝑙 2𝑓𝑓(𝒙𝒙′, 𝑡𝑡′)𝑑𝑑𝑥𝑥3′ 𝑑𝑑𝑥𝑥2′ 𝑑𝑑𝑥𝑥1′𝑑𝑑𝑡𝑡′

24

Data-driven modeling: problem setup - 1

• Comprehensive flow features are chosen to include all the relevant terms in the averaged conservation equations

• 19 local features selected as inputs: pressure gradient; momentum convection; energy convection; surface information

• Purpose of the desired DFNN is to use local flow features, which can be obtained in the MCFD solver without boiling closure relations, to predict the boiling heat transfer

25

Data-driven modeling: results - 2• Interpolation case• Well developed nucleate boiling• Individual nucleation sites can be

clearly identified which suggested the frequently activated nucleation location in the simulation.

• DNN with local features can give good prediction on local boiling process, as well as capture the global boiling patterns

Extrapolation case (1200 kW/m2)Interpolation case (800 kW/m2)

26

Observations & remarks

• The accuracy of the networks is tested through four case studies, including both interpolation and extrapolation cases

• DNN trained on local features have good performance on extrapolation cases– Some globally extrapolation case is locally interpolation– DNN can identify the intrinsic pattern of certain physical phenomena

• The data-driven approach based on deep feedforward network for boiling closures exhibits potential to learn from “rich data” obtained in high fidelity simulations to predict the boiling heat transfer with good accuracy

• The study also reveals limitations in MCFD solver in selecting and evaluating flow features, that ensure their scale invariance

27

For more detail description of the model and machine learning study:

Data-driven modeling (DDM)

For the “Big Data” to become useful in DDM, it has to undergo several processing steps:

– First, results of high-fidelity simulations and experiments need to be collected, categorized, and archived in an easily accessible storage format.

– Second, the value of data as information needs to be assessed, to establish their relevance to the conditions and models under consideration, so that these data become useful information.

– Third, data are processed by various methods (including ML) to recognize underlying correlations behind the information.

– The so-developed intelligence (e.g., in the form of closure relations) is used to enable thermal fluid simulation in applications

30

Intelligence

Machine learning

Information

Data

Mechanistic (theory-based) vs. Data-driven modeling Frameworks

Knowledge Base

Application(Lessons Learned)

Assessment(Validation, UQ)

Simulation

Data Analysis

Experiments Related Models and InsightsMachine Learning

Data-Driven Modeling Framework

Knowledge Base

Application(Lessons Learned)

Assessment(Validation, UQ)

Simulation

Mechanistic Description

ExperimentsRelated Models

and Insights

Model Development

(Semi-empirical)

Traditional FrameworkTaking years to

decades

Theory-based Modeling Framework Data-Driven Modeling Framework

31

[*] -potential to shorten the time and efforts required for development and assessment of predictive capability

[*]

Classification of machine learning frameworks

Type-I (physics-separated) ML

Type-II (physics-evaluated) ML

Type-III (physics-integrated) ML

Type-IV (physics-recovered) ML

Type-V (physics-discovered) ML

32

Criteria for framework classification

Classification Criteria Type I Type II Type III Type IV Type VIs PDE involved in thermal fluid simulation? Yes Yes Yes Yes No

Is the form of PDEs given? Yes Yes Yes No No

Is the PDE involved in the training of closure relations? No No Yes No No

Is a scale separation assumption required for the model development? Yes No No No No

34

Type I (physics-separated) ML frameworkElement 1. Collect data,

(xk, yk), k=1,2,…,n

Element 5. Use ML algorithms, ML(X) ≈ Y

Element 6. Apply physics constraints,g(ML(X))

Element 3. Select flow features as training inputs, X

Element 4. Prepare outputs,Y = f(X)

Element 2. Preprocess data

Element 7. Perform simulations with ML-based closure models

Conservation equations

ML-based thermal fluid closures

Conservation equations

ML-based thermal fluid closures

The framework is simplified.

35

Criteria for framework classification

Classification Criteria Type I Type II Type III Type IV Type VIs PDE involved in thermal fluid simulation? Yes Yes Yes Yes No

Is the form of PDEs given? Yes Yes Yes No No

Is the PDE involved in the training of closure relations? No No Yes No No

Is a scale separation assumption required for the model development? Yes No No No No

36

Type III (physics-integrated) ML frameworkElement 1. Collect from simulations or experiments,

(xk, yk), k=1,2,…,n

Element 2. Preprocess the data

Element 3. Prepare training targets from that corresponds to PDE solutions, Y = f(X)

Use ML algorithms

Element 8. Obtain field equations with ML-based fluid closures for predictions

Yes

Element 5. Adjust model parameters by ML algorithms, ML(X) ≈ Y

Element 6. Solve thermal fluid models with ML-based closures

ML-based closuresConservation equations

Element 6. Solve thermal fluid models with ML-based closures

ML-based closuresConservation equations

Element 7. Convergence?No

Element 4. Employ flow features as training inputs, X

37

Lessons learned from case studies performed to date

Type I ML Type II ML Type III ML Type V ML

Training data type SET SET IET IETData quantity requirement Low Low High High

Data quality requirement High Low High Low

Are NN-based closures iteratively queried while solving PDE?

Yes No Yes No

Are solutions constrained by PDE? Yes Yes Yes No

NotePreferable

when SET data quality is high.

Preferable when SET data quality is low.

CNN-based closures are preferable.

Potential for identifiability

issue

38

39

For more detail case studies for each Framework Type:

Concluding remarks

• Deep learning can assist in development of data-driven nuclear thermal-hydraulics simulation.– Allowing cost-effective model developments.– Using non-parametric models to capture underlying correlations behind a

substantial amount of data.

– Making use of information content of large datasets generated in advanced experiments and numerical simulations.

• Collection, qualification, and management of data are instrumental for the “Big Data” to become useful in data-driven modeling (DDM) (or for that matter, in any applications)

• The growing interest and potential application of DDM creates a new domain in V&V of scientific and technical computing, namely of Verification and Validation of codes that involve machine-learning-based data-driven models.

40

August 29, 2018 Nam Dinh, NCSU, Data-Driven Approach in Advanced Thermal-Hydraulics 41

Acknowledgment:

Support of the US DOE via CASL program and CINR program’s Integrated Research Project: “Development and Application of a Data-Driven Methodology for Validation of Risk-Informed Safety Margin Characterization Models”.

PhD research of Chih-Wei Chang and Yang Liu

Thank you!

42