MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A....
-
Upload
dominick-brooks -
Category
Documents
-
view
216 -
download
1
Transcript of MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A....
![Page 1: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/1.jpg)
MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system modelsA. Dozier, O. David, Y. Zhang, and M. Arabi
![Page 2: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/2.jpg)
Motivations
Different languages, frameworks, operating systems
Two-way feedbacks between legacy models
Refactoring legacy code is difficult
To program
Many lines of code
Reproducing model outputs correctly
To maintain
Updating model versions
To adopt
Little to no community adoption of code
When refactored by third party
When significantly changed
![Page 3: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/3.jpg)
Goals & Objectives
Goal
“to facilitate and abstract the legacy model integration process to include complex, multi-directional interactions between models… of different architectures and maintain model individuality”
Objectives
Design abstract interface for simplicity
Apply the interface
![Page 4: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/4.jpg)
Comparison between two methods of integration
Calling one as subroutine Using MODPI
Model 2
Read inputs
Execute
Write outputs
Model 1
Read inputs
Execute
Write outputs
Execute
Write outputs
Model 1
Read inputs
Execute
Write outputs
Model 2
Read inputs
Execute
Write outputs
MPI networkcommunication
![Page 5: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/5.jpg)
Model 2 Wrapper
MODPI Implementation
Model 1 Wrapper (User-specified)
MODPI Implementation
modpi_init()- Subscribes to
events by name- Initializes MPI
modpi_finalize()
MOdel Data Passing Interface (MODPI)
Model 1
Start
Daily Loop
…Perform work…
End Daily Loop
End
On Start
On Top Loop
On Bottom Loop
On End
Model 2
Start
Daily Loop
…Perform work…
End Daily Loop
End
On Top Loop
Senders
Rece
ivers
Even
ts
mpirun or mpiexec
![Page 6: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/6.jpg)
Sample implementation
DayCent-HYDRUS
DayCent estimates biogeochemical fluxes
HYDRUS provides physically-based representation of soil water content
Two different implementations
“SUB”: call HYDRUS as a subroutine within DayCent
“MODPI”: DayCent-HYDRUS linked using MODPI
![Page 7: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/7.jpg)
Sample implementation
Code changes
![Page 8: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/8.jpg)
Sample implementation
Runtime and overhead
![Page 9: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/9.jpg)
Discussion
Advantages
Framework and language independent
Multi-lingual and (hopefully) multi-platform
Non-intrusive event-based system
Built-in parallelization and communication via MPI
Limitations
May produce overhead in virtualized environments
Requires MPI
Framework must be compatible with MPI
May require some knowledge of MPI programming
![Page 10: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/10.jpg)
Future Work
Use “reflection” to lookup variable by string
Use a range of processes to broadcast values
Automate data transformations
Run tests across Windows and Linux
Generate Fortan or C code for initial model integration wrapper
![Page 11: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/11.jpg)
Questions?
![Page 12: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/12.jpg)
![Page 13: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/13.jpg)
References
Images
Language
Stop and listen
![Page 14: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/14.jpg)
Advantages
Simple concept
Simple programming
Disadvantages
Computationally inefficient
Convergence of iterations?
Option 1 – Iterative
Model 1
Model 2
Write new inputs
Converged?
Read outputs
Exit
No?
Yes!
![Page 15: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/15.jpg)
Model 2
Read inputs
Execute
Write outputs
Advantages
Computationally efficient
Disadvantages
Medium to hard concept
Medium to hard programming
Maintain separate code base
Option 2 – Call model as sub-model
Model 1
Read inputs
Execute
Write outputs
Execute
Write outputs
![Page 16: MODPI: A parallel MOdel Data Passing Interface for integrating legacy environmental system models A. Dozier, O. David, Y. Zhang, and M. Arabi.](https://reader036.fdocuments.net/reader036/viewer/2022082613/5697bf7a1a28abf838c82cd8/html5/thumbnails/16.jpg)
Implementing MODPI
Add events
Minimal model refactoring
Build sender and receiver subroutines
Build subroutine to point to events within the model
Build a wrapper program
Call modpi_init()
Run model
Call modpi_finalize()
Define input text files
Run mpirun or mpiexec