The Fujin

Development of

Parallel Coupler

Takashi Arakawa

arakawa@tokyo.rist.or.jp

Research Organization for Information Science & Technology

General Meaning of Fujin

• A god of wind• Inscribed in

Chinese/Japanese Kanji characters

風神

windgod

Painted by Khorin Ogata

God of Wind• Very popular name in Japan

– Used in various fields

Our Fujin

• Framework for meteorological model(s)

• Objectives– parallel computing, domain nesting,

coupling– with minimal effort and minimal time

• Information hiding– Modular and Layered Structure– Fortran 90 employed

What can you do with the Fujin?

• Parallelization– Two Dimensional domain decomposition

• Domain Nesting– Two way nesting– No limitation for nesting depth– Each coupled models can have different ne

sting depth

• Coupling– Multi-model coupling

Limitations of the Fujin• Structured Grid based model only

– FEM, Spectral Model etc. are not supported

• No implementation of interpolation scheme for Nested Domains/Coupled Models– interpolation scheme is meteorological

issue• not concerned problem for the Fujin

– Fujin provides only the simple interface

Parallel Package

Data Definition

Interface

existing model

Structure of the Fujin• 25 modules, 450

procedures• Three layers

– Parallel package layer• parallel library package using

MPI

– Data layer• definition of data structure• procedures for the data

– Interface layer• Utility modules

– Error handling,logging,file management

Detail of the Interface Layer

Coupler

Driver

Model Routines Model Routines

Data definition

Driver

• Coupler– task control– flux exchange

• Data Definition– definition of the global

variables

• Driver– Initialization, Integration,

Result output– Inserting procedures of

existing model

Grid Structure

• User is required to define the location of each global variable in a initialization subroutine

i_endi_start

i_start i_end+1

j_start

j_start

j_end

j_end+1

(1,1)

(1,1)

(ni,nj)

(ni+1,nj+1)

call InitData(pot_tmp,”potential temperature”, SCALAR_POINT)

Domain Nesting (1)Time Integration Scheme

• Two way self-nesting

①

②

③ ④

⑤

⑥ ⑦

Time

Domain 1

Domain 2

Domain 3

Domain Nesting (2)Data Exchange Scheme

• Effective data exchange scheme1 2

3 4

1 2

3 4

①

① ②

③ ④

①Interpolation

CouplingTask Control Strategy

• Each model is attached to one of these tasks– Models in the same task are computed serially– Models in the different tasks are computed parallelly

Coupler

Task 1 Task 2 Task 3

Model 2

Model 1Model 3 Model 4

• Models in the same task must have the same

number of domainHaving a different

decomposition pattern is allowed

Coupling (2)data exchange scheme

• Same as the scheme of domain nesting• Type-D ← similar to MPCCI of GMD• Difference from MPCCI is coupling strategy

CA

CA

CA

C O

C O

C O

C O

Connecting the Model• Target model is CReSS

– non-hydrostatic model developed by Nagoya Univ. and RIST

0

20

40

60

80

100

1 10 100number of processors

para

llel e

ffici

ency

• Parallel Efficiencyover 85% for 64 PEsCharacteristic of CReSS,

not the Fujin

Connecting the Model (2)

• Domain Nesting Test– Three dimensional mountain wave– Three domains– Interpolation and boundary condition←Meso-NH

Future Plan• Final goal

– To release the Fujin as free software

• Program structure is still complex

• Document