Francisco Castejón [email protected] CIEMAT As coordinator of NA4-Fusion:

33
User Forum 2006 Francisco Castejón Francisco Castejón [email protected] [email protected] CIEMAT CIEMAT As coordinator of NA4-Fusion: SW-Federation (CIEMAT, BIFI, UCM, INTA - Spain-), Russian Federation (Kurchatov Institute - Russia-), CEA (France), ENEA (Italy), Culham (UK), Korea (KISTI) Fusion activities in Fusion activities in the GRID the GRID (na4-egeeII) (na4-egeeII)

description

Fusion activities in the GRID (na4-egeeII). Francisco Castejón [email protected] CIEMAT As coordinator of NA4-Fusion: SW-Federation (CIEMAT, BIFI, UCM, INTA -Spain-), Russian Federation (Kurchatov Institute -Russia-), CEA (France), ENEA (Italy), Culham (UK), Korea (KISTI). - PowerPoint PPT Presentation

Transcript of Francisco Castejón [email protected] CIEMAT As coordinator of NA4-Fusion:

Page 1: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Francisco CastejónFrancisco Castejó[email protected]@ciemat.es

CIEMATCIEMAT

As coordinator of NA4-Fusion: SW-Federation (CIEMAT, BIFI, UCM, INTA -

Spain-), Russian Federation (Kurchatov Institute -

Russia-), CEA (France), ENEA (Italy), Culham (UK), Korea

(KISTI)

Fusion activities in the GRID Fusion activities in the GRID (na4-egeeII)(na4-egeeII)

Page 2: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

OutlineOutline

Motivation: Fusion on the GRID.Data storage and handling.

Fusion Deployment and VO setup. Applications: Computing in Plasma Physics.Future Applications in the grid. Final Remarks.

Motivation: Fusion on the GRID.Data storage and handling.

Fusion Deployment and VO setup. Applications: Computing in Plasma Physics.Future Applications in the grid. Final Remarks.

Page 3: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

MotivationMotivationLarge Nuclear Fusion installations: International

Cooperation among several Institutes. Generate ~ 1-10 GB/sec. Less than 30% of data

goes into processing. Distributed data storage and handling needed. Massive Distributed Calculation: A new way of

solving problems. (Physics Problems still without solution).

Fusion community (Science and Technology) needs new IT approaches to increase research productivity.

Large Nuclear Fusion installations: International Cooperation among several Institutes.

Generate ~ 1-10 GB/sec. Less than 30% of data goes into processing.

Distributed data storage and handling needed. Massive Distributed Calculation: A new way of

solving problems. (Physics Problems still without solution).

Fusion community (Science and Technology) needs new IT approaches to increase research productivity.

Page 4: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Pulse Preparation

Countdown

Pulse

Data Collection

Data Storage

Intershot Analysis

Control/Diagnostic

Setup/Validation

Offline Analysis

JET - PulseJET - PulsePulse

PreparationSimulatio

nModelling

Real-time control

Data Analysis

Data Acquisition

International ProjectRemote

ParticipationIT

SecurityNuclear Registered

Site

Data Management

Supercomputers

Page 5: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

NA4-Catania, 2006

ITER: Making decisions in real Time !!

ITER: Making decisions in real Time !!

Data Acquisition

and Storage (Grid, Supercomputers)

Data Acquisition

and Storage (Grid, Supercomputers)

Data Analysis and Reduction: Artificial Intelligence, Neural Network, Pattern Recognition

Simulations: Large codes in different platforms (Grid, Supercomputers)

Decision for present/next shot

One half an hour shot every one and half an hour: Decisions in real time.

Page 6: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

ITER PartnersITER Partners

Distributed Participation. Distributed Participation. Data access. Remote Control Rooms?Data access. Remote Control Rooms?

Page 7: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

NA4-Catania, 2006

DATA HANDLINGDATA HANDLING

Storage: Large data flux: 104 sensors x 20-50 kHz sampling= 1-10 GBy per second raw datax 0.5 h= 3 TBy per shot in ITER every 1,5 h

Supercomputing and Grid Computing --> Data Storage: Scratch and permanent.

Data Access & Sharing in Large Cooperative Experiments:A unified representation for experimental data native storages that LCG-2/gLite CE middleware could understand is needed.(get and put the data files from and into the storage). Storage should allow to do some basic processing: neural network, clustering…

Page 8: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Communications Communications

Remote Participation tools:Data AccessLocal VisualizationVideo Conferences and ChatsRemote ControlSECURITY & ROBUSTNESS

Page 9: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

PARTNERS and Resources for VOPARTNERS and Resources for VOSW Federation: CIEMAT, BIFI, UCM, INTA (Spain) Kurchatov (Russia). Culham Laboratory- UKAEA (UK) KISTI (South Korea). ENEA (Italy).CEA (France).

SW Federation: CIEMAT, BIFI, UCM, INTA (Spain) Kurchatov (Russia). Culham Laboratory- UKAEA (UK) KISTI (South Korea). ENEA (Italy).CEA (France).

Experience in using and developing Fusion Applications.Experience in porting applications and developing Grid Technologies.

--> Connection with EELA (Some fusion partners: Brazil, Mexico, Argentina)

Page 10: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

VO DeploymentVO Deployment

Present: CIEMAT: 27 KSpecInts; BIFI: 8 KSpecInts; INTA: 6 nodes Present: CIEMAT: 27 KSpecInts; BIFI: 8 KSpecInts; INTA: 6 nodes

http://grid.bifi.unizar.es/egee/fusion-vo/

http://www-fusion.ciemat.es/collaboration/egee/

Within less than 6 months: JET: 38 KSpecInts; BIFI: 32 KSpecInts; CEA-Cadarache ?KISTI?, INTA?, ENEA?

Within less than 6 months: JET: 38 KSpecInts; BIFI: 32 KSpecInts; CEA-Cadarache ?KISTI?, INTA?, ENEA?

Beginning of 2007:JET: 32 adtional cores; BIFI: 32 additional cores; CIEMAT ?;CEA-Cadarache ?(second phase already committed).

[email protected]

Beginning of 2007:JET: 32 adtional cores; BIFI: 32 additional cores; CIEMAT ?;CEA-Cadarache ?(second phase already committed).

[email protected]

Page 11: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

COMPUTING in the GRID: Present ApplicationsCOMPUTING in the GRID: Present Applications

Applications with distributed calculations: Monte Carlo, Separate estimates, …

Multiple Ray Tracing: e. g. TRUBA. Stellarator Optimization: VMEC Transport and Kinetic Theory: Monte Carlo Codes.

Applications with distributed calculations: Monte Carlo, Separate estimates, …

Multiple Ray Tracing: e. g. TRUBA. Stellarator Optimization: VMEC Transport and Kinetic Theory: Monte Carlo Codes.

Page 12: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Multiple Ray Tracing: TRUBAMultiple Ray Tracing: TRUBA

Single Ray (1 PE): Single Ray (1 PE): Hamiltonian Hamiltonian Ray Tracing Equations.Ray Tracing Equations.

Beam Simulation: Beam Simulation:

Bunch of rays with Bunch of rays with beam waist far beam waist far from the critical from the critical layer (100-200 layer (100-200 rays)rays)

Bunch of rays with Bunch of rays with beam waist close beam waist close to the critical layer to the critical layer (100-200 rays) x (100-200 rays) x (100-200 wave (100-200 wave numbers) numbers) ~10~1055

GRID PROBLEMGRID PROBLEM

Page 13: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

TRUBA: Multiple Ray TracingTRUBA: Multiple Ray TracingDifferent results with the two Different results with the two approximations.approximations.

(Also useful tool for looking for (Also useful tool for looking for Optimum Launching Position Optimum Launching Position in complex devices)in complex devices)

TRUBA for EBW: TRUBA for EBW: Collaboration between IOFAN and CIEMAT. Collaboration between IOFAN and CIEMAT. Useful for all Institutes with EBW heating (Culham, Useful for all Institutes with EBW heating (Culham, Princeton, Greifswald, CIEMAT,…)Princeton, Greifswald, CIEMAT,…)

Page 14: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

TRUBA: Multiple Ray TracingTRUBA: Multiple Ray TracingTRUBA for EBW: TRUBA for EBW: - Cylinder geometry: A single Non-relativistic ray (tens of Cylinder geometry: A single Non-relativistic ray (tens of sec.)sec.)-Real geometry in TJ-II:Coming from a supercomputer Real geometry in TJ-II:Coming from a supercomputer (VMEC).(VMEC).

- A single Non-relativistic ray (about 18’).A single Non-relativistic ray (about 18’).- A single relativistic ray (about 40’).A single relativistic ray (about 40’).

- Some problems with Geometry libraries.Some problems with Geometry libraries.

-See:See:J. L. VJ. L. Vázquez-Poleti. “Massive Ray Tracing in ázquez-Poleti. “Massive Ray Tracing in Fusion Plasmas on EGEE”. User Forum, 2006.Fusion Plasmas on EGEE”. User Forum, 2006.

Page 15: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

TJ-II: R= 1.5 m; a= 0.2 m, M=4, l=1.

HSX: R=1.2 m; a=0.15 m

LHD: R= 3.75 m; a= 0.65 m, M=10, l=2.

CHS: R= 1 m; a= 0.2 m; M=8; l=2.

Stellarator Optimization: Choosing the best concept

Page 16: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Optimised Stellarators QPS and NCSXSupercomputer Optimization

NCSXQPS

Page 17: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

The importance of Stellarator

optimization.The importance of Stellarator

optimization.

-Design of new Plasma Physics-Fusion devices under complex exigencies.

OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.

Many Labs looking for the optimized device.

-Design of new Plasma Physics-Fusion devices under complex exigencies.

OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.

Many Labs looking for the optimized device.

Page 18: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Stellarator optimization in the GridStellarator optimization in the Grid

-A lot of different Magnetic Configurations operating nowadays. OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.Every variant computed on a separate processor (~10’) VMEC (Variational Momentum Equilibrium Code)

120 Fourier parameters are varied.

-A lot of different Magnetic Configurations operating nowadays. OPTIMIZATION NECESITY BASED ON KNOWLEDGE OF STELLARATOR PHYSICS.Every variant computed on a separate processor (~10’) VMEC (Variational Momentum Equilibrium Code)

120 Fourier parameters are varied.

rB(ψ,θ,ϕ ) =

rBm,n

m,n∑ (ψ )ei(mθ−nϕ )

R(ψ ) = Rm,nm,n∑ (ψ )cos(mθ −nϕ )

Z(ψ ) = Zm,nm,n∑ (ψ )sin(mθ −nϕ )

Coils producing field confining the plasma may be optimised numerically by variation of the field parameters.

Coils producing field confining the plasma may be optimised numerically by variation of the field parameters.

Page 19: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

-Neoclassical Transport.- Bootstrap current.- Equilibrium vs. plasma pressure.- Stability (Balloning, Mercier,…)

-Neoclassical Transport.- Bootstrap current.- Equilibrium vs. plasma pressure.- Stability (Balloning, Mercier,…)

Partícle trajectories in W7-X

Optimization Criteria: Target Functions

-Genetic Algorithm to detect the optimum configuration for given criteria. Target Functions can be modified.

-Genetic Algorithm to detect the optimum configuration for given criteria. Target Functions can be modified.

Page 20: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

VMEC on Kurchatov GRIDVMEC on Kurchatov GRIDLCG-2 - based Russian Data Intensive Grid consortium

resources.About 7.500 cases computed (about 1.500 was not

VMEC-computable, i.e. no equilibrium). Each case took about 20 minutes. Up to 70 simultaneous jobs running on the grid.

- See:See:V. Voznesensky. “Genetic Stellarator V. Voznesensky. “Genetic Stellarator Optimisations in Grid”. User Forum, 2006.Optimisations in Grid”. User Forum, 2006.

LCG-2 - based Russian Data Intensive Grid consortium resources.

About 7.500 cases computed (about 1.500 was not VMEC-computable, i.e. no equilibrium).

Each case took about 20 minutes. Up to 70 simultaneous jobs running on the grid.

- See:See:V. Voznesensky. “Genetic Stellarator V. Voznesensky. “Genetic Stellarator Optimisations in Grid”. User Forum, 2006.Optimisations in Grid”. User Forum, 2006.

Page 21: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Kinetic TransportKinetic TransportFollowing independent particle orbits

Montecarlo techniques: Particles distributed according to experimental density and ion temperature profiles (Maxwellian distribution function)

SUITABLE PROBLEM FOR CLUSTER AND GRID TECHNOLOGIES

Following independent particle orbits

Montecarlo techniques: Particles distributed according to experimental density and ion temperature profiles (Maxwellian distribution function)

SUITABLE PROBLEM FOR CLUSTER AND GRID TECHNOLOGIES

rVD =

rE ×

rB

B2 +m2q

2v2 −v⊥2( )

rB×∇

rB

B3

Page 22: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Kinetic TransportKinetic Transport

Example of orbit in the real 3D Example of orbit in the real 3D TJ-II Geometry (single PE).TJ-II Geometry (single PE).

~1 GBy data, 24 h x 512 PE~1 GBy data, 24 h x 512 PE

Distribution function of parallel Distribution function of parallel velocity at a given position (Data velocity at a given position (Data Analysis).Analysis).

Page 23: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Kinetic transportKinetic transport

No collisions: 0.5 ms of trajectory takes 1 sec. CPU..No collisions: 0.5 ms of trajectory takes 1 sec. CPU..

Collisions: 1 ms of trajectory takes 4 sec CPU. Collisions: 1 ms of trajectory takes 4 sec CPU. Particle life: 150 - 200 ms. Single particle Particle life: 150 - 200 ms. Single particle ~ 10 min.~ 10 min.

Necessary statistics for TJ-II 10Necessary statistics for TJ-II 1077 particles. particles.

Page 24: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

NA4-Catania, 2006

Read from disk:

Input file (“input.lis”):

Number of files.Number of trajectories in each file (may be divided in blocks inside the file).

(total statistics = number of files x number of trajectories per file)Length of each trajectory.Integration algorithm and time discretization.Number of measures in time.Temperature of backgroud electrons and ions.Random seed.Initial distribution (in space and velocity space).Model used, including or not:

Collisions with backgroud ions.Electric field.T-density dependence.

Starting point according to a (space) distribution function (“perfil_densidad.in”).

Magnetic configuration (~ 20 MB file -> stored at the Fusion VO catalog) (“42_100_68.c3d”)

Magnetic field and derivatives.Magnetic coordinates -> electric field.Distance to the vaccum chamber.

Measurement times (“tiempos.lis”)

Read from disk:

Input file (“input.lis”):

Number of files.Number of trajectories in each file (may be divided in blocks inside the file).

(total statistics = number of files x number of trajectories per file)Length of each trajectory.Integration algorithm and time discretization.Number of measures in time.Temperature of backgroud electrons and ions.Random seed.Initial distribution (in space and velocity space).Model used, including or not:

Collisions with backgroud ions.Electric field.T-density dependence.

Starting point according to a (space) distribution function (“perfil_densidad.in”).

Magnetic configuration (~ 20 MB file -> stored at the Fusion VO catalog) (“42_100_68.c3d”)

Magnetic field and derivatives.Magnetic coordinates -> electric field.Distance to the vaccum chamber.

Measurement times (“tiempos.lis”)

Kinetic transport: The applicationKinetic transport: The application

Page 25: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Written on disk:

●“Raw” data ( “OUT????.DAT”) ●“Pictures” of the plasma at fixed times (persistence, total energy, kinetic energy, components of the velocity ...)●“Points” of escape of the particle from the TJ-II (according to several different criteria)●~ 0.9 KB / traj. (typically 1000 traj. / file -> ~ 900 KB)●writen every ~1 hour

●Histograms (“OUT????.HIS”)●Runtime histograms of serveral magnitudes (energies, componentes of velocity, fluxes ...) as a function of spacial coordinates (rho, phi, theta)●~ 0.16 KB / traj. (typically 1000 traj. / file -> ~160 KB)●writen every ~1 hour

●Debug (“RUNTIME.DAT”)●Measurements at the end of the trajectories, just for check●~ 0.2 KB / file OUT???.DAT●updated with every new OUT???.DAT (typically 1000 files -> ~200 KB)

●Individual trajectories (“ty_??_??_??.dat”) ●Complete information of each trajectory (time, position, velocity ...)●from ~10 MB to ~ GB / traj. depends on the length of the trajectory -> just for debug

Total disk space = ~ 1.3 GB

Page 26: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

COMPUTING in the GRID: Future applicationsCOMPUTING in the GRID: Future applications

EDGE2D Application for tokamaks

Transport Analysis of multiple shots (typically 104 shots) or Predictive Transport with multiple models: e. g. ASTRA. CIEMAT(Spa) + IPP(Ger) + Kurchatov(Rus) + EFDA(UE) + …

Neutral Particle Dynamics: EIRENE:CIEMAT(Spa) + IPP(Ger)

EDGE2D Application for tokamaks

Transport Analysis of multiple shots (typically 104 shots) or Predictive Transport with multiple models: e. g. ASTRA. CIEMAT(Spa) + IPP(Ger) + Kurchatov(Rus) + EFDA(UE) + …

Neutral Particle Dynamics: EIRENE:CIEMAT(Spa) + IPP(Ger)

Page 27: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

JET – Flagship of Worldwide Fusion: EDGE2D Equilibrium code.

JET – Flagship of Worldwide Fusion: EDGE2D Equilibrium code.

Page 28: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Cross section of present EU D-shaped tokamaks compared to the ITER project

EDGE2D: Determine plasma shape from Measurements: Plasma current, Pressure,

Magnetic field…

rj ×

rB=

r∇p

-EDGE2D code solves the 2 D fluid equations for the conservation of energy, momentum and particles in the plasma edge region.

-Ions, electrons and all ionisation stages of multiple species are considered.

-Interaction with the vessel walls is simulated by coupling to monte-carlo codes, to provide the neutral ion and impurity sources.

Page 29: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Massive Transport CalculationsMassive Transport Calculations

For Instance: Enhanced heat Confinement in TJ-II. Lower heat diffusivity for low electron density and high absorbed power density.

A different case on every PE.

0

0.5

1

1.5

2

-15 -10 -5 0 5 10 15

Pe-2562Pe-2565Pe-2558Pe-2560Pe-2559

<r> (cm)

0

0.5

1

1.5

-15 -10 -5 0 5 10 15

2562256525582560.2559

<r> (cm)0

0.2

0.4

0.6

0.8

1

1.2

-15 -10 -5 0 5 10 15

25592560255825652562

<r> (cm)

0.1

1

10

0 5 10 15

Chi-2562Chi-2565Chi-2558Chi-2560Chi-2559

r (cm)0

1

2

3

4

5

0.5 1 1.5n

0T

e-3/2 (1019 m-3 keV-3/2)

Page 30: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

EIRENEEIRENE CodeCode

Trayectory of a He atom in TJ-II. Vertical and horizontal proyections. It starts in the green point and is absorbed in the plasma by an ionization process. The real 3D geometry of TJ-II vacuum chamber is considerd.

Page 31: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

EIRENEEIRENE CodeCode

Two parts: 1) Following trajectories (Totally distributed) --> GRID

2) Reduction to put all together.

EIRENE Code comes from IPP (Jülich, Germany) and is extensively used by Fusion community.

0

1 109

2 109

3 109

4 109

5 109

6 109

7 109

0 0,2 0,4 0,6 0,8 1

Neutral densityHydrogen

enl13=0.65enl13=1.00enl13=1.80

enl (10

13

cm-3

)

r/a

0

5 109

1 1010

1,5 1010

2 1010

2,5 1010

3 1010

0 0,2 0,4 0,6 0,8 1

Neutral densityHelium

enl13=0.95

enl (10

13

cm-3

)

r/a

Fit

Radial profile of atoms of He in TJ-II plasmas. An average on every magnetic surface has been done

Radial profile of atoms of H in TJ-II

Page 32: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

User Forum 2006

Final RemarksFinal Remarks

GRID technologies will enhance Fusion Research: computing and data handling.

GRID technologies will win visibility when applied to large Fusion Experiments (like ITER).

Demonstration effect: If Fusion-Grid is succesful, GRID technologies will be extensively used by Fusion Community in the future.

FIRST APPLICATIONS ARE RUNNING IN THE GRID.

GRID technologies will enhance Fusion Research: computing and data handling.

GRID technologies will win visibility when applied to large Fusion Experiments (like ITER).

Demonstration effect: If Fusion-Grid is succesful, GRID technologies will be extensively used by Fusion Community in the future.

FIRST APPLICATIONS ARE RUNNING IN THE GRID.

Page 33: Francisco Castejón franciscostejon@ciemat.es CIEMAT As coordinator of NA4-Fusion:

NA4-Catania, 2006

Para ver esta película, debedisponer de QuickTime™ y deun descompresor TIFF (sin comprimir).