Recent Progresses in Numerical Weather Prediction and HPC at KMA

Hee-Dong YooKorea Meteorological Administration

26th WGNE MeetingOctober 18-22, 2010, Tokyo

Major Changes in Operational NWP System

Regional (Deterministic short-range)

Global (Deterministic medium-range)

Major NWP Changes

• T426L40 (GDAPS)– GSM from JMA– 3DVAR– Operation since 1997

• UM N320L50– The Unified Model from UKMO– 4DVAR– Operation since May 2010

• 30kmL33 (RDAPS)– MM5 Model– FDDA– Operation since 1997

• 10kmL40 (KWRF)– WRF ARW Model– Operation since 2007

• UM 12kmL38– The Unified Model from UKMO– Initialized from Global UM– Operation since May 2010

• 10kmL40 (KWRF)– WRF ARW Model– L.B.C. from Global UM

- 3 -

- 4 -

UM Implementation Background

KMA decided to import the Unified Model as a next-generation NWP

system (Q4 ’07)

UM Research license (Q4 ’07)

Collaboration Agreement – including Science Plan - between KMA and UK Met Office

(Operational License, Q2 ’08)

Routine operation of global/regional UM started (Q2 ’08)

Initialized from UK Met Office’s initial condition

Global D.A. cycle for UM including ODB implementation (’08~’09)

Migration of UM system to the 3rd supercomputer (Q4 ’09)

Parallel run of UM system on Cray XT5 Interim (March ’10~)

Operational run of UM system on Cray XT5 (14th May ’10~)

- 5 -

Cray XT5 (3rd HPC)

Global Model(UM N320L50)

10-day Forecast

Regional Model(UM 12km)

Parallel Suite : 2010.03 ~ 2010.05

ObservationPre-Processing

DataAssimilation

Obs. Pre-proc. System (OPS)

Atmos. 4DVAR (VAR)Surf. Analysis (SURF)

COMIS

GTS/FTP

Decoded Obs.(ODB)

Pre-/Post-Server(New)

Application/Statistical Models

Post-ProcessingRegional Model(KWRF 10km)

Operational Suite : 2010.05 ~

Operational UM System

- 6 -

Regional• Horiz. : ~12km (540x432 / 0.11°x0.11°)• Vert. : 38 layers (top ~ 39km)• +72hrs Forecast• Initialized from Global I.C.• Version : UM 6.6

Global• Horiz. : N320 (~40km / 0.5625°x0.375°)• Vert. : 50 layers (top ~ 60km)• +252hr Forecast• Initialized by 4DVAR• Version : UM 6.6

UM Configuration in KMA

MODELSHoriz. Resol.(Vert. Layers)

Forecast Length Target

UM (GLOBAL) 40km (50) 10 days Global Medium-range Fcst.

GDAPS

30km (40) 10 days Global Medium-range Fcst.

55km (40) 10 days Global Medium-range (EPS)

110km (21) 120 days Seasonal Forecasting

UM (REGIONAL) 12km (38) 72 hours East-Asia Short-range Fcst.

RDAPS 30/10/5km (33) 66/24/24 hours East-Asia Short-range Fcst.

KWRF 10 km (40) 66 hours East-Asia SRF – UM-based

KLAPS 5km 12 hours Korean Peninsula

Wave Models(Wave Watch-III)

60km 10 days GWW3(Global) – UM-based

8km 66 hours RWW3(E-Asia) – UM-based

1km 24 hours CWW3(Coastal) – UM-based

Sand Dust Model (ADAM)

30km 72 hoursYellow-Dust (East Asia) – UM-based

Tide and Storm Surge (RTSM)

8km 72 hoursRegional Tide & Storm Surge – UM-based

Typhoon Model(DBAR)

35km 72 hours Track & Intensity

Statistical/Digital Fcst. - 2-10 days UM-based

Main Operational Models (’10.5~)

- 8 -

UM-GLOBAL UM-REGIONAL

Governing Eq. Complete equation (Non-hydrostatic)

Horiz. Resolution N320 (40km 0.5625x0.375) 12km (0.11x0.11)

Vertical Layers L50 (top ~ 63km) L38 (top ~ 39km)

Forecast Length 10.5 days (252 hours) 3 days (72 hours)

Timestep Size 900 sec 240 sec

I.C./ Data Assimilation 4DVAR Downscaling from global initial condition

Spatial Discretization Finite Difference method

Time integ. / Advection Semi-implicit Semi-Lagrangian scheme

Radiation Process Edwards-Slingo general 2-stream scheme

Surface Process MOSES-II land-surface scheme

PBL Process MOSES-II Non-local PBL

Convection Process Mass flux convection with CAPE closure

Microphysics Mixed-phase precipitation

Gravity Wave Drag G.W. drag due to orography (GWDO)

Surface B. C. Surface Analysis + Climatology

Operation Frequency Twice daily (00/12 UTC) / 6hour D.A. cycle

Twice daily (00/12 UTC)

Operational NWP Models – Unified Model

- 9 -

GDAPS (Deterministic) GBEPS (Ensemble)

Base Model GSM (Global Spectral Model from JMA)

Governing Eq. Primitive Equation (Hydrostatic)

Horiz. Resolution T426 (0.28125x0.28125) T213 (0.5625x0.5625)

Vertical Layers L40 (top : 0.4 hPa)

Forecast Length 252 hours 240 hours

I.C./ Data Assimilation 3DVAR Breeding Method + Factor Rotation / 3DVAR

Ensemble Size - 17*2 (12 hour time-lag)

Spatial Discretization Spectral Transformation

Time integration Semi-implicit Scheme

Radiation Process SW : Lacis and Hansen (1974) / LW : JMA (Sugi et al., 1989)

Surface Process SiB (Simple Biosphere, Sellers, 1986)

PBL Process Non-Local PBL (Holtslag and Boville, 1993)

Convection Process Kuo Type (Kuo, 1974)

Microphysics Large-scale condensation (Sundqvist, 1978)

Gravity Wave Drag GWD due to orography (Iwasaki et al., 1989)

GWD due to cumulus convection (Chun and Baik, 1988)

Operation Frequency Twice daily (00/12 UTC)6hour 3DVAR cycle

Twice daily (00/12 UTC)6hour 3DVAR cycle

Operational NWP Models – Others / Global / Atmos.

RDAPS KWRF

30km 10km 5km

Base Model MM5 WRF ARW

Dynamic Frame Non-hydrostatic Non-hydrostatic

Horiz. Resolution 30km(171x191) 10km(160x178) 5km(141x141) 10km(574x514)

Vertical Layers 33 Layers / ~50 hPa 40 Layers / ~50 hPa

Forecast Length 66 hours 24 hours 24 hours 66 hours

I.C./Data Assim. FDDA 3hr cycle (IAU) 1-way interact. 3DVAR / DFI

Lateral B.C. Relaxation (12hr) Time & inflow/outflow dependent relaxation (3hr)

Updated every 6 hrs (global UM)

Spatial Discretization Finite Difference Finite Difference

Radiation Process Cloud Radiation RRTM

Surface Process 5-layer Soil Model Noah LSM

PBL Process MRF PBL YSU PBL

Convection Process New Kain-Fritsch None New Kain-Fritsch

Microphysics Mixed phase WSM6

Operational NWP Models – Others / Regional / Atmos.

- 10 -

Data Assimilation System – for Global UM

- 11 -

Observation Data and Global D.A.

• Observation DB : ODB CY32R3• Observations Used

– Surface (SYNOP/Ship/Buoy), Sonde (TEMP, Pilot, Wind Profiler), Aircraft, Satwind, AIRS (AQUA), Scatwind, IASI (MetOp), GPSRO (COSMIC), SSMIS

– ~80,000 observations per cycle (80~85% compared to UK Met Office)

• Analysis Scheme : 4-dimensional Variational Data Assimilation• Analysis Time : 00, 06, 12, 18 UTC• Cut-off Time : 2 hours 25 minutes for Early Analysis

6 hours 25 minutes for Update Analysis• Spatial Resolution (Inner Model) : N108(1.67x1.11deg) L50• Assimilation Window : -3 hours to +3 hours of Analysis Time

- 12 -

NWP Performance - Global

Forecast error comparison (GDAPS vs. UM)

- 13 -

NWP Performance - Global

+5days

+3days

RMS error for 500 hPa Geopotential Height / N. Hemisphere

Old (GDAPS)

New (UM)

NWP Performance - Global

• Verification Period– April ~ September 2010

• ~15% enhancement compared to GDAPS for 500 hPa Geopotential Height (+120 hours)

Verification against Analysis

• The Regional UM outperforms KWRF

• Light Precipitation is significantly over-estimated in the UM

KWRF(10km)

UM12

UM(12km) – KWRF(10km)

Precipitation Verification (against 76 ASOS obs.) – UM versus KWRF

- 15 -

NWP Performance - Regional (Precipitation)

New Supercomputer andUpcoming NWP Changes

NEC SX-5/28M21999.12 ~ 2005.11

1st Supercomputer

Theoretical performance

0.2 TFlops

×90

2nd Supercomputer

Theoretical performance

18.5 TFlops

CRAY X1E 1024/MSP2005.12 ~ 2010.11

3rd Supercomputer

Theoretical performance

682.9 TFlops

CRAY XE6 90,240 cores2010 ~

×37

- 17 -

Current Status (’10.10) - 1st phase of 3rd Supercomputer - Cray XT5 - 2,560 cores / 27.65 TFlops

KMA’s Supercomputer

Cray XT5 MPP System with Lustre Global Parallel File System– Processor : AMD 2.7GHz (4 core)

System 2Interim System

System 1Main Computational System

KMA’s 3rd Supercomputer – Initial Phase

- 18 -

Cray Next Generation XE6 (Baker) MPP System– Processor : AMD 2.1GHz (12 core)

KMA’s 3rd Supercomputer – Final Phase

System 1 - OperationalMain Computational System

System 3Interim System

System 2 – Research/BackupMain Computational System

- 19 -

Cray XE6

Cray XE6

• Construction : June 2008 ~ Jan 2010

• Official Opening Ceremony: Feb 2010

• Total Cost: 25.3 billion won(about 23M USD)

• Total Construction Area : 23,092 ㎡ / KMA HQ: ~16,500 ㎡

• Building area: 7,052 ㎡ , 3rd floors

Electricity: 250 % greater than HQ

UPS: 475 % greater than HQ

Cooling : 360 % greater than HQ

KMA’s Supercomputer Center

- 21 -

Upcoming NWP Changes

Global NWP System

• Improving the TC Bogussing Scheme for Global NWP– Current Bogussing Method : Wind Bogussing (Obs. Bogussing)

– New Development : Wind + Sea Level Pressure Bogussing

– Effect : Improvement in intensity (and track) of simulated TCs

– Operational Application : Q4 2010

Intensity errorreduced

Track errorreduced

- 22 -

Upcoming NWP Changes on the New Supercomputer

New Global Ensemble Prediction System

• GDAPS T213L40 (GBEPS)– Model : GDAPS– T213 (0.5625x0.5625) – Bred Vector + Factor Rotation– No Model Error Simulation– 16+1 members (x2, 00/12UTC)– 10 day Forecast– 6-hourly 3DVAR Cycle

00, 12UTC Forecast

• UM N320L50 (MOGREPS-G)– Model : The Unified Model– N320 (0.5625x0.375) – ETKF– Stochastic Physics included– 23+1 members– 15 day Forecast– Control I.C. from Deterministic Model

00, 12UTC Forecast

• Currently on its Pre-operational Real-time Test• Operational Run : Q1 2011

– Platform : Cray XE6 (final phase of KMA’s 3rd supercomputer)

RMSE

SPREAD

Solid : OPER

Dashed : N320L50

Results for Summer 2010

CPRSS_New

CPRSS_Old

- 23 -

Upcoming NWP Changes on the New Supercomputer

Resolution Change of Global NWP Model

• UM N320L50– N320 (640x481 / ~40km) – 50 Vertical Levels (~63km)– UM Version : vn6.6– 4DVAR Inner Loop : N108

• UM N512L70– N512 (1024x769 / ~25km) – 70 Vertical Levels (~80km)– UM Version : vn7.5 or later– 4DVAR Inner Loop : N144

• Operational Run : Q2 2011 on Cray XE6

N320 N512

• Max. Height in South Korea domain

– N320 orog. : 523m– N512 orog. : 692m

- 24 -

Upcoming NWP Changes on the New Supercomputer

Regional 4DVAR UM System

• UM 12kmL38– 12km (0.11x0.11 deg) – 38 Vertical Levels (~39km)– UM Version : vn6.6– Initialized from Global Model

(Cold-Start)– 00, 12UTC Forecast (+72 hours)

• UM-4DVAR 12kmL38– 12km (0.11x0.11 deg) – 70 Vertical Levels (~80km)– UM Version : vn7.5 or later– 6-hourly Atmos. 4DVAR +

Global Surface Analysis Downscaled– 00, 12UTC Forecast (+72 hours)

• Operational Run : Q2 2011 on Cray XE6

- 25 -

Upcoming NWP Changes on the New Supercomputer

Local High-Impact Weather Forecasting System

• UM 1.5kmL70– New NWP System

– Horizontal Resolution : 1.5km Variable Grid

– Vertical Levels : 70 Levels (~39km)

– UM Version : vn7.5 or later

– Data Assimilation : 3-hourly 3DVAR Cycle

Radar Data

• Pre-Operational Parallel Run : Q3 2011• Operational Run : Q2 2012

12km Domain

1.5km Domain

Two Tracks on the Future NWP at KMA

• Successful UM operation on new supercomputer• Modification and Improvement

• Development of our own NWP System• Own Techniques PBL scheme, Microphysics scheme,

Gravity wave, own dynamics (DFS)

- 26 -

- 27 -

Development of an Original NWP System

Long-Term Development Plan (’11~’19)

• 1st Phase : Initial Development Stage– Development of Core Modules

: Dynamics Core, Physical Parameterization Schemes, etc.

• 2nd Phase : Growing Stage– Production of a Prototype Numerical Model

: Combining Developed Modules and Preliminary Test

• 3rd Phase : Mature (Pre-Operational) Stage– Stabilization, Performance Enhancement

: Preparation for the Operational Implementation

# New organization out of KMA!

# Invitation for the position of the head of new organization soon!

Climate Prediction DivisionClimate Prediction DivisionClimate Prediction DivisionClimate Prediction Division

CPDCPD

LRFClimate Outlook

International

CooperationClimate Monitoring

Technical

Development

LRF and Climate OutlookLRF and Climate OutlookLRF and Climate OutlookLRF and Climate Outlook

1-month Forecast

- 3rd, 13th and 23rd of the month

3-month Forecast

- 23rd of the month

Seasonal Climate Outlook

- Feb. for Summer, May for Autumn, Aug. for Winter,

Nov. for Spring

ProcedureProcedureProcedureProcedure

Global SST Prediction Model Global Dynamic Model

Statistical Analysis Model

APCC MME(15 model, 8 contries) WMO LC-LRFMME(12 GPCs)

Discussion (CPD)

LRF ET Meeting (season) Int’l Joint Meeting (summer, winter)

Discussion

Press Release

Briefing

Disaster Prevention System

KMA Homepage Mass Media, …

「 World Best 365 」 하늘을 친구처럼 , 국민을 하늘처럼

WMO LC-LRFMMEWMO LC-LRFMMEWMO LC-LRFMMEWMO LC-LRFMME

http://www.wmolc.org

No. of Members: 106 members from 45 countries

「 World Best 365 」 하늘을 친구처럼 , 국민을 하늘처럼

Future PlanFuture PlanFuture PlanFuture Plan

UM based Atmospheric Model- Operational Forecast System is going to be replaced with the UM based atmospheric model(N144L38) by 2011

GloSea4- According to the agreement between UKMO and KMA on the joint seasonal forecast system(’10.6.22), KMA is currently working to move its seasonal forecast system to the GloSea4(HadGEM3N96L85/ORCA1L75)

< Major Plan > - Jan 2011: Migration of KMA-GloSea4 from Cray-XT5 to Cray-XE6- 2011: Testing of KMA-GloSea4 with real-time ICs from UKMO- 2012: Semi-operational run of KMA-Glosea4 with real-time ICs from UKMO - 2013: Operational run of KMA-GloSea4

Thank You for Your Attention

See you again in Korea or other countries!!