Collaborative Design of the Community Climate System Modelfor Software Performance PortabilitySponsor: Department of EnergyOffice of Biological and Environmental Research(OBER)Scientific Discovery through Advanced Computing(SciDAC)Principal InvestigatorsBob Malone (LANL) and John Drake (ORNL)Co-Principal InvestigatorsChris Ding (LBNL), Steve Ghan (PNNL), Doug Rotman (LLNL), John Taylor (ANL), Jeff Kiehl (NCAR), Warren Washington (NCAR),S.-J. Lin (NASA/DAO)

GoalsPerformance portability for CCSMOpen software design processLayered architecture to insulate modelingModern SWE practicesImprovement of methods and models supporting more comprehensive coupled climate simulations

Tasks Accomplished

Tasks Accomplished

CCA/ESMF ComponentsDriver has a GO portEach component must register its Use and Provide ports via an interface description and addProvidesPortregisterUsesPortgetPortremoveProvidesPortThe data structures are from ESMF/MCT infrastrucutreESMF_PhysGridESMF_DistGridESMF_Field

Coupler ArchitectureLayer 1aLayer 1bLayer 1cLayers 2-5main programcontrolmainDatamsgmapdiagfluxhistoryrestartcouplinginterfaceMCTwrappercalendarutilitiescsmsharedataTypesMCT derived objectsMCT base objectsMPEU utilitiesVendor utilities

Coupler DevelopmentModel Coupling ToolkitRelease of MCT 1.0, November 200212 new functions to retrieve subsets of data from MCT Attribute Vector and General Grid data typesadditional communication functions for the GlobalSegMap Rearranger module supports two forms of general data transfer between parallel components: between components that share no processors and between components that share all or a portion of their processor space MPH3 (multi-processor handshaking) library for coupling component modelsProTex was added, and a complete manual writtenMPH2 used in CCSM2MPH3 used in CSU coupled model development CPL6 DevelopmentCPL6 datatypes (so-called bundles) built upon the basic datatype (attribute vector) defined in MCT Initial timing of the message passing and mapping functions

MCT: Model Coupling ToolkitMajor Attributes:Maintains model decomposition descriptors (e.g., global to local indexing)Inter- and intra- component communications and parallel data transfer (routing)Flexible, extensible, indexible field storageTime averaging and accumulationRegridding (via sparse matrix vector multiply)The MCT eases the construction of coupler computational cores and component-coupler interfaces.www-unix.mcs.anl.gov/mctVersion 1.0 Released14 November 2002

MPH: Multi-Component Handshaking LibraryGeneral Features:Built on MPIEstablishes MPI communicator for each componentPerforms component name registrationAllows resource allocation for each componentSupports different execution modeswww.nersc.gov/research/SCG/acpi/MPHMPH allows generalized communicator assignment, simplifying the component model communication and inter-component communication setup process.

Mapping: ocn -> atmOcn (122,880 points) -> Atm (8192 points)bundle of 9 fields120 mapping calls

Mapping: atm -> ocn

Atm (8192 points) -> Ocn (122,880 points)bundle of 9 fields120 mapping calls

Parallel I/OParallel decompositions of ZioLibParallel performance of ZiolibParallel NetCDF in conjunction with SciDAC SDM Center

Evolution of Performance of the Community Atmospheric Model

CAM2.0 Eulerian Spectral Model at T42L26ORNL IBM p690 and PSC Compag AlphaServer SCHybrid MPI/OpenMP programming paradigmCache friendly chunks, load balance, improved algorithms

Performance of the CAM2 with FV Core

FV Core on IBM NHII in lat-vert decompositionDAOs mod_comm replacing PILGRIM

Performance of CAM-2 with Finite-Volume Dycore at 2-deg x 2.5-deg x 26 levels

Number of processors

Simulated days per day

1-D decomp., 4 thrds

1-D decomp., 8 thrds

2-D decomp., 4 thrds

2-D decomp., 8 thrds

Performance of CAM-2 with Finite-Volume Dycore at 2-deg x 2.5-deg x 66 levels (WACCM Configuration)

Atmospheric Model Resolution Increasing resolution to T85 (and beyond) in the operational model continues to be pursued. First coupled T85 study performed for CCSM workshop.Resolution studies at T170 planned. FV core resolution increased.

Subgrid Orography SchemeReproduces orographic signature without increasing dynamic resolutionRealisitic precipitation, snowcover, runoffMonth of March simulated with CCSM

Land Surface Model and River Transport Model Land Model Development Activity Community Land Model (CLM) Requirements Document reviewed SciDAC software engineering is focused on the interface and reduction of gather/scatters; communications bottleneck removedRTM is currently single processor -- designing parallel implementation and data structuresAnalysis of runoff in control simulation. Effect on July ocean salinity.

Chart4

134465.125120233189407

189168.9375147443177302

212335.234375171238175500

223868.625194071220130

241179.78125210292250108

193577.5210538216733

107056.8125199425145642

57494.20703117497183395

31120.35546913758149274

18992.94140610250555380

27182.6562593663104437

66360.140625100921168251

CCSM2

Observed

CCM3

Discharge (m3/s)

Amazon

data

JanFebMarAprMayJunJulAugSepOctNovDec

(31, 110)AmazonCCSM21344651891692123352238692411801935781070575749431120189932718366360125234

amazonObserved12023314744317123819407121029221053819942517497113758110250593663100921155240

amazon42189407177302175500220130250108216733145642833954927455380104437168251152,963

(50, 50)AmurCCSM25468.096682073.9946291585.511729.05273417268.0937516489.77148415923.71093820746.44335929659.76953131511.79687524710.55664114141.672852

amurObserved187912239393132139211609615585192102041716139592424079739

amur4222801266642556121440171911413112287113691133411699126811634616,959

(40,32)BrahmaputraCCSM2390940043617407643456780137762116224587197639257509210031

brahmaObserved462745775769105541694129811385104043931214202098679586618100

brahma4227796242031716111024759058304957468851208109192132877713,706

(48,84)ColumbiaCCSM2323383238032600323052410311401605334061730249187282254717507

columObserved31623433400657499786129188830498434232911296230985439

colum4210929139851700217278259553918235110228071318584676796789518,216

(29,4)CongoCCSM2126701124297126392135278139702130873898032953116204330406687410367793531

congoObserved47384377463465336880387583643431472310723644843342518635569340145

congo4213893415949916885719424620428218729315637711708879131545055376987547133,461

corrObserved20186208441902917920158061687916287143011483317042166461782517300

corr42276808597912152384323539113385521254142531062888848011978240,007

danubeObserved5941621973678574893883167122551947044447499658406499

danube422910270026482827337943615187527647834164366932493,763

(49,84)FraserCCSM21829715588142811453216635124324936275334598981174132053012486

(40,31)GangesCCSM2853080945572373540505779167173267836570245469849745813631

gangesObserved292324412079195121805581175983376135280155546426398610813

ganges4226408180001276396097778687520813293294376459832564493991227,628

irraObserved0

irra422360161411588807268051675389922651448305032481,927

(58, 45)LenaCCSM26369187844765370828861417183095216409163871923215215

lenaObserved2629197714721203557873996398022750825064139493430285416622

lena425295538950624496391934293067288929633413406547724,063

mac1Observed337930222873309412379181881604212881105608725522135228324

mac142125001166110723980893531002412117143771563015307145331351112,462

(56, 80)MackenzieCCSM28181491139194405202954351934325995675828644106191139013979

(31,37)MekongCCSM210359744551132869119910323235501835323889458961014532

mekongObserved23991862168515052347713714145218722128312113616436198011

mekong424233274320271623136912321313181926473728647568633,006

(36, 37)Mekong-realCCSM2-2341-3297-4310-32793548771230534142062035429265012-154314067

missObserved15772173142072623193234181603611402831674918430107731466714795

miss427846739568886463619562586767765784298641846881927,433

(42,96)MississippiCCSM21229112274126741429214331116258327642452995212638591219855

(34,0)NigerCCSM23091811773135711401409713382242831738043139945617

nigerObserved17001664137979234814219468814361443137915361058

niger4289491489413263967575933982145150911279458621,978

(43, 11)NileCCSM227693292742298512273498435926310132701907015850148881934015794

nileObserved1239103411261111117015401741155611901104107611551254

nile426338257382489823799527455236013264487589312010280136815146411293,067

(55,25)ObCCSM2390733932657410918789259303115-3036-6931458268140125527

obObserved45923807329433461459732577296962228914028101076069530012475

ob4236533310612621022099188941682317406219832831334524404734114827,956

obidosObserved12023314744317123819407121029221053819942517497113758110250593663100921155240

obidos426996580505961621269801227397774743970249641680314533215464391861,653

(35, 106)OrinocoCCSM21589768181866135346871134517458162301368116942225082323612668

OrinocoObserved12600892075209270184003080040500462004440035400271001970025068

Orinoco4242718261961582299266855751212010133322338345350648276149227,452

(19, 108)ParanaCCSM2-47001045241435708436223623833-6760-14901-21032-22227-18074-128089025

paranaObserved0

parana42122221909347452941781096548963961915400742650018670144831237845,522

(41, 93)Rio GrandeCCSM295718753745547703083244016051370118221389861125925402

saoObserved3244411651394330232318001587151115931740219021712645

sao424859793621450345952963118512112027548594051674532422212,966

(49, 104)St. LawrenceCCSM211332120021429922187230727461-3898-5822-27532255631690317957

St. Lawrenceobserved at Cornwall, Canada6689703674667774799080808043797279467832769173367654

terceroObserved86111867985971101219566586388

tercero42489460425390357329306293359437485502403

tocObserved15508248552785524257158607862444130832728341755121070512174

toc4255121042721822418756510152560339242186914832107537913606624,388

uruguayObserved5133727647324733103138301103238419103166941423251187153

uruguay42216210204198193188183178200225226223204

volgaObserved5751885982481431209010413313186154

volga4223972229632138819747191102065624335274742735325227243402435323,410

xinguObserved101471538321696185721497975913439185813631508243843488610

xingu421194429679779338931501273691389851016725843132471408478289112433391,311

(43,43)YangtzeCCSM227921181901796727587397425533266833601004858530600336433932838819

yangtzeObserved965092581163321150324504142547300369253302527000193501121525032

yangtze4262104432172774419445172542305847878649938935995283875487562354,459

(45, 42)YellowCCSM2134536385663910510169522392129918349513943053775529963264326798

yellowObserved5585319841023996897205228262774237213647461427

yellow4286577347636855535002488655067983121641374413825108568,491

(57,29)YeneseiCCSM2592928241009553108615722358447187698092964210729855416053

yeneseiObserved55805529529252472772675590268291770417141138466370534017683

yenesei425909430230092212174314391249121216332976513465913,117

zambeziObserved3503289246344096330940194558195419912293275740423337

zambezi42865779421029611830122401295114048156251674316225135111063512,559

graphs

graphs

6369.1518552629.25295

1877.82678219775389

446.5658261471.665062

64.722131202.684496

3707.7111825577.883919

28860.625739963429

41717.503906398023067

30951.720703275082889

16408.716797250642963

16386.769531139493413

19232.43753429.84065

15214.5214842853.84772

CCSM2

Observed

CCM3

Discharge (m3/s)

Lena

3907.2214364592.0936533

3392.5510253807.231061

2656.5556643294.3526210

4108.7451173346.4922099

18789.21093814596.5618894

25929.9062532576.7316823

3114.96264629695.6517406

-3036.15966822289.0721983

-692.68035914027.6528313

1458.24450710106.8734524

2681.2192386068.6540473

4012.2590335299.8941148

CCSM2

Observed

CCM3

Discharge (m3/s)

Ob

134465.125120233189407

189168.9375147443177302

212335.234375171238175500

223868.625194071220130

241179.78125210292250108

193577.5210538216733

107056.8125199425145642

57494.20703117497183395

31120.35546913758149274

18992.94140610250555380

27182.6562593663104437

66360.140625100921168251

CCSM2

Observed

CCM3

Discharge (m3/s)

Amazon

126701.46093847384.21138934

124296.57812537746.16159499

126392.25781234653.175168857

135277.687536880.35194246

139702.0937538757.5204282

130873.17187536434.115187293

89803.2812531471.83156377

29530.66601631071.635117088

16203.54003936448.42579131

33039.94140643341.53554505

66874.32031251863.4753769

103677.42187555692.7487547

CCSM2

Observed

CCM3

Discharge (m3/s)

Congo

8530.3505862923.4426408

8093.775879244118000

5571.886232078.6212763

3734.6435551950.569609

4050.041262180.447778

5779.0214845580.786875

16717.0546881759820813

32678.187533761.1129329

36569.81253528043764

24545.97265615554.1159832

9849.0009776425.7856449

7458.1752933985.8939912

CCSM2

Observed

CCM3

Discharge (m3/s)

Ganges

3909.3757324626.6727796

4003.8908694576.524203

3616.7751465768.8717161

4075.81591810553.7311024

4345.08007816941.477590

6780.008789298115830

13776.35449238509.54957

21162.32226640439.194688

24587.29492231213.535120

19763.17382820209.258109

9256.9472668678.9419213

5092.2260745865.8328777

CCSM2

Observed

CCM3

Discharge (m3/s)

Brahmaputra

5928.81445355805909

2824.0334475529.394302

1009.0085455292.453009

553.159185246.732212

10860.69628927725.711743

57222.88671975589.81439

58447.27343826828.571249

18768.58984417704.081212

8091.59033217140.821633

9641.89355513846.122976

10729.3105476370.415134

8554.3515625339.86591

CCSM2

Observed

CCM3

Discharge (m3/s)

Yenesei

8180.7724613379.0612500

4911.1088873021.8811661

3918.8386232873.2510723

4404.8994143094.199808

20295.13281212379.479353

43519.30078118188.1810024

34325.25781216042.2112117

9956.30468812881.4714377

7581.50048810559.9515630

8643.9667978724.6815307

10619.0644535221.4114533

11389.6435553521.7113511

CCSM2

Observed

CCM3

Discharge (m3/s)

Mackenzie

309.1887821699.57894

180.5533451664.03914

177.3621831378.66894

312.551697791.931326

571.372192348.223967

1401.018799142.35759

4097.227539194.383398

13382.456055687.642145

24283.1210941435.851509

17379.8925781442.811127

4313.0703121379.31945

993.7103271535.9862

CCSM2

Observed

CCM3

Discharge (m3/s)

Niger

27692.6640621239.4263382

29274.4082031034.4257382

22985.1132811126.0848982

12272.5888671111.0837995

4983.8759771170.2527455

3591.5043951540.1723601

6310.4980471740.7532644

13269.8388671556.0887589

19069.5234381190.33312010

15850.1123051103.92280136

14888.1835941076.1781514

19339.6933591154.8364112

CCSM2

Observed

CCM3

Discharge (m3/s)

Nile

11331.8007816689.2105263158

12002.409187035.947368421

14298.5742197465.8421052632

22186.781257773.5789473684

23072.4433597990.052631579

7461.2929698079.947368421

-3897.629158042.8421052632

-5822.4848637972.3157894737

-2753.3505867946.3684210526

2254.9721687831.5263157895

6316.3725597690.6842105263

9030.8593757335.5789473684

CCSM2

Observed (Cornwall)

Discharge (m3/s)

St. Lawrence

12290.88183615772.477846

12274.17968817313.877395

12674.09472720726.336888

14291.7158223192.536463

14330.758789234186195

11625.49121116035.536258

8327.11230511402.476767

6424.238778316.137657

5299.2705087491.48429

5211.6762784308641

6385.43994110773.448468

9120.68945314666.888192

CCSM2

Observed

CCM3

Discharge (m3/s)

Mississippi

27920.660156965062104

18190.3945319257.543217

17966.63476611632.527744

27587.031252115019445

39742.0351563245017254

55332.4296884142523058

66832.5546884730047878

60100.3398443692564993

48585.0273443302589359

30599.8925782700095283

33642.6484381935087548

39327.6601561121575623

CCSM2

Observed

CCM3

Discharge (m3/s)

Yangtze

13453.321289558.0258657

6385.491211530.6257347

6638.557617983.556368

10509.6474611023.2755553

16951.583984996.255002

23920.958984897.3754886

29917.7265622051.95506

34951.1757812826.457983

39429.9765622773.8512164

53774.8242192371.57513744

52996.1054691364.42513825

32642.558594745.610856

CCSM2

Observed

CCM3

Discharge (m3/s)

Yellow

-4699.53613320186.0912222

10452.3378912084419093

41435.49218819029.1847452

70842.718751792094178

62235.73828115805.91109654

23833.24609416879.0989639

-6759.61181616286.6461915

-14901.32226614300.7340074

-21032.04687514833.0926500

-22226.7656251704218670

-18073.6113281664614483

-12808.30761717825.1812378

CCSM2

Observed

CCM3

Discharge (m3/s)

Parana

5468.096681879.122801

2073.9946291223.0226664

1585.5939.2925561

11729.0527343132.3121440

17268.0937513920.9617191

16489.77148416095.7714131

15923.71093815585.1912287

20746.44335919210.1911369

29659.76953120417.3111334

31511.79687516138.8511699

24710.5566415923.8512681

14141.6728522406.7316346

CCSM2

Observed

CCM3

Discharge (m3/s)

Amur

15897.1386721260042718

6818.34668892026196

1866.153076752015822

1353.42907792709926

4687.349121184006855

11344.960938308007512

17457.6757814050012010

16230.1337894620013332

13681.2949224440023383

16942.0292973540045350

22507.8847662710064827

23235.5214841970061492

CCSM2

Observed

CCM3

Discharge (m3/s)

Orinoco

-2340.9272462398.874233

-3297.1850591862.12743

-4310.034181684.72027

-3278.6511231505.061623

354.3825382346.71369

8771.3798837136.631232

23052.70898414145.051313

41420.43359421872.381819

62034.64062521283.372647

42926.12109412113.383728

5012.3911136164.056475

-1542.7336433619.066863

CCSM2

Observed

CCM3

Discharge (m3/s)

Mekong

32337.9707033162.210929

32379.5683593432.9513985

32599.7363284006.1717002

32305.406255749.0417278

24103.4472669785.7725955

11401.28027312918.3139182

6052.6040048830.3535110

3406.2128914983.8622807

1730.1723633422.513185

2491.264162911.358467

8727.7265622962.256796

22547.2773443098.057895

CCSM2

Observed

CCM3

Discharge (m3/s)

Columbia

Chart2

12290.88183615772.477846

12274.17968817313.877395

12674.09472720726.336888

14291.7158223192.536463

14330.758789234186195

11625.49121116035.536258

8327.11230511402.476767

6424.238778316.137657

5299.2705087491.48429

5211.6762784308641

6385.43994110773.448468

9120.68945314666.888192

CCSM2

Observed

CCM3

Discharge (m3/s)

Mississippi

data

JanFebMarAprMayJunJulAugSepOctNovDec

(31, 110)AmazonCCSM21344651891692123352238692411801935781070575749431120189932718366360125234

amazonObserved12023314744317123819407121029221053819942517497113758110250593663100921155240

amazon42189407177302175500220130250108216733145642833954927455380104437168251152,963

(50, 50)AmurCCSM25468.096682073.9946291585.511729.05273417268.0937516489.77148415923.71093820746.44335929659.76953131511.79687524710.55664114141.672852

amurObserved187912239393132139211609615585192102041716139592424079739

amur4222801266642556121440171911413112287113691133411699126811634616,959

(40,32)BrahmaputraCCSM2390940043617407643456780137762116224587197639257509210031

brahmaObserved462745775769105541694129811385104043931214202098679586618100

brahma4227796242031716111024759058304957468851208109192132877713,706

(48,84)ColumbiaCCSM2323383238032600323052410311401605334061730249187282254717507

columObserved31623433400657499786129188830498434232911296230985439

colum4210929139851700217278259553918235110228071318584676796789518,216

(29,4)CongoCCSM2126701124297126392135278139702130873898032953116204330406687410367793531

congoObserved47384377463465336880387583643431472310723644843342518635569340145

congo4213893415949916885719424620428218729315637711708879131545055376987547133,461

corrObserved20186208441902917920158061687916287143011483317042166461782517300

corr42276808597912152384323539113385521254142531062888848011978240,007

danubeObserved5941621973678574893883167122551947044447499658406499

danube422910270026482827337943615187527647834164366932493,763

(49,84)FraserCCSM21829715588142811453216635124324936275334598981174132053012486

(40,31)GangesCCSM2853080945572373540505779167173267836570245469849745813631

gangesObserved292324412079195121805581175983376135280155546426398610813

ganges4226408180001276396097778687520813293294376459832564493991227,628

irraObserved0

irra422360161411588807268051675389922651448305032481,927

(58, 45)LenaCCSM26369187844765370828861417183095216409163871923215215

lenaObserved2629197714721203557873996398022750825064139493430285416622

lena425295538950624496391934293067288929633413406547724,063

mac1Observed337930222873309412379181881604212881105608725522135228324

mac142125001166110723980893531002412117143771563015307145331351112,462

(56, 80)MackenzieCCSM28181491139194405202954351934325995675828644106191139013979

(31,37)MekongCCSM210359744551132869119910323235501835323889458961014532

mekongObserved23991862168515052347713714145218722128312113616436198011

mekong424233274320271623136912321313181926473728647568633,006

(36, 37)Mekong-realCCSM2-2341-3297-4310-32793548771230534142062035429265012-154314067

missObserved15772173142072623193234181603611402831674918430107731466714795

miss427846739568886463619562586767765784298641846881927,433

(42,96)MississippiCCSM21229112274126741429214331116258327642452995212638591219855

(34,0)NigerCCSM23091811773135711401409713382242831738043139945617

nigerObserved17001664137979234814219468814361443137915361058

niger4289491489413263967575933982145150911279458621,978

(43, 11)NileCCSM227693292742298512273498435926310132701907015850148881934015794

nileObserved1239103411261111117015401741155611901104107611551254

nile426338257382489823799527455236013264487589312010280136815146411293,067

(55,25)ObCCSM2390733932657410918789259303115-3036-6931458268140125527

obObserved45923807329433461459732577296962228914028101076069530012475

ob4236533310612621022099188941682317406219832831334524404734114827,956

obidosObserved12023314744317123819407121029221053819942517497113758110250593663100921155240

obidos426996580505961621269801227397774743970249641680314533215464391861,653

(35, 106)OrinocoCCSM21589768181866135346871134517458162301368116942225082323612668

OrinocoObserved12600892075209270184003080040500462004440035400271001970025068

Orinoco4242718261961582299266855751212010133322338345350648276149227,452

(19, 108)ParanaCCSM2-47001045241435708436223623833-6760-14901-21032-22227-18074-128089025

paranaObserved0

parana42122221909347452941781096548963961915400742650018670144831237845,522

(41, 93)Rio GrandeCCSM295718753745547703083244016051370118221389861125925402

saoObserved3244411651394330232318001587151115931740219021712645

sao424859793621450345952963118512112027548594051674532422212,966

(49, 104)St. LawrenceCCSM211332120021429922187230727461-3898-5822-27532255631690317957

St. Lawrenceobserved at Cornwall, Canada6689703674667774799080808043797279467832769173367654

terceroObserved86111867985971101219566586388

tercero42489460425390357329306293359437485502403

tocObserved15508248552785524257158607862444130832728341755121070512174

toc4255121042721822418756510152560339242186914832107537913606624,388

uruguayObserved5133727647324733103138301103238419103166941423251187153

uruguay42216210204198193188183178200225226223204

volgaObserved5751885982481431209010413313186154

volga4223972229632138819747191102065624335274742735325227243402435323,410

xinguObserved101471538321696185721497975913439185813631508243843488610

xingu421194429679779338931501273691389851016725843132471408478289112433391,311

(43,43)YangtzeCCSM227921181901796727587397425533266833601004858530600336433932838819

yangtzeObserved965092581163321150324504142547300369253302527000193501121525032

yangtze4262104432172774419445172542305847878649938935995283875487562354,459

(45, 42)YellowCCSM2134536385663910510169522392129918349513943053775529963264326798

yellowObserved5585319841023996897205228262774237213647461427

yellow4286577347636855535002488655067983121641374413825108568,491

(57,29)YeneseiCCSM2592928241009553108615722358447187698092964210729855416053

yeneseiObserved55805529529252472772675590268291770417141138466370534017683

yenesei425909430230092212174314391249121216332976513465913,117

zambeziObserved3503289246344096330940194558195419912293275740423337

zambezi42865779421029611830122401295114048156251674316225135111063512,559

graphs

graphs

6369.1518552629.25295

1877.82678219775389

446.5658261471.665062

64.722131202.684496

3707.7111825577.883919

28860.625739963429

41717.503906398023067

30951.720703275082889

16408.716797250642963

16386.769531139493413

19232.43753429.84065

15214.5214842853.84772

CCSM2

Observed

CCM3

Discharge (m3/s)

Lena

3907.2214364592.0936533

3392.5510253807.231061

2656.5556643294.3526210

4108.7451173346.4922099

18789.21093814596.5618894

25929.9062532576.7316823

3114.96264629695.6517406

-3036.15966822289.0721983

-692.68035914027.6528313

1458.24450710106.8734524

2681.2192386068.6540473

4012.2590335299.8941148

CCSM2

Observed

CCM3

Discharge (m3/s)

Ob

134465.125120233189407

189168.9375147443177302

212335.234375171238175500

223868.625194071220130

241179.78125210292250108

193577.5210538216733

107056.8125199425145642

57494.20703117497183395

31120.35546913758149274

18992.94140610250555380

27182.6562593663104437

66360.140625100921168251

CCSM2

Observed

CCM3

Discharge (m3/s)

Amazon

126701.46093847384.21138934

124296.57812537746.16159499

126392.25781234653.175168857

135277.687536880.35194246

139702.0937538757.5204282

130873.17187536434.115187293

89803.2812531471.83156377

29530.66601631071.635117088

16203.54003936448.42579131

33039.94140643341.53554505

66874.32031251863.4753769

103677.42187555692.7487547

CCSM2

Observed

CCM3

Discharge (m3/s)

Congo

8530.3505862923.4426408

8093.775879244118000

5571.886232078.6212763

3734.6435551950.569609

4050.041262180.447778

5779.0214845580.786875

16717.0546881759820813

32678.187533761.1129329

36569.81253528043764

24545.97265615554.1159832

9849.0009776425.7856449

7458.1752933985.8939912

CCSM2

Observed

CCM3

Discharge (m3/s)

Ganges

3909.3757324626.6727796

4003.8908694576.524203

3616.7751465768.8717161

4075.81591810553.7311024

4345.08007816941.477590

6780.008789298115830

13776.35449238509.54957

21162.32226640439.194688

24587.29492231213.535120

19763.17382820209.258109

9256.9472668678.9419213

5092.2260745865.8328777

CCSM2

Observed

CCM3

Discharge (m3/s)

Brahmaputra

5928.81445355805909

2824.0334475529.394302

1009.0085455292.453009

553.159185246.732212

10860.69628927725.711743

57222.88671975589.81439

58447.27343826828.571249

18768.58984417704.081212

8091.59033217140.821633

9641.89355513846.122976

10729.3105476370.415134

8554.3515625339.86591

CCSM2

Observed

CCM3

Discharge (m3/s)

Yenesei

8180.7724613379.0612500

4911.1088873021.8811661

3918.8386232873.2510723

4404.8994143094.199808

20295.13281212379.479353

43519.30078118188.1810024

34325.25781216042.2112117

9956.30468812881.4714377

7581.50048810559.9515630

8643.9667978724.6815307

10619.0644535221.4114533

11389.6435553521.7113511

CCSM2

Observed

CCM3

Discharge (m3/s)

Mackenzie

309.1887821699.57894

180.5533451664.03914

177.3621831378.66894

312.551697791.931326

571.372192348.223967

1401.018799142.35759

4097.227539194.383398

13382.456055687.642145

24283.1210941435.851509

17379.8925781442.811127

4313.0703121379.31945

993.7103271535.9862

CCSM2

Observed

CCM3

Discharge (m3/s)

Niger

27692.6640621239.4263382

29274.4082031034.4257382

22985.1132811126.0848982

12272.5888671111.0837995

4983.8759771170.2527455

3591.5043951540.1723601

6310.4980471740.7532644

13269.8388671556.0887589

19069.5234381190.33312010

15850.1123051103.92280136

14888.1835941076.1781514

19339.6933591154.8364112

CCSM2

Observed

CCM3

Discharge (m3/s)

Nile

11331.8007816689.2105263158

12002.409187035.947368421

14298.5742197465.8421052632

22186.781257773.5789473684

23072.4433597990.052631579

7461.2929698079.947368421

-3897.629158042.8421052632

-5822.4848637972.3157894737

-2753.3505867946.3684210526

2254.9721687831.5263157895

6316.3725597690.6842105263

9030.8593757335.5789473684

CCSM2

Observed (Cornwall)

Discharge (m3/s)

St. Lawrence

12290.88183615772.477846

12274.17968817313.877395

12674.09472720726.336888

14291.7158223192.536463

14330.758789234186195

11625.49121116035.536258

8327.11230511402.476767

6424.238778316.137657

5299.2705087491.48429

5211.6762784308641

6385.43994110773.448468

9120.68945314666.888192

CCSM2

Observed

CCM3

Discharge (m3/s)

Mississippi

27920.660156965062104

18190.3945319257.543217

17966.63476611632.527744

27587.031252115019445

39742.0351563245017254

55332.4296884142523058

66832.5546884730047878

60100.3398443692564993

48585.0273443302589359

30599.8925782700095283

33642.6484381935087548

39327.6601561121575623

CCSM2

Observed

CCM3

Discharge (m3/s)

Yangtze

13453.321289558.0258657

6385.491211530.6257347

6638.557617983.556368

10509.6474611023.2755553

16951.583984996.255002

23920.958984897.3754886

29917.7265622051.95506

34951.1757812826.457983

39429.9765622773.8512164

53774.8242192371.57513744

52996.1054691364.42513825

32642.558594745.610856

CCSM2

Observed

CCM3

Discharge (m3/s)

Yellow

-4699.53613320186.0912222

10452.3378912084419093

41435.49218819029.1847452

70842.718751792094178

62235.73828115805.91109654

23833.24609416879.0989639

-6759.61181616286.6461915

-14901.32226614300.7340074

-21032.04687514833.0926500

-22226.7656251704218670

-18073.6113281664614483

-12808.30761717825.1812378

CCSM2

Observed

CCM3

Discharge (m3/s)

Parana

5468.096681879.122801

2073.9946291223.0226664

1585.5939.2925561

11729.0527343132.3121440

17268.0937513920.9617191

16489.77148416095.7714131

15923.71093815585.1912287

20746.44335919210.1911369

29659.76953120417.3111334

31511.79687516138.8511699

24710.5566415923.8512681

14141.6728522406.7316346

CCSM2

Observed

CCM3

Discharge (m3/s)

Amur

15897.1386721260042718

6818.34668892026196

1866.153076752015822

1353.42907792709926

4687.349121184006855

11344.960938308007512

17457.6757814050012010

16230.1337894620013332

13681.2949224440023383

16942.0292973540045350

22507.8847662710064827

23235.5214841970061492

CCSM2

Observed

CCM3

Discharge (m3/s)

Orinoco

-2340.9272462398.874233

-3297.1850591862.12743

-4310.034181684.72027

-3278.6511231505.061623

354.3825382346.71369

8771.3798837136.631232

23052.70898414145.051313

41420.43359421872.381819

62034.64062521283.372647

42926.12109412113.383728

5012.3911136164.056475

-1542.7336433619.066863

CCSM2

Observed

CCM3

Discharge (m3/s)

Mekong

32337.9707033162.210929

32379.5683593432.9513985

32599.7363284006.1717002

32305.406255749.0417278

24103.4472669785.7725955

11401.28027312918.3139182

6052.6040048830.3535110

3406.2128914983.8622807

1730.1723633422.513185

2491.264162911.358467

8727.7265622962.256796

22547.2773443098.057895

CCSM2

Observed

CCM3

Discharge (m3/s)

Columbia

Atmospheric Chemistry Gas-phase chemistry with emissions, deposiiton, transport and photo-chemical reactions for 89 species. Experiments performed with 4x5 degree Fvcore ozone concentration at 800hPa for selected stations (ppmv)Mechanism development with IMPACTA) Small mechanism (TS4), using the ozone field it generates for photolysis rates.B) Small mechanism (TS4), using an ozone climatology for photolysis rates.C) Full mechanism (TS2), using the ozone field it generates for photolysis rates.

Zonal mean Ozone, Ratio A/CZonal mean Ozone, Ratio B/C

Chemistry Validation Asterisks with error-bars are ozonesonde data. Solid line is full chemistry, dashed line is small chemistry using the climatology, and dotted line is small chemistry using its own ozone field

Sea Ice Model Incremental Remapping for Sea Ice and Ocean Transport Incremental remapping scheme that proved to be three times faster than MPDATA, total model speedup of about 30% --added to CCSM/CSIM Cache and vector optimizationsCICE3.0 restructered for vector Community Sea Ice ModelSensitivity analysis and parameter tuning test of the CICE code Automatic Differentiation (AD)-generated derivative code Major modeling parameters that control the sea ice thickness computation were the ice-albedo constants, densities and emissivities of ice and snow, and salinity constant Parameter tuning experiment with gradient information

POP Ocean ModelSoftware Engineering for POP and CICEDesign and implementation for the new ocean model (HYPOP) and CICE in progressOcean Model PerformancePOP2: new design involves a decomposition of the computational domain into blocks that can be sized to fit into cache On 1/10 degree, SGI (2x), IBM (1.25x), long vector gets 50% peak on FujitsuMLP in POPOn 1/10 degree, SGI(2x) HYPOP Model DevelopmentTreat purely Lagrangian dynamics of constant-mass layers as they inflate and deflate in regions intersecting bottom topography Pressure gradient is split into a 'baroclinic' part that vanishes and a 'barotropic' part that does not vanish when the density is uniform Comparison of surface height in Lagrangian and Eulerian vertical after 400 baroclinic steps

oldnewTotal115s55sBaroclinic93s38sBarotropic9s7s

MLP version of CCSMThe first bar reflects the run time of CCSM 2 after its conversion to MLP-based inter-module communication. The second is the reported result for the original code on the Power3 cluster performed at NERSC. The third result is the original code as executed on the 600 MHZ Origin. Origin executions utilized 208 CPUs. The IBM execution utilized 128CPUs.

Ocean generic Grid Generator - gCubedGUI in Tcl/TkTripole grid supported

Ocean Biogeochemistry

Iron Enrichment in the Parallel Ocean ProgramSurface chlorophyll distributions in POPfor 1996 La Nia and 1997 El Nio

Global DMS Flux from the Ocean using POP The global flux of DMS from the ocean to the atmosphere is shown as an annual mean. The globally integrated flux of DMS from the ocean to the atmosphere is 23.8 Tg S yr-1 .

BiogeochemistryFine-resolution POP simulationssingle phytoplankton-bin ecodynamics and embracing complete iron, nitrogen and sulfur cycles carbon cycle has been added, diatoms segregated as separate tracer so silicon cycle also includedDMS distributions analyzed and improvedCAM Analysisglobal atmospheric methane and carbon monoxide (distributions and fluxes) from IMPACT Marine Aerosol-Gas Phase Interactions(MAGPI) model extended to reactive bromine chemistry

CCSM2.0 Configuration

1 hour1 daypescomponentocniceatmlndcpl540 4123216(IBM Power3)

Coordination and ManagementManagment PlanMemorandum of Understanding between SciDAC and ESMFCoordination with SciDAC Integrated Software Infrastructure Centers and CollaboratoriesPERC, ESG, SDM, TSTT, CCA, APDEC, TOPS,

CCSM2 Coupled Simulations Coupled model released May 17, 2002

Simulation of Future Climates

Computing for this simulation was done at DOE's National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory, NCAR and Oak Ridge National Laboratory. CreditsAnimation and data management:Michael Wehner/Lawrence Berkeley National LaboratoryParallel Climate Model:Warren Washington, Jerry Meehl, Julie Arblaster, Tom Bettge and Gary Strand/National Center for Atmospheric ResearchVisualization softwareDean Williams/Lawrence Livermore National Laboratory

The End

CAM and CLM2 PlansRevisit architecture documents and bring into line with ESMF, CCA, etc.Block decomposition of dycores CAM, communicator interfacesMod_comm for CAMCLM2 M-2-N coupling and Parallel RTM using MCT types and utilitiesComprehensive performance portability/tuning Cache/vector optimizations

Emphasize that the NGC is a framework in which coupler instantiations can be easily built on top of the utilities.

For example, the current CCSM2 coupler is cpl5, a shared memory threaded application in which flexibility is not great.

Cpl6 is an instantiation of the NGC, which preproduces the functionality of cpl5.

Cpl6 is itself not the NGC, but rather a composition of the utiltities to achieve the same functionality of cpl5.

Cpl7, for example, might be a reproduction of cpl6, but with sequential execution rather than concurrent. Cpl8 might be one in which a new model has been added to the mix.

6,7,8 should be easy to build the top layer using the tools and utiltites provided within the cplx architecture.

Of course, some tools, utilities, functions, are more mature than others for example, the GUI interfacestory