PRELIMINARY DESIGN REVIEW NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP) OCTOBER 16,...

PRELIMINARY DESIGN REVIEW

NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP)

OCTOBER 16, 2012

PRESENTED BY:ANDREA SCHUMACHER1, MARK DEMARIA2, LIMIN ZHAO3

1. CIRA / COLORADO STATE UNIVERSITY2. NOAA/NESIDS/STAR/RAMMB

3. NOAA/NESDIS/OSPO

OUTLINE

Introduction Demaria Project Requirements Schumacher Operations Concept Zhao Algorithm Theoretical Basis Schumacher Software Architecture & Interfaces Schumacher Quality Assurance Zhao Risks and Actions Demaria Summary and Conclusions Demaria

Preliminary Design Review: NPP Microwave-based TC Products

Introduction

Presented by

Mark Demaria(STAR/CIRA)


INTRODUCTION

Project ObjectiveProject StakeholdersProject Plan


Provide TC intensity and structure estimates and mean layer winds from ATMS during the NPP and JPSS era to support the continuity of operation. The products include: Tropical cyclone maximum wind Minimum sea level pressure Radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center Balanced horizontal winds from 1000 to 200 mb within 600km of the

storm center

PROJECT OBJECTIVESPROJECT OBJECTIVES


PROJECT STAKEHOLDERS – DEVELOPMENT TEAM

STAR and CIRAMark DeMaria

Andrea Schumacher

John Knaff

OSPO Limin Zhao

Hazari Syed


PROJECT STAKEHOLDERS – O & M

OSPO Products Area Lead – Liqun Mao

OSPO QA Lead – Zhaohui Cheng

Operational Implementation and Maintenance Team Hazari Syed Jiande Wang

Science Maintenance Team Mark DeMaria Andrea schumacher


PROJECT STAKEHOLDERS – USERS

National Hurricane CenterMichael Brennan ([email protected]) Jack Beven ([email protected])

Joint Typhoon Warning CenterEdward Fukada ([email protected])

Central Pacific Hurricane CenterSatellite Analysis Branch

Mike Turk ([email protected])


Bob Kuligowski

Added Bob Jubach of HRC; he may call in instead of Kosta

IPT Lead: Mark DeMaria (STAR) IPT Backup Lead: Limin Zhao (OSPO)NESDIS Team:

STAR: Mark DeMaria, John Knaff OSPO: Limin Zhao, Liqun Ma (TC Products PAL) OSD: Tom Schott Data Center: Phil Jones Others: Andrea Schumacher (CIRA), Jiande Wang (SSAI), Hazari Syed (SSAI)

User Team Lead: Michael Brennan and Jack Beven (NWS/NHC)

Oversight Panel (OP) lead: ICAPOPOther Ops Involved: Soundings, Precipitation


PROJECT PLAN - IPT MEMBERS

PROJECT PLAN - STAKEHOLDER INVOLVEMENT

Development Team Coding and documentation Quality assurance Data verification and validation Configuration Management

Operation and Maintenance Code Acceptance Running system Interface with operations Monitoring Distribution

Users Supply customer request Review project development Archive data Communicate with development team


PROJECT PLAN -TASK AND SCHEDULES

Tasks: Defined in “FY13_NPP_Tropical_Cyclone_Products_v2.ppt” (SPSRB PSDI)

Schedule (key milestones): Preliminary Design Review – July, 2012 Oct 16, 2012 Critical Design Review – Nov, 2012 Software Review – Feb, 2013 System Readiness Review – July 2013 SPSRB Briefing – Aug 2013 Operations Commence – Aug 2013


Project Requirements

Presented by

Andrea Schumacher(STAR/CIRA)


SPSRB User Request #1009-0017 submitted by NHC Requesting the current AMSU-based TC intensity and structure estimates and mean layer winds be

generalized from ATMS during the NPP and JPSS era to support the continuity of operation.

User community National Hurricane Center and Central Pacific Hurricane Center (CPHC) Joint Typhoon Warning Center (JTWC) & Dept. of Defense NWS Offices

Benefit to user Continuity and potential improvement of TC products that provide guidance to NHC forecasters

NOAA Mission Goal supported Weather and Water, Commerce and Transportation

Mission priority Optimal


PROJECT REQUIREMENTS: USER REQUEST

NCEP GFS analyses and forecastsMIRS ATMS temperature profile retrievalsCurrent TC positions and intensities

NHC a-decks and b-decks, available at ftp://ftp.nhc.noaa.gov/atcf/ and currently being ingested by OSPO

JTWC a-decks and b-decks, currently being ingested by OSPO


PROJECT REQUIREMENTS: INGEST

Product name: ATMS-based TC ProductsAccuracy (same or better than current AMSU-based products)

Intensity estimate – MAE of 13.5 kt / 9 hPa Wind radii estimates – MAE of 32, 18, and 10 nmi for 34-, 50-, and 64-kt radii,

respectively Latency

Available ~ 3 hours after satellite passTimeliness

Run frequency: New product generated every 6 hours at synoptic times (0, 6, 12, 18 UTC)

Updated every 2 hours Run duration: Less than 5 minutes (~30 sec per active TC)


PROJECT REQUIREMENTS: PROCESSING

Product Dissemination NDE/ESPC DDS Text outputs will be integrated into Automated Tropical Cyclone Forecast

(ATCF) system. The data include: TC intensity estimates Surface wind radii estimates (R34, R50 and R64)* 2-d balance winds


PROJECT REQUIREMENTS: DISSEMINATION

*Will need to be hosted in real time on ftp site so they can be retrieved and integrated into ATCF

Code development CIRA/Colorado State and NOAA/NESDIS/STAR/RAMMB, Fort Collins, CO A. Schumacher and M. DeMaria

Operational implementation NTCP runs on the ESPC operational environment with only extra requirements for

disk storage and computing power tcfprod/vmtcfpdev (Linux VMWare)

Memory ~ 100 mb Job Run Time: ~ 30 sec per active TC ~50 GB disk storage

gp5/gp50 (Linux Web server) ~ 1 GB disk storage

DDS (IBM Data Distribution Server) ~ 1 GB disk storage


PROJECT REQUIREMENTS: SYSTEM

Operations ConceptPresented

by

Limin ZhaoNOAA/NESDIS/OSPO


OPERATIONS CONCEPT - OVERVIEW

Review the concept of operations that the customers/users has documented in the SPSRB user request Interact with customers/users to produce an initial algorithm/system

design that is consistent with their concept of operations Review the answers to the following questions based on customer/user

needs and expectations and production constraints What is the product? Why is this product being produced? How will this product be used? How should this product be produced (operational scenario)?

The operations concept will be refined by the NTCP IPT, in consultation with customers/users, as the product solution and design are matured through the development phase.


NPP TC PRODUCTS USERS

The product requirements are originated from NWS through the SPSRB user request, and its users include:National Hurricane Center and Central Pacific Hurricane Center

(CPHC) Joint Typhoon Warning Center (JTWC) & Dept. of Defense


WHAT IS THE PRODUCT?

Tropical Cyclone intensity and structure estimates and mean layer winds from NPP ATMS

Specific Requirements from users

Parameters Specifications

Environmental parameter Tropical cyclone maximum wind, minimum sea level pressure, radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center, balanced horizontal winds from 1000 to 200 mb within 600km of the storm center

Geographical coverage North Atlatic and Northeast Pacific Tropical cyclone basins

Vertical Resolution Point values for intensity/radii; mandatory levels for balanced wind

Horizontal resolution Point values for intensity/radii; 0.2 lat/lon grids for balanced winds

Measurement range Similar to current AMSU-based products

Measurement precision Similar to current AMSU-based products

Measurement accuracy Similar to current AMSU-based products

Latency 3 hours

Refresh 6 hours


WHY ARE THE PRODUCTS BEING PRODUCED?

Current operational TC products are available from NOAA-15/16/18

Users need the operational capability for TC products from the NPP/JPSS satellites in supporting the continuity of operation

The products will be produced using the MiRS ATMS temperature retrievals


HOW WILL THE PRODUCTS BE USED?

The NPP TC products will be used at NHC, JTWC, CPHC and WMO RMSCs for the diagnosis of tropical cyclone including intensity, size and warm/cold core structures

The NPP TC products will be integrated into ATCF for users to access.


HOW SHOULD THE PRODUCTS BE PRODUCED? (1/6)

Satellite Data for Product Generation ATMS-based temperature and moisture profiles from MiRS (netCDF4)

Ancillary Data for Product Generation ATCF TC positions provided by HRC

GFS Analysis and Forecasts from NCEP

Ancillary Data for Product Validation Hurricane flight reconnaissance intensities and wind radii (ground truth)

AMSU-derived products



There will be three distinct environments Development Environment (CIRA, Linux)

Development and testing of pre-operational codes on Redhat Linux OS

Test environment (ESPC, vmtcfpdev) Pre-operational code (DAP) received from CIRA will be tested on the

designated VMWare Linux machine at ESPC and modified as needed before it is promoted to operation

Operation Environment (ESPC, tcfprod) Operational DAP will be run to generate the products on the designated

VMWare machine at ESPC and the products will be distributed to user through ESPC distribution system (DDS)



Coverage Global Ocean

Formats ASCII, netCDF4

Latency 3 hours after satellite pass

Run time ~ 30 seconds per active TC

Resolution Point values for intensity and radii where storms exist

0.2 lat/lon grid for balanced winds

Products Tropical cyclone maximum wind

Minimum sea level pressure

Radii of 34, 50, 64 kt winds in 4 quadrants relative to the storm center

Balanced horizontal winds from 1000 to 200 mb within 600km of the storm centerPreliminary Design Review: NPP Microwave-based TC

Products


Production and Delivery Scenarios ESPC will pull the MiRS SND products from NDE following the existing

data access protocol NTCP will pull the required ancillary data that currently available on the

ESPC internal servers NTCP will collect the satellite inputs and required ancillary data to run the

TC algorithm The product will be generated in ASCII and netCDF formats The product users will be granted access to the ESPC distribution system

through the data access request submission process. ESPC will handle the distribution of the NPP TC products



Production Monitoring and Maintenance Scenarios The product process will be monitored by ESPC and email alerts will be

generated when an error occurs The NPP TC algorithm will provide overall quality control flags and quality

summary level information ESPC will cover the monitoring of input data latency, product latency, and

product distribution Offline Statistic comparison with NHC best track intensity and radii

estimates for cases within 3 hrs of aircraft fix The TC PAL and maintenance personnel at OSPO will monitor the product

quality and will coordinate with the STAR/CIRA scientists for any product quality issues



Production Archival Scenarios The TC intensity/structure products will be provided to NHC as a text

bulletin in Automated Tropical Cyclone Format (ATCF) and will be archived there (same as current AMSU products).

TC balanced winds products will be archived at NCDC, and a Data Submission Agreement will be developed and submitted to NCDC/CLASS for it.

Local archive requires to keep three day worth of data for real-time use


Algorithm Theoretical Basis

Presented byAndrea Schumacher

(STAR/CIRA)


ATMS-based TC products will be based on AMSU-based algorithm already in operations at NCEP

Intensity estimates (operational since 2003) Uses combination of AMSU-based soundings, the hydrostatic relationship, and statistics Described in

Demuth, J., M. DeMaria, and J.A. Knaff, 2006: Improvement of Advanced Microwave Sounding Unit Tropical Cyclone Intensity and Size Estimation Algorithms, Journal of Applied Meteorology and Climatology, 45, 1573-1581.

Demuth, J., M. DeMaria, J.A. Knaff and T. H. Vonder Haar, 2004: Validation of an advanced microwave sounder unit (AMSU) tropical cyclone intensity and size estimation algorithm, J. App. Met., 43, 282-296.

Wind radii estimates (operational since 2005) Estimated from 2-D winds created by AMSU-derived height field and solving non-linear balance equations Described in

Bessho, K., M. DeMaria, and J.A. Knaff, 2006: Tropical cyclone wind retrievals from the Advanced Microwave Sounder Unit: Application to Surface Wind Analysis. J. Appl. Meteor. and Clim., 45,399-415.


ALGORITHM OVERVIEW: SCIENCE DESCRIPTION

NOAA-15/16/18 AMSU-based balanced wind algorithms have been running operationally since 2005. Algorithm and validation details are provided in: Bessho, K., M. DeMaria, and J.A. Knaff, 2006: Tropical cyclone wind

retrievals from the Advanced Microwave Sounder Unit: Application to Surface Wind Analysis. J. Appl. Meteor. and Clim., 45,399-415.


ALGORITHM OVERVIEW: SCIENCE MATURITY


ALGORITHM OVERVIEW - BENEFITS

Algorithm updates Use of operational MIRS sounding products

Replaces Goldberg/Barnett statistical retrievals Will eliminate duplication of efforts in generating retrievals

Current Cal/Val project at CIRA has being evaluated impacts of this change Goal is that increased resolution of ATMS (vs. AMSU) will provide more accurate

estimates of TC intensity and structure Adding another source of polar orbiting satellite microwave-based TC intensity

and structure estimates will complement the existing AMSU-based products to improve the overall latency and availability of the NESDIS microwave-based TC products suite.

Validation planCompare ATMS-derived intensity estimates and wind radii to

AMSU-derived productsHurricane flight reconnaissance intensities and wind radii

(ground truth)ATMS-based products expected to exceed current AMSU-

based products Improved horizontal resolution


ALGORITHM OVERVIEW: PRELIMINARY VALIDATION PLAN

Software Architecture & Interfaces

Presented by

Andrea Schumacher(STAR/CIRA)


Languages and tools Fortran 90 (pgf90, ifort or g95 compilers)

Bash, Perl, and Python (version 2.7 or higher) shell scripts

Gnuplot

GrADS

HTML

Java /JavaScripts

Source of codes CIRA/CSU (most written in-house following the SPSRB F90 guidelines)


SYSTEM DESIGN

Directory Structure Use standard OPS directory structure

/bin Contains compiled executables

/data Contains input and output data files

/logs Contains all log files

/scripts Contains all script files

/setup Contains configuration and setup files

/src Contains all source code


SYSTEM DESIGN (CONT…)

Input data and formats NCEP GFS analysis (packed ASCII, 1-degree grid) MIRS ATMS vertical temperature and moisture profiles (netCDF4) TC positions and intensities (text a-deck and b-deck files)

Ancillary data and auxiliary files Statistical algorithm coefficient data files (text)

Output files and format TC intensity and radii (text) Balanced wind (netCDf4)

Run schedules New runs every 6 hours at synoptic times


SYSTEM DESIGN (CONT…)


IT SYSTEM ARCHITECTURE, DATA FLOW, SECURITY

39393939

DDSDDS

DisseminationDissemination NHC NHC

Input Data Includes: MIRS ATMS temperature profile retrievals, ATCF TC positions, GFS analyses

Input Data Includes: MIRS ATMS temperature profile retrievals, ATCF TC positions, GFS analyses

InternetOSPOInternetOSPO

Online QCMonitoringOnline QCMonitoring

ESPC IT EnvironmentESPC IT EnvironmentESPC IT EnvironmentESPC IT Environment

CLASS/NCDCCLASS/NCDC

ArchiveArchiveArchiveArchive

TC“CIRA Linux Server”

TC“CIRA Linux Server”

NTCP“Production ” NTCP“Production ” NTCP

“System Test” NTCP

“System Test”

NCEP GFSNCEP GFS

NPP ATMS TC ProductsNPP ATMS TC Products

NTCP “Development”

NTCP “Development”

ATMS TC

products

ATMS TC

products

NDE “DDS”NDE “DDS”

MiRS ATMS SND Products

DAP delivery


NTCP SOFTWARE FLOW DIAGRAM

40404040

GFS Analysis(PACK ASCII)

ATCF TC Positions

(text)

MIRS ATMS sounding(netCDF)

Sounding Retrieval

(ASCII)

Python script

Quality Control(Fortran)

Vertical and Horizontal

Integrations(Fortran)

Statistical Analysis (Fortran)

Bias Correction(Fortran)

Outputs (ASCII)

2D azimuthal mean radial/hgt cross sections (.RZA)

Balanced 2-D wind field (.XYA)

Outputs (ASCII)

TC Intensity and wind radii estimates (.FIX)

ATCF fix data (.AFX)

GFS Analysis(grib)

Perl script + Fortran

INPUT / OUTPUT FILES


File name Format Description Size (max)

G000{YY}_X/Y{MMDD}_PACK.DAT Packed ASCII, input 1-degree GFS analysis data file 9 MB

{swath/satellite ID}.nc NetCDF4, input MIRS retrieval data file 5 MB

aal{storm number}{YYYY}.dat ASCII text, input Official positions and intensities of current active global TCs

10 MB per active TC

{stormid}_{date}_{satellite}.FIX ASCII text, main output Swath used, intensity and wind radii estimates

1.4 kB

{stormid}_{date}_{satellite}.AFX ASCII text, main output ATCF fix file (i.e., intensity and wind radii estimate in ATCF format)

< 1 kB

{stormid}_{date}_{satellite}.RZA ASCII text, intermediate/output

Azimuthal mean radial/height cross sections (p, ρ, T, gradient wind)

20 MB

{stormid}_{date}_{satellite}.XYA ASCII text, intermediate/output

2-D balance U, V, T, and Z data 3.0 MB

{stormid}_{date}_{satellite}.STA ASCII text, intermediate Analysis statistics 1.1 kB

{stormid}_{date}_{satellite}.ret ASCII text, intermediate MIRS retrieval data rewritten in simple ASCII format

5 MB

{stormid}_{date}_{satellite}.LOC ASCII text, intermediate Swath lat/lon data 200 kB

Total < 1 Gb

Quality AssurancePresented

by

Limin ZhaoNOAA/NESDIS/OSPO


QA REVIEW

QA Purpose Problem Prevention Early Error Detection Risk Identification and Mitigation

QA Activities SPSRB Processes Configuration Management Project Monitoring System Monitoring Product Performance Monitoring


SPSRB PROCESS

Review Processes PDR CDR Software Code Review Operational System Readiness Review Documentations

Algorithm Theoretical Basis End User Manual Systems Maintenance Manual Archive Data Submission Agreement ESPC Monitoring Procedures

Documentation/Standards OSPO Code Standards SPSRB Documentation Templates


CONFIGURATION MANAGEMENT

CM Tool (CVS) CM Stakeholders

Dorothy Brown CM Lead

Zhaohui Cheng QA Lead

Joseph McIlvenny CM Administrator

Ed Ladd, Russ Lancaster CM installation

NTCP Team Developers CM Users

The ESPC CM Procedures will be enforcedPreliminary Design Review: NPP Microwave-based TC

Products

PROJECT MONITORING

Regular Team meeting to discuss the project status Identify and track the issues and risks throughout the lifecycleProvides managements with objective feedback regarding

process compliance to approved plans, procedures, standards, and analyses


Error tracking Intensity & wind radii estimates will be added to ATCF system

Can be routinely compared with similar estimates from other platforms Can be done by STAR/RAMMB, in coordination with SAB Real-time monitoring/feedback obtained from NHC

Nonlinear balance winds are input to the Multi-Platform Tropical Cyclone Surface Wind Analysis system Operational at NESDIS Includes automated input quality control that can be leveraged to routinely

monitor the nonlinear balance winds Near-real-time monitoring

24 x monitoring by ESPC help desk Monitor product log files each run cycle


QA – PRODUCT MONITORING

Annual verification will be performed by STAR / CIRA CIRA will work with NHC to perform seasonal verifications based on best

track wind intensities and aircraft measured pressures Verification statistics computed as compared to ATCF intensity and wind

radii diagnosesAnnual updates will be performed by STAR / CIRA

Bug fixes, updates to algorithm coefficients as neededSTAR / CIRA to provide near-real-time science support

E.g., answering algorithm-related questions, trouble-shooting As needed during normal business hours (M-F, 8-5pm MST)


QA - SCIENCE MONITORING

QA SUMMARY

QA activities will be performed throughout the software development life cycle.

Code development and documentation will follow the guidelines/standards of SPSRB.

The QA process assures faster and more efficient research to operation transitions.


Risks and ActionsPresented by

Mark DemariaNOAA/NESDIS/STAR


RISKS AND ACTIONS

PDR Risk 1: MiRS retrieval limitations may cause product degradationsRisk Assessment: Preliminary validation of AMSU MiRS data show

temperature/moisture retrievals don’t always converge near storm center

Impact: If too many soundings are not available near the storm center, product accuracy will be degraded

Likelihood: It is likely that some ATMS MiRS retrievals will be missing near the storm center. The impact on the product is unknown. Based on AMSU results, the most likely outcome is that the MiRS retrievals will provide tropical cyclone products of comparable accuracy to the current version with statistical retrievals.

Mitigation: Work with MIRS development team to improve MiRS retrieval in cloudy regions

if necessaryStatus: Open


Summary and Conclusions

Presented by

Mark DemariaNOAA/NESDIS/STAR


Project Plan has been reviewedOperations concept has been reviewedProject requirements have been reviewedAlgorithm theoretical basis has been reviewedSoftware system architecture and interfaces have been

reviewedPlans for quality assurance have been reviewedOpen Risks and Actions have been reviewed

REVIEW OBJECTIVES HAVE BEEN ADDRESSED


NEXT STEPS

Code Development PhaseCode developmentTest data generationUnit testsPrepare documentationCritical Design Review (Dec, 2012)


OPEN DISCUSSION

The review is now open for free discussion


PRELIMINARY DESIGN REVIEW NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP) OCTOBER 16,...

Documents

Transcript of PRELIMINARY DESIGN REVIEW NPP MICROWAVE SOUNDER-BASED TROPICAL CYCLONE PRODUCTS (NTCP) OCTOBER 16,...