The Evolution of AWIPS

Jason Tuell

Office of Science and Technology

National Weather Service

Overview

• What is AWIPS?

• Case For Change

• Where AWIPS is going?

• Roadmap for AWIP evolution

• Summary

What is AWIPS?

• AWIPS is the integrating element of the National Weather Service Modernization– Integrates data from modernized observations

systems, NEXRAD, ASOS, GOES into single processing and visualization environment

• Hardware, communications, software

AWIPS System Overview(Functional)

• Primary Field Systems (135 sites)– *122 Weather Forecast Offices (WFO)

• issue all routine forecasts and severe weather watches and warnings– **13 River Forecast Centers (RFC)

• Provide hydrologic and flood prediction and monitoring at the river basin level • National Centers and HQs (20 systems)

– *7 National Centers (NC)– **6 Regional Headquarters Offices (RHQ)– **2 HQ Operational Systems (NWSHQ)– 4 Training Center Systems (NTC)– 1 National Lab System (FSL)

• Development and Test Systems (14 systems)– 6 HQ Test and Development Systems (NWSHQ)– 3 Special Office Systems – *1 Network Control Facility System

• also have off site backup facility, BNCF– 4 Prime Contractor Test Systems

• 169 Total Systems* 24x7x365 operations** Occasional 24x7 operations during severe weather events.

AWIPS System Elements• WFO Configuration

– 3-6 workstation class computers per site.

– Standard servers, storage and peripherals at all sites

– Standard WFO software suite (Custom+COTS)

• RFC Configuration

– 6-10 workstation class computers per site

– More RAM and disk storage on standard servers/storage devices.

– Specialized “River Ensemble Processor” compute cluster at each site

– Specialized RFC software suite (Custom+COTS)

Major Features of AWIPS System Architecture

• Highly Distributed – Nearly identical hardware/software suite at all sites/systems

• Highly Redundant– Most data is distributed via Satellite Broadcast Network and stored

redundantly at all sites/systems

– Operational WFO’s provide site backup services to designated neighbors, in case of site failures.

– Most hardware w/in each site is redundant as well.

• Highly Available– 24x7x365 operations at all WFO’s, combined with mission requirements

for rapid distribution of watches and warnings, affecting life and property, drive system availability requirements. Critical system components require 99.9% availability.

• High Performance– During severe weather operations, minutes and seconds matter, thus end-

to-end system performance is paramount, especially during peak loads.

WFO/RFC Hardware Architecture (~July ’05)

DS1HP-UX10.2

Plaintree Switch

Ethernet 1000/100/10 Mbps SWITCH

LDADFirewall

LINUXPX1

LINUXXT

LINUXPX2

Serial MUX

LINUXDX1

LINUXDX2

Network Attached Storage (NAS)

LinuxCP

Router

WAN

LinuxAX

LDADSERVER

HP-UX 10.2

RFCREP

NexradRPG

< 1 yr old

1-2 yr old

2-3 yr old

3+ yr old

Legend

100Mbps FDDI

DS2HP-UX10.2

LINUXLX

AWIPS Communications• Satellite Broadcast Network (SBN)

– Provides broadcast and reliable multicast data transmission to field sites.• Transmitted data includes: Centrally collected radar data, GOES imagery,

NCEP model data, field observations, and watches and warnings– DVB-S

• Single channel solution.• Linearly scalable up to 43 Mbps, on demand

• AWIPS Terrestrial WAN– Dual homed, redundant, inter-meshed, hierarchal hub and spoke frame

relay network– Carries radar product data from all WFOs for central collection by the

NCF for dissemination over the SBN– Carries re-transmission frame/product requests to the Network Control

Facility (NCF) from WFOs for nonreceived SBN data– Carries forecast collaboration traffic between adjacent WFOs– Carries any other traffic deemed “operational.

Future changesAWIPS Communications

• AWIPS terrestrial WAN will be consolidated, along with all other NOAA line office WANs, into a single network solution– Consolidating bursty data into one network increases

overall network efficiency.– The new single network will be an MPLS IP VPN.

• Next generation replacement to obsolete frame relay.• VPN any-to-any architecture replaces inefficient hub-

spoke architecture.

AWIPS Software

• > 4.5 million SLOC• Mixed Languages –

– C/C++, FORTRAN, Python, Java, Perl, Tcl/TK, X and Motif• Mixed operating systems

– HP UX 10.20 and Red Hat 7.1+/7.2– Moving to all Linux environment built on Red Hat Enterprise 3.0

• Relational data base– Postgres/ Informix– Migrating off Informix

• Legacy architecture– Main architecture design dates from early to mid 90s

• 95% Government developed software– 5% prime contractor software

AWIPS Software• D2D – visualization environment• Warning Tools

– Warngen– Watch-Warning Advisory (WWA)– Riverpro

• Forecast preparation tools– GFE - Grid editing and formatters– AVNFPS – TAF preparation

• Decision Assistance Tools– SCAN – severe weather– FFMP – flash floods– SAFESEAS - maritime

• Infrastructure– Ingest, decode and storage– Informix– Communications routines

AWIPS Software Architecture

AWIPS MHS (message handling system)

LDAD

LAPS

IPC (socket)

AWIPS MC (remote monitoring)

CommsRouter

TextWorkstation

IPC

(sockets)

SiteMonitoring

Nexrad

TGAcq

serverGOES

Acqserver

RadarAcq

server

GRIB

TextDB

Text

Radar

Sat

METAR,

RAOB, +

IPC (socket)

MHSapps

SBNGOES W

SBN(TG)

DNS

NWWS

IFPS

Data Storage &Mgmt (netCDF,flat files. Informix)

D2D(Meteorological Analysis and Display)

EXTDISSEM

SBNGOES E

NCEP WANformatconvert

externalproviders

externalrecipients

extension

HIPS or other sat

Localization

NWSRFS WHFS

The Case for Change

• Checkpoint Analysis done Fall 2004 on AWIPS hardware, communications, software and data– AWIPS solid operational system, but ill poised

to meet future operational demands

• Architecture challenged to meet increasing data volumes, collaborative requirements and needs to accelerate the transition of research to operations

Present State: Summary Checkpoint Analysis

Hardware Compute Platforms Data Storage Devices LAN/WAN/SBN Interfaces Peripherals Architecture

? DocumentationData Inputs

Product Improvement Plans Requirements

Outputs Archives Architecture

Product Improvement PlansCommunications Infrastructure WAN SBN Architecture

Product Improvement Plans

Software Operating System Off The Shelf (Commercial and/or Open

Source) Baseline AWIPS

Performance Management and Control Requirements

Local Applications Performance Management and Control

Architecture Documentation Product Improvement Plans Cost effectiveness

Where is AWIPS going?

• New contract awarded August 2005 - Raytheon– Performance based, firm fixed price contract– Contract transition August - October 2005– Five base years with five options years

• Contract components– Core O&M

• Network Control Facility Operations• Satellite Broadcast Network Operations• Software Integration and Test

– Sustaining Engineering– Software maintenance option– Continuous Technology Refresh (CTR) option

• Hardware & communications improvement• Software re-architecture

Software Re-architecture

• AWIPS moving to Service Oriented Architecture (SOA)– Raytheon to implement J2EE Enterprise

Service Bus– Raytheon to deliver development environment

and software development kit to support software migration and development

• SOA will provide a more flexible and robust infrastructure for AWIPS

Approach to Migrating AWIPS to a New Operational Concept

AWIPS for

the 1990sAWIPS for

the year 2010

FY-08 PAC

AWIPS Evolution

Budget Initiatives

AWIPS Re-Compete

O&M Cost Savings

funding migration

to an SOA WFO Centric Architecture Little AWIPS/NAWIPS Integration High software Maintenance Costs Poor RTO efficiency Fire Hose Data Distribution

Supports New Ops Concept More flexible in Production/Delivery Increased AWIPS/NAWIPS Integration Improved RTO efficiency Increased access to data for decision making Reduced software O&M costs Flexible data delivery

A Services Oriented Architecture is necessary but not sufficient to

get us to a new Operational Concept and a more flexible AWIPS

AWIPS’ Needs

• Data delivery and information architecture– Introduce a more flexible data retrieval paradigm

• Visualization– Provide a consolidated foundation for visualization of

data and products within the AWIPS environment• Collaboration

– Provide an infrastructure for collaborative services with internal NWS, NOAA and external trusted partners

• Information Generation– Provide an infrastructure for the generation of NWS

products and services

Data Delivery/Information Architecture

• Move to “push-pull” data delivery paradigm– Expanding AWIPS beyond push capability (SBN) only– Exploring use of OpenDap as a technology to enable a

push-pull paradigm

• Business and performance cases to dictate final implementation– Delivering all the data still may be most cost effective

solution– Latency may make “pull” only approach impractical to

support the warning mission

Visualization

• Common visualization tools needed– “Incompatible” visualization environments

being used between applications– Lack of common look and feel

• Possible solution– “Plug in” architecture, based on Forecast

Systems Laboratory advanced prototyping– Potential benefits of reduced migration costs

with increased flexibility

CAVE Display Window

LSR DB

FFMP

SCAN

Data Ingest(OSIP #05-040)

Gridded, NetCDF,AWIPS DBs

CommonLook & Feel

GFE

SBN/WAN

SAFESEAS

Warngen

Hydroview

Plugin Applications

GFE DB

HydroDB

Send Grids

LSR DisplayWindow

ClimateDisplayWindow

LSR

Data Send,Communicate

Look & Feel

Climate

Look & Feel

Send Products

FX-C DB

Climate DB

WDSSII

RadarImagery

SatelliteImagery

ModelImagery

Visual Data Types

Other...

Depictables

CAVE DisplayServices

CommonGIS

3D Graphics(openGL)

Socket

FX-C

Warngen/WWADB

FFMP DB

SCAN DB

SAFESEAS DB

ATCF

Plugins

ADS

Socket

WFO/NCEPGraplical Object Exchange

XML GOER/W Plugin

Earth CentricMapping

SloshPainter

GOES-RComposites

IGC

NewServices

Modified Existing Service

Unchanged Existing Services

First Decomposition Context DiagramCAVE

Multi-ColorGraphic Overlay

Collaboration

• Goals– Provide infrastructure for real time graphical collaboration

between WFOs, RFCs and centers for enhanced internal collaboration

– Provide infrastructure for real time collaboration with other NOAA entities

– Provide infrastructure for collaboration with trusted partners, e.g., Emergency Managers

• Tools– Leverage current tools such as FX-Net (low bandwidth, “AWIPS

on a laptop” and FX-C)– Chat, Whiteboarding, remote briefing capabilities

Information Generation

• Needs– Standardize infrastructure for generation of NWS

products and services

– Enable more rapid adoption and integration of new dissemination technologies

• Outcomes– Reduction of number of unique product templates

– More responsiveness to customer driven changes in products

Information GenerationContent

GenerationMetadata Product

Generation

Forecasts

(GFE)

Warnings(GHG,

WARNGEN,

RiverPro)

Grids

GIS?

XML?

Web

NWR

NWWS

OtherOther

AWIPS Evolution Outcomes

• Increased integration of AWIPS and NAWIPS• Improved research to applications throughput• Increased access to all environmental data for

decision making• Reduction of software O&M costs and reduced

tech refresh costs• Increased flexibility to seamlessly transfer

operational functions and responsibilities between WFOs and National Centers for new concepts of operation

Challenges and Risks

• Migration of operational system to new architecture– Changing the wheel on the car at 60 mph

• Performance requirements– Defining and measuring performance requirements

against which to measure new architecture

• Training of management, developers to work in new environment

Summary

• AWIPS moving to a Services Oriented Architecture

• Under CTR option, Raytheon will deliver new architecture and infrastructure under the new contract

• Migration of current baseline and future baselines to be done FY10

• Enhancement of AWIPS capabilities within new architecture critical to meeting current and future mission needs