AWIPS II Evolution Overview

16 May 2008

Steve SchotzOffice of Science and Technology

2

Purpose

• AWIPS II Evolution Scope

• AWIPS II Evolution Schedule

• AWIPS II Evolution Improvements

• AWIPS II High Level Architecture

AWIPS Evolution Scope• AWIPS Evolution (FY2005 – FY2014)

– A long-term project which delivers a modern, robust software infrastructure that provides the foundation for future system level enhancements for the entire NWS enterprise

Phase 1: AWIPS II – Migration of WFO/RFC AWIPS (FY2007-FY2009)– Implements a modern Services Oriented Architecture (SOA) infrastructure– First output of AWIPS Evolution and provides the foundation for all subsequent

improvements

Phase II: AWIPS II Extended - Creation of a seamless weather enterprise spanning NWS operations (FY2009-FY2010)

– Migration of NAWIPS into the AWIPS II SOA– Delivery of thin client

• Support to the Weather Service Offices and Center Weather Support Units– Integration of “orphan” systems (e.g., Weather Event Simulator)– CHPS Integration into AWIPS II SOA

Phase III: AWIPS II enterprise enhancements (FY2009 – FY2014)– Data delivery enhancements

• “Smart push-smart pull” data access– Integrated visual collaboration– Visualization enhancements– Information generation enhancements

2007 2008 2009 2010 2011 2012 2013 2014

AWIPS Evolution

Roadmap

AWIPS IIExtended

= Fiscal Year

= Calendar Year

OTE / Deployment

AWIPS II

AWIPS II

Baseline Application Migration

Governance Model

NAWIPS Migration

Thin Client

WES Integration

AWIPS IIEnhancements

Smart push-smart pull data access

CHPS

IOC FOC

NWS Integrated Collaboration Phase 2 Phase 3

Streamlined Generation of Products IOC FOC

3-D Visualization IOC

AWIPS II – Open Source

Collaboration with Emergency Managers

Integrated support to FAA, Fire Weather and WSOs

AWIPS IIRoadmap

MigrationStrategy

2007 2008 201020092006

SW CTR (AWIPS II)

AE OSIP Gates 2 3 4a 4b

= Fiscal Year

= Calendar Year

MPLS Meshed Topology

12/05/07

OBx 7 8 9 10

PIP

Analysis

Migration Planning

RTS IRAD ADE Development

“User” Functional Tests

Baseline Application Migration

8 8.3

ADE Training

Field Ops Training-- ITO, ESA

DeploymentOTE

C & A

Deployment Planning

Local App Migration

ADE Local App Training

New Release Paradigm

O & M Transition

NWS New Capability Development in ADE

OB 9 Dev & Test Deployment

O & M Transition Prep & Coordination

ADE Training Development

OTE / Deployment Support

AWIPS II 1.0

Note: Task bar colors are For speaker reference only

6

AWIPS EvolutionNWS Enterprise Improvements

• More responsive to partner needs – reduce development time of new products by 50%

• Direct and integrated visual collaboration with all levels of NWS operations - National Centers, RFCs, WFOs, WSOs

• Streamlined generation of products in industry standard formats– CAP, GIS, etc.

• Expanded access to data for NWS and external partners– SBN enhancements, smart push-smart pull

• Improved and integrated incident support for Emergency Managers and DHS

• Better weather support for the FAA at CWSUs through enterprise level integration

7

AWIPS EvolutionInternal Improvements

• Free operations from technology constraints– Seamless, flexible and extensible weather enterprise that

integrates all levels of NWS operations– Integrated service delivery via the migration of AWIPS and

NAWIPS into a common Service Oriented Architecture• Improved continuity of operations

– Streamlined service back up – Faster software installations– More stable operations – fewer bugs

• Put new science into operations faster – more accurate warnings and forecasts – Enable collaborative development between local, national

and outside developers– Open source – closed community – open up development

to local apps developers and those outside traditional development community, e.g. NASA, academia

SOA Architecture – Logical Layered ViewLayers Separated By Simple APIsClient/Presentation Services

Platform Layer

Mission Services Layer

Data Access Layer

Metadata Index

Data PersistenceStore

En

terp

ris

e S

erv

ice

Bu

s -

Co

mm

un

ica

tio

n

Se

cu

rity

Se

rvic

es

/D

em

ilit

ari

zed

Zo

ne

(D

MZ

)

SpatialIndex

Hydro Models

LAPS

FORTRAN/C/C++Command Line

Programs

External Programs

JMX

<<Java>>

DataLayer

PostgreSQLHDF5

<<abstract>>

BaseDao

Hibernate

<<Java>>

HDF5DataStore

HDF5 API

IngestSrv

PersistSrv

IndexSrv

ProductSrv

AdapterSrv

NotifySrv

SubscribeSrv

AutoBldSrv

PurgeSrv

Mbean

CAVE

StagingSrv

UtilitySrv

LocalizationStore

AWIPS-II ADE High Level System Services SOA Services Running in an ESB Container

IngestSrvStagingSrv

PersistSrvPurgeSrv

IndexSrv ProductSrv

NotifySrv AutoBldSrv Collaboration

CAVEVisualization

Client

CORE SOA Services

Mbean Mbean Mbean Mbean

Mbean Mbean

JMX RemoteService Management

Client

METAR Plug InRadar Plug InSatellite Plug In

FileSystemRDBMS via JDBC

Meta Data IndexSpatial Data Base

uEngineManage Subscription

Subscription Notify

Data RenderingCAVE Bundles

CAVE ProceduresWx DrawingWx Warning

Data Interrogation

Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints

AdapterSrv

Mbean

Exec AdapterJNI Adapter

Radar All Tilts

ADE 1.0

uEngine

Services Independent of End Points

Enterprise Service Bus – HTTP, JMS, Virtual Memory, File Endpoints

HDF5 persistence

XMPP

Updated T06

Hydro Visualization

WAN Visible

GRIB Plug In

UtilitySrv

Mbean

Localization Data

GFE IFPServer

Color TablesMaps and Topo

10

Open Source Projects Used in ADE

OS Project Functionality ADE Role

ANT Build scripting Build system

Mule + Spring Enterprise Service Bus + Container Runtime Environment

ActiveMQ Java Messaging System Broker JMS Broker

Jibx XML to Object Serialization Canonical XML Message

GeoTools + JTS GIS capabilites GIS primitives

Tomcat Web Server Test Client Server

Baltik Scalable Vector Graphics Tools SVG tools

Ehcache Event Driven Clusterable Cache Cache Framework

Log4j Java Logging API Log manager

Jogl Java API to OpenGL CAVE rendering interface

Eclipse RCP GUI plug-in based framework CAVE framework

Eclipse IDE Java Integrated Development Development environment

MC4J Console JMX Management Console Remote management console

11

AWIPS MigrationRe-Architecture Approach

•Preserve existing AWIPS functionality •Minimize changes to user interfaces – “Black

Box” conversion– Minimizes user training of forecasters and

administrators

•Leverage use of AWIPS II services, functionality and development tools to optimize migration– AWIPS II uses open source projects - No proprietary

code– AWIPS II is platform independent

•AWIPS functionality incrementally to AWIPS II to reduce risk– Multiple deliveries planned using RTS Task Orders

12

• Dynamic load balancing– Failover handled automatically!!!– Takes full advantage of all available hardware– Scalable hardware (clustering)

• Significant performance improvements– Borrows techniques from video games -

mathematically intensive calculations handed off to the graphics card; image remapping on the fly

• Google Earth-like disclosure of imagery, grids and observations– Allows zooming in of satellite imagery with full resolution– Architecture is scale independent

• Integrated drawing and graphical collaboration– Tools built into the infrastructure, implemented in 2011

• Improved reliability – LESS CODE - Potential order of magnitude reduction in amount of software– Reduced code complexity

TM

AWIPS II InfrastructurePromising Attributes Thus Far

13

Summary

•New SOA architecture defined and infrastructure delivered– ADE 1.0 delivered June 2007– TO8 Partial D2D Functionality Feb 2008

•AWIPS I migration underway– AWIPS baseline migration to be completed FY09

•NAWIPS Migration and “smart push- smart pull” work to start in FY09, delivery in FY11

•Long term technology roadmap defined that supports many of the 4-D cube concepts

14

QUESTIONS???

15

Backup Slides

16

AWIPS EvolutionData Delivery

•OSIP Project 05-040•Enables “smart push - smart pull” data delivery

– Implements a discovery service within the SOA– Access to data not available locally– Freedom from the tyranny of the SBN– Enables consideration of new data delivery architecture

• What data to you broadcast over SBN?• What data do you make available on servers?

•Schedule– IOC - 2011 - software implementation for remote data

access– FOC 2012 - enterprise configuration (servers, comms,

etc.) that enables remote data access

17

AWIPS EvolutionCollaboration

•OSIP Project 05-041•Objective

– Integrated graphical collaboration throughout the NWS Weather Enterprise and beyond

– Phase 1 - Integrated collaboration between all levels of NWS operations

– Phase 2 - Collaboration between NWS offices and other NOAA entities

– Phase 3 -Collaboration between NWS offices and trusted external partners, e.g., Emergency Managers

•Schedule– Phase 1 IOC - 2011– Phase 2 IOC - 2012– Phase 2 IOC - 2013

18

AWIPS EvolutionInformation Generation & Visualization

•OSIP Projects 05-042 (IG) and 05-021 (Vis)• Information Generation objective

– Re-architect generation of all NWS products and services

– Separation of content generation from formatting and dissemination

– Enable faster response to emerging customer demands

•Visualization objective– Common user interface - standardize User Interfaces

across applications– 3-D visualization– Improve user interfaces based on latest principles and

research

19

AWIPS EvolutionGovernance Model

• What is it?– Governance model controls the development, test, integration,

configuration management, deployment and support of the new system -- both hardware and software

• Why?– AWIPS II offers new levels of flexibility and extensibility– New rules needed to exploit system capabilities, define limits– Tension between unlimited modifications and ability to support

the baseline system• Sample issues for consideration

– Monolithic configurations no longer required -- how do we manage site specific configurations?

– Plug ins can be down loaded and installed on demand– Scripting that modifies AWIPS menus, functions

• Flexible Governance Model enhances open source implementation– Open source – closed community model– Path to baseline defined for local and externally developed

applications– Definition of “sand box” for development and execution of local

applications without impacting operations

20

AWIPS IITesting

• Objective: Deliver a system that delivers today’s functionality with no impact on operations

• Layered testing strategy– Different testing phases overlap – functionality tested multiple times in

different settings– Significant field participation in testing

• ADE usage– Not formal testing, but expect to get feedback as developers use the ADE

• Algorithm Verification & Validation – – Assist algorithm choice– Verify port of algorithms

• Verification & Validation– Executed by dev orgs in controlled setting– Performance testing – Evaluate system against known and developing

benchmarks– Acceptance testing – part of the task order acceptance process

• Field Evaluation– Side by side testing of new system by field personnel in a “lab”

environment– Testing at limited number sites in manner that doesn’t impact operations

• Operational Test & Evaluation (OT&E)– Formal testing of the entire system– Tests system interfaces, operations, support, training, etc.