Available online at www.sciencedirect.com

s 69 (2008) 146–153www.elsevier.com/locate/jmarsys

Journal of Marine System

Norwegian METOC distribution system: Sophisticated integrationthrough simple mechanisms

Jan Kristian Jensen a,⁎, Kjell Røang c, Reinoud Bokhorst b,Richard Olsen a, Morten Torsås a

a Norwegian Defense Research Establishment (FFI), P.O. Box 115, N-3191, Horten, Norwayb Norwegian Meteorological Institute, P.O. Box 43 Blindern, N-0313 Oslo, Norway

c Christian Michelsen Research, P.O. Box 6031, N-5892 Bergen, Norway

Received 1 July 2005; accepted 20 December 2006Available online 27 February 2007

Abstract

An exceedingly versatile and flexible system for meteorological and oceanographic data distribution and processing has beenbuilt for the Norwegian Navy, with special emphasis on how operational forecasting products may be distributed effectively tomarine users. This distribution system meets an extremely wide range of user requirements, from visualization in web browser tosophisticated integration with weapons and sensors systems. Important factors for this versatility are the use of open standards andformats and utilizing the robust data retrieval mechanisms of the web (http) protocol.© 2007 Elsevier B.V. All rights reserved.

Keywords: Weather forecasting; Web Mapping Services; Prediction; Meteorology; Oceanography; Remote sensing

1. Introduction

There is an ever increasing demand for environmentaldata, in particular real-time services with meteorologicaland oceanographic information. New applications andservices that require more sophisticated environmentaldata are being developed: Search and rescue drift modelsand Decision Support Systems, ship routing and generalsafety regulations. For military users, increasinglysophisticated Decision Support Systems, informationsystems, command and control (C2iS) systems andweapon and sensor systems require the development ofimproved services for meteorological and oceanographic(METOC) data. For marine users – or mobile users in

⁎ Corresponding author.E-mail address: jje@ffi.no (J.K. Jensen).

0924-7963/$ - see front matter © 2007 Elsevier B.V. All rights reserved.doi:10.1016/j.jmarsys.2007.02.019

general – the major challenge lies in developing aninfrastructure that is capable of distributing near real timedata over limited bandwidth and connectivity. This paperpresents the latest developments from the NorwegianDefence Research Establishment (FFI), the NorwegianMeteorological Institute and Christian Michelsen Re-search (CMR). In our experience, a flexible and effectivedistribution system for meteorological and oceanographicdata should be built emphasizing flexibility and simplic-ity, while using open standards and formats. The http-protocol is independent of the peculiarities of file system,network connections and operating systems. It is easy toimplement: a web client can bewritten in less than 30 linesof code in any modern programming language. Further-more, the http protocol is well suited for situations withlow bandwidth and broken connectivity often experi-enced by marine users. Following this philosophy, we

147J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

have had considerable success developing web-basedservices for mobile users on low to moderate throughputnetwork connections. These services were originallydesigned to meet requirements of the Royal NorwegianNavy, but are now operational services beneficial to thewhole Norwegian community. The services have beendeveloped along two lines that may ultimately becombined to one service in the future.

• The first line is a fairly traditional approach to aproduction web server, serving a pre-defined set oftextual (html) and graphical products. This service hasbeen augmented to make selection and download ofMETOC products as efficient as possible. This in-volves simple page designs, reusable definitions ofdownload packages and support for partial downloadsof files due to interrupted downloads or frequent up-dates to some files. This service is referred to asKildenbelow, the Norwegian word for “the source”.

• The second line is based on Open GIS ConsortiumWeb Mapping Services, for the purpose of interactivedefinition of data sets and end user products. Below,we refer to this as the METOC WEB/WMS server.

The following two sections of this paper give adescription of these two services, with special emphasison the benefits to maritime users.

2. The product web server Kilden

Awide range of textual and graphical meteorologicaland oceanographic products are available through theweb-server Kilden, which is a service run by theNorwegian Meteorological Institute for their publiccustomers. Users now include Norwegian ArmedForces, Airport Authorities, Rescue HQ and severalothers. Each of these institutions has a unique account atKilden, the contents of which are tailored for eachcustomer's needs. Although quite sophisticated func-tionality has been added to the online web GUI (Table 1),

Table 1Features and functionality of the Kilden web interface

Features of Kilden web interface

Download functionality (see next section)Automatic refresh of menu and productsVisual indication of outdated products (product is older than specified

expiration time)Update alert: user may set an alarm when selected products are

updated. Alarm is pop-up window or visual indication in the menuSearch functionality for aviation textual products (TAF, METAR, IGA

code)

Kilden is not principally different from production webservers that have existed for years, like the services ofNOGAPS FNMOC (FNMOC, 2005) or GermanWeather Service GWD (GWD, 2005).

Originally developed for the Royal Norwegian Navy,a lot of effort has gone into making the web interface aslittle bandwidth-demanding as possible. Viewing graph-ical products online require some minimum bandwidthto be of any practical use — but browsing the Kildenmenu and viewing textual products is still feasible atInternet connection speeds of less than 10 kb/s.

2.1. Offline availability of Kilden

For marine users, problems with connectivity as wellas limited network bandwidth are to be expected. Thismeans that on-line services like Kilden will often not beavailable or usable. It is therefore vital to have an easy-to-use and efficient method to make the serviceavailable off-line. Kilden addresses this problem byproviding a sophisticated functionality known as‘download packages’. As we will show in the followingsections, download packages make it possible tomaintain an offline version of Kilden on a local filesystem in a robust and bandwidth saving manner.

2.1.1. Data download using packagesA download package is an arbitrary subset of

products packed into a single archive file. The packagecreation and download is a two-step process, initiatedon-demand by the user through the download form(Fig. 1). First the user selects the products that he wishesto include. An html index, that reflects the same menustructure as the online menu, is generated for thisparticular subset and will be included as well. Secondly,an archive file containing the selected products iscreated dynamically and made available for downloadby a simple URL. The archive file is compressed usingzip or tar.gz to save costly bandwidth. After down-loading and uncompressing the archive file to the localfile system, the html index can be used in a web browseras a simple interface to the downloaded products.

Often it is wanted to download the same subset ofproducts more than once to keep the offline product filesup-to-date. To that end a package definition (i.e. theproduct list and some other options that the user selectedin the first step) may be saved for later use in a databaseon the Kilden server. However, the actual archive filethat constitutes the Kilden package is dynamicallyrecreated at download time which guarantees that italways contains the latest product updates. Any usermay save any number of package definitions, packages

Fig. 1. Web GUI of Kilden download center.

148 J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

will be linked to the user account atKilden. Some packagedefinitions may be password protected against unautho-rized editing— but any user may download any package.

2.1.2. Automatic download of data from KildenSaving a package definition makes it available for

automatic download utilizing a client-side script. Automat-ic download of a particular Kilden-package is a matter ofcomposing and sending correctly formatted http-requests.First the client sends an initial request that contains 5parameters. The most important parameters are the ID ofthe package definition and the if-Modified-Since parameter(see next section). The remaining parameters control theformat of the package (zip or tar.gz) and namingconventions for the final package. The client must alsosupply valid user credentials (basic http authentication).

The response of the Kilden web-server is to generatethe archive file that contains the products and html indexaccording to the package definition and the requestedparameters, and return a link (URL) to this file. Theclient will use this link (which is valid for a limited time)for the actual data download. After download, the data

should be automatically uncompressed to the local filesystem. For this purpose, a wide range of commercial orfreeware software is available.

2.1.3. The if-modified-since mechanismApart from using compression, the size of the archive

files may be further decreased by limiting the includedproducts based on their modification date. This is useful forrepeated downloads of the same products where one onlyneeds to retrieve the products that were updated since thelast download. This is known as the ‘if-modified-since’mechanism.

The http protocol supports this mechanism via its ‘If-Modified-Since’ header. But the contents of each Kildenpackage are created dynamically at download time.Therefore, the mechanism of the http-protocol could notbe used directly. Instead, the modification time is includedas the required parameter ifModifiedSince (=year, month,date, hour, minute) of the URL syntax. If this parameter isset to zero, all the contents of the package are downloaded.This mechanism allows for frequent checks for new datawith optimum use of bandwidth. If the html index of a

149J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

particular package is newer than the ifModifiedSinceparameter, the html index will also be included in thedownload. This implies that the package definition may bechanged without corrupting existing automatic downloadroutines.

2.1.4. Partial download of dataFor maritime users, problems with connectivity as

well as limited bandwidth are to be expected. It istherefore vital to support partial download of data,meaning that if (or rather, when) the data connection isbroken during a download process the download can beresumed after re-establishing the connection. The actualarchive file that constitutes the Kilden package iscreated dynamically at download time, and is thereforea “snapshot” of the product state at the time of downloadrequest. Some products of the Kilden web-server areupdated continuously, others at regular or irregularintervals. It is therefore to be expected that the contentsof a particular Kilden package are updated during thedownload process. The two-step download processdescribed in Section 2.1.2 – where a data downloadlink may be re-used any number of times for a limitedtime – is a simple and robust way of supporting partialdownload of continuously updated data.

Fig. 2. METOCWEB/WMSweb GUI for visualizing 2D and 3D meteorologilayer (approximately 1400 m a.s.l.) over Scandinavia for the 18th of June, 2

2.2. Installation and maintenance, Kilden productserver

The Kilden product server consist of a web server,an administration database and a large collection ofproduct files. The system runs on Debian Linux and isbuilt using reliable and well proven open sourcecomponents like Apache, mod_perl (CGI using Perl)and MySQL. These components are available for allmajor platforms and use a platform-independentsyntax. Installation should therefore be relativelystraightforward on any platform. Kilden has beenrunning at the Norwegian Meteorological Institutewithout any major hickups or downperiods since itslaunch in July 2003.

Maintenance tasks include updating the administra-tion database and product collection. The databasecontains information about user accounts and metadataabout the available products. It is maintained manuallythrough a web interface. The product collection isupdated automatically by the central data distributionsystem of the Meteorological Institute. To date, Kildencontains approximately 1000 different products, mainlytext files (including html) and image files, that areupdated continuously.

cal and oceanographic data. This screenshot shows wind in the 850 hPa005. Note the tooltip information at cursor position over mid-Sweden.

150 J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

2.3. Future development, Kilden

The Kilden product server will continue to evolvebased on changes in user requirements. For the mainpart, user feedback and requirements are about im-proved products and minor enhancements of function-ality and graphical layout. However, some requirementspoint toward a map-based, interactive interface, withfunctionality resembling that of the METOC WEB/WMS server described in the next section.

3. Sophisticated data manipulation, distribution andvisualization: the METOC WEB/WMS server

In the following sections we will describe thecapabilities of the METOC WEB/WMS server formanipulation, distribution and visualization of meteo-rological and oceanographic (METOC) data:

Data manipulation The server can extract and visual-ize data for arbitrary points, lines,areas, vertical profiles and verticalcross sections (Section 3.2).

Data distribution The METOCWEB/WMS server isan important infrastructure compo-

Fig. 3. METOC WEB/WMS web GUI for selecting parameters and sub-sa

nent (Jensen et al., 2004) for theNorwegian Armed Forces throughthe GRIB export functionality(Section 3.3) and the data integra-tion functionality (Section 3.4)

Visualization The server offers visualizationthrough the WMS protocol (Sec-tion 3.1) or through the web GUI(Section 3.2).

An important aspect for both for maintenance andoperational use is that the METOC WEB/WMS servereasily can be installed locally, i.e. on board a vessel.Using the data distribution mechanisms (Section 3.3),the local server can download data whenever it isconnected to the central server. Users on board thevessel will access the local METOC WEB/WMSserver, which will always have the freshest possibledata that the network infrastructure allows. A serverinstalled locally also ensures reliable services for localapplications that need METOC input data (Section3.4).

Presently, the METOC WEB/WMS server supportsgridded 2D and 3D data sets in WMO GRIB (WMO,2005) format and observations from surface (SYNOP)

mpling in space or time for data distribution in WMO GRIB format.

151J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

as well as wind and wave data from several remotesensing sources.

3.1. Visualization using the Web Mapping Services(WMS) protocol

Along with the other standards from the OpenGeospatial Consortium,1 the Web Mapping Services(WMS) protocol (OGC, 2004) provides a “plug-and-play” concept for sharing geospatial data via the httpprotocol. The services of a WMS server is viewed aspartly transparent “layers” that can be added to existingmap layers in a WMS client. Sophisticated GISfunctionality can be built using the three basic requestsof the WMS.

3.2. Sophisticated web GUI

Most users access the METOC WEB/WMS serverthrough the web GUI (Fig. 2). As with many otherWMS servers with a web GUI front end, the GUI isimplemented as a WMS client that can combine maplayers from several WMS servers. In our case, we needto show a background map as well as the METOC layer.The most important feature of this GUI are:

• Choose which data set and background map todisplay.

• Basic map control (zoom, pan etc.).• Temporal control.• Print screen• Tooltip information (data values at cursor position)• Meteogram, time series plot• Elevation control (for 3D data)• Plot of vertical profile or cross section (for 3D data)• Export of vertical profile, cross section to lybin (for3D oceanographic data, see Section 3.4.1)

3.3. Flexible data distribution: GRIB export

The idea behind GRIB export is to have an onlineservice that distributes arbitrary subsets of its griddeddata sets in WMO GRIB format (WMO, 2005). Newsubsets may be defined any time through the webinterface. The user defines a bounding box rectanglewhere he wants the data and selects the appropriate dataset, then selects which parameters that should beincluded (Fig. 3). The data may also be sub-sampledin space or time. The response is an URL which is usedto download the actual data. This link will also be used

1 http://www.opengeospatial.org/.

to check for new data using the generic http ifModi-fiedSince mechanism. Partial download (as discussed inSection 2.1.4) is also supported.

It is also possible to automatically generate newsubsets without using the manual web GUI. The syntaxof the GRIB export URL is quite straightforward togenerate in an automated process. This procedure doesrequire some knowledge of which data sets that exist onthe METOC WEB/WMS server — which is easilyqueried using the getCapabilities request (OGC, 2004).

3.4. Sophisticated integration using the web protocol

There are an increasing number of applications thatneed some sort of environmental data. However,implementing generic support for ingesting 2D or 3Doceanographic and meteorological (METOC) data intothese applications is time-consuming and expensive.Our approach is to use http based data extractionrequests to provide a programming interface that is easyto implement and use. The problem of ingesting therequired METOC data is then reduced to correctlyformatting and sending http requests and parsing thereceived XML file.

The basic premise for the METOC WEB/WMS dataextraction is the ability to interpolate data for arbitrarypoints, lines and planes based on gridded 2D or 3D datasets. As an example, a correctly formatted getPoint httprequest will returns an XML file with data for thespecified point in space and time. For 3-dimensionaldata, the requests getZProfile and getZCrossSectionextract vertical profiles and cross sections. Similarrequests also exist for time series and horizontal areas.Gridded data for horizontal areas may be re-sampled tomost standard projections. Further, all data exportrequests have a corresponding plotting routine (e.g.plotTimeSeries corresponds to getTimeSeries).

There is no standard or protocol for this kind of datacommunication at present. Our strategy for providing a“standardized” programming interface is to use syntaxelements from the OGC WMS and the Web CoverageService (OGC, 2003) protocols when defining thesyntax of data export URLs and XML return objects.Before composing the data export request the OCGWMS GetCapabilities request should be used to querywhat data exists on the METOC WEB/WMS server.

3.4.1. Customized integration to specific applicationsThe generic data export mechanism described above

may be further “customized” to specific applications byadding “filters” that convert to application-specificformats. This may also hide some of the complexity

152 J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

involved in using the generic data extraction requests.Adding new application specific http requests to theMETOC WEB/WMS server is usually done within anhour or two of programming. As an example, we haveadded the request getLybin-Profile (Fig. 4) for use withthe Norwegian Navy's sonar performance tool Lybin.This request has only one required parameter (lat, loncoordinate) as well as several optional parameters. Theresponse is to return a profile with temperature andsalinity from a 3D ocean data set at a format than can beread directly by Lybin. A similar request extracts avertical cross section and surface data (wind, waves)along a client-specified line for use with range-dependent version of Lybin. Since January 2005, thelybin-specific requests have been integrated with thesonar exercise evaluation tool SIMSON, which com-pares collected real-life sonar data with LYBINsimulations.

3.5. Installation and maintenance, METOC WEB/WMSserver

METOC WEB/WMS is written in Java as a Javaservlet, intended to be part of a web server that supports

Fig. 4. A simple and robust programming interface is provided between Mthrough defining plain http-request such as getLybinProfile.

Java servlet containers. With the exception of GRIBexport (which are pre-compiled C routines) and userauthentication (which use features of the tomcat webserver), all components of the METOC WEB/WMSserver is independent of the type of underlying webserver or operating system.

The background maps used in METOC WEB/WMSserver comes from a different WMS server, which maybe installed locally on the system or on the net.Typically, a marine user would have the backgroundWMS map server installed locally. A wide range ofsoftware exist for this purpose.

The maintenance consists mainly of keeping the datafiles (GRIB files) of the system up-to-date. If the datacomes from a remote web server – such as GRIB export(Section 3.3) of a remote installation of METOC WEB/WMS server – the local server has an integrated httpclient with a web interface that can be used to automate(or manually control) this task.

In our experience, the installation of aMETOCWEB/WMS server locally is a relatively straightforward taskthat still should be undertaken by a system administratoror relatively experienced user. However, the mainte-nance of the system requires very little computer skills.

ETOC WEB/WMS server and sonar prediction tool lybin is provided

153J.K. Jensen et al. / Journal of Marine Systems 69 (2008) 146–153

3.6. Further development, METOC WEB/WMS server

Due to the rapid evolvement OGC–WMS basedservices and web-based services in general, there are amultitude of possible development strategies for theMETOC WEB/WMS server. Here we summarize whatstrategies that are being pursued:

• Enhancements to the web GUI (layout, functionality)• Extend the getCapabilities to document the special-ized data export functions (Section 3.3, 3.4) usingXML technology.

• Streamlined installation package customized forlocal installation at PC at naval vessel (Section 3.5),

• Improve GRIB export (Section 3.3):· Support GRIB files with polar stereographic,rotated lat/lon grids

· GUI enhancements• Support oceanographic observation data (CTD,XBT)

• Distribution of observational data (weather observa-tions, oceanographic observations) with functionalitysimilar to GRIB export (Section 3.3).

• Utilizing METOC WEB/WMS server as visualiza-tion platform for other maritime information· Search and rescue areas and resources· Marine Automatic Identification System (AIS)· Simulations of drifting objects and particles (personin water (PIW), oil drift, drifting vessels, mines etc.).

The question of merging the Kilden and METOCWEB/WMS servers at some future time has been raisedseveral times. A more likely outcome is to keep theservers as separate units, but develop a new web GUIthat present one integrated service. Whether this willhappen depends on the demands of the user community,but for the moment this is not an immediate requirement.

4. Conclusions

Experience gained in a close working relationshipbetween researchers, operational forecasters and Navypersonnel shows that timely end efficient provision ofMETOC data can be useful operationally in the field. Inthis paper we describe key features of two systemsdesigned to meet Royal Norwegian Navy requirementsfor an operational METOC service. To date, thesystems have been developed and tested on unclassi-fied networks, with some testing on classified systemsat the SECRET level. FFI has further recommendedthat the web services should be deployed on the Nor-wegian classified RESTRICTED and SECRET net-works (Jensen et al., 2004), and steps are being taken tofollow this up in the near future.

References

FNMOC, 2005. World wide web site for official information aboutfleet numerical meteorology and oceanography center. https://www.fnmoc.navy.mil/PUBLIC/.

GWD, 2005. Web site with operational meteorological forecasting prod-ucts run by the German weather service. http://www.wetterzentrale.de/topkarten.

Jensen, J.K., Olsen, R., Torsås, M., 2004. Recommended infrastructurefor operative METOC services in the Norwegian armed forces.Technical Report FFI-2004/00945. Norwegian Defence ResearchEstablishment. in Norwegian, NATO RESTRICTED.

OGC, 2003. Web coverage services. https://portal.opengeospatial.org/files/?artifact_id=3837. Open Geospatial Consortium Implemen-tation specification. Reference number OGC 03-065r6.

OGC, 2004. Web map service. http://portal.opengeospatial.org/files/?artifact_id=5316. Open Geospatial Consortium Publicly AvailableStandard. Reference number OGC 04-024.

WMO, 2005. World meteorological organization: a guide to the codeform fm92-ix ext. grib. http://www.wmo.ch/web/www/WMOCodes/Guides/GRIB/GRIB1-Contents.html.