1/9

Development of a New Automatic Weather Observing System in support of Air Navigation at AEMET

Eduardo Monreal Agencia Estatal de Meteorología (AEMET)

C/Leonardo Prieto Castro 8, Ciudad Universitaria, 28003 Madrid, Spain emonrealf@aemet.es

Abstract As a provider of Meteorological Service in support of Air Navigation in more than 60 aerodromes all over Spain, the Spanish National Meteorological Service (henceforth AEMET) requires the use of tools like Automatic Weather Observing Systems which provide continuous and real time measurements of several meteorological parameters through a wide range of sensors. To achieve a greater responsiveness to new user demands and a higher capability of evolving to meet future changes in user requirements and in regulations (ICAO Annex 3 / WMO), AEMET has chosen to design and develop its own system which shall enable to tackle the full automation of observation and the increasing demand for greater reliability, consistency and quality of information. Designed as a modular system with loosely coupled components or subsystems and based on the adoption of standards and the use of open source software, the new system also leverages AEMET operational code. Its main features include: - Integration of all kind of sensors from any manufacturer and reuse of already installed sensors - Integration of information from other observation networks, especially remote sensing networks

(lightning, radar) operated by AEMET - Interoperability with other systems using standard and widely spread data formats - Full automation: auto METAR and TAF derived TREND - Integration of metadata - New algorithms: improved detection of present weather, computation of parameters

representative of conditions at the aerodrome, cloud cover, etc. - Centralized data stream management with diverse data delivery communications:

concentration of all data and metadata from different Automatic Weather Observing Systems - Optimization of procurement - Lower operating costs: replacement of components at the end-of-life - Remote monitoring: less in-situ maintenance - Integration of aeronautical and synoptic/climatological observation - Redundancy of processing elements and of communication links, possibility of sensors

duplication - Multiplatform access This paper describes the history of the project, the main elements of the design and the current status of its implementation. Introduction AEMET, the Spanish National Meteorological Service, is responsible for providing aeronautical meteorological information in more than 60 aerodromes all over Spain, among which are large airports, military airbases and heliports. At present, Automatic Weather Observing Systems providing continuous and real time measurements of various meteorological parameters are in operation in more than 50 of these facilities. Deployment of that systems, based in commercial and proprietary software, began in the late 80s and although its operation has been in general satisfactory, a number of drawbacks and the need to tackle the full automation of observation and

2/9

the increasing demand for greater reliability, consistency and quality of information has made AEMET to raise the design and development of its own system. This paper provides a brief description of the project, the background and objectives, outlines the new functionalities and main elements of the design and presents the progress in its implementation. Background Installation and renewal of Automatic Weather Observing Systems in aerodromes has been addressed at AEMET for each location independently, so that software is purchased in the same tender, along with sensors and infrastructure. This has led to a great heterogeneity of systems with very limited capability of evolution as any improvement AEMET requires ─like a new functionality or a new type of sensor─ usually involve a procurement procedure with the software manufacturer. In 2007 AEMET began to consider the development of its own system as the best solution to these drawbacks, which should also enable a greater responsiveness to new user demands and a higher capability of evolving the system to meet future changes in user requirements and in regulations (ICAO Annex 3 / WMO / Single European Sky). After a couple of previous initiatives, the project for the Development of a New Automatic Weather Observing System in support of Air Navigation, or NSIM for its acronym in Spanish, started in late 2012. A project team was set with experts in aeronautical meteorology, automatic observing systems, telecommunications and software engineering. The team was committed to achieve, through an in-house development, “a new system to cope with AEMET current and future needs as a provider of meteorological services in support of air navigation". Objectives After analyzing AEMET needs and requirements for the near future that NSIM should implement, the following main objectives were set for the project: - Full automation of observation: auto METAR and TAF derived TREND. - Integration of all kind of sensors from any manufacturer. - Integration of aeronautical and of synoptic/climatological observation. - Optimization of procurement: different procurement tenders for infrastructures, sensors,

hardware and software. Templates for ITT documents (specification of technical and legal requirements).

- Merging information from other observational networks operated by AEMET, especially remote sensing networks (lightning, radar).

- Inclusion of metadata. - Interoperability with other systems. - Lower operating costs: reuse of already installed sensors still in useful life, replacement of

elements at the end of life. - Continue and remote monitoring: reducing in-situ maintenance. - A simplified upgrading process, including remote software upgrading. - Centralized repository of data and metadata from all the Automatic Weather Observing

Systems running at different aerodromes. - User friendly GUI and multiplatform access. - Increased reliability through redundancy of processing elements and of communication links.

Use of diverse paths and of duplicated sensors when required. - Use of standard hardware and open source software. - Development of new algorithms: improved detection of present weather, computation of

parameters representative of conditions at the aerodrome, cloud cover, etc.

3/9

General overview As any other Automatic Weather Observing Systems in support of Air Navigation, NSIM consists of a series of sensors installed around the airfield, data acquisition units and computers for data processing and display, all linked through a telecommunications network (see fig. 1).

Figure 1. Scheme of an Evolved Automatic Weather Observing System at an aerodrome

In the design, AEMET deep experience in operating observing systems and networks has been taken into account when choosing a modular architecture with three loosely coupled components or subsystems, remote, distributed and central (see fig. 2), that make up the system. They are, respectively, in charge of data acquisition from sensors, management of all data sources, quality control and pre-processing; data processing, computation of new parameters, product generation, information display and product dissemination; and gathering and archive of information from all sites NSIM is operational. As the three subsystems are independent, it is possible to replace or to make changes in one of them without the others being affected and with no disruption at all. Communication between subsystems and its components is based on TCP/IP protocol stack, AMQP protocol, and JSON as the lightweight data and metadata interchange message format.

4/9

Figure 2. NSIM high level overview

The remote subsystem consists of several aeronautic weather stations (EMA for its acronym in Spanish), and software to perform the data acquisition and a first level of quality control. The EMA, generally installed in the airfield, is composed of datasources, being a datasource either a sensor or a data acquisition unit (data logger) and its associated sensors. An essential part of the remote subsystem are the drivers, specific programs for each datasource that encapsulate and isolate the particularities of accessing each datasource, allowing the integration of all kind of meteorological sensors available in market and avoiding the dependence on a particular data logger. The distributed subsystem, which runs on computers located at each NSIM installation, as aforementioned, is responsible of data processing and product generation. Through the drivers, the distributed subsystem collects data from the remote subsystem and then makes quality control, computes derived information and runs product generation. It provides real time access to users to all local information and also provides temporal (one year) data storage. It can operate either in fully automated or in semi-automated mode, in which the observer can supplement or modify the information that is to be incorporated in METAR/SPECI reports. The central subsystem, that runs on computers located at AEMET headquarters, acts as a central repository providing permanent storage. It gathers and archives information (data and products) from all distributed subsystems of the different sites where NSIM is operational for further dissemination to other systems and for user access to non real time information. It is also responsible of channeling to the different distributed subsystems information from remote sensing networks to be incorporated in product generation and shown in displays. A bit of history Due to the complex nature of the project —integration of weather observing systems along with software development— and the high uncertainty involved, a prototype oriented spiral development

5/9

life cycle was selected (see fig. 3). First of all, a very basic prototype of the EMA was to be developed as a test bed for alternative technologies and design options. In the next phase, a prototype of the whole system was to be build and install to demonstrate the viability of the chosen design. And finally, the whole NSIM was to be developed and deployed in a pilot facility where the different components could be integrated, tested and validated. Upon user feedback, a new cycle of software development shall start in order to refine the system before to proceed with the deployment in the rest of facilities. The choice of the location for the pilot installation was conducted through a thoroughly assessment of all aerodromes where AEMET provides meteorological services in support to air navigation; Armilla airbase near Granada was finally selected. After decision, in order to reduce potential risks, implications thereof were also evaluated.

Figure 3. NSIM development life cycle

The first phase (or first interaction of the spiral cycle) lasted until the end of 2013 and was mainly devoted to gathering requirements and to system design: - The different types of users of NSIM, with their information needs and preferred modes of

access, were identified and categorized. - A comprehensive elicitation of requirements was made by thoroughly reviewing the regulations

(ICAO Annex 3 / WMO / Single European Sky) and agreements with aeronautical users, both civilian and from the Army; interviews with relevant internal users were also carried out.

- The functional requirements for user interaction with the system were compiled in use cases and paper prototypes (or mockups) were also made to help in the design of the user graphic interface. Figure 4 is an example of mockup for a display showing an overview of the area surrounding an aerodrome merged with information from remote sensing networks (radar and lightning data).

- A test bed for meteorological equipment was set and alternate technologies for data acquisition were assessed; as a result it was decided to build the EMA using data loggers.

6/9

Figure 4. Example of NSIM user interface mockup: Overview of the area surrounding an aerodrome showing visibility references and radar and lightning information

The second phase, until the end of 2014, was dedicated to complete system design and to the functional specification of components: - The functional, operational and metrological requirements to be met by all kind of sensors used

in NSIM were specified, as well as the protocols and formats for data transmission between system components.

- The functional specifications for data logger programs and drivers, the software components of the remote subsystem, were defined afterwards. Basic prototypes for the EMA, based on that specification, were implemented and tested.

- All the meteorological parameters involved: raw data from sensors and derived information, and products (including alarms linked to thresholds overcoming or to the operational status of system components) were comprehensively identified and documented.

- In the same way, work started to develop and to specify, in a very detailed fashion, all the underlying algorithms to obtain each variable or to generate all the products, including the fully automation of METAR/SPECI and TAF derived TREND. In this work we have followed all WMO/ICAO existing rules and recommendations about the use of instruments and observing systems at aerodromes.

- Finally, the project for the pilot installation was drafted including tender documents for two different ITT: the first was for all the infrastructures, including network equipment and cabling, the second one was to supply and to install all meteorological equipment (sensors).

7/9

During the third phase, where NSIM project is running now, the algorithms specification has been finished, a complete prototype of EMA ─comprising all the sensors usually installed in a location on the airfield─ has been installed in field, and substantial progress has been made towards the final target of system implementation, and its deployment and testing in pilot installation. When writing this document, the status is as follows: - Pilot installation in Armilla airbase: in 2015 the two ITT were issued and contracts awarded to

different contractors. Works on the infrastructure are finished and accepted on July 2016, after verification. Computing equipment, two servers for redundancy, were installed and configured on June and all the meteorological equipment (sensors) and data loggers are to be installed during August.

- It is expected to deploy, configure and integrate all software components during September and to perform system validation during last quarter of 2016.

Design main features As above mentioned, NSIM was thought as a modular system with three loosely coupled components or subsystems: remote, distributed and central, which should also leverage AEMET operational code as much as possible. Communication between NSIM components or subsystems is based on: • TCP/IP protocol stack • AMQP protocol, an open standard application level protocol for message-oriented middleware.

The defining features of AMQP are message orientation, queuing, routing, reliability and security. AMQP brings NSIM with point-to-point, publish-and-subscribe communication design patterns

• JSON, the lightweight data and metadata interchange message format NSIM can be defined as a Big Data System. Not only NSIM uses data sets that are so large or complex that traditional data processing techniques are inadequate, but also NSIM analyzes, captures, computes and visualizes continuous streams of data originated in NSIM remote subsystem (see fig. 5). Data ingestion, real time analytics and computation of derived parameters are implemented as data microservice applications called modules that operates on streams or flows of data. These data microservice applications can be combined independently to orchestrate multiple computing patterns on data flow. Generation of products is implemented as tasks, which are batch microservice applications developed independently that can be combined to form processing chains. The definition of flows, modules and tasks is done remotely through command line by an administrator which gives great flexibility. An OOP (Object Oriented Programming) and AOP (Aspect Oriented Programming) approach is intensively used in NSIM code; Interface-Driven design has been selected to reduce coupling. As shown in figure 6, objects and classes are structured in:

a) the Domain Object Model which provides the domain model of NSIM. The domain model is

a representation of meaningful real-world concepts pertinent to the domain that need to be modeled in NSIM software. Concepts include the data involved in NSIM and the rules of use in relation to that data,

b) the Data Access Layer which provides simplified access to data stored in persistent storage of some kind, such as an entity-relational database or a NoSQL solution,

c) the Service Layer, where business logic is encapsulated. Provides a unified access interface to the application functionality. The Service Layer is the middle layer between presentation and data storage. The idea behind such a layer is to have an architecture that can support multiple presentation layers such as web, desktop applications, mobile, REST interfaces, etc.

8/9

Figure 5. NSIM as a Big Data Application

Figure 6. Layer structure in NSIM

From data persistence point of view, NSIM requires of schemaless databases because of the heterogeneous and time variable nature of data stored. Thus the use of relational databases was discarded because of its lack of flexibility. Instead, an open source, non-SQL, schemaless and document-oriented database, in which data structures are similar to JSON messages, is used. To ensure high availability a configuration with a distributed database and automatic data replication has been setup. User interaction with NSIM is done through different interfaces that allow access to the same Service Layer logic. There are REST interfaces, and Responsive Web Design applications that can be displayed on a wide variety of devices: from PC to tablets or smartphones.

9/9

Future work The last iteration or phase of the project should start just after validation of pilot installation. Tasks planned for that stage include: - Analysis of users experience with pilot installation. A new cycle of software development for the

application and user interface may start to include new requirements. - Preparation of the plan for the progressive deployment of NSIM in all aerodromes where

AEMET provides met services in support of air navigation. This plan should take into account priorities derived from the operational needs of AEMET, the status of current systems, the difficulties to be addressed in each location and the estimated costs thereof; and should include the corresponding training plan.

Acknowledgement The author would like to thank all the other members of NSIM project team: Guillermo Ballester, Javier Casado, Francisco Escribá, Guillermo García, Marcelino González, Juan Iglesias, Fortunato Márquez, Jesús Montero, Eduardo Morales, Juan Ignacio Pérez, Ana Portillo, Irene Sanz and Carlos Torres, for their contribution to the project development. Bibliography WMO (1990). Guide to Practices for Meteorological Offices Serving Aviation (OMM-No.732). World Meteorological Organization (Geneva). WMO (2006). Guide to Meteorological Observing and Information Distribution Systems for Aviation Weather Services (OMM-No.731). 2nd Edition. World Meteorological Organization ( Geneva). WMO (2014). Guide to Meteorological Instruments and Methods of Observation (OMM-No.8). 2008 edition, updated in 2010. World Meteorological Organization (Geneva). WMO (2011). Technical Regulations - Fundamental Documents No.2 – Volumen II (OMM-No.49). 2010 Edition. World Meteorological Organization (Geneva). ICAO (2003). Aeronautical Information Services Manual (Doc 8126). 6th Edition. International Civil Aviation Organization (Montreal). ICAO (2005a). Manual on Low-level Wind Shear (Doc 9817). International Civil Aviation Organization (Montreal). ICAO (2005b). Manual of Runway Visual Range Observing and Reporting Practices (Doc 9328). 3rd Edition. International Civil Aviation Organization (Montreal). ICAO (2006a). Aerodrome Design – Part 6 – Frangibilty. 1st Edition. International Civil Aviation Organization (Montreal). OACI (2006b). Manual of Aeronautical Meteorological Practice (Doc 8896). 7th Edition. International Civil Aviation Organization (Montreal). ICAO (2011). Manual On Automatic Meteorological Observing Systems Of Aerodromes (Doc 9837). 2nd Edition. International Civil Aviation Organization (Montreal). ICAO (2013). Meteorological Service for International Air Navigation (Anex 3). 18th Edition. International Civil Aviation Organization (Montreal).