Online Data Monitoring Framework Based on Histogram Packaging in Network Distributed Data

Acquisition Systems

Tomoyuki Konno1, Anatael Cabrera2

Masaki Ishitsuka1, Masahiro Kuze1,

Yasunobu Sakamoto3

1. Tokyo Institute of Technology,

2. CNRS/IN2P3-APC Laboratory (Paris),

3. Tohoku Gakuin University

CHEP2010@ taipei, 10/20/2010

Network-distributed data acquisition system

• DAQ subsystems and monitoring information are distributed in local network

• Requirements for online monitoring

1. A lot of connections to DAQ sub-systems must be handled

2. Monitoring information must be available from outside of firewall

CHEP@Taipei, 2010/10/20 2

DA

Q s

ub

-sys

tem

s

Local area network

off site personsfirewall

on site shifter

Online monitor framework• General software framework for online data monitoring

– Collects information from network distributed systems

– All information are handled as collections of histograms

• ”HistogramPackage” : Common interface via network

– Displays to shifters far from experimental sites via internet

• DAQ sub-systems

– Create and update HistogramPackages

• Monitor Server

– Collects HistogramPackages from DAQ sub-systems

• Monitor Viewer

– Connect to Monitor Server and create graphical plots

– Two types with different technologies: Java, web browserCHEP@Taipei, 2010/10/20 3

HistogramPackage

• Common interface via network

– DAQ sub-systems to Monitor Server

– Monitor Server to Monitor Viewers

• Collections of histograms

– Package name (String) : used for identification

– Update serial ID (Integer) : used for checking updates

– Array of Monitored Objects

• Histograms (1-D and 2-D)

• Graphs (normal graph and timed graph)

• Updates all histograms at the same time

– Send updates of the histograms at once

CHEP@Taipei, 2010/10/20 4

CHEP@Taipei, 2010/10/20

Overview of Online Monitor framework

web serverDAQ sub-systems

Java version ofMonitor Viewer

TCP

Local area network

www

HTTPXML

Monitor Server

5

Available from everywhere via Internet

web version ofMonitor Viewer

Monitor Server

• Several DAQ sub-systems can communicate in parallel

• Monitor Viewer gets all HistogramPackages with one connectionCHEP@Taipei, 2010/10/20 6

MonitorSkelton

MonitorProvider

Monitor Server

shared memory

Monitor Viewers

DAQ sub-systems

thread

process

Data transition to Viewers

• DAQ sub-systems have triggers to start updating data

CHEP@Taipei, 2010/10/20 7

MonitorSkelton

MonitorProvider

Monitor Server

shared memory

Monitor Viewer

DAQ sub-systems

notify updatesvia condition

variables

wait for updates of the

packages

always wait for updates of the

packages

always wait for updates of the

package

Monitor Viewer• Available from everywhere via Internet

– Independent to platform (Windows, Mac OS, Linux)

– Free from additional plug-ins or libraries

• Technologies– Java : Swing with Java “web start” technologies

– web : Google web toolkit, Ajax, Canvas (HTML5)

• Create graphical plots on viewer sides– Dynamic control of plots (zooming , changing colors, …)

• GUI layouts are automatically generated by XML files– Panels, tables ,tabs, scroll bars etc..

– Specifics of objects (colors, width, font, draw options, …)

– Additional objects (legend, axis, text, …)

8CHEP@Taipei, 2010/10/20

Data stream to Viewer

CHEP@Taipei, 2010/10/20 9

XMLXML

XML

Monitor Server Java Viewer

TCP binary

Web browser Viewer

XMLXML

XML

web Server

XMLXML

XMLHTTP / Ajax

MonitorServer Proxy

High performance

Easyto access

Screen shots ofMonitor Viewer

10CHEP@Taipei, 2010/10/20

dummy data

dummy data Java version

Web browser versionon Google Chrome

GUI generation (Tabbed panels)

CHEP@Taipei, 2010/10/20 11

<package title=“Dummy"><tabbedPanel>

<tab title=“Main"><tabbedPanel>

<tab title=“Readout">…</tab><tab title="Disk">…</tab><tab title="Dead channels">…

</tab></tabbedPanel>

</tab><tab title=“Module">…

</tab></tabbedPanel>

</package>

GUI generation (Tables)

CHEP@Taipei, 2010/10/20 12

<package title=“Dummy"><tabbedPanel>

<tab title=“Main"><tabbedPanel>

<tab title=“Readout"><table col=“2” row=“1”>

<table col=“2” row=“2”><canvas …/><canvas …/><canvas …/><canvas …/>

</table><canvas …/>

</table></tab>…

</tab></tabbedPanel>

</package>

first table

second table

CHEP@Taipei, 2010/10/20 13

<canvas name="c_disk" title="Disk usage"><histo name="g_disk_available" maximum="600" minimum="0“ line-color="black" fill-color="light_gray” draw-option="AL" /><histo name="g_disk_used" line-color="red" fill-color="pink“ linked-axis="a_used" draw-option="AL" /><graph name="g_disk_full" draw="0, 500: 60, 500“ line-color="green" draw-option="AL" /><y2-axis name="a_used" direct="Y" pos="right" title="Data rate [GB]“ minimum="0" maximum="10" line- color="red" line-width="1“ font-color="red" label-font-color="red" />

<legend list="g_disk_available, g_disk_used" /></canvas>

dummy data

Additional line for reference

extra y-axis

Legend of objects

Draw Option : AL

Control of graphical objects

14CHEP@Taipei, 2010/10/20

dummy data

<canvas name="c_module_state"><histo name="histo_2d" draw-option="AL" maximum="4" minimum="0"

line-color="white" line-width=“4”/><color-axis color-pattern="blue,green,yellow,red"

label="OFF,STABLE,WARNING,ERROR;0.5,1.5,2.5,3.5” label-font-size="0.9" line-color="none" />

</canvas>

Control of graphical objects

Color axis

2-D histogram

Summary

• Developed general software framework for online monitoring

– Network-distributed systems create “HistogramPackage”s

– Monitor Server collects the packages

– Monitor Viewer makes graphical plots from the packages

• Two types of Monitor Viewer were developed

– Available from everywhere via Internet

– The GUI layouts are controlled by XML files

• Developed for the Double Chooz experiment

– Already being used on-site for the detector commissioning

CHEP@Taipei, 2010/10/20 15

Technologies for Monitor Viewer

• Java version– Swing

– Java web start

• web browser version– HTML technologies (Ajax + Canvas)

• Google Web Toolkit

– compiler from Java to HTML + java script

– GUI components

– GUI event handlers

• Ajax : Unsynchronized communication to web servers

• Canvas : Vector graphics on web pages

CHEP@Taipei, 2010/10/20 16