1 ASTRON is part of the Netherlands Organisation for Scientific Research (NWO)

Netherlands Institute for Radio Astronomy

LOFAR Operations and Schedule

Antonis Polatidis

2

Rollout Status

Can observe with 14 core, 6 remote and 3 international stationsUp to 34 (NL) + 2 (DE) HBA antenna fields

3

Towards integrated operations:1st Control and Metadata Busy Week

Test the flow of an observation starting with an astronomer defining a project, interaction with the Radio Observatory, data taking and the flow of processed data through the online and offline systems up to the data archive.

Simulated an astronomer using MoM to describe an observation (LEA128 as an example), interaction with RO support scientists and observers, and observations taking place under SAS/MAC (multiple observations).

FIRST RESULTS (15-23 Feb and beyond)

Many issues identified; some solved, others entered in CRAFT for further work.

Definition of observations through MoM and SAS functioning OK - Overhaul to add Beam-forming modes.

Interaction between scientist, observers, friend & scheduler going far more smoothly.

Now able to schedule HBA through MoM & SAS Data management on the agenda Archiving of LOFAR datasets again in progress after a long period

4

Towards integrated operations:2nd Control and Metadata Busy Week

Put more emphasis to the flow of processed data through the online and offline systems up to the data archive.

Simulate planning, scheduling and executing of multiple observations interfacing the LOFAR SCHEDULER to MoM and SAS.

Reduce the setup time between individual observations (from min to sec).

Test multiple beam functionality. Integrate SAS with the pipeline and process data all the way to the archive.

Develop data scrutiny protocols. Observations:

Many, short observations alternating in frequency and mode (LBA, HBA)

Long, multiple beam observation Parallel interferometric and beam formed observations Parallel interferometric and TBB dumps

5

6

MoM - Defining details of observation

7

Current Issues

Problems with Blue Gene rack allocation - Became obvious when recording data at high rates ( >30 antenna fields with HBAs)

A failed process causes the whole subrack to freeze and hence the observations can not recover.

Stations continue streaming data and overflow the network switches

Solution seeked: Hardware problem: Node replaced (8 March)

Operating System: IBM notified

Arrange network to avoid data overflows

Check if process hung and re-allocate the sub-rack switching off only the affected part.

Seems that Pulsar observations are not affected by this problem

Amplitude ripples Ripples in HBA XX polarization in core stations (when splitter is on)

8

Coordination of CEP offline resource usage

With increased number of stations, and large data rates require to write to large number of storage nodes simultaneously.

Reshuffling of data is highly inefficient.

Simultaneous use of storage nodes for observing and processing expected to severely degrade the performance of the systems (will test this further).

Current strategy of appointing subclusters to user groups is no longer sufficient

Solutions under consideration

Keep /data1 in each storage node free for observations - Process observations and move data out as soon as possible.

Stick with the “dedicated” subcluster allocation for usual work and Allocate additional resources to larger observations for limited time (ie spare subclusters, all /data1 partitions):

User groups should assign a “spokeperson” for faster communication.Assign to user groups permission to access the additional resources for a limited time (and clear up the additional disk space).

Assign a “friend” of resources (support scientist)

Will consult with the user groups and implement a policy soon.

9

Commissioning Activities March - April 2010

10

Proposal Review

Feedback from the Programm Committee sent Feb. 2nd

LEA proposals are incorporated in the general commissioning plan. This plan is FLEXIBLE so that it can accommodate the needs and requests of commissioners as well as the available resources of the Radio Observatory.

RO will contact proposers to arrange when they are available to schedule their observations and presence in Busy Weeks.

Reviews of Long Term Proposals will start reaching Pis by end of March. This will be the first step of an iteration that will lead to guidelines for writting observing proposals

Feedback with the Programme Committee: T. Van der Hulst

vdhulst_AT_astro.rug.nl

Discuss with the observatory regarding observing schedule: lofarscheduler_AT_astron.nl

11

Proposal Rating: CLASS

Class A: proposals judged to be those projects with high commissioning value or scientific impact and will be scheduled early in the term.

Class B: proposals judged to be those projects with significant commissioning value or scientific impact, but requiring either adjustment or system capabilities not available till later in the observing period. These projects will be scheduled on a best effort basis.

Class C: Projects that were not recommended for observations because the LOFAR Programme Committee found them of low commissioning value.

Where applicable, some of the aims of these projects can be achieved by getting access rights and processing resources for existing or previously scheduled data sets.

12

Proposal Rating: GROUPA Scheduling Indication

Group 1: Projects that can be scheduled in the next few months with the hardware and software available before MSSS

Group 2: Projects that can be scheduled concurrently or use data taken during MSSS

Group 3: Projects that require hardware and functionality that will be available after the completion of MSSS.

Group 4: Projects judged to require functionality that will be available well after MSSS is completed or that wish to test modes that will be under development later than the end of 2010.

These projects will not be scheduled in this round of commissioning.

13

14

LOFAR Schedule Viewer

The current schedule of LOFAR can be seen online:

“www.astron.nl/lofar-schedule/schedule/schedule.php”

LOFAR Management of Measurements

“http://lofar.astron.nl/mom3/”

15

16