Observations, Database, and Pipeline for EoR

Jacqueline N. HewittMIT Kavli Institute for Astrophysics and

Space Research

Note: With the descope of LOFAR. MWA-512 is now the most sensitive pathfinder for power spectrum detection

Minimum Goal (NSF deliverable!) : subtract foregrounds and have noise-limited EoR limitGoal: detect the power spectrum at some redshift

From NSF proposal in 2004

Goal: detect the evolution of the power spectrum with redshift

McQuinn et al. (2008), ApJ, 680, 962

Power spectrum slope vs z

Power spectrum amplitude vs z

Goal: Detect quasar “bubbles”(Attempt is NSF deliverable)

Wyithe, Loeb, and Barnes (2006)

z=6.5

• Cross-correlation with galaxy surveys

• Non-gaussianity of galaxy and star formation

• Non-gaussianity of some inflation models

• Separating “gastro-” physics from “real” physics

• Energy injection via dark matter decay

Other Topics of Interest

Challenge for all Science Goals: Foreground Subtraction

Extensive simulations have been carried out that include• Realistic point source population - above and below confusion limit• Realistic Galactic emission• Realistic primary and synthesized beams (frequency dependence)

In these simulations, residual foreground is well below expected EoR signal

We have not yet included• Antenna calibration errors• Polarization

Bowman, Morales & Hewitt 2008; Liu, Tegmark & Zaldarriaga 2008; Geil et al. 2008; Jelic et al. 2008; Gleser et al. 2007; Wang et al. 2006; McQuinn et al. 2006; Santos et al. 2005; Di Matteo et al. 2004; Zaldarriaga et al. 2004; Morales & Hewitt 2004; Frulanetto & Briggs 2004; Gnedin & Shaver 2004; Oh & Mack 2003; Di Matteo et al. 2002

Purpose of This Session

DipolesFlakey maps

Sensitivity calculations Theory

Foreground Subtraction sims

MAPS & RTSsims of EoR field

Bridge the gap!!Get pipeline in place so we can hit the ground

running at 512T!!We will present software teams with Grand

Challenge simulation

David Barnes – data transport, bubble searchesFrank Briggs – general builder, data quality, foreground subtractionJudd Bowman – foreground subtraction, power spectrumChris Carilli - ?Angelica de Oliveira-Costa – foreground modelingSteve Furlanetto – theory, simulationsBryan Gaensler – foreground subtractionLincoln Greenhill – RTS, bubble searches, survey supportLars Hernquist – theoryJackie Hewitt – data archive, data quality, power spectrumAvi Loeb – theoryColin Lonsdale – general builderMiguel Morales – former RTS, M&C, foreground subtraction, power spectrumShiv Sethi – drift scan strategies, data qualityMax Tegmark – optimum mapping, foreground subtraction, power spectrumRachel Webster – data archive, bubble searchesStuart Wyithe – foreground subtraction, bubble searchesMatias Zaldarriaga - theory

Collaboration Members and Their Responsibilities

Organized intoSub-work Packages and Coordinators

Data Archive – Jackie HewittData Quality – Jackie HewittOptimum Mapping – Max TegmarkForeground Subtraction – Miguel MoralesPower Spectrum – Miguel MoralesBubble Searches – Stuart WyitheForeground Support – Lincoln GreenhillTheory and Simulation - …….

First half of EoR pipeline (work in progress)

How to divide time between full sky and EoR field?Is it right not to do the ionospheric correction?

Should confusion-level sources be subtracted in uv or image plane?Polarized foregrounds??????????Verification: what is the appropriate “practice” simulation and Grand ChallengeSimulation?

On-Site Storage

Monitor and Control

RTS Mapper

RTS Ionosphere

solution

Real-Time Computer

EoR Real-Time Module

On-line quality assessment, editing, and averaging

RTS Calibration

solution

Eor Real-Time

Module

Supported by MWA Construction Grant

• Commissioning - Q1, Q2 and Q3 2010

• Season 1 - Q4 2010 - Q2 2011

• Season 2 - Q4 2011 - Q2 2012

• Target first science results mid- to late-2012

• Proposed first public data release (“MWA1”) on 1 Jan 2013

EoR Data Acquisition

Assume construction of 512 tiles is complete by early 2010EoR observing “season” is 1 October - 1 JuneTwo target fields1200+500 = 1700 hours of observations of EoR fields per season

Supported by MWA construction grant

EoR Archives

• Repository of data at MIT Kavli Institute, with copy at Melbourne U.

During commissioning, lots of uv dataAfter commissioning, some uv data for algorithm tests2-minute integrations of RTS maps

• MetadataMonitor data baseEnvironmentAntenna calibration and ionosphere solutions, point source subtraction record

• Off-line quality assessment

• Later, processed data (foregrounds subtracted)

• Software for accessSupported by NSF MRI and $260k MIT contribution

11 September MWA Memo (Morgan & Hewitt) documents database plans

1.1 Maps: 2-minute integrations of I, Q, U, V MWAHealpix maps, FITS header.50,000 maps per year gives 400 TB

1.2 Metadata1.2.1 Monitor and Control data1.2.1.1 Commands and instrument response1.2.1.2 Environmental and system data1.2.1.3 RTS outputs

1.2.1.3.1 Calibrator refraction measurements1.2.1.3.2 Calibrator gain measurements and peeling record1.2.1.3.3 Calibrator Faraday rotation measurements (if done)1.2.1.3.4 Primary beam fit parameters1.2.1.3.5 Ionospheric rubber sheet fit parameters (if done)1.2.1.3.6 Ionospheric Faraday rotation fit parameters

1.3 Data quality: functional derivatives of level 0 data

1.4 Foreground subtracted maps

Problem: Calibration solution for one 8-second interval is 164 Mbytes(100 sources X 100 channels X 512 tiles X 4 polprods X 8 bytes) which1.8 Tbytes/day. Not practical to store this.

Possible problem: How much information do we have to store for peeled source list?

Technical decisions that need to be made:

How to divide observing time between full sky and EoR field?Do we do the ionospheric correction?How do we save the calibration and peeling information?

Others will surely arise during this discussion.

Collaboration issues:

We are developing parallel analyses that have the same goal. Is our publication policy adequate to handle this? (Policy is one must post paper two weeks before submission)

Do we need “protection” for student thesis projects?

EoR Committee membership no longer represents MWA partners (the partner list has changed).

Other issues?

We have only 39% of EoR membership here, unfortunately. Need for virtual meeting.