Post on 01-Apr-2015
July 2006 Instrument Scientist Retreat, Santiago
Quality Control: closing the loopReinhard Hanuschik, DFO/QC
1. Principles
2. Processes
3. Critical components
4. Conclusions
July 2006 Instrument Scientist Retreat, Santiago
• purpose of calibration:– calibrate the science observations
= remove (atm+tel)*(ins+det)– know the status of tel+ins+det
maintenance, intervention– calibrate the instrument
• purpose of QC:– ensure that calibs can do that – to a known and predictable accuracy
1. Principles
July 2006 Instrument Scientist Retreat, Santiago
• QC group:– 7+1 scientists– data processing
and quality control– roughly 400 GB
raw data per month, 11,500 processing jobs
– 80 SM packages– 9 VLT, 2 VLTI; 2
survey telescopes
July 2006 Instrument Scientist Retreat, Santiago
• aspects of QC:– formal compliance of file format:
PSO, DMD/DBCM, DMD/SAO– compliance with user constraints:
PSO (OB grading)– check raw files against reference files:
PSO– check pipeline products, do optimized
association, optimized processing: QC group
– compare to similar data (trending):QC group
July 2006 Instrument Scientist Retreat, Santiago
• standard QC
Paranal SciOps
QC Garching
raw proc (on-line pipe)
raw proc (off-line pipe)
certification
trending
10 days
fits file transfer
July 2006 Instrument Scientist Retreat, Santiago
• standard QC
• shared QC: close the loop
• feedback to enable corrections
Paranal SciOps
QC Garching
raw proc (on-line pipe)
raw proc (off-line pipe)
certification
trending
10 days
fits file transfer
HC process
HC monitor
< 1 day
op
s l
og
ftp
July 2006 Instrument Scientist Retreat, Santiago
2. Processes
Components:
• On-site QC
• Health Check monitor
• QC processing and certification
• QC trending
10 days!
July 2006 Instrument Scientist Retreat, Santiago
• On-site QC– day/night astronomers:
• completeness check • visual quality checks: exposure level, patterns • inspection of pipeline products: are there any?
do look reasonable?
July 2006 Instrument Scientist Retreat, Santiago
• Reference frames– www.eso.org/qc/ALL/ref_frames/ref_frames.html
July 2006 Instrument Scientist Retreat, Santiago
• Health Check monitor– www.eso.org/qc/ALL/daily_qc1.html– day astronomer + QC scientist:
• how do new calibs compare to previous ones
• do new calibs show up at all
• outliers, alerts? trigger actions
• currently being upgraded to better organize information, closeups etc.
July 2006 Instrument Scientist Retreat, Santiago
• upgraded: Giraffe, VIMOS
July 2006 Instrument Scientist Retreat, Santiago
• QC process, certification– QC scientists in Garching:
• process all calibration data, SM science data• associate best-possible calibs to new data
– best: closest in time, certified, dynamically updated– correct dependencies (following the cascade)
• process with best-possible pipeline setup– optimized, customized parameters– maybe patched/improved beta versions
• measure and assess product quality– customized QC reports, experience– trending analysis
• certify products– released for archiving and delivery– released for usage in processing other data
July 2006 Instrument Scientist Retreat, Santiago
• Trending– done and maintained by QC group
• set of current and history trending plots• customized for each product type or instrument component• access to ASCII data
– www.eso.org/qc
July 2006 Instrument Scientist Retreat, Santiago
– useful for:• maintenance planning (“by how much do the
zeropoints degrade over a year by dust accumulation”)
• complete instrument history (“how often in the past has grating XZ shown this kind of shift”)
• InsScientists, PIs, archive users (“how does my current dispersion rms compare to earlier observations in P74”)
July 2006 Instrument Scientist Retreat, Santiago
3. Critical components, challenges
• calibration plan– definition of calibs and frequency must be
complete– must include Health Check calibs– execution must be complete:
• daytime (calOBbuilder)• twilight/nighttime (…)
July 2006 Instrument Scientist Retreat, Santiago
• on-site QC, Health Check monitor– define within IOT:
• regular set of HC calibs (regular spacing, daily or so)• set of key QC1 params to be included (cannot be all,
must focus on most critical ones):– DET, FILT, GRISM/GRAT, LAMPs, system throughput etc.
– don’t expect all trending info on the HC monitor• simply too much …• find under www.eso.org/qc/:• full history, all plots, all tutorial info, all text files
July 2006 Instrument Scientist Retreat, Santiago
• pipelines – QC1 parameters need to be pipeline-
produced and included in ops log files, to show up on the Health Check monitor
• Health Check monitor, trending:– maintenance continuous effort– complexity is a challenge!
July 2006 Instrument Scientist Retreat, Santiago
• Data transfer– delay by 8-14 days– large impact on QC task:
• large data volume affected by any problem discovered in Garching
• delivery bursts processing delays may happen
dvdMonitor
July 2006 Instrument Scientist Retreat, Santiago
• complexity– supporting 9 VLT + 2 VLTI plus 2 VST/VISTA– publish and maintain ~600 web pages– we are largely on our own with maintenance/
development of most components
July 2006 Instrument Scientist Retreat, Santiago
• IOTs– extremely important for setting up and
maintaining the QC process– shared QC requires each side to know what
the other side is doing and why
– we visit you come and visit us!
July 2006 Instrument Scientist Retreat, Santiago
• shared QC: – in excellent shape– but needs continuous efforts– can still be improved …
4. Conclusions
• key issues: – data delivery pattern– daytime QC/HC monitor– operational stability