Monthly Operations Report€¦ · Contract No. 4000111291/14/I-LG - 1310174...
Transcript of Monthly Operations Report€¦ · Contract No. 4000111291/14/I-LG - 1310174...
MONTHLY OPERATIONS
REPORT
MOR#47
Reporting period from 16-Oct-2017 to 15-Nov-2017
Reference: PROBA-V_D5_MOR-47_11_v1.0 Author(s): Dennis Clarijs, Sindy Sterckx, Alex Geboers, Erwin Wolters Version: 1.0 Date: 20/11/2017
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 2 of 20
DOCUMENT CONTROL
Signatures Author(s) Dennis Clarijs, Sindy Sterckx, Alex Geboers, Erwin Wolters Reviewer(s) Dennis Clarijs Approver(s) Dennis Clarijs Issuing authority
Change record
Release Date Pages Description Editor(s)/Reviewer(s)
1.0 20/11/2017 All Initial version
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 3 of 20
TABLE OF CONTENT
1. Summary .............................................................................................................................................. 4 2. System Infrastructure ........................................................................................................................... 4 3. Image Processing Services .................................................................................................................... 5 3.1. Ingested and archived products ........................................................................................................... 5 3.2. Generated and archived products ........................................................................................................ 5 3.3. Backup and archiving service ............................................................................................................... 6 3.4. Dissemination service .......................................................................................................................... 7 3.5. End-user activity .................................................................................................................................. 7 4. Image Calibration services ................................................................................................................... 10 4.1. Radiometric Calibration ...................................................................................................................... 10 4.2. Geometric Calibration ......................................................................................................................... 16 5. Anomalies ........................................................................................................................................... 17 5.1. System related issues .......................................................................................................................... 17 5.2. Image processing issues ...................................................................................................................... 18 6. Scheduled activities for the next period(s) .......................................................................................... 20 7. Operational remarks ........................................................................................................................... 20
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 4 of 20
1. Summary In this reporting period, the majority of the the synthesis products were nearly complete. Most missing data in the synthesis products were caused by geometric errors and some automatic recoveries of the platform. In the first week of November the amount of geometric errors peaked. The reason for this is still under investigation. On October 24, a platform 90° yaw calibration manoeuvre was executed over the Niger 1 site. The left camera was looking Nadir this way. This calibration acquisition was performed to verify and improve the camera’s radiometric calibration settings. An upgrade of database scheme of the VITO distribution portal was performed on Tuesday 14 November. This should improve the overall improvement of the portal for all users. Acquisitions of the Antarctica continent have started since October 24. This continent is imaged using calibration requests preventing the segments to be processed in the nominal production workflow as this would lead to processing errors. The idea is to acquisition Antarctica during it’s summertime. All data will be post-processed to L2A data and distributed to the end-user soon. Communication about these acquisitions to the relevant community is currently being prepared. The final validation report on the Collection 1 data will be published on the PROBA-V site soon. There were no major issues with the image quality during this reporting period for the radiometric or geometric quality. No new developments are expected in the coming period.
2. System Infrastructure
Category % Up Time % Down Time
Switches 100.0 0.0
Database Servers 100.0 0.0
Mid Term File Servers 100.0 0.0
Short Term File Servers 100.0 0.0
Master Servers 100.0 0.0
Worker Nodes 100.0 0.0
PDF 100.0 0.0
Table 1: System Infrastructure availability for this reporting period
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 5 of 20
3. Image Processing Services
3.1. Ingested and archived products
Product Type Total Received Missing data, ingested by VITO
Archived
METEO 248 248 0 248
TFF 310 310(*) 0 310
Table 2: Ingested and archived products for this reporting period
(*) TFF 14738 - missing TFF number due to wrong numbering (no data missing)
3.2. Generated and archived products Product Type Total Processed Error Archived
PROBAV_L1A - Calibration 687 682 5 686
PROBAV_L1A - Nominal 2628 2622 6(*) 2626
PROBAV_L1C 2622 2622 0 2621
PROBAV_L2A_100M 911 911 0 911
PROBAV_L2A_300M 2622 2622 0 2622
PROBAV_L2A_1KM 2622 2622 0 2622
PROBAV_L3_S1_TOA_100M 31 30 0 30
PROBAV_L3_S1_TOC_100M 31 30 0 30
PROBAV_L3_S1_TOC_NDVI_100M 31 30 0 30
PROBAV_L3_S5_TOA_100M 6 5 0 5
PROBAV_L3_S5_TOC_100M 6 5 0 5
PROBAV_L3_S5_TOC_NDVI_100M 6 5 0 5
PROBAV_L3_S1_TOA_300M 31 31 0 31
PROBAV_L3_S1_TOC_300M 31 31 0 31
PROBAV_L3_S10_TOC_300M 3 3 0 3
PROBAV_L3_S10_TOC_NDVI_300M 3 3 0 3
PROBAV_L3_S1_TOA_1KM 31 31 0 31
PROBAV_L3_S1_TOC_1KM 31 31 0 31
PROBAV_L3_S10_TOC_1KM 3 3 0 3
PROBAV_L3_S10_TOC_NDVI_1KM 3 3 0 3
Table 3: Generated and archived products for this reporting period
(*) 6 x L1A errors due to geometric processing
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 6 of 20
3.3. Backup and archiving service
Product type Total Files Total File Size (GB)
TFF 300 771.24
L1A 3186 1541.68
Database transaction logs 768 87.33
Database incremental back-up 46 14.95
Database full back-up 8 534.35
Table 4: Back-up data volumes for this reporting period
Product type Total Files Total File Size (GB)
PROBAV_TRANSFERFRAMES 290 801.71
PROBAV_L1A 3063 1593.46
PROBAV_L1C 2453 2859.66
PROBAV_L2A_100M 2420 2332.90
PROBAV_L2A_300M 7523 1298.16
PROBAV_L2A_1KM 7510 166.84
PROBAV_L3_S1_TOA_100M 80 2335.72
PROBAV_L3_S1_TOC_100M 85 2337.97
PROBAV_L3_S1_TOC_NDVI_100M 80 247.56
PROBAV_L3_S5_TOA_100M 18 1915.12
PROBAV_L3_S5_TOC_100M 19 1997.17
PROBAV_L3_S5_TOC_NDVI_100M 21 249.16
PROBAV_L3_S1_TOA_300M 82 1050.58
PROBAV_L3_S1_TOC_300M 88 1093.89
PROBAV_L3_S10_TOC_300M 14 276.20
PROBAV_L3_S10_TOC_NDVI_300M 10 17.76
PROBAV_L3_S1_TOA_1KM 81 134.85
PROBAV_L3_S1_TOC_1KM 85 136.26
PROBAV_L3_S10_TOC_1KM 11 29.22
PROBAV_L3_S10_TOC_NDVI_1KM 9 1.96
ICP_GEOMETRIC_CENTRE 0 0
ICP_ GEOMETRIC _LEFT 0 0
ICP_ GEOMETRIC _RIGHT 0 0
ICP_RADIOMETRIC_CENTRE 1 0.04
ICP_RADIOMETRIC_LEFT 1 0.04
ICP_RADIOMETRIC_RIGHT 1 0.04
METEO_ECMWF 231 0.29
METEO_METEOSERVICES 232 1.24
POLARMOTION 1 0.00
Table 5: Archived data volumes for this reporting period
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 7 of 20
3.4. Dissemination service Product type Added to catalogue Ordered Delivered
PROBAV_L1C 2611 192 185
PROBAV_L2A_100M 908 97 130
PROBAV_L2A_300M 2611 1306 1373
PROBAV_L2A_1KM 2611 0 278
PROBAV_L3_S1_TOA_100M 31 71 44
PROBAV_L3_S1_TOC_100M 31 2835 1946
PROBAV_L3_S1_TOC_NDVI_100M 31 2399 1737
PROBAV_L3_S5_TOA_100M 6 19 20
PROBAV_L3_S5_TOC_100M 6 1292 1267
PROBAV_L3_S5_TOC_NDVI_100M 6 76 239
PROBAV_L3_S1_TOA_300M 31 357 332
PROBAV_L3_S1_TOC_300M 31 374 416
PROBAV_L3_S10_TOC_300M 3 103 99
PROBAV_L3_S10_TOC_NDVI_300M 3 188 194
PROBAV_L3_S1_TOA_1KM 31 132 138
PROBAV_L3_S1_TOC_1KM 31 451 474
PROBAV_L3_S10_TOC_1KM 3 54 63
PROBAV_L3_S10_TOC_NDVI_1KM 3 221 367
Table 6: Ordered and delivered products for this reporting period
3.5. End-user activity 14 new user(s) were registered in this reporting period. The total number of users registered for PROBA-V data and that have ordered data is 1273 with 114 different nationalities representing 956 different companies/universities.
Product type Africa Asia Europe N-America Oceania S-America
PROBAV_L1C 0 385.72 170.66 0 0 0.12
PROBAV_L2A_100M 0.21 2.05 2.05 24.59 0 0
PROBAV_L2A_300M 0 0.02 4.91 10.07 0 0
PROBAV_L2A_1KM 0 0 0 5.51 0 0
PROBAV_L3_S1_TOA_100M 0 0.11 5.91 0 0.09 0
PROBAV_L3_S1_TOC_100M 0.00 660.84 2823.59 681.06 0 3.82
PROBAV_L3_S1_TOC_NDVI_100M 13.89 10.61 4.40 0 0 0.00
PROBAV_L3_S5_TOA_100M 0.04 0 550.20 0 0 0.01
PROBAV_L3_S5_TOC_100M 36.15 118.54 531.82 7.23 0 249.66
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 8 of 20
PROBAV_L3_S5_TOC_NDVI_100M 0.43 4.45 25.85 0 0 2.07
PROBAV_L3_S1_TOA_300M 0 2.74 1068.60 0 0 1347.95
PROBAV_L3_S1_TOC_300M 0 0 999.19 297.00 0 0
PROBAV_L3_S10_TOC_300M 3.59 1.04 191.27 0 0 0
PROBAV_L3_S10_TOC_NDVI_300M 2.84 0.56 0.00 13.96 0 0.04
PROBAV_L3_S1_TOA_1KM 0.00 0.00 155.30 0 0 0
PROBAV_L3_S1_TOC_1KM 0.00 0 263.36 0 0 0
PROBAV_L3_S10_TOC_1KM 0 0.94 17.81 0.85 0 0
PROBAV_L3_S10_TOC_NDVI_1KM 0.09 1.26 1.32 0.14 0 0.01
Table 7: Data download (GB) in total per Origin of the User for the reporting period
Product Type Global
L1C 556.51
PROBAV_L2A_100M 28.91
PROBAV_L2A_300M 15.01
PROBAV_L2A_1KM 5.51
PROBAV_L3_S1_TOA_100M 6.11
PROBAV_L3_S1_TOC_100M 4169.31
PROBAV_L3_S1_TOC_NDVI_100M 28.89
PROBAV_L3_S5_TOA_100M 550.25
PROBAV_L3_S5_TOC_100M 943.39
PROBAV_L3_S5_TOC_NDVI_100M 32.80
PROBAV_L3_S1_TOA_300M 2419.29
PROBAV_L3_S1_TOC_300M 1296.20
PROBAV_L3_S10_TOC_300M 195.91
PROBAV_L3_S10_TOC_NDVI_300M 17.41
PROBAV_L3_S1_TOA_1KM 155.31
PROBAV_L3_S1_TOC_1KM 263.36
PROBAV_L3_S10_TOC_1KM 19.60
PROBAV_L3_S10_TOC_NDVI_1KM 2.81
Table 8: Data download (GB) in total for the reporting period
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 9 of 20
Company # Downloads
VITO 1873
VUB 1414
DEBRE TABOR UNIVERSITY 875
ARID LAND RESEARCH CENTER 640
CSIR-IHBT 595
LEGOS 440
AVIA-GIS 416
DESCARTES LABS 375
SPACE RESEARCH INSTITUTE RAS (IKI RAN) 303
CNCT 186
Table 9: Top 10 user companies for the reporting period
Country # Users
CHINA 122
BELGIUM 104
ITALY 62
UNITED STATES 60
FRANCE 60
BRAZIL 56
INDIA 54
UNITED KINGDOM 48
GERMANY 46
NETHERLANDS 45
Table 10: Top 10 countries with most registered users List of issues raised by users:
southward shift in PROBA-V S5 100m Feb 2016
PV MEP VM
Empty FTP folder
Index validation with PROBA-V
password username
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 10 of 20
4. Image Calibration services
4.1. Radiometric Calibration
Calibration request type Total Processed Not received Error
CLOUDS 20 12 8 0
DARK CURRENT 32 19 13 0
MOON 2 2 0 0
RAYLEIGH 69 38 31 0
SNOW 239 156 83 0
SUN_GLINT 0 0 0 0
Table 11: Calibration Image requests for this reporting period
Calibration image type Total Valid Invalid
PROBA_V_L1A_CALIBRATION 2 2 0
PROBA-V_L1B_CALIBRATION 680 642 31
PROBA-V_L1B_INTERSECTION 773 227 545
PROBA-V_L1B_OVERLAPREGION 0 0 0
Table 12: Processed calibration images for this reporting period
(*) Due to insufficient overlap with the calibration region of interest, not enough pixels (e.g. clouds contamination), site not sufficiently uniform (illumination), etc. Long-term monthly Libya-4 mean plots for different cameras are given in Figure 1 and Figure 2 and Figure 3. Deep convective clouds interband calibration results are given in Figure 4. Although the application of a degradation model for LEFT and CENTER BLUE bands a slight trend is still observed. Also for the RIGHT BLUE strip a minor trend is observed. These trends will be further monitored in order to decide on the actions to be taken. Pixel non-uniformity correction parameters have been successfully derived from the yaw maneuver data for LEFT and RIGHT camera. A verification analysis on nominal images is on-going. If all these analyses are successful, the equalization parameters in the ICP file will be updated accordingly. A slight vertical stripe is observed in the CENTER SWIR3 strip and LEFT SWIR1 strip. It will be evaluated if an update of the dark current values will be sufficient to correct for the stripes. If not, it might be decided to assign the status BAD to these pixels.
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 11 of 20
Radiometric ICP file Both the SWIR and BLUE LEFT/CENTER absolute calibration coefficients will be updated following a linear degradation model. Furthermore the dark currents will be updated. The current ICP files are
PROBAV_ICP_RADIOMETRIC#LEFT_20171101_V01
PROBAV_ICP_RADIOMETRIC#CENTER_20171101_V01
PROBAV_ICP_RADIOMETRIC#RIGHT_20171101_V01
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 12 of 20
Figure 1. Libya-4 desert calibration results: LEFT monthly averaged results (collection 1)
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 13 of 20
Figure 2. Libya-4 desert calibration results: CENTER monthly averaged results (collection 1)
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 14 of 20
Figure 3. Libya-4 desert calibration results: RIGHT monthly averaged results (collection 1)
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 15 of 20
Figure 4. DCC inter-band calibration results: LEFT, CENTER and RIGHT camera (collection 1)
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 16 of 20
4.2. Geometric Calibration
Calibration image type Total Processed Error
PROBA-V_L1C_INTERSECTION 14255 14255 0
Table13: Processed calibration images for this reporting period
Due to processing issues, we can unfortunately only report on the geolocation accuracy for the period 9/10 – 29/10. To tackle the processing issues, extra processing capacities are implemented shortly. We will report on the period 30/10 – 15/12 in the next MOR. During the period 9/10 – 29/10, the geometric performance was stable and average ALE was < 66 m (σ < 84 m). The daily ALE evolution (see Figure 5) shows three ALE peaks (on 11/10, 16/10, and 26/10), but these can be considered minor peaks, as the highest values are still < 75 m. The geometric accuracy was within the requirement of < 300 m, with an average compliance for all cameras of 99.41% (99.10 – 99.75% from BLUE to SWIR). Daily values showed the normal minima synchronous with the ALE maxima, with lowest values of 98.8 – 99.6 (BLUE to SWIR) on 26/10 (BLUE – NIR) and 29/10 (SWIR). Geometric ICP file
PROBAV_ICP_GEOMETRIC#LEFT_20160907_V01
PROBAV_ICP_GEOMETRIC#CENTER_20160907_V01
PROBAV_ICP_GEOMETRIC#RIGHT_20160907_V01
Figure 5 - Daily ALE evolution for all PROBA-V spectral bands
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 17 of 20
5. Anomalies
5.1. System related issues A detailed description of each issue is available in the issue tracking system http://jira.vgt.vito.be
Key Summary Status Created Component/s
PROBAVUS-7 Very small images fail to process Resolved 10/01/2014 General
PROBAVUS-63 Cloud shadow detection at high solar zenith angles not working properly
Open 11/05/2016 Software
PROBAVUS-65 Processing statuses L2 products In Progress
16/09/2016 Software
PROBAVUS-66 Cloud cover percentages on PDF products are not reliable
Resolved 19/10/2016 Software
PROBAVUS-68 ICP file version not taken into account when processing
In Progress
20/03/2017 Software
PROBAVUS-69 Version number of segment not filtered when querying for syntheses
Open 20/03/2017 Software
PROBAVUS-70 Investigate L2A artefact in data Open 31/05/2017 Software
PROBAVUS-72 Status mask of data with decompression error is not correctly set
Open 22/06/2017 Software
PROBAVUS-73 Large geometirc shift (20160218-20160219)
Open 30/08/2017 Geometry
PROBAVUS-74 Take suboptimal meteo data when invalid values in meteo files
Open 07/11/2017 Software
PROBAVUS-75 Clear pixel edge in cloud shadow detection
Open 15/11/2017 Software
2 new issues were logged during this reporting period 0 issue(s) was resolved and closed during this reporting period 3 issues are resolved but remain to be closed formally 1 issues are resolved but remain in the list logging purposes 7 issue(s) is open and remain to be solved
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 18 of 20
5.2. Image processing issues A detailed description of each issue is available in the Weekly Report and the image processing tracking system https://juniper.vgt.vito.be/ciptools The below table gives an overview of the S1’s of this reporting period:
# S1 Dates
Major Gaps (> 21600 km² (missing TFF)) 0
Large Gaps (< 21600 km²) 1 24/10
Medium Gaps (< 10000 km²) 2 07/11, 14/11
Minor Gaps (< 3600 km²) 5 29/10, 21/10, 28/10, 13/11, 22/10
Negligible Gaps (< 1000 km²) 21 30/10, 04/11, 09/11, 23/10, 08/11, 03/11, 17/10, 27/10, 02/11, 12/11, 16/10, 26/10, 01/11, 06/11, 11/11, 25/10, 31/10, 05/11, 15/11, 10/11,
19/10
Complete synthesis (no gaps) 2 18/10, 20/10
Table14: Overview of S1 for this reporting period
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 19 of 20
Synthesis Missing Decom. Error
Geom. Error
Missing TFF
Autom. Recovery
VC4 Missing
Create Contours
Other
20171016 1.16% 2 3
20171017 0.73% 2
20171018 0.93%
20171019 1.87% 1
20171020 2.40%
20171021 0.91% 1 3 1
20171022 0.70% 7 1
20171023 0.01% 1 1
20171024 7.62% 3 4 3
20171025 0.59% 1
20171026 0.47% 2
20171027 0.78% 4
20171028 6.33% 4 6 1
20171029 3.53% 7 6 1
20171030 4.02% 2 1
20171031 3.69% 1 1 1
20171101 0.02% 4 1
20171102 0.01% 2
20171103 0.36% 5
20171104 0.01% 1 3
20171105 0.04% 1 4
20171106 0.34% 5 10
20171107 4.59% 2 12
20171108 2.31% 22
20171109 1.75% 3 22
20171110 3.93% 24
20171111 0.02% 1 5
20171112 1.12% 2
20171113 2.54% 8 19 1 1
20171114 0.47% 7 4 4
20171115 1.64% 3 3 1
Table 15: List of synthesis with an error overview of the missing percentages and errors for this reporting period
Monthly Operations Report PROBA-V Operations Contract No. 4000111291/14/I-LG - 1310174
PROBA-V_D5_MOR-47_11_v1.0 Monthly Operations Report Page 20 of 20
6. Scheduled activities for the next period(s)
Software upgrades: No software upgrades planned.
Hardware: No hardware upgrades planned.
Development: No developments are planned.
No other activities scheduled.
7. Operational remarks
No operational remarks.