DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov,...
Transcript of DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov,...
![Page 1: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/1.jpg)
DIDBase: Intelligent, Interactive Archiving Technology for Ionogram
Data
B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov
Environmental, Earth, and Atmospheric Sciences Department, Center for Atmospheric Research, University of Massachusetts Lowell, USA
![Page 2: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/2.jpg)
Our choice
• RDBMS Server - Firebird (Interbase)
• JDBC - Database Client
• Language – Java– Applications– Applets– Servlets
• Application Server - Tomcat
![Page 3: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/3.jpg)
Ionogram processing software• DIDBase (ionograms, scaling data,
ionospheric characteristics)
• Ionogram software suite– SAO Explorer– DIDB Fill, Smart Backup– DIDB Request Processor (CalVal project)– Analyze Scaling Quality– Representative Ionospheric Characteristics– Visualization programs: Ion2Png, Drg2Png,
DrgMaker
![Page 4: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/4.jpg)
Data Management Automation
• DIDBase data ingestion
• ADRES
• Smart Backup
• Database backup
Extended data quality analysis
• Bad data• Incomplete data• Future data
![Page 5: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/5.jpg)
ADRESDIDBase
at UMLCAR
Digisondes
DIDBase Request Processor
Requests(REQ files)
Reports(SAO files)
Offline archivesAutomated
Data RequestExecution System
USER
Online archives
DIDB Fill
SAO Explorer
Query only data for requests
Save scaled data
ADRES
![Page 6: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/6.jpg)
Ion2Png thumbnail mode
![Page 8: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/8.jpg)
Station list
![Page 9: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/9.jpg)
Inventory for Station
![Page 10: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/10.jpg)
Calendar list
![Page 11: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/11.jpg)
One day
![Page 12: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/12.jpg)
Ionogram example
![Page 13: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/13.jpg)
LoginStep 1 Step 2
Step 3
![Page 14: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/14.jpg)
Manual scaling
• Total manually scaled data – 144297 (more than 4 years of 15 min data)
![Page 15: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/15.jpg)
DMSP F16 weather satellite
• DMSP (Defense Meteorological Satellite Program ) F16 weather satellite was launched 18 October 2003
• Orbital parameters are period 101.9 min , apogee 853 km, perigee 843 km, and inclination 98.9°.
• Two optical instruments onboard:– SSULI - limb scanning ultraviolet imager / spectrometer
(built by the Naval Research Laboratory)
– SSUSI - nadir scanning ultraviolet imager /spectrometer and photometer (built by the Applied Physics Laboratory at Johns Hopkins University)
![Page 16: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/16.jpg)
• SSULI and SSUSI sensors make measurements of far and extreme ultraviolet airglow, which are converted into electron and neutral density profiles by ground processing software.
• Measurement calibrated and validated with ground truth– Digisondes
– Incoherent Scatter Radars (ISR)
– F16 SSJ5 Particle Detector Data for auroral measurements
CalVal campaign
![Page 17: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/17.jpg)
Digisonde network
• 50 Digisondes in the network• 27 Digisondes are participating in CalVal
campaign• Products:
– Ionospheric characteristics (foF2, foE, hmF2, …)– Regular profiles– Auroral profiles
![Page 18: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/18.jpg)
CalVal Requests
• Nominally 4 requests per Station per Day when overpass happen for limb and surface measurements on ascending and descending part of orbit
• Each Request has one hour time interval (± 30 min from overpass time)
• CalVal is planned for period from Dec 15 2002 to Mar 2005
• Some “dry run” data for 2001
![Page 19: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/19.jpg)
Statistics
Request status Total, as of Oct 2004
Total requests 12230
Request data loaded to DIDBase 10826
Manually scaled and reported requests
10027
Manually scaled and reported ionograms
50598
![Page 20: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/20.jpg)
Interactive Ionogram Processing
• Ionogram Data Visualization– ionogram display with zoom-in and details-on-demand– ionogram surveys (using “thumbnails”)– ionogram movies
• Interactive Ionogram Scaling• Profile Inversion• Time series of Scaled Data
– Ionospheric characteristics– Plasma density contour/profilogram– Directogram– Text tables for ASCII export
![Page 21: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/21.jpg)
Data Formats Supported
• Ionograms– MMM
– BEM
– SBF
– RSF-flex
– PGH
– RSF
• Scalings– SAO 3.0 - 4.3
– ART
– ADP binary
![Page 22: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/22.jpg)
Ionogram surveys
Profilogram
Char Table
Ionogram Editor
Directogram
Welcome to SAO-X
Contours
Characteristics Plot
![Page 23: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/23.jpg)
Main screen
List of records
Ionosphericcharacteristics set.
Drag and drop to change order.
Scaler (DIDB only)
![Page 24: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/24.jpg)
Ionogram frame
Acton buttons
Mode controls
Context menu(mouse right button)
Details on mouse position
![Page 25: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/25.jpg)
Database connection
• Read-only connection to DIDB over the Internet
• Remembers last connection information
![Page 26: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/26.jpg)
Database queries• List of available data
with the station name and the start/end times
• DIDB inventory available
• Use of ionospheric characteristics as search criteria
• Query only manual data
• By default, the queries bring the data of the highest available quality
Manually scaled
Select measurements with auroral layer
![Page 27: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/27.jpg)
Inventory tree (DIDB only)
• Simple navigation
• Easy to check what is stored in DIDB
• Comprehensive information about data
• “One click” to set station/date in the Query dialog
![Page 28: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/28.jpg)
DIDB write access
• Scaler password protected login
• Submit current record
• Auto submit option
![Page 29: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/29.jpg)
SAO-X homepage
![Page 30: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/30.jpg)
Installation
http://car.uml.edu/Installation/SAO-X_3/install.htm
InstallAnywhere:commercial strength installers for various platforms.
Windows, Solaris: installers with Java VM included.
Others: JVM has to be installed first.
![Page 31: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/31.jpg)
Digisonde and Internet
• Web Publishing
(via homepage)– Latest displays
– Retro displays
– SAO archive
• Dissemination
(by FTP deliveries)– Near real-time
– Works from behind a firewall
![Page 32: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/32.jpg)
Digisonde Homepage
1997
1995
1998
1996
![Page 33: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/33.jpg)
Digisonde Homepage (2)
1999
2000
2002
2001
![Page 34: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/34.jpg)
Digisonde Homepage (3)
2003 - 2004
![Page 35: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/35.jpg)
2004 Digisonde Homepage
![Page 36: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/36.jpg)
Digisonde Data Tree
DIGISONDERaw Ionograms
DFT: Raw DriftSAO: Scaled data
Time Series
Contours
Profilograms
MUF
MMM, BEM, PGH, SBF, RSF
Ionogram displays
DRG: Directograms
Old Formats ION, IHT, IOS, TEC, ADP, ART
HousekeepingBIT, ERR
SKY: Skymaps
DVL: Velocities
TLT: Tilt
online pictures
16 channel D-256
![Page 37: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/37.jpg)
References
• UMLCAR home page http://ulcar.uml.edu
• Digisonde home page http://ulcar.uml.edu/digisonde.html
• SAOExplorer home page http://ulcar.uml.edu/SAO-X/SAO-X.html
• DIDBase home page http://ulcar.uml.edu/DIDBase/
![Page 38: DIDBase: Intelligent, Interactive Archiving Technology for Ionogram Data B. W. Reinisch, G. Khmyrov, I. A. Galkin, and A. Kozlov Environmental, Earth,](https://reader030.fdocuments.net/reader030/viewer/2022032805/56649ee75503460f94bf7bf5/html5/thumbnails/38.jpg)
Publications
• Reinisch, B. W., I. A. Galkin, G. Khmyrov, A. Kozlov, and D. F. Kitrosser, Automated collection and dissemination of ionospheric data from the digisonde network, Adv. Radio Sci. (2004), 2: 241-247.