Access to NCAR’s Historical Upper Air Archive By Joey Comeaux Steve Worley Bob Dattore
34
Access to NCAR’s Historical Upper Air Archive By Joey Comeaux Steve Worley Bob Dattore Will Spangler Dennis Joseph Roy Jenne DSS/CISL/NCAR AMS - 2010
description
Access to NCAR’s Historical Upper Air Archive By Joey Comeaux Steve Worley Bob Dattore Will Spangler Dennis Joseph Roy Jenne DSS/CISL/NCAR AMS - 2010. Objective : Provide access to approximately 80 million upper air soundings archived in NCAR’s Research Data Archive (RDA). - PowerPoint PPT Presentation
Transcript of Access to NCAR’s Historical Upper Air Archive By Joey Comeaux Steve Worley Bob Dattore
Access to NCAR’s Historical Upper Air Archive
ByJoey ComeauxSteve Worley
Bob DattoreWill Spangler
Dennis JosephRoy Jenne
DSS/CISL/NCARAMS - 2010
Objective :
Provide access to approximately 80 million upper air soundings archived in NCAR’s Research Data Archive (RDA)
TD52
ADP
NCEP_B-3
C-CARDS
TD390_Raobs
French
South_African_Raobs
RUSSIAN_UA
CARDS542
U.S._Control
Russian_ships
MIT
Navy_Spot
Navy_Kunia
SHERHAG
TD53
TD54
PERMANENT_SHIPS
Countries
USAF
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
Sources of U/A Data in RDA
Year
Content of Upper Air Archive
1920
1922
1924
1926
1928
1930
1933
1935
1937
1939
1941
1943
1946
1948
1950
1952
1954
1956
1959
1961
1963
1965
1967
1969
1972
1974
1976
1978
1980
1982
1985
1987
1989
1991
1993
1995
1998
0
200
400
600
800
1000
1200
Stations Reporting 30 Times/ Month or More
YEAR
# S
tatio
nsContent of Upper Air Archive
Global Coverage Plots : 1920-1980
Global Coverage of Upper Air ArchiveGlobal Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Global Coverage of Upper Air Archive
Foundation for Work
• RDA U/A data Several reanalysis projects – Data collected since 1960’s – Much data added in early 1990’s – Some systematic problems fixed
Problems• Varying Units• Multiple Formats• Varying Station ID Metadata• Inconsistent QC• Duplicates
– Intersource• Discovery
TD52ADP
NCEP_B-3C-CARDS
TD390_RaobsFrench
South_African_Raobs
RUSSIAN_UACARDS542
U.S._ControlRussian_ships
MITNavy_Spot
Navy_KuniaSHERHAG
TD53TD54
PERMANENT_SHIPS
CountriesUSAF
1920 1930 1940 1950 1960 1970 1980 1990 2000 2010
Sources of U/A Data in RDA
Year
GOALS1. Provide an archive of U/A data that contains
homogeneous units, format(s), station metadata and QC
2. Support usage of data in multiple ways– Output in synoptic or time series – Multiple discovery options– Access various sources and/or merged version
3. Support both research and reanalysis communities
4. Ability to track all reports5. Easy to Update
– New Data– Bias corrected data– Metadata
USAF NCEP ADPB3
MITTD53
Navy
TD52
TD54
Relational Database
Why Relational DB?•Data ManagementoSeparate sources, yet togetheroReproducibilityoTrackingoEasily expandable
• Users oMultiple access options
Synoptic or by TSDomain, Region, Country…
Current DB Status• ✓Homogeneous units and format(s)
• Support usage of data in multiple ways✓Output in synoptic or time series ✓Multiple discovery options✓Access various sources
• ✓Ability to track all reports
• ✓Easy to Update– New Data
• ✓Prototype Interface
http://dss.ucar.edu/datasets/ds370.0
Temporal Range
• ~ 1 month lag with real time
• Only 2 source, no overlap
19732011-01-04
http://dss.ucar.edu/datasets/ds370.0
Request Sent -> You Receive
•Email summary right away
•Email with link to data when ready
Data File•Single ASCII format•All Units Homogenized• Record and Version tracking
Future Work• Improve Station Meta Data• Improve Interface • Improve QC• Coordination of Efforts with NCDC• Ingest New Sources
o Bias Corrections
• Include Aircraft Data • More Output OptionsoNetCDF
Station MetadataProblems• Primarily focusing on IDs • Archive contains WMO, WBAN and other IDs• Multiple sources use varying WMO libraries• Not trivial to access full available POR for a
station WMO IDs for Belfast
Station Lat Lon 3917 54.65 353.78 3918 54.50 353.66 3920 54.48 353.90
Distance : 15-20KM
1940 1950 1960 1970 1980 1990 2000 2010
3917
3920
3918
WMO IDs for BELFAST
1940 1950 1960 1970 1980 1990 2000 2010
3917
3920
3918
Combined 1948 - Cont
WMO IDs for BELFAST
1920
1922
1924
1926
1928
1930
1932
1934
1936
1938
1940
1942
1944
1946
1948
1950
1952
1954
1956
1958
1960
1962
1964
1966
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
-200
0
200
400
600
800
1000
1200
1400
DSS - IGRA Monthly Station Counts for Stations with atleast 30 Obs / Month
DSS_30IGR_30DSS_30 - IGRA_30
Year
# o
f St
atio
ns
