Post on 03-Feb-2016
description
June 2012
Jörg TrentmannSatellite Application Facility on
Climate Monitoring (CM SAF)
Deutscher Wetterdienst
Technical Introduction
June 2012
•Introduction to
•CM SAF data
•Software tools (cdo, R)
•Example scripts
•Suggestions/ideas for exercises
Overview
June 2012
Are you planning to use CM SAF data during the
CM SAF Event Week?
Introduction
Yes
No(but will use
CM SAF data later)
No(no plans to useCM SAF data)
June 2012
Are you a registered user of CM SAF?
Yes No
Have you ever ordered CM SAF data through the Web
User Interface (WUI: www.cmsaf.eu/wui)?
Yes No
Introduction
June 2012
CM SAF data files
How does a CM SAF data file look like, have you ever
seen a CM SAF data file??
Yes No
June 2012
CM SAF data files
The file names of CM SAF data follow a certain naming
convention:
June 2012
CM SAF data files
The file names of CM SAF data follow a certain naming
convention:
June 2012
Example:
CM SAF data files
June 2012
hdf(http://www.hdfgroup.org/HDF5/)
Data Formats
+ supports additional data compression
+ commonly used in satellite data community
+ provides a standard
– not very common– only limited software
available to access the data
– complex to read data
June 2012
hdf(http://www.hdfgroup.org/HDF5/)
Data Formats
+ supports additional data compression
+ commonly used in satellite data community
+ provides a standard
– not very common– only limited software
available to access the data
– complex to read data
netcdf (http://www.unidata.ucar.edu/software/netcdf
+ growing number of meteorological / climatological users
+ wider range of software available to access the data, incl. GIS
+ standard for climate data available
– no data compression, larger files
June 2012
hdf(http://www.hdfgroup.org/HDF5/)
Data Formats
+ supports additional data compression
+ commonly used in satellite data community
+ provides a standard
– not very common– only limited software
available to access the data
– complex to read data
netcdf (http://www.unidata.ucar.edu/software/netcdf
+ growing number of meteorological/climatological users
+ wider range of software available to access the data, incl. GIS
+ standard for climate data available
– no data compression, larger files
It is recommended to order CM SAF data in netcdf-format via the Web
User Interface (WUI)!
June 2012
Software tools
The free, open-source climate data operators (cdo)
and the statistical software tool R are the main tools
supported to analyse CM SAF data.
•cdo: https://code.zmaw.de/projects/cdo
•R: http://www.r-project.org
June 2012
Software tools
Have you ever used the
climate data operators (cdo) / R?
Yes
No
Very little
Yes
No
Very little
cdo R
June 2012
Climate Data Operators• developed at Max Planck Institute for Meteorology, Hamburg
• freely available including extensive documentation:
https://code.zmaw.de/projects/cdo
• works with multiple file formats, including grib, netcdf etc.
• collection of operators for processing climate (model) data e.g.spatial interpolationdata selectionSubsamplingstatistical and arithmetical functions…
• is accessible from the ‘command line’ (‘prompt’)
• cdo does currently not include a graphical user interface and can
not visualize data
tools: cdo
June 2012
R• programming language and software environment for
statistical computing and graphics: http://www.r-project.org/• includes an interface to netcdf
• has extensive statistical and graphical capabilities, mainly through additional packages
• supports the use of scripts
• extensive online-documentation available on the official webpage and on other webpages
• has been developed for use in statistics (biology, medicine, etc.);
• has not been developed for use in meteorology / climatology:-> terminology sometimes not intuitive -> rather slow for large data sets
tools: R
June 2012
tools: cdo + R
Which operating system are you using?
Windows
Linux/Unix
MAC OSX
XP 7
2000 Vista
June 2012
tools: cdo + R
Is cdo / R available on your computer system?
Yes
No
Yes
No
cdo R
June 2012
•A collection of software scripts including example CM SAF data is provided on the Course page
•scripts are based on Windows, rearranging them for Unix/Linux/Mac-operating systems should be straight forward.
•Before running these scripts, install cdo and R on your local computer system (see further instructions on the Course Page)!
•Suggestion: install panoply (visualize netcdf-files) and Notepad++ (text-editor).
•Have a look at the screencasts (provided on the Course page) that describe the use of the scripts!
tools: cdo + R
June 2012
1. Order data in netcdf-format on a regular lon-lat grid from the Web User Interface
Step-by-step
June 2012
1. Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
Step-by-step
June 2012
1. Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
3. Extract the time series for a single location (cdo)
4. Calculate the temporal / spatial means (cdo)
5. Calculate the multi-year monthly averages (cdo)
6. Calculate the monthly anomalies (cdo)
Step-by-step
June 2012
1. Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface
2. Combine the individual time steps into one netcdf-file (cdo)
3. Extract the time series for a single location (cdo)
4. Calculate the temporal / spatial means (cdo)
5. Calculate the multi-year monthly averages (cdo)
6. Calculate the monthly anomalies (cdo)
7. Plot time series data (R)
8. Analyse the time series data, e.g., calculate average annual cycles, trends (R)
9. Visualize the 2D Data, e.g., monthly means / temporal means / anomalies (R)
Step-by-step
June 2012
Step-by-step
Examples
Time Series Analysis
Regional Plot
Surface Solar Radiation, Offenbach, Germany
June 2012
• Use the scripts and the data provided• Order more data that fits your purpose using the WUI• Best way of learning new tools is using them!
Now it is your turn!
June 2012
• Use the scripts and the data provided• Order more data that fits your purpose using the WUI• Best way of learning new tools is using them!
• Some ideas to start with:- Have a look at your home town:
What is the average annual cycle of cloud cover, solar radiation, humidity?
Is there a trend in solar surface radiation? Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?Which was the ‘best’ summer / winter in the last five years?
Now it is your turn!
June 2012
• Use the scripts and the data provided• Order more data that fits your purpose using the WUI• Best way of learning new tools is using them!
• Some ideas to start with:- Have a look at your home town:
What is the average annual cycle of cloud cover, solar radiation, humidity?
Is there a trend in solar surface radiation? Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?Which was the ‘best’ summer / winter in the last five years?
- Which region in your country has the highest cloud cover (e.g., in July)? Is this the region with the highest solar radiation?
Now it is your turn!
June 2012
• Use the scripts and the data provided• Order more data that fits your purpose using the WUI• Best way of learning new tools is using them!
• Some ideas to start with:- Have a look at your home town:
What is the average annual cycle of cloud cover, solar radiation, humidity?
Is there a trend in solar surface radiation? Are there temporal/spatial correlations between solar
radiation, cloud cover, humidity in your country?Which was the ‘best’ summer / winter in the last five years?
- Which region in your country has the highest cloud cover (e.g., in July)? Is this the region with the highest solar radiation?
Now it is your turn!
Create your own project, we are curious to learn from you!
June 2012
Now it is your turn!
Have fun playing with the data!
Any Questions?
We will be back later today (14 UTC) in Centra for further comments / questions.
See also the course page:
•Download CM SAF Sample File including instructions (screencasts).
•Forum at Course page for further discussions.