Tutorial SAXS data reduction and analysis software (UM2016)
Transcript of Tutorial SAXS data reduction and analysis software (UM2016)
![Page 1: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/1.jpg)
![Page 2: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/2.jpg)
Page 2 Users Meeting 2016 - Michael Sztucki, ESRF
M. Sztucki
ESRFthe European Synchrotron
Grenoble (France)
Tutorial: SAXS data reduction and analysis
![Page 3: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/3.jpg)
Page 3 Users Meeting 2016 - Michael Sztucki, ESRF
Outline
➔ ID02 data and metadata concept
➔ understanding of the data formats (EDF, HDF5)
➔ detector dependent corrections (non-linearity, dark current, flat field, spatial distortion, etc.) as well as the normalisation to an absolute intensity scale and the azimuthal integration to obtain one dimensional SAXS profiles
➔ data reduction schemes+ SPD (saxs programs)+ DAHU (PyFAI)
➔ SAXSutilities+ 1D, 2D visualization
+ 1D, 2D data reduction
➔ More advanced features include direct modelling (standard models in small-angle scattering)
![Page 4: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/4.jpg)
Page 4 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 data and metadata concept
SAXS, WAXS and USAXS
![Page 5: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/5.jpg)
Page 5 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 data and metadata concept
data.edf data.h52 data formats (edf, h5)
![Page 6: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/6.jpg)
Page 6 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 data and metadata concept
data.edf
![Page 7: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/7.jpg)
Page 7 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 data and metadata concept
data.h5
![Page 8: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/8.jpg)
Page 8 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 data and metadata concept
data.h5
![Page 9: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/9.jpg)
Page 9 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
http://www.esrf.fr/files/live/sites/www/files/Instrumentation/software/data-analysis/OurSoftware/SAXS/SX_parametrization-ref-short_20130125.pdf
![Page 10: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/10.jpg)
Page 10 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
![Page 11: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/11.jpg)
Page 11 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
http://www.esrf.fr/files/live/sites/www/files/Instrumentation/software/data-analysis/OurSoftware/SAXS/SaxsKeywords-V2.459P2.033E2.229.pdf
special case ehf/edf files
easily accessible metadata in edf
![Page 12: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/12.jpg)
Page 12 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
http://www.nexusformat.org
https://www.hdfgroup.org
data.h5
![Page 13: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/13.jpg)
Page 13 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
ESRF NeXus filesESRF NeXus files
NXroot top level, one per file
NXentry one group per measurement
NXinstrument only one per NXentry
NXdetector NXdata one NXdata group per plot
measurement (@NXcollection) flattened view of everything measured - only one per NXentry
NXsample user (@NXuser)
NXprocess for analysis
![Page 14: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/14.jpg)
Page 14 Users Meeting 2016 - Michael Sztucki, ESRF
understanding of the data formats (EDF, HDF5)
raw data (*_raw.h5) [NXdata]:/entry_0000/measurement/id02-XXXX/data
raw metadata (*_raw.h5)/entry_0000/measurement/id02-XXXX/parameters
raw metadata (*_scalers_*_raw.h5)/entry_0000/id02/MCS/entry_0000/id02/TFG/entry_0000/id02/parameters
processed data [NXdata]:/entry_0000/PyFAI/process_XXXX/data/entry_0000/PyFAI/process_XXXX/q/entry_0000/PyFAI/process_XXXX/t
processed metadata:/entry_0000/PyFAI/process_XXXX/parameters/entry_0000/PyFAI/process_XXXX/MCS/entry_0000/PyFAI/process_XXXX/TFG
Note: static and dynamic metadataare written in two different files
locating data and metadata in h5
![Page 15: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/15.jpg)
Page 15 Users Meeting 2016 - Michael Sztucki, ESRF
(online) data reduction
(A) detector specific corrections
(B) scattering specific corrections
(C) sample and beam specific corrections
![Page 16: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/16.jpg)
Page 16 Users Meeting 2016 - Michael Sztucki, ESRF
(A) detector specific corrections
![Page 17: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/17.jpg)
Page 17 Users Meeting 2016 - Michael Sztucki, ESRF
(B) scattering specific corrections
![Page 18: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/18.jpg)
Page 18 Users Meeting 2016 - Michael Sztucki, ESRF
(C) sample and beam specific corrections
![Page 19: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/19.jpg)
Page 19 Users Meeting 2016 - Michael Sztucki, ESRF
(C) sample and beam specific corrections
WAXS projection
In Waxs projection the scattering pattern of a tilted detector can be geometrically analysed (azimuthal regrouping etc.) like a small angle scattering pattern.
![Page 20: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/20.jpg)
Page 20 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 21: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/21.jpg)
Page 21 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 22: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/22.jpg)
Page 22 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 23: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/23.jpg)
Page 23 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 24: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/24.jpg)
Page 24 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 25: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/25.jpg)
Page 25 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 26: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/26.jpg)
Page 26 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
![Page 27: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/27.jpg)
Page 27 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction steps
all automatised in online data reduction
![Page 28: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/28.jpg)
Page 28 Users Meeting 2016 - Michael Sztucki, ESRF
data reduction schemes
+ SPD (saxs programs)
+ DAHU (PyFAI)
http://www.esrf.fr/Instrumentation/software/data-analysis/OurSoftware/SAXS/SaxsDownloads.html
→ project for 2016: online variance calculation in PyFAI
![Page 29: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/29.jpg)
Page 29 Users Meeting 2016 - Michael Sztucki, ESRF
SPD – saxs programs
ID02 online data reduction (SPD) package was developed by P. Boesecke,A. Sole and R. Wilcke during 2001-2005
saxs programs are command line oriented programs (C) to perform certain tasks for data reduction
example:
P. Bösecke, "Reduction of two-dimensional small- and wide-angle X-ray scattering data," J. Appl. Cryst. 40, 423-427 (2007)
propagation of statistical errors
GUI is available for most common tasks in SAXSutilities
![Page 30: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/30.jpg)
Page 30 Users Meeting 2016 - Michael Sztucki, ESRF
saxs programs
further info:
http://www.esrf.fr/files/live/sites/www/files/Instrumentation/software/data-analysis/OurSoftware/SAXS/SaxsPrograms-V2.459P2.033E2.229.pdf
P. Bösecke, "Reduction of two-dimensional small- and wide-angle X-ray scattering data," J. Appl. Cryst. 40, 423-427 (2007)
![Page 31: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/31.jpg)
Page 31 Users Meeting 2016 - Michael Sztucki, ESRF
PyFAI - Fast Azimuthal Integration in Python
[1] J. Kieffer and D. Karkoulis, PyFAI, a versatile library for azimuthal regrouping, Journal of Physics: Conference Series 425, 202012 (2013)
[2] J. Kieffer and J.P. Wright, PyFAI: a Python library for high performance azimuthal integration on GPU, Powder Diffraction 28, S339 (2013)
PyFAI is a novel library for azimuthal integration which already provides geometric equivalence with SPD (Bösecke, 2007) and Fit2D (Hammersley et al., 1996).
● designed to reduce SAXS, WAXS and XRPD images into 1D curves (azimuthal integration)● as a library, the aim of pyFAI is to be integrated into other tools with a clean pythonic interface
(e.g. beamline control systems)● both local and total intensities are conserved in order to obtain accurate quantitative results during
the transformation from cartesian space to polar space● parallelized on graphic cards (GPU)● shares the geometry definition of SPD● features also command line tools for batch processing (however not directly compatible with ID02
metadata concept)
![Page 32: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/32.jpg)
Page 32 Users Meeting 2016 - Michael Sztucki, ESRF
ID02 online data processing
two routes for online data reduction
raw:
reduced:
ehf/edf h5
SPD DAHU (PyFAI)
edf h5
![Page 33: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/33.jpg)
Page 33 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities
http://www.saxsutilities.eu
![Page 34: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/34.jpg)
Page 34 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities
http://www.saxsutilities.eu
![Page 35: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/35.jpg)
Page 35 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 2D visualisation (EDFplot)
![Page 36: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/36.jpg)
Page 36 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 2D visualisation (EDFplot)
Saxs COORDINATES < > Saxs PROJECTION
Saxs PROJECTION (scattering pattern on a flat 2D detector)
(distance of pixel from poni on detector) wavelength0 SAXS coordinate = -----------------------------------------* ----------- (distance of poni from sample) wavelength where wavelength0 is 1e-9 m
SAXS(Saxs) coordinate ~ s*nm for 2Theta<<1
Waxs PROJECTION (projection of the scattering pattern from the EWALD sphere to a plane perpendicular to the incident beam)
In the Waxs projection, the scattering pattern of the detector isdistorted in such a way that the SAXS coordinate of the new pattern isequal to s:
SAXS(Waxs) coordinate = s*nm
In Waxs PROJECTION the scattering pattern of a tilted detector can begeometrically analysed (azimuthal regrouping etc.) like a smallangle scattering pattern.
![Page 37: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/37.jpg)
Page 37 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 2D visualisation (EDFplot)
![Page 38: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/38.jpg)
Page 38 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 2D visualisation (EDFplot)
![Page 39: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/39.jpg)
Page 39 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 2D tools (EDFtools)
![Page 40: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/40.jpg)
Page 40 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 1D visualisation (BHplot)
![Page 41: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/41.jpg)
Page 41 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – 1D tools (DataTools)
![Page 42: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/42.jpg)
Page 42 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – special routines (ASAXS)
![Page 43: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/43.jpg)
Page 43 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities – special routines (ASAXS)
radial distribution of
F0(r) : non-resonant scattering units (macroion + constant part of counterions)
v(r) : resonant scattering units (radial distribution of counterions)
TotalIntensity
“Self term”of the counterions
“Cross-term”Intensity measuredfar below the edge SAXS-Intensity
)()()()()()(2)(),( 2220
20 qvEfEfqvqFEfqFEqI
X-ray scattering factor of a single counterion:
Energy dependent scattering length density profile:
)()()( 0 EfiEffEf
)()(),( EfrrvEr e
)()()()(),( 0 qvEfiEfqFEqF Scattering amplitude:
12000 13000 14000-10
-8
-6
-4
-2
0
2
4
f ',
f ''
E [eV]
f '' f '
Br K-edge
13474 keV
![Page 44: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/44.jpg)
Page 44 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities - modelling
![Page 45: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/45.jpg)
Page 45 Users Meeting 2016 - Michael Sztucki, ESRF
SAXSutilities - modelling
![Page 46: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/46.jpg)
Page 46 Users Meeting 2016 - Michael Sztucki, ESRF
Conclusions
Online data reduction and analysis are crucial for the productivity of any Small-Angle X-ray Scattering instrument especially when dealing with large amount of data or taking decision during a time-resolved experiment or radiation damage testing. The developed programs and tools do not only work at the ESRF beamline, but can also be installed at the home laboratory for more detailed data analysis.
http://www.esrf.fr/Instrumentation/software/data-analysis/OurSoftware/SAXS/SaxsDownloads.html
![Page 47: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/47.jpg)
Page 47 Users Meeting 2016 - Michael Sztucki, ESRF
Acknowledgements
P. Boesecke, A. Sole, R. Wilcke, J. Kieffer
T. Narayanan and all present and former ID02 staff
BCU (beamline control unit), DAU (data analysis unit)
![Page 48: Tutorial SAXS data reduction and analysis software (UM2016)](https://reader031.fdocuments.net/reader031/viewer/2022021508/5849756d1a28aba93a8f51ee/html5/thumbnails/48.jpg)