Skip to Content

Changes for page The FLASH HDF5 structure

Last modified by sndueste on 2026-01-19 14:35
From version 106.2
edited by sndueste
on 2026-01-09 15:30
Change comment: There is no comment for this version
To version 98.2
edited by sndueste
on 2025-09-08 14:27
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -1,3 +1,6 @@
1 +{{layout}}
2 +{{layout-section ac:type="single"}}
3 +{{layout-cell}}
1 1  == Contents ==
2 2  
3 3  
... ... @@ -5,26 +5,10 @@
5 5  {{toc/}}
6 6  
7 7  
8 -== General information about HDF5 and how to look at it ==
11 +== General information about HDF5 ==
9 9  
10 10  HDF5 is a data format maintained by the HDF group. For detailed general information see [[here>>url:https://portal.hdfgroup.org/display/support||shape="rect"]]
11 11  
12 -The easiest way to look at the HDF5 files from your beamtime is to use the **Jupyter notebook** interface to Maxwell:  [[https:~~/~~/max-jhub.desy.de/>>https://max-jhub.desy.de/]]
13 -
14 - The file content can be browsed by double clicking on the file.
15 -
16 -Typically the Jupyter notebook starts at your home directory and you have to create a (soft) link to your beamtime folder: open a terminal window (File->New-> Terminal) and create the link via :
17 -
18 -{{code language="none"}}> ln -sn beamtime path linkname
19 -e.g.
20 -> ln -sn /asap3/flash/gpfs/fl23/2025/data/11024321/ ~/beamtime_11024321 {{/code}}
21 -Then a “beamtime_11024321” folder should appear in your home directory.
22 -
23 -[[image:https://mail.desy.de/service/home/~~/?auth=co&loc=de&id=400586&part=2||height="219" width="500"]]
24 -
25 -
26 -Alternativaly you can also use a conventional **HDF viewer:**
27 -
28 28  To download the HDF5 Viewer click [[here>>url:https://portal.hdfgroup.org/display/support/Download+HDFView||shape="rect"]]
29 29  
30 30  For use on the DESY Maxwell or PAL (max-fsc or pal) **hdfview** is available in the xray module :
... ... @@ -74,17 +74,25 @@
74 74  {{/expand}}
75 75  
76 76  === HDF examples: ===
64 +{{/layout-cell}}
65 +{{/layout-section}}
77 77  
67 +{{layout-section ac:type="three_equal"}}
68 +{{layout-cell}}
78 78  * ADC data as example for **fast** **data** (10 Hz):            
79 79  
80 80  [[image:attach:image2020-11-16_15-26-28.png||height="250"]]
72 +{{/layout-cell}}
81 81  
74 +{{layout-cell}}
82 82  * The //average// FEL pulse energy as example for the **slow** **data**. Here the data is not saved with 10 Hz  - thus not for every FEL pulse train.  The data is typically saved with 1 Hz if the values are changing (like the FEL energy) and every about 20 sec if there is no change for longer time ( e.g. motor set values). Thus to use the data together with the "fast" one, one has to interpolate the data as explained in the examples in the repos below show (e.g. {{code language="none"}}df['GMD_T_average'] = df['GMD_T_average'].interpolate(method='linear'){{/code}})
83 83  
84 84   ( as example only every 10th train ID is listed in the HDF group "index")
85 85  
86 86  [[image:attach:image2020-11-16_15-31-45.png||height="250"]]
80 +{{/layout-cell}}
87 87  
82 +{{layout-cell}}
88 88  * (((
89 89  //zraw// group contains the **original DAQ (DOOCS) names**
90 90  )))
... ... @@ -93,7 +93,11 @@
93 93  
94 94  
95 95  [[image:attach:image2020-11-16_16-26-3.png||height="400"]]
91 +{{/layout-cell}}
92 +{{/layout-section}}
96 96  
94 +{{layout-section ac:type="single"}}
95 +{{layout-cell}}
97 97  {{info}}
98 98  === There are different options that help you to work with the FLASH HDF5 data in Python ===
99 99  
... ... @@ -100,8 +100,11 @@
100 100  * The currently developed option for large data sets: [[the FAB package>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]] ... see below
101 101  * and for smaller projects:  (% class="Object" %)[[https:~~/~~/gitlab.desy.de/christopher.passow/flash-daq-hdf>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]]
102 102  
103 -(% class="Object" %)See also the collection of Demo data and sample scripts : [[Collection of HDF5 sample data from different beamlines>>doc:FLASHUSER.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]] and [[DEMO - Working with FLASH data>>doc:FLASHUSER.Offline data analysis (DAQ).DEMO - Working with FLASH data.WebHome]]
102 +(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]] and [[doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).DEMO - Working with FLASH data.WebHome]]
104 104  {{/info}}
104 +{{/layout-cell}}
105 +{{/layout-section}}
106 +{{/layout}}
105 105  
106 106  
107 107  
... ... @@ -109,7 +109,7 @@
109 109  
110 110  The complete list for the relation between DOOCS names and HDF5 names for the recordable parameters can be found in [[DESY's Repository>>url:https://stash.desy.de/projects/CS/repos/pah/browse/src/camp/data/channel2HdfName.dat||shape="rect"]].
111 111  
112 -== Most popular FLASH parameters and their names in HDF5, DOOCS and (.raw) DAQ ==
114 +== Most popular FLASH parameters and their names in HDF5, DOOCS and (raw) DAQ ==
113 113  
114 114  {{id name="DOOCSparameters"/}}
115 115  
... ... @@ -121,7 +121,7 @@
121 121  
122 122  The previously used naming scheme (2024 and before) can be found here:
123 123  
124 -{{expand expanded="false" title="FLASH1 naming scheme used until 2024"}}
126 +{{expand expanded="false" title="FLASH1 Naming scheme used until 2024"}}
125 125  ==== (% id="cke_bm_8853497S" style="display:none" %) (%%)Beamline info (FLASH1) ====
126 126  
127 127  {{code language="none"}}/FL1/Beamlines/Attenuator/pressure{{/code}}
... ... @@ -875,23 +875,6 @@
875 875  desc : The bunch number of the bunch used for the wavelength calculation
876 876  units :
877 877  
878 -{{expand expanded="false" title="More detailed info on OPIS properties"}}
879 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL370,'%','relative width of photoline in percent of the FEL photon energy ROI1, rising flank'
880 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL375,'%','relative width of photoline in percent of the FEL photon energy ROI1, tailing flank'
881 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL470,'%','relative width of photoline in percent of the FEL photon energy ROI1, mean'
882 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL371,'%','relative width of photoline in percent of the FEL photon energy ROI2, rising flank'
883 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL376,'%','relative width of photoline in percent of the FEL photon energy ROI2, tailing flank'
884 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL471,'%','relative width of photoline in percent of the FEL photon energy ROI2, mean'
885 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL372,'%','relative width of photoline in percent of the FEL photon energy ROI3, rising flank'
886 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL377,'%','relative width of photoline in percent of the FEL photon energy ROI3, tailing flank'
887 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL472,'%','relative width of photoline in percent of the FEL photon energy ROI3, mean'
888 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL373,'%','relative width of photoline in percent of the FEL photon energy ROI4, rising flank'
889 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL378,'%','relative width of photoline in percent of the FEL photon energy ROI4, tailing flank'
890 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL473,'%','relative width of photoline in percent of the FEL photon energy ROI4, mean'
891 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL374,'%','relative width of photoline in percent of the FEL photon energy ROI5, rising flank'
892 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL379,'%','relative width of photoline in percent of the FEL photon energy ROI5, tailing flank'
893 -FLASH.UTIL/STORE/FL2.TUNNEL.OPIS/VAL474,'%','relative width of photoline in percent of the FEL photon energy ROI5, mean'
894 -{{/expand}}
895 895  
896 896  If Opis is running typically on the the averaged data is saved. For several experiments it may make sense to save the information for each single bunch. This is up to now done by saving the complete ADC trace of the TOF setup. This is a huge amount of data and needs processing. This has to be performed after the beamtime in close contact to [[Markus Braune>>mailto:markus.braune@desy.de||shape="rect"]] ( responsible for [[OPIS>>url:http://photon-science.desy.de/facilities/flash/photon_diagnostics/opis_spectrometer/index_eng.html||shape="rect"]])
897 897  
... ... @@ -936,7 +936,7 @@
936 936  desc: gap value of the undulators. This can be used to follow up how many undulators were closed and if there was a taper. 
937 937  units: mm
938 938  
939 -The gap values are saved for all 12 undulators (Nr 2 to 13). Undulator 13 is the one closest to the experimental hall.
924 +The gap values are saved for all 12 undulators (Nr 2 to 14). Undulator 14 is the one closest to the experimental hall.
940 940  
941 941  =====
942 942  arrival time (BAM) =====
... ... @@ -1220,7 +1220,7 @@
1220 1220  
1221 1221  {{view-file att--filename="FSLADAQ-DAQParameterlistFL24-2.pdf"}}{{/view-file}}
1222 1222  
1223 -internal link to parameter list: [[FS-LA DAQ - XWiki>>url:https://xwiki.desy.de/xwiki/bin/view/FSLADAQ/]]
1208 +
1224 1224  {{/info}}
1225 1225  
1226 1226  {{expand title="Laser parameters used until 2023"}}
image.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.sndueste
Size
... ... @@ -1,1 +1,0 @@
1 -1.1 MB
Content