Changes for page The FLASH HDF5 structure

Last modified by sndueste on 2025/09/10 11:43

From 89.11 to 89.10 From 97.3 to 97.2

From version 97.2

edited by sndueste
on 2025/05/19 17:20

Change comment: There is no comment for this version

To version 89.11

edited by sndueste
on 2025/04/23 14:55

Change comment: Update document after refactoring.

Raw
Rendered

Summary

Page properties (1 modified, 0 added, 0 removed)

Details

Page properties

Content

@@ -103,29 +103,27 @@
  {{/info}}
  {{/layout-cell}}
  {{/layout-section}}
--{{/layout}}
++{{layout-section ac:type="single"}}
++{{layout-cell}}
++
--
  == Complete list of recordable parameters ==
  The complete list for the relation between DOOCS names and HDF5 names for the recordable parameters can be found in [[DESY's Repository>>url:https://stash.desy.de/projects/CS/repos/pah/browse/src/camp/data/channel2HdfName.dat||shape="rect"]].
++
  == Most popular FLASH parameters and their names in HDF5, DOOCS and (raw) DAQ ==
  {{id name="DOOCSparameters"/}}
++Note, the HDF group and data set names apply to our HDF tree version since vers. 0.3.0.
++
  === FLASH1 ===
++==== Beamline info (FLASH1) ====
--In the Shutdown 2024 / 2025  the complete photon diagnostic and experiment control was renewed and thus the Doocs names and also some HDF5 names had to be changed. The new naming will be documented here as soon as the systems are online again.
--
--The previously used naming scheme (2024 and before) can be found here:
--
--{{expand expanded="false" title="FLASH1 Naming scheme used until 2024"}}
--==== (% id="cke_bm_8853497S" style="display:none" %) (%%)Beamline info (FLASH1) ====
--
  {{code language="none"}}/FL1/Beamlines/Attenuator/pressure{{/code}}
  //always saved (PBD)//
  DOOCS prop : {{code language="none"}}FLASH.FEL/GAS_DOSING/FL1.ATTENUATOR/PRESSURE{{/code}}
@@ -132,6 +132,7 @@
  DAQ channel: {{code language="none"}}FLASH.FEL/GAS_DOSING/FL1.ATTENUATOR/PRESSURE{{/code}}
  desc: set pressure in the gas attenuator
  units: mbar
++
  {{code language="none"}}/FL1/Beamlines/BL/Fast shutter/open{{/code}}
  //always saved (PBD)//
@@ -153,6 +153,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BLFILTERS1/FILTER1/FW.MOTOR.POS{{/code}}
  desc: Position of the BL filter wheel 1 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/BL/Filters/BL filter wheel/position filter 2{{/code}}
  //always saved (PBD)//
@@ -160,6 +160,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BLFILTERS2/FILTER2/FW.MOTOR.POS{{/code}}
  desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 1{{/code}}
  //always saved (PBD)//
@@ -167,6 +167,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BDAF1/BDA.F1/FW.MOTOR.POS{{/code}}
  desc: Position of the PG filter wheel 1
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 2{{/code}}
  //always saved (PBD)//
@@ -174,6 +174,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BDAF2/BDA.F2/FW.MOTOR.POS{{/code}}
  desc: Position of the PG filter wheel 2
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 3{{/code}}
  //always saved (PBD)//
@@ -246,8 +246,8 @@
  {{code language="none"}}/FL1/Photon Diagnostic/GMD/Average energy/energy tunnel{{/code}}
  //always saved (PBD)//
--(% style="color:#000000" %)DOOCS prop : {{code language="none"}}FLASH.FEL/XGM.PHOTONFLUX/FL1.TUNNEL/PHOTONFLUX.UJ{{/code}} (%%)
--(% style="color:#000000" %)DAQ channel: {{code language="none"}}FLASH.FEL/XGM.PHOTONFLUX/FL1.TUNNEL/PHOTONFLUX.UJ{{/code}}
++(% style="color:#000000" %)DOOCS prop : {{code language="none"}}FLASH.FEL/XGM.PHOTONFLUX/FL1.TUNNEL/PHOTONFLUX.UJ{{/code}} (%%)
++(% style="color:#000000" %)DAQ channel: {{code language="none"}}FLASH.FEL/XGM.PHOTONFLUX/FL1.TUNNEL/PHOTONFLUX.UJ{{/code}}
  desc : calibrated average SASE Energy/pulse measured in the TUNNEL upstream the gas attenuator
  units : microJ
@@ -256,8 +256,8 @@
  {{code language="none"}}/FL1/Photon Diagnostic/GMD/Pulse resolved energy/energy tunnel{{/code}}
  //always saved (PBD)//
--(% style="color:#000000" %)DOOCS prop : (%%)
--(% style="color:#000000" %){{code language="none"}}FLASH.FEL/XGM.INTENSITY/FL1.TUNNEL/INTENSITY.TD{{/code}}DAQ channel: {{code language="none"}}FLASH.FEL/XGM.INTENSITY/FL1.TUNNEL/INTENSITY.TD{{/code}}
++(% style="color:#000000" %)DOOCS prop :
++{{code language="none"}}FLASH.FEL/XGM.INTENSITY/FL1.TUNNEL/INTENSITY.TD{{/code}}DAQ channel: {{code language="none"}}FLASH.FEL/XGM.INTENSITY/FL1.TUNNEL/INTENSITY.TD{{/code}}
  desc : Energy per pulse measured in the Tunnel.  In addition measurement errors and beam position are included for EACH pulse in the pulse train  !! (see below)
  units : a.u. (more or less µJ but need to be calibrated with the "Average energy" for good precision)** [[see here for help>>doc:FS-FLASH USER tmp.jddd-linked help pages.Calibrating the pulse resolved (electron) data from GMD.WebHome]]**
@@ -264,6 +264,7 @@
  {{info title="GMD pulse resolved data structure"}}
  For every pulse in the pulse train the information is saved:
++
 . **Intensity per pulse** (a.u. (more or less µJ ))
 . Intensity per pulse (auxillary GMD) - not used
@@ -277,7 +277,8 @@
  The pulse energy and the error are plotted for the first bunch of the pulse trains saved in this
  file
-- [[image:image2021-2-9_10-51-6.png]]
++
++[[image:attach:image2021-2-9_10-51-6.png||height="250"]]
  {{/info}}
@@ -469,8 +469,8 @@
  //always saved (PBD)//
  (% style="color:#172b4d" %)**FL1.SFELC**(%%)
--(% style="color:#000000" %)channel: FLASH.SDIAG/BAM/(% style="color:#172b4d" %)FL1.SFELC(% style="color:#000000" %)/ARRIVAL_TIME.ABSOLUTE.SA1.COMP(%%)
--(% style="color:#000000" %){{code language="none"}} DOOCS prop : FLASH.SDIAG/BAM//ARRIVAL_TIME.ABSOLUTE.SA1.COMPDAQ {{/code}}(% style="letter-spacing:0.0px" %)desc: Electron bunch arrival time measured  with the BAM  before the undulator (pulse resolved data). This one was newly installed in 2020.. The property contains only the arrival time of the bunches sent to FL1 (e.g. if there are 30 bunches in FL1 the first 30 values are the arrival time the remaining numbers still may have arbitrary numbers looking like a signal which they are not). These are the same values as the "raw" data below - just "cleaned". The values show a very good correlation to the arrival time  of the XUV pulses in the experiment (see help).
++(% style="color:#000000" %)channel: FLASH.SDIAG/BAM/(% style="color:#172b4d" %)FL1.SFELC(% style="color:#000000" %)/ARRIVAL_TIME.ABSOLUTE.SA1.COMP
++{{code language="none"}} DOOCS prop : FLASH.SDIAG/BAM//ARRIVAL_TIME.ABSOLUTE.SA1.COMPDAQ {{/code}}(% style="letter-spacing:0.0px" %)desc: Electron bunch arrival time measured  with the BAM  before the undulator (pulse resolved data). This one was newly installed in 2020.. The property contains only the arrival time of the bunches sent to FL1 (e.g. if there are 30 bunches in FL1 the first 30 values are the arrival time the remaining numbers still may have arbitrary numbers looking like a signal which they are not). These are the same values as the "raw" data below - just "cleaned". The values show a very good correlation to the arrival time  of the XUV pulses in the experiment (see help).
  units: fs (bigger numbers (typically) indicate later arrival times of the electrons).
@@ -506,6 +506,7 @@
  DAQ channel: {{code language="none"}}PBD.BEAM.PROF.ML/DAQ.OUT.PROF.CCCED{{/code}}
  desc: temporal profile of electron bunch, y axis in Ampers (FLASH1)
  units: pixel
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron bunch profile/Expert stuff/TDS calibration constant{{/code}}
  //always saved (PBD) - IF LOLA is ON and ACTIVATED in the PBD DAQ (talk to expert)//
@@ -513,6 +513,7 @@
  DAQ channel: = PBD.BEAM.PROF.ML/CCT=
  desc: TDS calibration constant for the x-axis of the profiles: fs per pixel
  units: fs per pixel
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron bunch profile/TDS profile width rms{{/code}}
  //always saved (PBD) - IF LOLA is ON and ACTIVATED in the PBD DAQ (talk to expert)//
@@ -528,7 +528,8 @@
  DOOCS prop : {{code language="none"}}TTF2.DAQ/ENERGY.DOGLEG/E_INTRA_MEAN/VAL{{/code}}
  DAQ channel: {{code language="none"}}PBD.ENERGY.DOGLEG/E_MEAN{{/code}}
  desc: electron bunch energy (average over the bunch train)
--units: (% class="twikiNewLink" %)MeV
++units: (% class="twikiNewLink" %)MeV(%%)
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron energy/pulse resolved energy{{/code}}
  //always saved (PBD)//
@@ -544,27 +544,26 @@
  DAQ channel: {{code language="none"}}PBD.ENERGY.DOGLEG/LAMBDA_MEAN{{/code}}
  desc: Wavelength calculated by the electron bunch energy (average over the bunch train) (FLASH1)
  units: nm
++
  [[Contents>>doc:||anchor="Contents"]]
--====   ====
--
--
  ==== Timing information, rep rate etc.  (FLASH1) ====
  ===== bunch repetition rate =====
--{{code language="none"}}/Timing/repetition rate{{/code}}
++{{code language="none"}}/FL1/Timing/repetition rate{{/code}}
  //always saved (PBD)//
  DOOCS prop : {{code language="none"}}FLASH.DIAG/TIMER/FLASHCPUTIME1.0/REP_RATE_KHZ.1{{/code}}
--DAQ channel: {{code language="none"}}FLASH.DIAG/TIMER/FLASHCPUTIME1.0/REP_RATE_KHZ.1{{/code}}
++DAQ channel: {{code language="none"}}TTF2.UTIL/LASER.CONTROL/GUN/PULSE_FREQ{{/code}}
  desc: repetition rate of the bunches / pulses within the burst (FLASH1)
  units: kHz
++
  ===== set number of pulses =====
--(% style="color:#ff6600" %)(HDF5 name not yet implemented - see zraw)(% style="color:#ff0000" %){{code language="none"}}/Timing/set number of bunches {{/code}}(%%)
++(% style="color:#ff6600" %)(HDF5 name not yet implemented - see zraw)(% style="color:#ff0000" %){{code language="none"}}/FL1/Timing/set number of bunches {{/code}}(%%)
  //always saved (PBD)//
  DOOCS prop : {{code language="none"}}FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1 [4th number]{{/code}}
  DAQ channel: {{code language="none"}}FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1 [4th number]{{/code}}
@@ -579,6 +579,7 @@
  DAQ channel: {{code language="none"}}TTF2.DIAG/PBD.TOROID.ML/12EXP{{/code}}
  desc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH1)
  units:
++
  ===== actual pulse pattern recorded after the undulator =====
@@ -588,6 +588,7 @@
  DAQ channel: {{code language="none"}}TTF2.DIAG/PBD.TOROID.ML/12EXP{{/code}}
  desc: The bunch pattern as function of time in a burst recorded by toroid diagnostic BEHIND the undulator. (FLASH1)
  units:
++
  ===== Train ID =====
@@ -597,34 +597,24 @@
  DAQ channel: {{code language="none"}}none{{/code}}
  desc: Each 10 Hz burst has its unique train ID. For the HDF5 data set the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !)
  units:
++
--===== (% style="color:#e67e22" %)Train time(%%) =====
++===== Train time =====
--(% style="color:#e67e22" %){{code language="none"}}/Timing/train time{{/code}}
++{{code language="none"}}/Timing/train time{{/code}}
  desc:Local time as array of day, hour, minute, second, and centisecond. This data set is meant for visualization purposes only. For correlations use the train ID or the Unix time of the time stamp
  units: d h min s cs
++
--(% style="color:#e67e22" %){{code language="none"}}/Timing/time stamp{{/code}}
++{{code language="none"}}/Timing/time stamp{{/code}}
  desc:first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date ~-~-date='@1553617729' or matlab, python etc
  second column: microseconds
  third column: Train ID of FLASH
++
--currently it is saved as Unix time in : /zraw/FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1/dGroup/time
++[[Contents>>doc:||anchor="Contents"]]
--{{expand expanded="false" title="Timestamp help for python"}}
-- import time
--# epoch time is found in  /zraw/FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1/dGroup/time  #!!! a leading 1 has to be added !!!
--# The time and date of the start of the data taking in the HDF file is encoded in the filename - to roughly check the time
--epoch_time = 1709051499.17  # Replace with your epoch time
--
--formatted_time = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(epoch_time))
--print(formatted_time)
--{{/expand}}
--
--
--
--
  ==== Pump Probe Laser (FLASH1) ====
  **PIGLET (PG laser)**
@@ -740,9 +740,10 @@
  * {{code language="none"}}number of samples{{/code}}: total number of samples recorded for each 10 Hz trigger
  [[Contents>>doc:||anchor="Contents"]]
--{{/expand}}
++
++
  === FLASH2 ===
  There is analog to FLASH1 a permanently running "PhotonDagnostic DAQ FLASH2" (PBD2) and 2 User DAQs
@@ -750,12 +750,15 @@
  ==== Beamline info (FLASH2) ====
  {{code language="none"}}/FL2/Beamlines/Attenuator/pressure  {{/code}}
++(% style="color:#000000" %)//always saved (PBD2)// (%%)
  (% style="color:#000000" %)DOOCS prop : {{code language="none"}}FLASH.FEL/ATT.GAS_DOSING/FL2.HALL/PRESSURE{{/code}} (%%)
  (% style="color:#000000" %)DAQ channel:  {{code language="none"}}FLASH.FEL/ATT.GAS_DOSING/FL2.HALL/PRESSURE{{/code}} (%%)
  (% style="color:#000000" %)desc: set pressure in the gas attenuator (%%)
--(% style="color:#000000" %)units: mbar
++(% style="color:#000000" %)units: mbar(%%)
++
  {{code language="none"}}/FL2/Beamlines/FL20/Shutter/open{{/code}}
++//always saved (PBD2)//
  DOOCS prop : {{code language="none"}}FLASH.FEL/ADC.SIS.FL2FS/FL20.SHUTTER/CH00.TD{{/code}}
  DAQ channel: {{code language="none"}}FLASH.FEL/ADC.SIS.FL2FS/FL20.SHUTTER{{/code}}
  desc: BL Beamline Fast shutter state: 1 is open, 0 is closed ( for technical reasons there are 100 vales of this state saved ...)
@@ -763,12 +763,15 @@
  {{code language="none"}}/FL2/Beamlines/Filter wheel/position wheel 1{{/code}}
++//always saved (PBD2)//
  DOOCS prop : {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR1.MOT3/FPOS{{/code}}
  DAQ channel: FLASH.FEL/FL20H.PH.MOTOR/MOTOR1.MOT3/FPOS
  desc: Position of the BL filter wheel 1 - to correlate with the filter material please look **[[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]**
  units: degree
++
  /FL2/Beamlines/Filter wheel/position wheel 2
++always saved (PBD2)
  DOOCS prop : {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR2.MOT3/FPOS{{/code}}
  DAQ channel: {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR2.MOT3/FPOS{{/code}}
  desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
@@ -784,6 +784,7 @@
  ==== Photon Diagnostics SASE ([[XGMD>>url:http://photon-science.desy.de/facilities/flash/photon_diagnostics/gmd_intensity_and_position/index_eng.html||shape="rect"]] - FLASH2) ====
  {{code language="none"}}/FL2/Photon Diagnostic/GMD/Average energy/energy tunnel{{/code}}
++//always saved (PBD2)//
  DOOCS prop : {{code language="none"}}FLASH.FEL/XGM.PHOTONFLUX/FL2.TUNNEL/PHOTONFLUX.UJ{{/code}}
  DAQ channel:{{code language="none"}} FLASH.FEL/XGM.PHOTONFLUX/FL2.TUNNEL/PHOTONFLUX.UJ{{/code}}
  desc : calibrated average ( ~~ 20 sec averaging time ) SASE Energy/pulse measured in the TUNNEL before the attenuator (ion current)
@@ -799,6 +799,7 @@
  {{info title="GMD pulse resolved data structure"}}
  For every pulse in the pulse train the information is saved:
++
 . **Intensity per pulse** (a.u. (more or less µJ ))
 . Intensity per pulse (auxillary GMD) - not used
@@ -1096,6 +1096,7 @@
  DAQ channel: {{code language="none"}}FLASH.DIAG/PBD2.TOROID.ML/3GUN/NUMBEROFBUNCHES.FLASH2{{/code}}
  desc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH2)
  units:
++
  ===== actual pulse pattern recorded after the undulator =====
@@ -1105,6 +1105,7 @@
  DAQ channel: {{code language="none"}}FLASH.DIAG/PBD2.TOROID.ML/9FL2BURN.PULSEPATTERN{{/code}}
  desc: The bunch pattern as function of time in a burst recorded by toroide diagnostic BEHIND the undulator. (FLASH2)
  units:
++
  ===== Train ID =====
@@ -1114,40 +1114,26 @@
  DAQ channel: {{code language="none"}}none{{/code}}
  desc: Each 10 Hz burst has its unique train ID. For the HDF5 dataset the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !)
  units:
++
--===== (% style="color:#e67e22" %)Train time(%%) =====
++===== Train time =====
--(% style="color:#e67e22" %)//always saved (PBD2)//(%%)
--(% style="color:#e67e22" %){{code language="none"}}/Timing/train time{{/code}}
++//always saved (PBD2)//
++{{code language="none"}}/Timing/train time{{/code}}
  desc:Local time as array of day, hour, minute, second, and centisecond. This dataset is meant for visualisation purposes only. For correlations use the train ID or the Unix time of the time stamp
  units: d h min s cs
++
  {{code language="none"}}
  /Timing/time stamp
  {{/code}}
--(% style="color:#e67e22" %)//always saved (PBD2)//(%%)
--(% style="color:#e67e22" %)desc: first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date ~-~-date='@1553617729' or matlab, python etc
++//always saved (PBD2)//
++desc: first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date ~-~-date='@1553617729' or matlab, python etc
  second column: microseconds
  third column: Train ID of FLASH
++
--
--(% id="cke_bm_2455529S" style="display:none" %) (%%)currently it is saved as Unix time in : /zraw/FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1/dGroup/time
--
--{{expand expanded="false" title="Timestamp help for python"}}
-- import time
--
--# epoch time is found in  /zraw/FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1/dGroup/time  #!!! a leading 1 has to be added !!!
--# The time and date of the start of the data taking in the HDF file is encoded in the filename - to roughly check the time
--epoch_time = 1709051499.17  # Replace with your epoch time
--
--formatted_time = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(epoch_time))
--print(formatted_time)
--{{/expand}}
--
--
--
--
  [[Contents>>doc:||anchor="Contents"]]
@@ -1691,3 +1691,8 @@
  )))
  [[Contents>>doc:||anchor="Contents"]]
++
++
++{{/layout-cell}}
++{{/layout-section}}
++{{/layout}}

Changes for page The FLASH HDF5 structure

Summary

Details

Applications

Navigation