Changes for page The FLASH HDF5 structure

Last modified by sndueste on 2025/09/10 11:43

From 89.4 to 89.3 From 90.1 to 89.13

From version 89.13

edited by sndueste
on 2025/04/24 13:24

Change comment: There is no comment for this version

To version 89.4

edited by sndueste
on 2025/04/23 14:28

Change comment: Renamed back-links.

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Parent

@@ -1,1 +1,1 @@
--FLASHUSER.Offline data analysis (DAQ).WebHome
++FS-FLASH USER tmp.Offline data analysis (DAQ).WebHome

Content

@@ -99,7 +99,7 @@
  * The currently developed option for large data sets: [[the FAB package>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]] ... see below
  * and for smaller projects:   (% class="Object" %)[[https:~~/~~/gitlab.desy.de/christopher.passow/flash-daq-hdf>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]]
--(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]] and [[doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).DEMO - Working with FLASH data.WebHome]]
++(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]] and [[doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).DEMO - Working with FLASH data.WebHome]]
  {{/info}}
  {{/layout-cell}}
  {{/layout-section}}
@@ -111,6 +111,7 @@
  == Complete list of recordable parameters ==
  The complete list for the relation between DOOCS names and HDF5 names for the recordable parameters can be found in [[DESY's Repository>>url:https://stash.desy.de/projects/CS/repos/pah/browse/src/camp/data/channel2HdfName.dat||shape="rect"]].
++
  == Most popular FLASH parameters and their names in HDF5, DOOCS and (raw) DAQ ==
@@ -129,6 +129,7 @@
  DAQ channel: {{code language="none"}}FLASH.FEL/GAS_DOSING/FL1.ATTENUATOR/PRESSURE{{/code}}
  desc: set pressure in the gas attenuator
  units: mbar
++
  {{code language="none"}}/FL1/Beamlines/BL/Fast shutter/open{{/code}}
  //always saved (PBD)//
@@ -148,15 +148,17 @@
  //always saved (PBD)//
  DOOCS prop : {{code language="none"}}TTF2.FEL/BLFILTERS1/FILTER1/FW.MOTOR.POS{{/code}}
  DAQ channel: {{code language="none"}}TTF2.FEL/BLFILTERS1/FILTER1/FW.MOTOR.POS{{/code}}
--desc: Position of the BL filter wheel 1 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
++desc: Position of the BL filter wheel 1 - to correlate with the filter material please look [[here>>doc:FLASHUSER.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/BL/Filters/BL filter wheel/position filter 2{{/code}}
  //always saved (PBD)//
  DOOCS prop : {{code language="none"}}TTF2.FEL/BLFILTERS2/FILTER2/FW.MOTOR.POS{{/code}}
  DAQ channel: {{code language="none"}}TTF2.FEL/BLFILTERS2/FILTER2/FW.MOTOR.POS{{/code}}
--desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
++desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FLASHUSER.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 1{{/code}}
  //always saved (PBD)//
@@ -164,6 +164,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BDAF1/BDA.F1/FW.MOTOR.POS{{/code}}
  desc: Position of the PG filter wheel 1
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 2{{/code}}
  //always saved (PBD)//
@@ -171,6 +171,7 @@
  DAQ channel: {{code language="none"}}TTF2.FEL/BDAF2/BDA.F2/FW.MOTOR.POS{{/code}}
  desc: Position of the PG filter wheel 2
  units: degree
++
  {{code language="none"}}/FL1/Beamlines/PG/Filters/position filter 3{{/code}}
  //always saved (PBD)//
@@ -261,6 +261,7 @@
  {{info title="GMD pulse resolved data structure"}}
  For every pulse in the pulse train the information is saved:
++
 . **Intensity per pulse** (a.u. (more or less µJ ))
 . Intensity per pulse (auxillary GMD) - not used
@@ -396,7 +396,7 @@
  arrival time (BAM) =====
  {{info title="BAM information: updates 2022 (status 2025)"}}
--* see: [[Info collection about the BAMs and how to use the BAM data>>doc:FS-FLASH USER tmp.jddd-linked help pages.Info collection for the BAM.WebHome||shape="rect"]]
++* see: [[Info collection about the BAMs and how to use the BAM data>>doc:FLASHUSER.jddd-linked help pages.Info collection for the BAM.WebHome||shape="rect"]]
  * The data format of the BAM has been completely altered in the 2022 shutdown
  * before 2022 BAMs were always saving the arrival time information for each 1µs bucked regardless if there were electrons in the accelerator or not. IN addition the arrival times for  FL1 and FL2 were saved in the same parameter ...
  * THIS is now different. There are new parameters saving only the arrival times for pulses that go to FL1 and to FL2 (in detail: first time slot of the accelerator and second)
@@ -405,7 +405,7 @@
  ** FL1: 1SFELC →  FL1.SFELC
  ** FL2: FL2XTDS → (% style="color:#172b4d" %)FL2.SEED5
  * for more Info: [[LINK to detailed infos from MSK>>doc:SDiag.How-to articles.BAM Data Structure.WebHome||shape="rect"]]
--* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments>>doc:FS-FLASH USER tmp.Additional helpful things1.FLASH beamlines and instruments references.WebHome]]
++* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments>>doc:FLASHUSER.Additional helpful things.FLASH beamlines and instruments references.WebHome]]
  * a recent  [[talk about the working principle of the BAM>>attach:BAM-basics and outlook-2018_DESY-template_16-9Format.pdf]]
  {{/info}}
@@ -504,6 +504,7 @@
  DAQ channel: {{code language="none"}}PBD.BEAM.PROF.ML/DAQ.OUT.PROF.CCCED{{/code}}
  desc: temporal profile of electron bunch, y axis in Ampers (FLASH1)
  units: pixel
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron bunch profile/Expert stuff/TDS calibration constant{{/code}}
  //always saved (PBD) - IF LOLA is ON and ACTIVATED in the PBD DAQ (talk to expert)//
@@ -511,6 +511,7 @@
  DAQ channel: = PBD.BEAM.PROF.ML/CCT=
  desc: TDS calibration constant for the x-axis of the profiles: fs per pixel
  units: fs per pixel
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron bunch profile/TDS profile width rms{{/code}}
  //always saved (PBD) - IF LOLA is ON and ACTIVATED in the PBD DAQ (talk to expert)//
@@ -526,7 +526,8 @@
  DOOCS prop : {{code language="none"}}TTF2.DAQ/ENERGY.DOGLEG/E_INTRA_MEAN/VAL{{/code}}
  DAQ channel: {{code language="none"}}PBD.ENERGY.DOGLEG/E_MEAN{{/code}}
  desc: electron bunch energy (average over the bunch train)
--units: (% class="twikiNewLink" %)MeV
++units: (% class="twikiNewLink" %)MeV(%%)
++
  {{code language="none"}}/FL1/Electron Diagnostic/Electron energy/pulse resolved energy{{/code}}
  //always saved (PBD)//
@@ -542,6 +542,7 @@
  DAQ channel: {{code language="none"}}PBD.ENERGY.DOGLEG/LAMBDA_MEAN{{/code}}
  desc: Wavelength calculated by the electron bunch energy (average over the bunch train) (FLASH1)
  units: nm
++
  [[Contents>>doc:||anchor="Contents"]]
@@ -556,6 +556,7 @@
  DAQ channel: {{code language="none"}}TTF2.UTIL/LASER.CONTROL/GUN/PULSE_FREQ{{/code}}
  desc: repetition rate of the bunches / pulses within the burst (FLASH1)
  units: kHz
++
  ===== set number of pulses =====
@@ -574,6 +574,7 @@
  DAQ channel: {{code language="none"}}TTF2.DIAG/PBD.TOROID.ML/12EXP{{/code}}
  desc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH1)
  units:
++
  ===== actual pulse pattern recorded after the undulator =====
@@ -583,6 +583,7 @@
  DAQ channel: {{code language="none"}}TTF2.DIAG/PBD.TOROID.ML/12EXP{{/code}}
  desc: The bunch pattern as function of time in a burst recorded by toroid diagnostic BEHIND the undulator. (FLASH1)
  units:
++
  ===== Train ID =====
@@ -592,25 +592,20 @@
  DAQ channel: {{code language="none"}}none{{/code}}
  desc: Each 10 Hz burst has its unique train ID. For the HDF5 data set the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !)
  units:
++
--===== (% style="color:#e67e22" %)Train time(%%) =====
++===== Train time =====
--(% style="color:#e67e22" %){{code language="none"}}/Timing/train time{{/code}}
++{{code language="none"}}/Timing/train time{{/code}}
  desc:Local time as array of day, hour, minute, second, and centisecond. This data set is meant for visualization purposes only. For correlations use the train ID or the Unix time of the time stamp
  units: d h min s cs
++
--(% style="color:#e67e22" %){{code language="none"}}/Timing/time stamp{{/code}}
++{{code language="none"}}/Timing/time stamp{{/code}}
  desc:first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date ~-~-date='@1553617729' or matlab, python etc
  second column: microseconds
  third column: Train ID of FLASH
--
--currently it is saved as Unix time in
--
--|(((
--/zraw/FLASH.DIAG/TIMINGINFO/TIME1.BUNCH_FIRST_INDEX.1/dGroup/time
--
--)))
  [[Contents>>doc:||anchor="Contents"]]
@@ -688,7 +688,7 @@
  ===== GHz ADCs =====
--ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be found** [[here>>doc:FS-FLASH USER tmp.Data Acquisition and controls 1.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]]**
++ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be found** [[here>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]]**
  //saved on DEMAND in the user DAQ//
  \\The HDF5 names for the ADC traces are depending on the beamline :
  \\PG Beamline:
@@ -745,7 +745,8 @@
  (% style="color:#000000" %)DOOCS prop : {{code language="none"}}FLASH.FEL/ATT.GAS_DOSING/FL2.HALL/PRESSURE{{/code}} (%%)
  (% style="color:#000000" %)DAQ channel:  {{code language="none"}}FLASH.FEL/ATT.GAS_DOSING/FL2.HALL/PRESSURE{{/code}} (%%)
  (% style="color:#000000" %)desc: set pressure in the gas attenuator (%%)
--(% style="color:#000000" %)units: mbar
++(% style="color:#000000" %)units: mbar(%%)
++
  {{code language="none"}}/FL2/Beamlines/FL20/Shutter/open{{/code}}
  //always saved (PBD2)//
@@ -759,14 +759,15 @@
  //always saved (PBD2)//
  DOOCS prop : {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR1.MOT3/FPOS{{/code}}
  DAQ channel: FLASH.FEL/FL20H.PH.MOTOR/MOTOR1.MOT3/FPOS
--desc: Position of the BL filter wheel 1 - to correlate with the filter material please look **[[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]**
++desc: Position of the BL filter wheel 1 - to correlate with the filter material please look **[[here>>doc:FLASHUSER.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]**
  units: degree
++
  /FL2/Beamlines/Filter wheel/position wheel 2
  always saved (PBD2)
  DOOCS prop : {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR2.MOT3/FPOS{{/code}}
  DAQ channel: {{code language="none"}}FLASH.FEL/FL20H.PH.MOTOR/MOTOR2.MOT3/FPOS{{/code}}
--desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FS-FLASH USER tmp.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
++desc: Position of the BL filter wheel 2 - to correlate with the filter material please look [[here>>doc:FLASHUSER.jddd-linked help pages.Filter-Units.Filter wheels in FLASH1 and FLASH2.WebHome]]
  units: degree
@@ -795,6 +795,7 @@
  {{info title="GMD pulse resolved data structure"}}
  For every pulse in the pulse train the information is saved:
++
 . **Intensity per pulse** (a.u. (more or less µJ ))
 . Intensity per pulse (auxillary GMD) - not used
@@ -934,7 +934,7 @@
  arrival time (BAM) =====
  {{info title="BAM information: updates 2022 (status 2025)"}}
--* see: [[Info collection about the BAMs and how to use the BAM data>>doc:FS-FLASH USER tmp.jddd-linked help pages.Info collection for the BAM.WebHome||shape="rect"]]
++* see: [[Info collection about the BAMs and how to use the BAM data>>doc:FLASHUSER.jddd-linked help pages.Info collection for the BAM.WebHome||shape="rect"]]
  * The data format of the BAM has been completely altered in the 2022 shutdown
  * before 2022 BAMs were always saving the arrival time information for each 1µs bucked regardless if there were electrons in the accelerator or not. IN addition the arrival times for  FL1 and FL2 were saved in the same parameter ...
  * THIS is now different. There are new parameters saving only the arrival times for pulses that go to FL1 and to FL2 (in detail: first time slot of the accelerator and second)
@@ -943,7 +943,7 @@
  ** FL1: 1SFELC →  FL1.SFELC
  ** FL2: FL2XTDS → (% style="color:#172b4d" %)FL2.SEED5
  * for more Info: [[LINK to detailed infos from MSK>>doc:SDiag.How-to articles.BAM Data Structure.WebHome||shape="rect"]]
--* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments>>doc:FS-FLASH USER tmp.Additional helpful things1.FLASH beamlines and instruments references.WebHome]]
++* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments>>doc:FLASHUSER.Additional helpful things.FLASH beamlines and instruments references.WebHome]]
  * a recent  [[talk about the working principle of the BAM>>attach:BAM-basics and outlook-2018_DESY-template_16-9Format.pdf]]
  {{/info}}
@@ -968,7 +968,7 @@
  * The BAM 4DBC3 measures the arrivaltime of** FLASH 1 and FLASH2** in the same data set (thus also sorted in at /FL1/ !).
  * The BAM 8FL2XTDS measures only for FLASH2 BUT has the same data structure as the other BAMS ... thus there are also (random) values in in the  FLASH1 time slot
  ** Structure: The first values are for FLASH1 bunches. After a gap of about 70 micros ( 70 colums)  with  as entry for the switching between FLASH 1 and 2 the values for the electrons used in FLASH2 start.  The start time of FLASH2 is also recorded in the DAQ. FLASH1 start time is for historic reasons 700. thus if e.g. the start time of FLASH2 ( property name see below) is 1200 it means that FLASH starts at column 500 (1200-700) ... .In case of doubt ask your local contact
--* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments >>doc:FS-FLASH USER tmp.Additional helpful things1.FLASH beamlines and instruments references.WebHome]]
++* [[Link a collection of papers related to the BAM and the analysis of pump-probe experiments >>doc:FLASHUSER.Additional helpful things.FLASH beamlines and instruments references.WebHome]]
  * [[LINK to detailed infos from MSK (may only work inside DESY network~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|height="12" width="13"~]~]>>url:http://www.desy.de/~~mbock/pages/BAM_daq_channel_descriptions.html||shape="rect"]]
  * a recent  [[talk about the working principle of the BAM>>attach:BAM-basics and outlook-2018_DESY-template_16-9Format.pdf]]
  {{/info}}
@@ -1092,6 +1092,7 @@
  DAQ channel: {{code language="none"}}FLASH.DIAG/PBD2.TOROID.ML/3GUN/NUMBEROFBUNCHES.FLASH2{{/code}}
  desc: Number of bunches measured BEHIND the undulator. If pulses are used for diagnostic of the protection system of the accelerator limits the number of bunches to be accelerated this is the actual number that created XUV radiation.The number is calculated by the DAQ middle layer server, (FLASH2)
  units:
++
  ===== actual pulse pattern recorded after the undulator =====
@@ -1101,6 +1101,7 @@
  DAQ channel: {{code language="none"}}FLASH.DIAG/PBD2.TOROID.ML/9FL2BURN.PULSEPATTERN{{/code}}
  desc: The bunch pattern as function of time in a burst recorded by toroide diagnostic BEHIND the undulator. (FLASH2)
  units:
++
  ===== Train ID =====
@@ -1110,6 +1110,7 @@
  DAQ channel: {{code language="none"}}none{{/code}}
  desc: Each 10 Hz burst has its unique train ID. For the HDF5 dataset the ID is the same for all parameters with the same index (note camera images may be shifted by 1 ID - talk to the experts !)
  units:
++
  ===== Train time =====
@@ -1117,6 +1117,7 @@
  {{code language="none"}}/Timing/train time{{/code}}
  desc:Local time as array of day, hour, minute, second, and centisecond. This dataset is meant for visualisation purposes only. For correlations use the train ID or the Unix time of the time stamp
  units: d h min s cs
++
  {{code language="none"}}
  /Timing/time stamp
@@ -1126,6 +1126,7 @@
  desc: first column: Local time in unix time. To get day, hour, minute, second you can use unix: e.g. date ~-~-date='@1553617729' or matlab, python etc
  second column: microseconds
  third column: Train ID of FLASH
++
  [[Contents>>doc:||anchor="Contents"]]
@@ -1140,7 +1140,7 @@
  ===== GHz ADCs =====
--ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be found** [[here>>doc:FS-FLASH USER tmp.Data Acquisition and controls 1.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]]**
++ADC traces of the (SPDevices 412) GHZ ADCs available for the users . More information about the ADCs can be found** [[here>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]]**
  //saved on DEMAND in the user DAQ//
  \\Up to now there are 4 channels available at FL24
  \\\\{{code language="none"}}/FL2/Experiment/MTCA-EXP1/ADQ412 GHz ADC/CH00/TD{{/code}}

Changes for page The FLASH HDF5 structure

Summary

Details

Applications

Navigation