Skip to Content
Last modified by sndueste on 2025-09-08 15:36
From version 26.1
edited by sndueste
on 2021-06-07 08:45
Change comment: There is no comment for this version
To version 28.1
edited by sndueste
on 2022-09-08 11:18
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -7,7 +7,7 @@
7 7  == General Concept ==
8 8  
9 9  A large number of parameters which are essential for ALL Experiments is continuously saved (pulse energy, arrival time, Beamline settings ...) in the [[ PHOTONDIAGNOSTIC FLASH DAQ>>doc:FLASHUSER.FLASH1 PhotDiag stream]] system.
10 -Other [[experiment related information>>doc:FLASHUSER.Configuring the user DAQs]] is ONLY saved by the users ON DEMAND in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and one at FLASH 2 (FLASH2_USER1)
10 +Other [[experiment related information>>doc:FLASHUSER.Configuring the user DAQs]] is ONLY saved by the users **ON DEMAND **in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and two at FLASH 2 (FLASH2_USER1, FLASH2_USER2)
11 11  How to save this experiment related "User-Data" will be explained briefly in this page:
12 12  
13 13  \\
... ... @@ -20,19 +20,20 @@
20 20  
21 21  === DAQ control panel ===
22 22  
23 -To get to the User DAQ control one has to go to {{code language="none"}}Experiment control{{/code}} -> {{code language="none"}}User DAQ tab{{/code}} -> {{code language="none"}}FLxUSERy DAQ CTRL{{/code}}
23 +To get to the User DAQ control one has to go to {{code language="none"}}Experiment control{{/code}} -> {{code language="none"}}User DAQ tab{{/code}} -> {{code language="none"}}FLxUSERy DAQ CTRL {{/code}}
24 24  
25 -[[image:attach:jddd_DAQ-control-manual-how_to_get.png||height="400"]]
25 +[[image:attach:jddd_DAQ-control-manual-how_to_get.png||width="500"]][[image:attach:image2022-9-8_10-50-53.png||width="450"]]
26 26  \\The following window opens. Here the User DAQ can be started and stopped as well as the incoming data can be visualized, information about data rates and saved properties is presented as well as the panel is automatically printed in to the logbook:
27 27  
28 -[[image:attach:jddd_DAQ-control-manual2.png||height="400"]]
28 +[[image:attach:jddd_DAQ-control-manual2.png||width="800"]]
29 29  \\\\
30 30  
31 31  \\
32 32  
33 -* By pressing the start and stop button, besides sending the commands to the DAQ the button prints the panel to the respective beamline / experiments logbook. Here you can later on keep track of the runs. Links to the Beamline logbooks can be found [[here>>doc:FLASH.Electronic Logbooks]] (DESY internal link)
34 -* After starting, the stop button appears only after saving abut 200 shots to prevent too early stopping which may disturb the DAQ.
35 -* Each run has a unique run number. This number is needed to retrieve the data later on. So it is important to keep track which number is associated to what experimental parameters. Note that the run numbers are used by all FLASH DAQs (currently about 10). It might thus well be, that two consecutive runs from you do have run numbers which are spaced by more than one if another DAQ has been started in-between ...
33 +* By pressing the start and stop button, besides sending the commands to the DAQ the button prints the panel to the respective beamline / experiments logbook. Here you can later on keep track of the runs. Links to the Beamline logbooks can be found [[here>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.E-Logbooks.WebHome]].
34 +* here a sample entry in the **logbook**. The run number, "run comment" and several other parameters are printed as (searchable) text in addition with the screenshot of the DAQ control to get all relevant data for this run in the logbook: [[image:attach:image2022-9-8_10-57-56.png||thumbnail="true" height="250"]]
35 +* NOTE: After starting, the stop button appears only after saving abut 200 shots to prevent too early stopping which may disturb the DAQ.
36 +* NOTE: Each run has a unique run number. This number is needed to retrieve the data later on. So it is important to keep track which number is associated to what experimental parameters. Note that the run numbers are used by all FLASH DAQs (currently about 10). It might thus well be, that two consecutive runs from you do have run numbers which are spaced by more than one if another DAQ has been started in-between ...
36 36  
37 37  \\
38 38  
... ... @@ -81,7 +81,7 @@
81 81  
82 82  The DAQ Data GUI is a general tool to select and visualize data that was saved in the DAQ . It has in addition simple analysis options like histogram, mean, min or max values as function of time etc. Also correlations between different parameters can be analyzed in the tool. More details can be found [[How to Use the FLASH DAQ Data GUI?~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=FLASHDAQdataGUI||shape="rect"]] and [[ here (DAQdataGUI link collection) >>doc:FLASH.How to get the data out of the DAQ]]
83 83  
84 -The tool (separate Java program) can be started [[DAQdataGUI~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local (Desy) PC. In some cases the access via Windows does not work. An option which should always work is to log onto {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} and start {{code language="none"}}DAQdataGUI{{/code}} from the command line.
85 +The tool (separate Java program) can be started [[DAQdataGUI~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local user console (Linux or Windows) or log  onto the COntrol machines: {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} (there you also may start via  {{code language="none"}}DAQdataGUI{{/code}} from the command line).
85 85  
86 86  
87 87  \\To access the desired data one has to:
... ... @@ -99,16 +99,15 @@
99 99  The (% class="twikiNewLink" %)DAQdataGui(%%) is a powerful tool to visualize the saved data and to do very preliminary analysis. however for more detailed analysis the data has to be read into analysis programs ( matlab, python, Origin ...) here are different options one can use (what option is the best and how to set it up has to be discussed before hand with the FLASH DAQ experts ...)
100 100  
101 101  * The FLASH DAQ system records the data in binary " {{code language="none"}}.raw{{/code}} " files.
102 -** [[reading the data via Matlab >>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]] works not for all data types equally well and needs precise knowledge of the desired Parameters (useful to have a quick look / preliminary analysis on few parameters).
103 +** (% class="confluence-link" %)reading the .raw files [[directly via Python>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=PythonDAQClientInterface||shape="rect"]], or [[via Matlab >>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]](%%)(no longer recommended) works, yet not for all data types equally well, and needs precise knowledge of the desired Parameters (useful to have a quick look / preliminary analysis on few parameters).
103 103  ** Note that the raw data is only accessible from certain computers within the DESY network and thus not very handy for offline analysis later on ...
104 104  
105 105  \\
106 106  
107 107  * To ease the data analysis we IN ADDITION provide //ON REQUEST// the data from the FLASH DAQ as HDF5 format
108 -** ONLINE HDF5s: To make efficient use of the HDF5 files for analysis during the beamtime and afterwards we provide a [[very useful tool (Beamtime Daq Access)>>doc:FLASHUSER.Near-Online data analysis]]. Using this tool (working for Python and Matlab) one can get the data of parameters per run or for specific bunch ID intervals etc. without having to care from which subset of HDF5 files the data is read ...
109 -** Summary HDF5 files include complete runs. They will be put after the beamtime on the gpfs storage and are available from computers within DESY (for registered Beamtime participants) and also access
110 -** Intro to the [[HDF5 structure used at FLASH>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]]. This contains also a list of the most used Parameters saved at FLASH for analysis - including the HDF5 names, DOOCS and DAQ names ...
111 -** [[Short examples>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] and [[user provided examples>>doc:FLASH.Collection of user codes]] on how to read HDF5 in Matlab
109 +** ONLINE HDF5s: Intro to the [[HDF5 structure used at FLASH>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]]. This contains also a list of the most used Parameters saved at FLASH for analysis - including the HDF5 names, DOOCS and DAQ names ...
110 +** [[Examples>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]] and an [[API>>url:https://gitlab.desy.de/christopher.passow/fdh-builder||shape="rect"]] to make use of the online HDF5s
111 +** [[Short examples>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] and [[user provided examples>>doc:FLASH.OUTDATED - Collection of user codes]] on how to read HDF5 in Matlab (no longer recommended).
112 112  
113 113  The HDF5 files (online and summary) are saved in the [["gpfs" system ~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Architecture||shape="rect"]]. it can be accessed from [[Windows and Linux computers at DESY~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Mounting+the+storage+system+at+the+beamline||shape="rect"]] by the persons ( logins) which are registered for the beamtime in DOOR ( functional accounts
114 114  
... ... @@ -136,12 +136,16 @@
136 136  [[image:attach:RCGUI_2.jpg||height="400"]]
137 137  \\
138 138  
139 -Using the jddd DAQ control panel one can start and stop runs but in order to configure the DAQ which data to save one has to use a a separate *DAQ run control * java application. Since it needs write access to DOOCS internal file systems, which can not be made available elsewhere one has to start the system on our DOOCS control computers {{code language="none"}}flashlxuser1.desy.de, flashlxuser2.desy.de {{/code}} or {{code language="none"}}flashlxuser3.desy.de{{/code}}.
140 -Log in with the beamline account (bl1user, bl2user, ..., fl24user...) either via X-Win32 from windows or via ssh from Linux machines.
141 -Once logged in to {{code language="none"}}flashlxuser1,2,3{{/code}} the DAQ run control can be started. Depending on the DAQ you want to use the commands are:
142 -
143 143  \\
144 144  
141 +Using the jddd DAQ control panel one can start and stop runs but in order to configure the DAQ which data to save one has to use a a separate *DAQ run control * java application. Since it needs write access to DOOCS internal file systems, which can not be made available elsewhere one has to start the system on our linux user consoles at the beamline or on the  DOOCS control computers {{code language="none"}}flashlxuser1.desy.de, flashlxuser2.desy.de {{/code}} or {{code language="none"}}flashlxuser3.desy.de{{/code}}.
142 +Log in with the beamline account (bl1user, bl2user, ..., fl24user...) either via fastX3 or {{code language="none"}}Remote desktop{{/code}} from windows or via ssh from Linux machines.
143 +Once logged in to {{code language="none"}}flashlxuser1,2,3{{/code}} the DAQ run control can be started in the "advanced"  Tab of the DAQ control :
144 +
145 +[[image:attach:image2022-9-8_11-14-7.png||border="true" width="343" height="250"]]
146 +
147 +Or in the command line:
148 +
145 145  (% class="wrapped" %)
146 146  |(((
147 147  Unix command
... ... @@ -176,7 +176,16 @@
176 176  )))|(((
177 177  FLASH2_USER1 DAQ
178 178  )))
183 +|(% colspan="1" %)(((
184 +{{code language="none"}}
185 +RCGUIFL2USER2
186 +{{/code}}
187 +)))|(% colspan="1" %)(((
188 +FLASH2_USER2 DAQ
189 +)))
179 179  
191 +\\
192 +
180 180  Which DAQ to use is decided by the FLASH DAQ team according to the beamtime schedule.
181 181  
182 182  [[image:attach:RCGUI_properties.jpg||height="400"]]
... ... @@ -183,7 +183,7 @@
183 183  
184 184  \\
185 185  
186 -Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the 4 MHZ ADC channels available at BL1. To get a detailed information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>doc:FLASHUSER.Configuring the user DAQs]].
199 +Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the four  MHZ ADC channels available at BL1. To get a detailed information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>doc:FLASHUSER.Configuring the user DAQs]].
187 187  
188 188  \\
189 189