Skip to Content
Last modified by sndueste on 2025-09-08 15:36
From version 34.4
edited by sndueste
on 2025-02-05 16:53
Change comment: There is no comment for this version
To version 33.1
edited by sndueste
on 2024-03-14 15:13
Change comment: There is no comment for this version

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,0 @@
1 -FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).WebHome
Content
... ... @@ -31,6 +31,7 @@
31 31  * NOTE: After starting, the stop button appears only after saving abut 200 shots to prevent too early stopping which may disturb the DAQ.
32 32  * NOTE: Each run has a unique run number. This number is needed to retrieve the data later on. So it is important to keep track which number is associated to what experimental parameters. Note that the run numbers are used by all FLASH DAQs (currently about 10). It might thus well be, that two consecutive runs from you do have run numbers which are spaced by more than one if another DAQ has been started in-between ...
33 33  
34 +
34 34  === Checklist for proper saving of the data ===
35 35  
36 36  * Check that some 20 seconds after starting a run the DAQ control shows that the internal checks did not show errors -> the **DAQ traffic light is green** (... still there may be undetected problems !!!)
... ... @@ -69,9 +69,9 @@
69 69  [[image:attach:daqgui3.png||height="250"]][[image:attach:DAQgui2.jpg||height="250"]][[image:attach:DAQgui1.jpg||height="250"]]
70 70  
71 71  
72 -The DAQ Data GUI is a general tool to select and visualize data that was saved in the DAQ . It has in addition simple analysis options like histogram, mean, min or max values as function of time etc. Also correlations between different parameters can be analyzed in the tool. More details can be found [[How to Use the FLASH DAQ Data GUI?>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=FLASHDAQdataGUI||shape="rect"]] and [[ here (DAQdataGUI link collection) >>doc:FLASH.How to get the data out of the DAQ]]
73 +The DAQ Data GUI is a general tool to select and visualize data that was saved in the DAQ . It has in addition simple analysis options like histogram, mean, min or max values as function of time etc. Also correlations between different parameters can be analyzed in the tool. More details can be found [[How to Use the FLASH DAQ Data GUI?~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|height="12" width="13"~]~]>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=FLASHDAQdataGUI||shape="rect"]] and [[ here (DAQdataGUI link collection) >>doc:FLASH.How to get the data out of the DAQ]]
73 73  
74 -The tool (separate Java program) can be started [[DAQdataGUI>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local user console (Linux or Windows) or log  onto the Control machines: {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} (there you also may start via  {{code language="none"}}DAQdataGUI{{/code}} from the command line).
75 +The tool (separate Java program) can be started [[DAQdataGUI~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|height="12" width="13"~]~]>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local user console (Linux or Windows) or log  onto the COntrol machines: {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} (there you also may start via  {{code language="none"}}DAQdataGUI{{/code}} from the command line).
75 75  
76 76  
77 77  
... ... @@ -83,20 +83,22 @@
83 83  * browse through the data tree and select your important parameters and drag them with the mouse in the {{code language="none"}}selected{{/code}} window. The list of important parameters should have been defined before the experiment with the FLASH stuff together. This list can also be found in the offline monitor (lower part of DAQ control) )
84 84  * press {{code language="none"}}Start Display{{/code}} to open a new window with your data. In case it was a long run and / or a large amount of data per shot one can limit the number of shown events to a fraction: {{code language="none"}}Events -> Reading Options -> Event Interval{{/code}} provides the option to show only every //n// th dataset.
85 85  
87 +
86 86  === How to get the data out for analysis (short summary) ===
87 87  
88 88  The (% class="twikiNewLink" %)DAQdataGui(%%) is a powerful tool to visualize the saved data and to do very preliminary analysis. however for more detailed analysis the data has to be read into analysis programs ( matlab, python, Origin ...) here are different options one can use (what option is the best and how to set it up has to be discussed before hand with the FLASH DAQ experts ...)
89 89  
90 90  * The FLASH DAQ system records the data in binary " {{code language="none"}}.raw{{/code}} " files.
91 -** (% class="confluence-link" %)reading the .raw files [[directly via Python>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=PythonDAQClientInterface||shape="rect"]], or [[via Matlab >>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]](%%)(no longer recommended) works, yet not for all data types equally well, and needs precise knowledge of the desired Parameters (useful to have a quick look / preliminary analysis on few parameters).
93 +** (% class="confluence-link" %)reading the .raw files [[directly via Python>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=PythonDAQClientInterface||shape="rect"]], or [[via Matlab >>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]](%%)(no longer recommended) works, yet not for all data types equally well, and needs precise knowledge of the desired Parameters (useful to have a quick look / preliminary analysis on few parameters).
92 92  ** Note that the raw data is only accessible from certain computers within the DESY network and thus not very handy for offline analysis later on ...
93 93  
96 +
94 94  * To ease the data analysis we IN ADDITION provide //ON REQUEST// the data from the FLASH DAQ as HDF5 format
95 -** ONLINE HDF5s: Intro to the [[HDF5 structure used at FLASH>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]]. This contains also a list of the most used Parameters saved at FLASH for analysis - including the HDF5 names, DOOCS and DAQ names ...
98 +** ONLINE HDF5s: Intro to the [[HDF5 structure used at FLASH>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]]. This contains also a list of the most used Parameters saved at FLASH for analysis - including the HDF5 names, DOOCS and DAQ names ...
96 96  ** [[Examples>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]] and an [[API>>url:https://gitlab.desy.de/christopher.passow/fdh-builder||shape="rect"]] to make use of the online HDF5s
97 -**
100 +** [[Short examples>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] and [[user provided examples>>doc:FLASH.OUTDATED - Collection of user codes]] on how to read HDF5 in Matlab (no longer recommended).
98 98  
99 -The HDF5 files (online and summary) are saved in the [["gpfs" system >>https://docs.desy.de/asap3/||shape="rect"]]. it can be accessed from Windows and Linux computers at DESY by the persons ( logins) which are registered for the beamtime in DOOR ( functional accounts
102 +The HDF5 files (online and summary) are saved in the [["gpfs" system ~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|height="12" width="13"~]~]>>url:https://confluence.desy.de/display/ASAP3/Architecture||shape="rect"]]. it can be accessed from [[Windows and Linux computers at DESY~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|height="12" width="13"~]~]>>url:https://confluence.desy.de/display/ASAP3/Mounting+the+storage+system+at+the+beamline||shape="rect"]] by the persons ( logins) which are registered for the beamtime in DOOR ( functional accounts
100 100  
101 101  The path to your data is structured the following way:
102 102  
... ... @@ -169,6 +169,7 @@
169 169  FLASH2_USER2 DAQ
170 170  )))
171 171  
175 +
172 172  Which DAQ to use is decided by the FLASH DAQ team according to the beamtime schedule.
173 173  
174 174  [[image:attach:RCGUI_properties.jpg||height="400"]]
... ... @@ -278,16 +278,20 @@
278 278  * in several cases the reason is a configuration problem of the server. Then 2 servers are sending on the same "address". Then one property usually is saved without loss while the second one is only saved once in a while ( 1-5%). Here the server has to be fixed. As fast cure one can only exclude the less important subsystem and save the more important properly ...
279 279  
280 280  
285 +
286 +
281 281  === Restarting the DAQ ===
282 282  
283 283  
284 284  * If the RCGUI does not respond (and also the clock display does not count up) one has to restart the GUI. Usually the DAQ is not affected by the RCGUI hang up ...
285 285  
292 +
286 286  * Generally there** 3 ways to "restart" the DAQ**
287 287  ** 1. The least interfering way is the "**all to INITIALIZE**" that tries to reset the DAQ processes ... helps for simple "hickups"
288 288  ** 2. The "**re-start DAQ**" really reboots the whole DAQ process. The settings still should be as before ... this is the "Usual" restart
289 289  ** 3. In very hard and severe cases there is the **ultimate restart button: "re-start DAQ (no RCSHM)"**
290 290  
298 +
291 291  //**Here in more detail:**//
292 292  
293 293  
... ... @@ -303,7 +303,7 @@
303 303  ==== **all to INITIALIZE** ====
304 304  
305 305  
306 -*
314 +*
307 307  ** First, one may try the "**all to INITIALIZE"** button. It starts a script window showing the progress of the initialization. The script sets all subcomponents to initialize. It is helpful if one subcomponent got stuck.
308 308  ** If there is no window coming up or closing right away there may be a problem with the rights. In this case you:
309 309  *** may try to login at the flashlxuser machines and try there again
... ... @@ -334,8 +334,11 @@
334 334  ** goto "Subsystems" tab and load your Whitelist
335 335  ** and start a run ...
336 336  
345 +
337 337  * If there are still components not working after the "re-start DAQ (no RCSHM)" you have to call the photon coordinator (4455) and pray ...
338 338  
348 +
349 +
339 339  ----
340 340  
341 341  == [[Some more details for experts ...>>doc:FLASH.Collection of Detailsof the DAQ from various times \.\.\. not for public use]] ==