Skip to Content
Last modified by sndueste on 2025-09-08 15:36
From version 5.1
edited by sendels
on 2019-05-15 12:53
Change comment: There is no comment for this version
To version 7.1
edited by sendels
on 2019-05-15 13:07
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -25,8 +25,8 @@
25 25  
26 26  == General Concept ==
27 27  
28 -A large number of parameters which are essential for ALL Experiments is continuously saved (pulse energy, arrival time, Beamline settings ...) in the [[ PHOTONDIAGNOSTIC FLASH DAQ>>doc:FLASHUSER.PBD DAQ]] system.
29 -Other [[experiment related information>>doc:FLASHUSER.Description of the Properties in the FLASH1_USER1, 2 and 3 for FLASH1 Users]] is ONLY saved by the users ON DEMAND in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and one at FLASH 2 (FLASH2_USER1)
28 +A large number of parameters which are essential for ALL Experiments is continuously saved (pulse energy, arrival time, Beamline settings ...) in the [[ PHOTONDIAGNOSTIC FLASH DAQ>>doc:FLASHUSER.FLASH1 PhotDiag stream]] system.
29 +Other [[experiment related information>>doc:FLASHUSER.Configuring the user DAQs]] is ONLY saved by the users ON DEMAND in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and one at FLASH 2 (FLASH2_USER1)
30 30  How to save this experiment related "User-Data" will be explained briefly in this page:
31 31  
32 32  \\
... ... @@ -117,16 +117,16 @@
117 117  
118 118  * The FLASH DAQ system records the data in binary " {{code language="none"}}.raw{{/code}} " files.
119 119  ** [[reading the data via Matlab >>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]] works not for all data types equally well and needs precise knowledge of the desired Parameters (useful to have a quick look / preliminary analysis on few parameters).
120 -** Examples how to read from {{code language="none"}}.raw{{/code}} files can also be found in [[the user code collection>>url:http://hasfweb.desy.de/bin/view/Setup/UserCodeCollection||shape="rect" class="twikiLink"]]
120 +** Examples how to read from {{code language="none"}}.raw{{/code}} files can also be found in [[the user code collection>>doc:FLASHUSER.Collection of user codes]]
121 121  ** Note that the raw data is only accessible from certain computers within the DESY network and thus not very handy for offline analysis later on ...
122 122  
123 123  \\
124 124  
125 125  * To ease the data analysis we IN ADDITION provide //ON REQUEST// the data from the FLASH DAQ as HDF5 format
126 -** ONLINE HDF5s: To make efficient use of the HDF5 files for analysis during the beamtime and afterwards we provide a [[very useful tool (Beamtime Daq Access)>>url:http://hasfweb.desy.de/bin/view/Setup/BeamtimeDaqAccess||shape="rect" class="twikiLink"]]. Using this tool (working for Python and Matlab) one can get the data of parameters per run or for specific bunch ID intervals etc. without having to care from which subset of HDF5 files the data is read ...
126 +** ONLINE HDF5s: To make efficient use of the HDF5 files for analysis during the beamtime and afterwards we provide a [[very useful tool (Beamtime Daq Access)>>doc:FLASHUSER.Near-Online data analysis]]. Using this tool (working for Python and Matlab) one can get the data of parameters per run or for specific bunch ID intervals etc. without having to care from which subset of HDF5 files the data is read ...
127 127  ** Summary HDF5 files include complete runs. They will be put after the beamtime on the gpfs storage and are available from computers within DESY (for registered Beamtime participants) and also access
128 128  ** Intro to the [[HDF5 structure used at FLASH>>url:http://hasfweb.desy.de/bin/view/Setup/HDF5main||shape="rect" class="twikiLink"]]. This contains also a list of the most used Parameters saved at FLASH for analysis - including the HDF5 names, DOOCS and DAQ names ...
129 -** [[Short examples>>url:http://hasfweb.desy.de/bin/view/Setup/HDF5main#Samples_how_to_read_HDF5_with_Ma||shape="rect" class="twikiAnchorLink"]] and [[user provided examples>>url:http://hasfweb.desy.de/bin/view/Setup/UserCodeCollection||shape="rect" class="twikiLink"]] on how to read HDF5 in Matlab
129 +** [[Short examples>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] and [[user provided examples>>doc:FLASHUSER.Collection of user codes]] on how to read HDF5 in Matlab
130 130  
131 131  The HDF5 files (online and summary) are saved in the [["gpfs" system ~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Architecture||shape="rect"]]. it can be accessed from [[Windows and Linux computers at DESY~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Mounting+the+storage+system+at+the+beamline||shape="rect"]] by the persons ( logins) which are registered for the beamtime in DOOR ( functional accounts
132 132  
... ... @@ -135,7 +135,7 @@
135 135  * /asap3/flash/gpfs/ //beamline used// / //year// /data/ //Beamtime application number// /raw/hdf/ (the //Beamtime application number// can be fund in the pdf schedule sent around by Rolf ...)
136 136  * as example: {{code language="none"}}/asap3/flash/gpfs/bl2/2017/data/11003163/raw/hdf/{{/code}}
137 137  
138 -More details and links can be found in the [[Summary of DAQ / control options>>url:http://hasfweb.desy.de/bin/view/Setup/Index||shape="rect" class="twikiLink"]].
138 +More details and links can be found in the [[doc:FLASHUSER.User overview]].
139 139  
140 140  \\
141 141  
... ... @@ -203,7 +203,7 @@
203 203  
204 204  \\
205 205  
206 -Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the 4 MHZ ADC channels available at BL1. To get a detailled information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>url:http://hasfweb.desy.de/bin/view/Setup/DAQExp2Description||shape="rect" class="twikiLink"]].
206 +Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the 4 MHZ ADC channels available at BL1. To get a detailled information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>doc:FLASHUSER.Configuring the user DAQs]].
207 207  
208 208  \\
209 209  
... ... @@ -269,7 +269,7 @@
269 269  
270 270  === Setting the logbook to print in / putting the Whitelist name in jddd ===
271 271  
272 -The jddd DAQ Control should be set up that it prints by default the relevant run information in the logbook associated to your beamline [[Logbooks>>url:http://hasfweb.desy.de/bin/view/Setup/ELogs||shape="rect" class="twikiLink"]]
272 +The jddd DAQ Control should be set up that it prints by default the relevant run information in the logbook associated to your beamline [[Logbooks>>doc:FLASHUSER.Electronic Logbooks]]
273 273  
274 274  Define logbook / and whitelist for display in the jddd DAQ control (jddd otherwise does not "know" which White list was used ... this has to be put in by hand ...up to now)
275 275  [[image:url:http://hasfweb.desy.de/pub/Setup/DAQjddd/setting_parameters.jpg||alt="setting_parameters.jpg" width="500" height="300"]]
... ... @@ -305,7 +305,7 @@
305 305  === Data seems not properly saved ===
306 306  
307 307  
308 -[[image:url:http://hasfweb.desy.de/pub/Setup/DAQjddd/RC_logfile.jpg||alt="RC_logfile.jpg" width="518" height="300"]]When checking the saved run with the (% class="twikiNewLink" %)[[DAQdataGUI>>url:http://hasfweb.desy.de/bin/edit/Setup/DAQdataGUI?topicparent=Setup.DAQjddd;nowysiwyg=0||rel="nofollow" shape="rect"]](%%) it may happen that the property shows up in the data tree but no events are saved. Then (most likely) something is wrong with the sending of the data.
308 +[[image:url:http://hasfweb.desy.de/pub/Setup/DAQjddd/RC_logfile.jpg||alt="RC_logfile.jpg" width="518" height="300"]]When checking the saved run with the (% class="twikiNewLink" %)DAQdataGUI(%%) it may happen that the property shows up in the data tree but no events are saved. Then (most likely) something is wrong with the sending of the data.
309 309  
310 310  * A very helpful tool is the {{code language="none"}}RC logfile{{/code}}. in the RCGUI on the lower right side there is a button to start an editor with the log file. besides lots of standard entries there is usually also a hint on where to find the problem ...
311 311  * check if the device is working properly. Is it connected? Can you see data in jddd / DOOCS ? . If not you have to fix the DOOCS part ... DAQ can only save what is properly implemented in DOOCS.