Changes for page How to save data in the User DAQ

Last modified by sndueste on 2025-09-08 15:36

From 26.1 to 27.1 From 30.1 to 31.1

From version 27.1

edited by erlandm
on 2021-06-17 15:46

Change comment: There is no comment for this version

To version 30.1

edited by sndueste
on 2022-09-09 11:41

Change comment: There is no comment for this version

Raw
Rendered

Summary

Page properties (2 modified, 0 added, 0 removed)

Details

Page properties

Author

@@ -1,1 +1,1 @@
--XWiki.erlandm
++XWiki.sndueste

Content

@@ -6,8 +6,8 @@
  == General Concept ==
--A large number of parameters which are essential for ALL Experiments is continuously saved (pulse energy, arrival time, Beamline settings ...) in the [[ PHOTONDIAGNOSTIC FLASH DAQ>>doc:FLASHUSER.FLASH1 PhotDiag stream]] system.
--Other [[experiment related information>>doc:FLASHUSER.Configuring the user DAQs]] is ONLY saved by the users ON DEMAND in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and one at FLASH 2 (FLASH2_USER1)
++A large number of parameters which are essential for ALL Experiments is continuously saved (pulse energy, arrival time, Beamline settings ...) in the [[ PHOTONDIAGNOSTIC FLASH DAQ>>doc:FLASH.FLASH1 PhotDiag stream]] system.
++Other [[experiment related information>>doc:FLASH.Configuring the user DAQs]] is ONLY saved by the users **ON DEMAND **in the FLASH DAQ system. To record this type of data we have 3 User DAQs at FLASH1 (FLASH1_USER1,FLASH1_USER2 and FLASH1_USER3) and two at FLASH 2 (FLASH2_USER1, FLASH2_USER2)
  How to save this experiment related "User-Data" will be explained briefly in this page:
  \\
@@ -20,19 +20,20 @@
  === DAQ control panel ===
--To get to the User DAQ control one has to go to {{code language="none"}}Experiment control{{/code}} -> {{code language="none"}}User DAQ tab{{/code}} -> {{code language="none"}}FLxUSERy DAQ CTRL{{/code}}
++To get to the User DAQ control one has to go to {{code language="none"}}Experiment control{{/code}} -> {{code language="none"}}User DAQ tab{{/code}} -> {{code language="none"}}FLxUSERy DAQ CTRL {{/code}}
--[[image:attach:jddd_DAQ-control-manual-how_to_get.png||height="400"]]
++[[image:attach:jddd_DAQ-control-manual-how_to_get.png||width="500"]][[image:attach:image2022-9-8_10-50-53.png||width="450"]]
  \\The following window opens. Here the User DAQ can be started and stopped as well as the incoming data can be visualized, information about data rates and saved properties is presented as well as the panel is automatically printed in to the logbook:
--[[image:attach:jddd_DAQ-control-manual2.png||height="400"]]
++[[image:attach:jddd_DAQ-control-manual2.png||width="800"]]
  \\\\
  \\
--* By pressing the start and stop button, besides sending the commands to the DAQ the button prints the panel to the respective beamline / experiments logbook. Here you can later on keep track of the runs. Links to the Beamline logbooks can be found [[here>>doc:FLASH.Electronic Logbooks]] (DESY internal link)
--* After starting, the stop button appears only after saving abut 200 shots to prevent too early stopping which may disturb the DAQ.
--* Each run has a unique run number. This number is needed to retrieve the data later on. So it is important to keep track which number is associated to what experimental parameters. Note that the run numbers are used by all FLASH DAQs (currently about 10). It might thus well be, that two consecutive runs from you do have run numbers which are spaced by more than one if another DAQ has been started in-between ...
++* By pressing the start and stop button, besides sending the commands to the DAQ the button prints the panel to the respective beamline / experiments logbook. Here you can later on keep track of the runs. Links to the Beamline logbooks can be found [[here>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.E-Logbooks.WebHome]].
++* here a sample entry in the **logbook**. The run number, "run comment" and several other parameters are printed as (searchable) text in addition with the screenshot of the DAQ control to get all relevant data for this run in the logbook: [[image:attach:image2022-9-8_10-57-56.png||thumbnail="true" height="250"]]
++* NOTE: After starting, the stop button appears only after saving abut 200 shots to prevent too early stopping which may disturb the DAQ.
++* NOTE: Each run has a unique run number. This number is needed to retrieve the data later on. So it is important to keep track which number is associated to what experimental parameters. Note that the run numbers are used by all FLASH DAQs (currently about 10). It might thus well be, that two consecutive runs from you do have run numbers which are spaced by more than one if another DAQ has been started in-between ...
  \\
@@ -81,7 +81,7 @@
  The DAQ Data GUI is a general tool to select and visualize data that was saved in the DAQ . It has in addition simple analysis options like histogram, mean, min or max values as function of time etc. Also correlations between different parameters can be analyzed in the tool. More details can be found [[How to Use the FLASH DAQ Data GUI?~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=FLASHDAQdataGUI||shape="rect"]] and [[ here (DAQdataGUI link collection) >>doc:FLASH.How to get the data out of the DAQ]]
--The tool (separate Java program) can be started [[DAQdataGUI~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local (Desy) PC. In some cases the access via Windows does not work. An option which should always work is to log onto {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} and start {{code language="none"}}DAQdataGUI{{/code}} from the command line.
++The tool (separate Java program) can be started [[DAQdataGUI~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/DAQdataGUI-experimental.jnlp||shape="rect"]] or in the DAQ control panel ( lower right) on your local user console (Linux or Windows) or log  onto the COntrol machines: {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}}  (there you also may start via  {{code language="none"}}DAQdataGUI{{/code}} from the command line).
  \\To access the desired data one has to:
@@ -135,12 +135,16 @@
  [[image:attach:RCGUI_2.jpg||height="400"]]
  \\
--Using the jddd DAQ control panel one can start and stop runs but in order to configure the DAQ which data to save one has to use a a separate *DAQ run control * java application. Since it needs write access to DOOCS internal file systems, which can not be made available elsewhere one has to start the system on our DOOCS control computers {{code language="none"}}flashlxuser1.desy.de, flashlxuser2.desy.de {{/code}} or {{code language="none"}}flashlxuser3.desy.de{{/code}}.
--Log in with the beamline account (bl1user, bl2user, ..., fl24user...) either via X-Win32 from windows or via ssh from Linux machines.
--Once logged in to {{code language="none"}}flashlxuser1,2,3{{/code}}  the DAQ run control can be started. Depending on the DAQ you want to use the commands are:
--
  \\
++Using the jddd DAQ control panel one can start and stop runs but in order to configure the DAQ which data to save one has to use a a separate *DAQ run control * java application. Since it needs write access to DOOCS internal file systems, which can not be made available elsewhere one has to start the system on our linux user consoles at the beamline or on the  DOOCS control computers {{code language="none"}}flashlxuser1.desy.de, flashlxuser2.desy.de {{/code}} or {{code language="none"}}flashlxuser3.desy.de{{/code}}.
++Log in with the beamline account (bl1user, bl2user, ..., fl24user...) either via fastX3 or {{code language="none"}}Remote desktop{{/code}} from windows or via ssh from Linux machines.
++Once logged in to {{code language="none"}}flashlxuser1,2,3{{/code}}  the DAQ run control can be started in the "advanced"  Tab of the DAQ control :
++
++[[image:attach:image2022-9-8_11-14-7.png||border="true" width="343" height="250"]]
++
++Or in the command line:
++
  (% class="wrapped" %)
  |(((
  Unix command
@@ -175,7 +175,16 @@
  )))|(((
  FLASH2_USER1 DAQ
  )))
++|(% colspan="1" %)(((
++{{code language="none"}}
++RCGUIFL2USER2
++{{/code}}
++)))|(% colspan="1" %)(((
++FLASH2_USER2 DAQ
++)))
++\\
++
  Which DAQ to use is decided by the FLASH DAQ team according to the beamtime schedule.
  [[image:attach:RCGUI_properties.jpg||height="400"]]
@@ -182,7 +182,7 @@
  \\
--Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the 4 MHZ ADC channels available at BL1. To get a detailed information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>doc:FLASHUSER.Configuring the user DAQs]].
++Using the {{code language="none"}}Add{{/code}} and {{code language="none"}}Rem(ove){{/code}} buttons one can move the desired "subsystems" in the included side - meaning that they are saved. The "subsystems" contain typically several DOOCS parameters. E.g. {{code language="none"}}EXPERIMENT_MHZ_ADC_BL1{{/code}} contains all relevant DOOCS properties of the four  MHZ ADC channels available at BL1. To get a detailed information about the saved parameters one can have a look to the "Show Properties in Subsystems" button . In addition, a list of all possible subsystems and their description can be found [[here>>doc:FLASH.Configuring the user DAQs]].
  \\
@@ -199,9 +199,9 @@
  \\
--To START a run - meaning starting to save the experiment related data - one has to press the "start" button in the Run Control GUI. This opens a confirmation window asking if you really want to start the run ("in default mode"). Press yes. Now it takes about 20 seconds for the system to start up and start recording data. Indication is the "Run Control: RUN" and "DAQ: RUN" greenish indicators on the upper left side of the run control.
++**To START** a run - meaning starting to save the experiment related data - one has to press the "start" button in the Run Control GUI. This opens a confirmation window asking if you really want to start the run ("in default mode"). Press yes. Now it takes about 20 seconds for the system to start up and start recording data. Indication is the "Run Control: RUN" and "DAQ: RUN" greenish indicators on the upper left side of the run control.
--To STOP a run one has to press the "stop" button. This opens a window for comments. This contains already the actual Experiment name by default. Please leave this information in. Add a comment for your run. This comment will be printed into the logbook and can later on be used to identify individual runs ... note you are required to type in something otherwise the run will not stop.
++**To STOP** a run one has to press the "stop" button. This opens a window for comments. This contains already the actual Experiment name by default. Please leave this information in. Add a comment for your run. This comment will be printed into the logbook and can later on be used to identify individual runs ... note you are required to type in something otherwise the run will not stop.
  The stopping also needs about 20 seconds.*_IT IS IMPORTANT_ not to start and stop the DAQ while it is still ramping up or down. So please wait before restarting the DAQ.*
@@ -211,7 +211,7 @@
  === Load/Save the configuration ===
--[[image:attach:whitelist.jpg||height="400"]]
++[[image:attach:whitelist.jpg||height="250"]]
  Once a list of subsystems is defined and tested this parameter combination should then be saved in a so-called //white list//. By loading this white list one can then easily configure the DAQ after changes / restarts ... to the initially defined settings to save the correct data.
  The whitelist can be saved/loaded with the Run Control GUI by: {{code language="none"}}File -> Group White List -> Save/Load{{/code}}. Note, while the top left rectangle's background of time and date is yellow, don't try any further controls of the GUI, you might confuse the program. It is busy reading the configuration from the run control database. The background changes back to grey when it has finished.
@@ -252,11 +252,13 @@
  To assign which DAQ is used at what beamline one has to use the dropdown menu in the {{code language="none"}}parameter monitor{{/code}}. In each Beamline overview tab (BL1,2,3) there is a button (upper right) that starts the {{code language="none"}}parameter monitor{{/code}} panel. In the upper middle is a DAQ field in which the appropriate DAQ can be assigned to the Beamline.
++[[image:attach:image2022-9-9_9-51-8.png||height="150"]]
++
  \\
  === Setting the logbook to print in / putting the Whitelist name in jddd ===
--The jddd DAQ Control should be set up that it prints by default the relevant run information in the logbook associated to your beamline [[Logbooks>>doc:FLASH.Electronic Logbooks]]
++The jddd DAQ Control should be set up that it prints by default the relevant run information in the logbook associated to your beamline [[Logbooks>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.E-Logbooks.WebHome]]
  Define logbook / and whitelist for display in the jddd DAQ control (jddd otherwise does not "know" which White list was used ... this has to be put in by hand ...up to now)
@@ -268,11 +268,9 @@
  == In case something goes wrong: Troubleshooting ==
--\\
--
  === No //START// or //STOP// buttons are visible in the DAQ control panel ===
--~* The jddd GUI only shows the Start and stop button if the DAQ is in the "Idle" state. sometimes the DAQ "just" gets stuck in the stopping phase. Then stopping again or starting form the not "Idle" state can help. For this goto the "Advanced" tab of the jddd DAQ control . there the START and STOP buttons are always available ... independent of the state of the DAQ. If this does not help you may have to restart the DAQ ...
++The jddd GUI only shows the Start and stop button if the DAQ is in the "Idle" state. Sometimes the DAQ "just" gets stuck in the stopping phase. Then stopping again or starting form the not "Idle" state can help. For this goto the "Advanced" tab of the jddd DAQ control . there the START and STOP buttons are always available ... independent of the state of the DAQ. If this does not help you may have to restart the DAQ ...
  \\
@@ -280,9 +280,9 @@
  \\
--* Mainly the DAQ works fine but one of the properties that should be saved is not available or somehow corrupted. There the {{code language="none"}}RC logfile{{/code}} helps (see below) to determine where to localize the problem. The pragmatic way is to exclude the subsystems which may contain the properties with problems. Thus exclude the "critical" subsystems. Start a run. If this works, include one by one the excluded subsystems and start runs. That way one should find out which subsystem causes the problem. Now one can investigate this one in more detail. Is the property available in DOOCS? Does it update? maybe restart the server providing the property ...
++* Mainly the DAQ works fine but one of the properties that should be saved is not available or somehow corrupted. There the {{code language="none"}}RC logfile{{/code}} helps (see below) to determine where to localize the problem. The pragmatic way is to exclude the subsystems which may contain the properties with problems. Thus exclude the "critical" subsystems. Start a run (starting with the RCGUI not with jddd DAQ control ). If this works, include one by one the excluded subsystems and start runs. That way one should find out which subsystem causes the problem. Now one can investigate this one in more detail. Is the property available in DOOCS? Does it update? maybe restart the server providing the property ...
--\\
++        as example: [[image:url:https://ttfinfo.desy.de/FELPGMelog/data/2017/15/11.04/2017-04-11T02:56:05-00.PNG||height="250"]]
  \\
@@ -313,10 +313,10 @@
  \\
--* Generally there 3 ways to "restart" the DAQ
--** 1. The least interfering way is the "all to INITIALIZE" that tries to reset the DAQ processes ... helps for simple "hickups"
--** 2. The "re-start DAQ" really reboots the whole DAQ process. The settings still should be as before ... this is the "Usual" restart
--** 3. In very hard and severe cases there is the ultimate restart button: "re-start DAQ (no RCSHM)"
++* Generally there** 3 ways to "restart" the DAQ**
++** 1. The least interfering way is the "**all to INITIALIZE**" that tries to reset the DAQ processes ... helps for simple "hickups"
++** 2. The "**re-start DAQ**" really reboots the whole DAQ process. The settings still should be as before ... this is the "Usual" restart
++** 3. In very hard and severe cases there is the **ultimate restart button: "re-start DAQ (no RCSHM)"**
  \\//**Here in more detail:**//
@@ -324,47 +324,52 @@
  \\
  * if the DAQ gets stuck in the starting or stopping procedure one can try to initialize the DAQ or in the worst case restart it.
--** **to be able to restart the DAQ one has to run jddd on the {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}} (up to now ( 2021) the restat on our consoles (cons0...12) is NOT reliably possible ) : log in at one of these machines with your beamline account (e.g. bl1user or fl24user). Start jddd with the {{code language="none"}}flash{{/code}} command. Click on "Photons" -> Experimental Hall "Photons main panel"**
--** The DAQ "expert" panel can be reached from the User DAQ control panel.
++** **to be able to restart the DAQ one has to run jddd on Linux in the DESY network, meaning wither on our User consoles in the Experimental halls (cons0...12)  or on  {{code language="none"}}flashlxuser1,flashlxuser2{{/code}} or {{code language="none"}}flashlxuser3{{/code}}. Log in at one of these machines with your beamline account (e.g. bl1user or fl24user). **(On the flashlxuser machines there are no desktop icons to start jddd. Here you can use  the  shell command: {{code language="none"}}flash ->{{/code}} Click on "Photons" -> "Photons main panel" )
++** To Restart the DAQs you need the **"expert" Tab** of the  User DAQ control panel.
--               [[image:attach:DAQ_expert.jpg||height="400"]]
++               [[image:attach:image2022-9-9_10-57-31.png||height="400"]]
  \\
--*
--** Then you see the following "DAQ expert panel":
--** first one may try the "all to INITIALIZE" button. it starts a script window and asks for the passwd (the beamline accounts (blxuser) should have the according rights. vuvfuser has it for sure. The script sets all subcomponents to initialize. It is helpful if one subcomponent got stuck.
++There is some description of the different ways to restart the DAQ in the panel:
--               [[image:attach:UserDAQ_status.jpg||height="400"]]
--\\
++==== **all to INITIALIZE** ====
  \\
--* If this does not help you have to "re-start DAQ". This completely restarts all processes and takes a while. A script window appears and shows what steps are actually done. At the end is sais: "Cleanup finished".
++*
++** First, one may try the "**all to INITIALIZE"** button. It starts a script window showing the progress of the initialization. The script sets all subcomponents to initialize. It is helpful if one subcomponent got stuck.
++** If there is no window coming up or closing right away there may be a problem with the rights. In this case you:
++*** may try to login at the flashlxuser machines and try there again
++*** or use the red "DAQ restart" Button one below. This button reacts technically different and can circumvent rights problems ...
--     [[image:attach:DAQ_restart_msg.jpg||height="400"]]
++
++        Initialize script: [[image:attach:image2022-9-9_11-6-38.png||height="250"]]  parts of the DAQ are in different "states":[[image:url:https://ttfinfo.desy.de/Flash_THzStreakingelog/data/2020/49/2020-12-02T13:53:01.jpg||height="250"]]
  \\
--* For the "re-start DAQ" the DAQ should come up in the same state as before. Meaning that the subsystems (ADCs, cameras ..) are still loaded and displayed. If this is not the case you can try to load your white list and start a new run. If the data is saved correctly you are done.
++==== Restart the DAQ ====
  \\
--* **IF the DAQ still does not work properly** the final option is the: "re-start DAQ (no RCSHM)". This button does the same as the "re-start DAQ" but the Run Controller shared memory is **not** re-stored. It means that after the DAQ is back no run configuration is selected.
--** in the jddd control the list of subsystems sais "empty"
--** in the RCGUI in the lower part it sais "RUN mode is not set"
++* If the "initialize"  does not help,  you have to "re-start DAQ". This completely restarts all processes and takes a while. A script window appears and shows what steps are actually done. At the end is sais: "Cleanup finished".
++* There are **actually 2 buttons** to "restart" the DAQ. They start the same restarting script - however in different ways. Thus, if one for what ever reason does not work with your configuration you may use the second one ...
--               [[image:attach:restart_RCM_no_runmode.jpg||height="400"]]
--\\
++    Script window:  [[image:attach:DAQ_restart_msg.jpg||height="400"]]  Restart:  [[image:attach:image2022-9-9_11-30-33.png||height="400"]]
  \\
--*
--** one has to go to the "Run Modes" tab and select the run mode {{code language="none"}}FLASH1_EXP{{/code}} by clicking on it and choose the desired file size (stored settings)
--** then one has to go back to the {{code language="none"}}Subsystems{{/code}} . initially all subsystems are included.
--** finally you have to select your White list and start a run ...
++* For the "re-start DAQ" the DAQ should come up in the same state as before. Meaning that the subsystems (ADCs, cameras ..) are still loaded and displayed. If this is not the case you can try to load your white list and start a new run. If the data is saved correctly you are done.
--\\
++==== "Ultimate" restart ====
++* **IF the DAQ still does not work properly** the final option is the: " "Ultimate" re-start DAQ (no RCSHM)". This button does the same as the "re-start DAQ" but the Run Controller shared memory is **not** re-stored. It means that after the DAQ is back no run configuration is selected.
++** in the jddd control the list of subsystems sais "empty" [[image:attach:image2022-9-9_11-36-1.png||thumbnail="true" height="250"]]
++** and you have to reload the parameters to be saved from the  Whitelist.
++** you can use the "Runcontrol"  button right below the  "Ultimate" re-start DAQ (no RCSHM)" to start the RunControl window
++** choose the files ize ( best default: 1GB)[[image:attach:image2022-9-9_11-39-3.png||thumbnail="true" height="250"]]
++** goto "Subsystems" tab and load your Whitelist
++** and start a run ...
++
  \\
  * If there are still components not working after the "re-start DAQ (no RCSHM)" you have to call the photon coordinator (4455) and pray ...

Changes for page How to save data in the User DAQ

Summary

Details

Applications

Navigation