Changes for page DAQ and data analysis

Last modified by sndueste on 2025/02/05 14:56
From version 20.1
edited by sndueste
on 2020/01/07 13:03
Change comment: There is no comment for this version
To version 13.1
edited by sndueste
on 2019/08/08 11:50
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -2,9 +2,22 @@
2 2  
3 3  == Contents ==
4 4  
5 +\\
5 5  
7 +* Control and DAQ at FLASH (web version)
8 +** [[Overview>>doc:||anchor="Overview"]]
9 +** [[Things to consider before the beamtime>>doc:||anchor="Things to consider before the beamtime"]]
10 +** [[Beamline control and photon diagnostics>>doc:||anchor="Beamline control and photon diagnostics"]]
11 +** [[IT infrastructure>>doc:||anchor="IT infrastructure"]]
12 +** [[Data access>>doc:||anchor="Data access"]]
13 +** [[Online>>doc:||anchor="Online"]]
14 +** [[Data acquisition>>doc:||anchor="Data acquisition"]]
15 +** [[nearOnline>>doc:||anchor="nearOnline"]]
16 +*** [[.raw files>>doc:||anchor=".raw files"]]
17 +*** [[HDF5 files>>doc:||anchor="HDF5 files"]]
18 +** [[Offline>>doc:||anchor="Offline"]]
6 6  
7 -{{toc/}}
20 +\\
8 8  
9 9  \\
10 10  
... ... @@ -14,7 +14,7 @@
14 14  
15 15  [[image:attach:overview_flash_control_and_daq.png]]
16 16  
17 -The FLASH accelerator, beamlines and experiments are using the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://doocs-web.desy.de/index.html||shape="rect"]]). Beamline and experiment devices are mainly operated via graphical user interfaces created with the Java DOOCS Data Display ( [[jddd~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://jddd.desy.de/||shape="rect"]]) by the FLASH team or the users themself. Further it is provided a [[DOOCS client API~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=DOOCS%20Wiki||shape="rect"]] in Java, Python, Matlab, C or C++ to the users to write own software for data access, online data or device control. User-provided devices can be run in parallel or can be implemented in DOOCS. For each beamline and accelerator is a dedicated [[electronic logbook~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/elog/showElogs.jsp||shape="rect"]] available in which jddd and other software can print directly.
30 +The FLASH accelerator, beamlines and experiments are using the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://doocs-web.desy.de/index.html||shape="rect"]]). Beamline and experiment devices are mainly operated via graphical user interfaces created with the Java DOOCS Data Display ( [[jddd~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://jddd.desy.de/||shape="rect"]]) by the FLASH team or the users themself. Further it is provided a [[DOOCS client API~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=DOOCS%20Wiki||shape="rect"]] in Java, Python, Matlab, C or C++ to the users to write own software for data access, online data or device control. User-provided devices can be run in parallel or can be implemented in DOOCS. For each beamline and accelerator is a dedicated [[electronic logbook~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/ttf_apps/showElogs.jsp||shape="rect"]] available in which jddd and other software can print directly.
18 18  
19 19  The data acquisition system (DAQ) collects beamline and experimental data and writes them to hard disk. All data are stored along with a time stamp and, in case of train specific data, along with a (pulse) train ID. There are several ways to access beamline and experimental data stored in the DAQ during or after the beamtime be it programmatically via DAQ access libraries or after data conversion to HDF5 format. While the data transport to the user's home institution is possible, DESY hosts high power computing resources which can be used for analysis.
20 20  
... ... @@ -22,7 +22,7 @@
22 22  
23 23  == Things to consider before the beamtime ==
24 24  
25 -In consultation with the user's local contact the user should check our [[beamtime preparation checklist>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Before beamtime Checklist.WebHome]] and fill in the [[beamtime preparation questionnaire>>attach:Before beamtime questionnaire.pdf]].
38 +In consultation with the user's local contact the user should check our [[beamtime preparation checklist>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Before beamtime Checklist.WebHome]] and fill in the [[beamtime preparation questionnaire>>attach:Before_beamtime_questionnaire.pdf]].
26 26  
27 27  [[Contents>>doc:||anchor="Contents"]]
28 28  
... ... @@ -38,7 +38,7 @@
38 38  |=(((
39 39  [[DOOCS>>url:https://doocs-web.desy.de/index.html||shape="rect"]]
40 40  )))|=(((
41 -(% style="color: rgb(51,204,204);" %)[[url:https://jddd.desy.de/||style="color: rgb(51,204,204);" shape="rect"]](%%)jddd
54 +[[jddd>>url:https://jddd.desy.de/||shape="rect"]]
42 42  )))
43 43  
44 44  [[Contents>>doc:||anchor="Contents"]]
... ... @@ -120,7 +120,7 @@
120 120  
121 121  To monitor individual parameters online, e.g. ADCs or cameras, the use of jddd is recommended. For more complex tasks the users can use the DOOCS client API for [[Matlab>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Online data analysis.MatlabDOOCS.WebHome]], [[Python>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Online data analysis.PythonDOOCS.WebHome]] and [[Labview>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Online data analysis.LabviewDOOCS (no longer supported).WebHome]]. For accessing the control system [[DOOCS addresses>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] are required.
122 122  
123 -A collection (Matlab [ [[TOF & camera GUIs~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://stash.desy.de/projects/FSFL/repos/matlab-guis/browse||shape="rect"]]] / [[Juypter notebooks~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://stash.desy.de/projects/FSFL/repos/jupyter-notebook/browse||shape="rect"]]) for common use cases is available.
136 +A collection (Matlab [ [[scan tool>>doc:FLASHUSER.Matlab Scantools]] , [[TOF & camera GUIs~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://stash.desy.de/projects/FSFL/repos/matlab-guis/browse||shape="rect"]], [[others>>doc:FLASH.Collection of user codes]]] / [[Juypter notebooks~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://stash.desy.de/projects/FSFL/repos/jupyter-notebook/browse||shape="rect"]]) for common use cases is available.
124 124  
125 125  [[Contents>>doc:||anchor="Contents"]]
126 126  
... ... @@ -155,7 +155,7 @@
155 155  
156 156  ==== .raw files ====
157 157  
158 -The .raw files are only accessible from certain computers within the DESY network and in generell it is not recommended to use them directly. For checking the file content and doing simple analysis, e.g. histogram, line outs, we provide the [[DAQdataGUI>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=FLASHDAQdataGUI||shape="rect"]]. If the user already have very precise knowledge of the desired parameters and it's types it is possible [[to read with Matlab from the files directly>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]].
171 +The .raw files are only accessible from certain computers within the DESY network and in generell it is not recommended to use them directly. For checking the file content and doing simple analysis, e.g. histogram, line outs, we provide the [[DAQdataGUI>>doc:FLASH.How to get the data out of the DAQ]]. If the user already have very precise knowledge of the desired parameters and it's types it is possible [[to read with Matlab from the files directly>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).For experts\: many ways to get Data from the DAQ.How to Access the DAQ Using Matlab.WebHome]].
159 159  
160 160  FLASH's DAQ .raw files are saved locally and with a time delay of a few minutes backupped via tape ( [[dCache~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Managing+data+taken+before+April+2015||shape="rect"]]) by the DESY central IT.
161 161