Changes for page DAQ and data analysis

Last modified by sndueste on 2025/02/05 14:56
From version 8.1
edited by sendels
on 2019/06/25 14:53
Change comment: Saved some attachments
To version 10.1
edited by sndueste
on 2019/08/06 13:14
Change comment: There is no comment for this version

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -Summary of DAQ / control options
1 +User overview
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.sendels
1 +XWiki.sndueste
Content
... ... @@ -1,9 +7,3 @@
1 -\\
2 -
3 -\\
4 -
5 -= Control and DAQ at FLASH (web version) =
6 -
7 7  Before starting a beamtime at FLASH the users have to consider several options regarding the control and data acquisition system where the users are supported by [[the controls group~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://photon-science.desy.de/facilities/flash/beamlines/daq_and_controls/index_eng.html||shape="rect"]].
8 8  
9 9  == Contents ==
... ... @@ -19,9 +19,8 @@
19 19  ** Online
20 20  ** Data acquisition
21 21  ** nearOnline
22 -***
23 -**** .raw files
24 -**** HDF5 files
16 +*** .raw files
17 +*** HDF5 files
25 25  ** Offline
26 26  
27 27  \\
... ... @@ -46,7 +46,7 @@
46 46  
47 47  Use existing jddd panel provided by FLASH or use the jddd editor to create a new panel according to the user's own needs.
48 48  
49 -[[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jddd_logo.png~|~|alt="jddd Photon Beamline Controls" width="125" height="80"~]~]~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd_xml/run_jnlp.jsp?run=true&file=/svn/FLASH/Hasylab/Photons_beamline_control_main.xml||shape="rect"]][[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jdddEditor.png~|~|alt="jdddEditor.png" width="125"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd.jnlp?run=true||title="jddd editor" shape="rect"]](% class="twikiNewLink" %)[[url:http://hasfweb.desy.de/bin/edit/Ttfinfo2/desy/de/common/applications/jddd/Jnlp?topicparent=Setup.Index;nowysiwyg=0||rel="nofollow" shape="rect"]]
42 +[[ ~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd_xml/run_jnlp.jsp?run=true&file=/svn/FLASH/Hasylab/Photons_beamline_control_main.xml||shape="rect"]][[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jdddEditor.png~|~|alt="jdddEditor.png" width="125"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd.jnlp?run=true||title="jddd editor" shape="rect"]](% class="twikiNewLink" %)[[url:http://hasfweb.desy.de/bin/edit/Ttfinfo2/desy/de/common/applications/jddd/Jnlp?topicparent=Setup.Index;nowysiwyg=0||rel="nofollow" shape="rect"]]
50 50  
51 51  \\
52 52  
... ... @@ -59,10 +59,12 @@
59 59  
60 60  == IT infrastructure ==
61 61  
62 -During a beamtime at FLASH we have two IT infrastructures each with different purpose. In the FLASH halls you have [[local machines>>url:http://hasfweb.desy.de/bin/view/Setup/ComputingResourcesUserGuide||shape="rect" class="twikiLink"]] which are used with functional accounts and they have access to the [[beamline files-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]] for your current experiment. For more demanding task we could also provide workstations which can be dedicated to a single user experiment. For offline and nearOnline analysis the [[Maxwell cluster~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/IS/Maxwell||shape="rect"]] for high performance computing is available. On the Maxwell cluster you have to work with personal accounts as this regulates data access to the [[core file-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]].
55 +During a beamtime at FLASH we have two IT infrastructures each with different purpose. In the FLASH halls you have [[local machines>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.ComputingResourcesUserGuide.WebHome]] which are used with functional accounts and they have access to the [[beamline files-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]] for your current experiment. For more demanding task we could also provide workstations which can be dedicated to a single user experiment. For offline and nearOnline analysis the [[Maxwell cluster~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/IS/Maxwell||shape="rect"]] for high performance computing is available. On the Maxwell cluster you have to work with personal accounts as this regulates data access to the [[core file-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]].
63 63  
64 -[[image:url:http://hasfweb.desy.de/pub/Setup/Index/it-infrastructure2.png||alt="it-infrastructure2.png" width="800" height="393"]]
57 +[[image:attach:it-infrastructure2.png||height="400"]]
65 65  
59 +\\
60 +
66 66  The FLASH control & DAQ system supports [[several devices>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).DOOCS Device Support.WebHome]] and at each beamline are [[doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]] available. With the MTCA technology it is possible to [[synchronize own devices in respect to the FEL>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).Trigger Howto.WebHome]].
67 67  If the user is using his or her own DAQ system it is further possible to [[receive the trainId via network>>doc:FLASHUSER.TrainID]] for the purpose of synchronization.
68 68  
... ... @@ -70,7 +70,7 @@
70 70  
71 71  (% class="wrapped" %)
72 72  |=(((
73 -[[local machines>>url:http://hasfweb.desy.de/bin/view/Setup/ComputingResourcesUserGuide||shape="rect" class="twikiLink"]]
68 +[[local>>attach:jddd.jnlp]][[ machines>>attach:jddd.jnlp]]
74 74  )))|=(((
75 75  (% class="content-wrapper" %)
76 76  (((
... ... @@ -77,9 +77,9 @@
77 77  [[Maxwell cluster~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/IS/Maxwell||shape="rect"]]
78 78  )))
79 79  )))|=(((
80 -[[MTCA ADCs>>url:http://hasfweb.desy.de/bin/view/Setup/MtcaAdc||shape="rect" class="twikiLink"]]
75 +[[doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]]
81 81  )))|=(((
82 -[[supported devices>>url:http://hasfweb.desy.de/bin/view/Setup/DOOCS_supported_devices||shape="rect" class="twikiLink"]]
77 +[[supported devices>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).DOOCS Device Support.WebHome]]
83 83  )))|=(((
84 84  (% class="content-wrapper" %)
85 85  (((
... ... @@ -86,9 +86,9 @@
86 86  [[supported cameras~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=Camera%20server||shape="rect"]]
87 87  )))
88 88  )))|=(((
89 -[[trigger via MTCA>>url:http://hasfweb.desy.de/bin/view/Setup/TriggerHowto||shape="rect" class="twikiLink"]]
84 +[[trigger via MTCA>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).Trigger Howto.WebHome]]
90 90  )))|=(((
91 -[[trainId via network>>url:http://hasfweb.desy.de/bin/view/Setup/TrainID||shape="rect" class="twikiLink"]]
86 +[[trainId via network>>doc:FLASHUSER.TrainID]]
92 92  )))
93 93  
94 94  \\
... ... @@ -95,9 +95,11 @@
95 95  
96 96  == Data access ==
97 97  
98 -[[image:url:http://hasfweb.desy.de/pub/Setup/Index/timescale_dataAccess.png||alt="timescale_dataAccess.png" width="1000" height="539"]]
93 +\\
99 99  
100 -At the Free-electron Laser Hamburg ( [[FLASH~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://flash.desy.de/||shape="rect"]]) we use the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/doocs/doocs.html||shape="rect"]]). Devices are implemented via DOOCS server and via an API ( [[ONLINE>>url:http://hasfweb.desy.de/bin/view/Setup/OnlineAPIs||shape="rect" class="twikiLink"]]) it is possible to request data directly from the DOOCS server by knowing the DOOCS address.
95 +[[image:attach:timescale_dataAccess.png||height="400"]]
96 +
97 +At the Free-electron Laser Hamburg ( [[FLASH~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://flash.desy.de/||shape="rect"]]) we use the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/doocs/doocs.html||shape="rect"]]). Devices are implemented via DOOCS server and via an API ( [[ONLINE>>doc:FLASHUSER.Online access to DOOCS properties]]) it is possible to request data directly from the DOOCS server by knowing the DOOCS address.
101 101  As correlations of different physical properties are often required all data at FLASH are indexed by the [[trainID>>doc:FLASHUSER.TrainID]], which identify each of FLASH's pulse trains. The during a beamtime recorded data are stored via a Data Acquisition System ( [[DAQ>>doc:FLASHUSER.Information about the FLASH DAQ system]]) which sort all events from the individual DOOCS server by trainID. When requested [[HDF files>>doc:FLASHUSER.Information about the FLASH HDF5 structure]] are created after the beamtime which includes the important data from [[the accelerator and its diagnostic>>doc:FLASHUSER.PBD DAQ]] as well as the data created by the [[users>>doc:FLASHUSER.Description of the Properties in the FLASH1_USER1, 2 and 3 for FLASH1 Users]]. This time scale we define as {{code language="none"}}offline{{/code}} as the HDF files are converted after the beamtime is over. For synchronous data during an experiment it is possible to create shorter HDF slices via a {{code language="none"}}nearOnline{{/code}} converter within a few minutes. For working with this partially incomplete HDF slices we provide an API called [[BeamtimeDaqAccess>>doc:FLASHUSER.Beamtime DAQ Access]]. Reading synchronous data via an {{code language="none"}}online{{/code}} API is possible via a configurable DAQ middle layer server, the [[DAQmonitor>>doc:AIDA2020ESST.DAQ Monitor]], which feeds the correlated data back in the control system while it provides a ring buffer with 32 events in size.
102 102  
103 103  \\
... ... @@ -140,9 +140,9 @@
140 140  
141 141  \\
142 142  
143 -(% class="wrapped" %)
140 +(% class="wrapped relative-table" %)
144 144  |=(((
145 -[[DAQ introduction>>url:http://hasfweb.desy.de/pub/Setup/DAQmain/Operator_training_DAQ_May_2016.pdf||shape="rect"]]
142 +[[DAQ introductions>>attach:Operator_training_DAQ_May_2016.pdf]]
146 146  )))|=(((
147 147  [[FLASH1 PBD>>doc:FLASHUSER.PBD DAQ]]
148 148  )))|=(((
... ... @@ -177,4 +177,11 @@
177 177  
178 178  ----
179 179  
180 -**//NOTE~://** Content below the STOPPUBLISH variable is not published. Use this space as a scrap area.
177 +**//
178 +//**
179 +
180 +\\
181 +
182 +\\
183 +
184 +\\