Changes for page DAQ and data analysis

Last modified by sndueste on 2025/02/05 14:56
From version 1.1
edited by sendels
on 2019/05/15 09:06
Change comment: There is no comment for this version
To version 3.1
edited by sendels
on 2019/05/15 16:07
Change comment: There is no comment for this version

Summary

Details

Page properties
Content
... ... @@ -10,19 +10,19 @@
10 10  
11 11  \\
12 12  
13 -* [[Control and DAQ at FLASH (web version)>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Control_and_DAQ_at_FLASH_web_ver||shape="rect"]]
14 -** [[Overview>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Overview||shape="rect"]]
15 -** [[Things to consider before the beamtime>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Things_to_consider_before_the_be||shape="rect"]]
16 -** [[Beamline control and photon diagnostics>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Beamline_control_and_photon_diag||shape="rect"]]
17 -** [[IT infrastructure>>url:http://hasfweb.desy.de/bin/view/Setup/Index#IT_infrastructure||shape="rect"]]
18 -** [[Data access>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Data_access||shape="rect"]]
19 -** [[Online>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Online||shape="rect"]]
20 -** [[Data acquisition>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Data_acquisition||shape="rect"]]
21 -** [[nearOnline>>url:http://hasfweb.desy.de/bin/view/Setup/Index#nearOnline||shape="rect"]]
13 +* Control and DAQ at FLASH (web version)
14 +** Overview
15 +** Things to consider before the beamtime
16 +** Beamline control and photon diagnostics
17 +** IT infrastructure
18 +** Data access
19 +** Online
20 +** Data acquisition
21 +** nearOnline
22 22  ***
23 -**** [[.raw files>>url:http://hasfweb.desy.de/bin/view/Setup/Index#raw_files||shape="rect"]]
24 -**** [[HDF5 files>>url:http://hasfweb.desy.de/bin/view/Setup/Index#HDF5_files||shape="rect"]]
25 -** [[Offline>>url:http://hasfweb.desy.de/bin/view/Setup/Index#Offline||shape="rect"]]
23 +**** .raw files
24 +**** HDF5 files
25 +** Offline
26 26  
27 27  \\
28 28  
... ... @@ -32,7 +32,7 @@
32 32  
33 33  == Overview ==
34 34  
35 -[[image:url:http://hasfweb.desy.de/pub/Setup/Index/overview_flash_control_and_daq.png||alt="overview_flash_control_and_daq.png" width="748" title="overview_flash_control_and_daq.png" height="823"]]
35 +[[image:url:http://hasfweb.desy.de/pub/Setup/Index/overview_flash_control_and_daq.png||alt="overview_flash_control_and_daq.png" width="748" height="823"]]
36 36  
37 37  The FLASH accelerator, beamlines and experiments are using the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/doocs/doocs.html||shape="rect"]]). Beamline and experiment devices are mainly operated via graphical user interfaces created with the Java DOOCS Data Display ( [[jddd~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://doocs-web.desy.de/access.html||shape="rect"]]) by the FLASH team or the users themself. Further it is provided a [[DOOCS client API~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/DOOCSWiki/Wiki.jsp?page=DOOCS%20Wiki||shape="rect"]] in Java, Python, Matlab, C or C++ to the users to write own software for data access, online data or device control. User-provided devices can be run in parallel or can be implemented in DOOCS. For each beamline and accelerator is a dedicated [[electronic logbook~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/ttf_apps/showElogs.jsp||shape="rect"]] available in which jddd and other software can print directly.
38 38  
... ... @@ -40,16 +40,17 @@
40 40  
41 41  == Things to consider before the beamtime ==
42 42  
43 -In consultation with the user's local contact the user should check our [[beamtime preparation checklist>>url:http://hasfweb.desy.de/bin/view/Setup/BeforeBeamtimeChecklist||shape="rect" class="twikiLink"]] and fill in the [[beamtime preparation questionnaire>>url:http://hasfweb.desy.de/pub/Setup/ITManuals/Before_beamtime_questionnaire.pdf||shape="rect"]].
43 +In consultation with the user's local contact the user should check our [[beamtime preparation checklist>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Before beamtime Checklist.WebHome]] and fill in the [[beamtime preparation questionnaire>>url:http://hasfweb.desy.de/pub/Setup/ITManuals/Before_beamtime_questionnaire.pdf||shape="rect"]].
44 44  
45 45  == Beamline control and photon diagnostics ==
46 46  
47 47  Use existing jddd panel provided by FLASH or use the jddd editor to create a new panel according to the user's own needs.
48 48  
49 -[[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jddd_logo.png~|~|alt="jddd Photon Beamline Controls" width="125" title="jddd Photon Beamline Controls" height="80"~]~]~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd_xml/run_jnlp.jsp?run=true&file=/svn/FLASH/Hasylab/Photons_beamline_control_main.xml||shape="rect"]][[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jdddEditor.png~|~|alt="jdddEditor.png" width="125"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd.jnlp?run=true||title="jddd editor" shape="rect"]](% class="twikiNewLink" %)[[url:http://hasfweb.desy.de/bin/edit/Ttfinfo2/desy/de/common/applications/jddd/Jnlp?topicparent=Setup.Index;nowysiwyg=0||rel="nofollow" shape="rect"]]
49 +[[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jddd_logo.png~|~|alt="jddd Photon Beamline Controls" width="125" height="80"~]~]~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd_xml/run_jnlp.jsp?run=true&file=/svn/FLASH/Hasylab/Photons_beamline_control_main.xml||shape="rect"]][[~[~[image:url:http://hasfweb.desy.de/pub/Setup/Index/jdddEditor.png~|~|alt="jdddEditor.png" width="125"~]~]>>url:http://ttfinfo2.desy.de/common/applications/jddd.jnlp?run=true||title="jddd editor" shape="rect"]](% class="twikiNewLink" %)[[url:http://hasfweb.desy.de/bin/edit/Ttfinfo2/desy/de/common/applications/jddd/Jnlp?topicparent=Setup.Index;nowysiwyg=0||rel="nofollow" shape="rect"]]
50 50  
51 51  \\
52 52  
53 +(% class="wrapped" %)
53 53  |=(((
54 54  [[DOOCS>>url:http://tesla.desy.de/doocs/index.html||title="DOOCS home" shape="rect"]]
55 55  )))|=(((
... ... @@ -60,23 +60,30 @@
60 60  
61 61  During a beamtime at FLASH we have two IT infrastructures each with different purpose. In the FLASH halls you have [[local machines>>url:http://hasfweb.desy.de/bin/view/Setup/ComputingResourcesUserGuide||shape="rect" class="twikiLink"]] which are used with functional accounts and they have access to the [[beamline files-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]] for your current experiment. For more demanding task we could also provide workstations which can be dedicated to a single user experiment. For offline and nearOnline analysis the [[Maxwell cluster~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/IS/Maxwell||shape="rect"]] for high performance computing is available. On the Maxwell cluster you have to work with personal accounts as this regulates data access to the [[core file-system~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/ASAP3/Directory+Structure||shape="rect"]].
62 62  
63 -[[image:url:http://hasfweb.desy.de/pub/Setup/Index/it-infrastructure2.png||alt="it-infrastructure2.png" width="800" title="it-infrastructure2.png" height="393"]]
64 +[[image:url:http://hasfweb.desy.de/pub/Setup/Index/it-infrastructure2.png||alt="it-infrastructure2.png" width="800" height="393"]]
64 64  
65 -The FLASH control & DAQ system supports [[several devices>>url:http://hasfweb.desy.de/bin/view/Setup/DOOCS_supported_devices||shape="rect" class="twikiLink"]] and at each beamline are [[MTCA ADCs>>url:http://hasfweb.desy.de/bin/view/Setup/MtcaAdc||shape="rect" class="twikiLink"]] available. With the MTCA technology it is possible to [[synchronize own devices in respect to the FEL>>url:http://hasfweb.desy.de/bin/view/Setup/TriggerHowto||shape="rect" class="twikiLink"]].
66 +The FLASH control & DAQ system supports [[several devices>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).DOOCS Device Support.WebHome]] and at each beamline are [[doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).MTCA ADCs.WebHome]] available. With the MTCA technology it is possible to [[synchronize own devices in respect to the FEL>>url:http://hasfweb.desy.de/bin/view/Setup/TriggerHowto||shape="rect" class="twikiLink"]].
66 66  If the user is using his or her own DAQ system it is further possible to [[receive the trainId via network>>url:http://hasfweb.desy.de/bin/view/Setup/TrainID||shape="rect" class="twikiLink"]] for the purpose of synchronization.
67 67  
68 68  \\
69 69  
71 +(% class="wrapped" %)
70 70  |=(((
71 71  [[local machines>>url:http://hasfweb.desy.de/bin/view/Setup/ComputingResourcesUserGuide||shape="rect" class="twikiLink"]]
72 72  )))|=(((
75 +(% class="content-wrapper" %)
76 +(((
73 73  [[Maxwell cluster~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://confluence.desy.de/display/IS/Maxwell||shape="rect"]]
78 +)))
74 74  )))|=(((
75 75  [[MTCA ADCs>>url:http://hasfweb.desy.de/bin/view/Setup/MtcaAdc||shape="rect" class="twikiLink"]]
76 76  )))|=(((
77 77  [[supported devices>>url:http://hasfweb.desy.de/bin/view/Setup/DOOCS_supported_devices||shape="rect" class="twikiLink"]]
78 78  )))|=(((
84 +(% class="content-wrapper" %)
85 +(((
79 79  [[supported cameras~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://ttfinfo.desy.de/FLASHWiki/Wiki.jsp?page=Camera%20server||shape="rect"]]
87 +)))
80 80  )))|=(((
81 81  [[trigger via MTCA>>url:http://hasfweb.desy.de/bin/view/Setup/TriggerHowto||shape="rect" class="twikiLink"]]
82 82  )))|=(((
... ... @@ -87,7 +87,7 @@
87 87  
88 88  == Data access ==
89 89  
90 -[[image:url:http://hasfweb.desy.de/pub/Setup/Index/timescale_dataAccess.png||alt="timescale_dataAccess.png" width="1000" title="timescale_dataAccess.png" height="539"]]
98 +[[image:url:http://hasfweb.desy.de/pub/Setup/Index/timescale_dataAccess.png||alt="timescale_dataAccess.png" width="1000" height="539"]]
91 91  
92 92  At the Free-electron Laser Hamburg ( [[FLASH~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://flash.desy.de/||shape="rect"]]) we use the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/doocs/doocs.html||shape="rect"]]). Devices are implemented via DOOCS server and via an API ( [[ONLINE>>url:http://hasfweb.desy.de/bin/view/Setup/OnlineAPIs||shape="rect" class="twikiLink"]]) it is possible to request data directly from the DOOCS server by knowing the DOOCS address.
93 93  As correlations of different physical properties are often required all data at FLASH are indexed by the [[trainID>>url:http://hasfweb.desy.de/bin/view/Setup/TrainID||shape="rect" class="twikiLink"]], which identify each of FLASH's pulse trains. The during a beamtime recorded data are stored via a Data Acquisition System ( [[DAQ>>url:http://hasfweb.desy.de/bin/view/Setup/DAQmain||shape="rect" class="twikiLink"]]) which sort all events from the individual DOOCS server by trainID. When requested [[HDF files>>url:http://hasfweb.desy.de/bin/view/Setup/HDF5main||shape="rect" class="twikiLink"]] are created after the beamtime which includes the important data from [[the accelerator and its diagnostic>>url:http://hasfweb.desy.de/bin/view/Setup/DAQPbdDescription||shape="rect" class="twikiLink"]] as well as the data created by the [[users>>url:http://hasfweb.desy.de/bin/view/Setup/DAQExp2Description||shape="rect" class="twikiLink"]]. This time scale we define as {{code language="none"}}offline{{/code}} as the HDF files are converted after the beamtime is over. For synchronous data during an experiment it is possible to create shorter HDF slices via a {{code language="none"}}nearOnline{{/code}} converter within a few minutes. For working with this partially incomplete HDF slices we provide an API called [[BeamtimeDaqAccess>>url:http://hasfweb.desy.de/bin/view/Setup/BeamtimeDaqAccess||shape="rect" class="twikiLink"]]. Reading synchronous data via an {{code language="none"}}online{{/code}} API is possible via a configurable DAQ middle layer server, the [[DAQmonitor>>url:http://hasfweb.desy.de/bin/view/Setup/ConfigureDAQmonitor||shape="rect" class="twikiLink"]], which feeds the correlated data back in the control system while it provides a ring buffer with 32 events in size.
... ... @@ -94,6 +94,7 @@
94 94  
95 95  \\
96 96  
105 +(% class="wrapped" %)
97 97  |(((
98 98  online
99 99  )))|(((
... ... @@ -131,6 +131,7 @@
131 131  
132 132  \\
133 133  
143 +(% class="wrapped" %)
134 134  |=(((
135 135  [[DAQ introduction>>url:http://hasfweb.desy.de/pub/Setup/DAQmain/Operator_training_DAQ_May_2016.pdf||shape="rect"]]
136 136  )))|=(((