Changes for page Data Access at FLASH

Last modified by sndueste on 2025/02/05 11:22
From version 5.1
edited by sndueste
on 2019/08/06 13:50
Change comment: There is no comment for this version
To version 4.1
edited by sendels
on 2019/06/26 08:39
Change comment: added attachments

Summary

Details

Page properties
Title
... ... @@ -1,1 +1,1 @@
1 -Data Access at FLASH (DAQ, gpfs,...)
1 +Data Access at FLASH
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.sndueste
1 +XWiki.sendels
Content
... ... @@ -3,7 +3,7 @@
3 3  [[~[~[image:url:http://hasfweb.desy.de/pub/Setup/DataAccessGeneralConcept/timescale_dataAccess.png~|~|alt="timescale_dataAccess.png"~]~]>>attach:FLASHUSER.User overview@timescale_dataAccess.png]]
4 4  
5 5  At the Free-electron Laser Hamburg ( [[FLASH~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:https://flash.desy.de/||shape="rect"]]) we use the Distributed Object Oriented Control System ( [[DOOCS~[~[image:url:http://hasfweb.desy.de/pub/TWiki/TWikiDocGraphics/external-link.gif~|~|width="13" height="12"~]~]>>url:http://tesla.desy.de/doocs/doocs.html||shape="rect"]]). Devices are implemented via DOOCS server and via an API ( [[ONLINE>>doc:FLASHUSER.Online access to DOOCS properties]]) it is possible to request data directly from the DOOCS server by knowing the DOOCS address.
6 -As correlations of different physical properties are often required all data at FLASH are indexed by the [[trainID>>doc:FLASHUSER.Data Acquisition and controls.Controls (DOOCS, jDDD,\.\.\.).How to read Train IDs at FLASH.WebHome]], which identify each of FLASH's pulse trains. The during a beamtime recorded data are stored via a Data Acquisition System ( [[DAQ>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).WebHome]]) which sort all events from the individual DOOCS server by trainID. When requested [[HDF files>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] are created after the beamtime which includes the important data from [[the accelerator and its diagnostic>>doc:FLASHUSER.FLASH1 PhotDiag stream]] as well as the data created by the [[users>>doc:FLASHUSER.Configuring the user DAQs]]. This time scale we define as {{code language="none"}}offline{{/code}} as the HDF files are converted after the beamtime is over. For synchronous data during an experiment it is possible to create shorter HDF slices via a {{code language="none"}}nearOnline{{/code}} converter within a few minutes. For working with this partially incomplete HDF slices we provide an API called [[BeamtimeDaqAccess>>doc:FLASHUSER.Near-Online data analysis]]. Reading synchronous data via an {{code language="none"}}online{{/code}} API is possible via a configurable DAQ middle layer server, the [[doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Online data analysis.DAQmonitor.WebHome]], which feeds the correlated data back in the control system while it provides a ring buffer with 32 events in size.
6 +As correlations of different physical properties are often required all data at FLASH are indexed by the [[trainID>>doc:FLASHUSER.TrainID]], which identify each of FLASH's pulse trains. The during a beamtime recorded data are stored via a Data Acquisition System ( [[DAQ>>doc:FLASHUSER.Information about the FLASH DAQ system]]) which sort all events from the individual DOOCS server by trainID. When requested [[HDF files>>doc:FLASHUSER.Information about the FLASH HDF5 structure]] are created after the beamtime which includes the important data from [[the accelerator and its diagnostic>>doc:FLASHUSER.PBD DAQ]] as well as the data created by the [[users>>doc:FLASHUSER.Description of the Properties in the FLASH1_USER1, 2 and 3 for FLASH1 Users]]. This time scale we define as {{code language="none"}}offline{{/code}} as the HDF files are converted after the beamtime is over. For synchronous data during an experiment it is possible to create shorter HDF slices via a {{code language="none"}}nearOnline{{/code}} converter within a few minutes. For working with this partially incomplete HDF slices we provide an API called [[BeamtimeDaqAccess>>doc:FLASHUSER.Beamtime DAQ Access]]. Reading synchronous data via an {{code language="none"}}online{{/code}} API is possible via a configurable DAQ middle layer server, the [[doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Online data analysis.DAQmonitor.WebHome]], which feeds the correlated data back in the control system while it provides a ring buffer with 32 events in size.
7 7  
8 8  \\
9 9