For experts: many ways to get Data from the DAQ

Version 18.2 by sndueste on 2025/02/05 11:21

 

Typical way to get to DAQ data: HDF5 files

Generally we provide the DAQ data as easy to access HDF5 files. For  more info goto   HDF5 file docu. and DEMO - Working with FLASH data

To get direct access to the saved .raw files there are several (sometimes not anymore supported) ways for experts:

Manuals Overview for experts

DAQ Data Access Related Graphical User Interfaces

How to Use the FLASH DAQ Data GUI?

How to Use the FLASH DAQ Data GUI? (DOOCS Wiki, Rybnikov, 2013)

The DAQ Data GUI is a general tool to select and visualise DAQ data. It also contains a DAQ File Manager. The File Manager is used to explore files stored on disk or tape and to retrieve data files from the tape archive too. The DAQ Data GUI can display data retrieved remotely from DOOCS servers. It also can access data stored locally in ROOT formatted .raw files, or files stored locally in HDF5 format.

Start the DAQdataGUI via Java Webstart.

Use the DAQ data GUI locally for staged files: login at flashlxuser1 (others may not work). One should just put in the run numbrt and define the directory ( with the field in the upper right) and then press get files. (No hooks for local mode or dates or something else ...!!)

How to Use the DAQ Converter GUI?

The DAQ Converter GUI is used to convert DAQ data stored in the ROOT format, a self describing hierarchical binary format optimised for fast access, into ASCII formatted data for offline analysis. The GUI allows to select the files and channels from files to be converted. The GUI also has an interface to plug-in converters to write other data formats (Rybnikov et al., 2010).

Start the DAQexperiemntGUI via Java Webstart.

How to Stage Data From DAQ?

How to Stage Data From DAQ? (Brenner, 2013)

This manual explains step by step how to use the DAQ File Manager GUI started via the DAQdataGUI to copy DAQ data files from tape to disk for offline analysis, if one does not want to use one of the APIs listed below.

How to Discover Channels Recorded by the DAQ?

You can discover the channels recorded by the DAQ using the DAQdataGUI or the DAQexperiemntGUI, if you have access to the DESY network.

If you also have access to the .raw files directly, you can use the library jdaq-use to create a searcheable text document. The library includes a command line application producing a list of channels discovered in a given .raw file or at a given point in time. Download this jdaq-use jar. On a computer with access to the MCS-4 NFS filesystem the following command creates the list of channels as follows:

java -ea -jar jdaq-use-0.0.4-j1.7.jar FLASH1_EXP /daq_data/fel/EXP2 FLASH1_EXP_stream_2_run17666_file1_20170610T065636.1.raw

In case you get no access, you could also use the DAQ Channel Info Database as explained on the help pages by Ned Arnold of the Argonne Advanced Photon Source.

TODO: Explain how to use "rpc_test" to discover DAQ data and channels.

Application Program Interfaces to Access DAQ Data

  • Java

DAQ Run Control Graphical User Interfaces

How to Start the DAQ Status GUI?

The DAQ Status GUI displays the current DAQ data flow from frontend through collectors, event builder to the dCache. The GUI can be started via Java Webstart for the Photon Beam DAQ and the secondary experiment DAQ, respectively.

How to Start the Run Control GUI?

The Run Control GUI is used to configure the DAQ run controller reflected in the run control data base. The Run Control GUI can be run on the central DOOCS user machines only. To start the GUI login with X-forwarding enabled to e.g. "flashlxuser1" and execute the command "RCGUIPBD" for the photon beam DAQ run control or the command "RCGUIEXP2" for the secondary experiment DAQ.

How to Monitor the Experiment DAQ?

How to Monitor the Experiment DAQ? (DOOCS Wiki, NN, 2011)

This page explains, how to check, if the DAQ is running, i.e. if the DAQ is collecting and writing data. Possibly the page is not quite up to date -- the uninitiated user is hardly able to follow the instructions given.

General Introductions to the FLASH DAQ

Other Resources for DAQ and dCache

N.B. The general DESY info on its dCache do not directly apply to the FLASH DAQ dCache.

Contact Info

for Questions / Problems: