DEMO - Working with FLASH data

Version 41.2 by sndueste on 2025/02/05 11:21
Warning: For security reasons, the document is displayed in restricted mode as it is not the current version. There may be differences and errors due to this.

Experimental data is recorded as HDF files[link] on the GPFS file system[link]. The access rights are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[link]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh, Maxwell-Display Server[link] or JuyterHub[link]. We recommend using the JupyterHub for data exploration and the SLURM resources for high performances computing - see FAB for easy usage of the infrastructure. 

How to login JupyterHub

Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.

Failed to execute the [velocity] macro. Cause: [The execution of the [velocity] script macro is not allowed. Check the rights of its last author or the parameters if it's rendered from another script.]. Click on this message for details.


There are different options that help you to work with the FLASH HDF5 data in Python

See also the collection of Demo data and sample scripts : Collection of HDF5 sample data from different beamlines


image2023-9-29_11-1-37.png


older ideas ...

(object oriented) https://gitlab.desy.de/christopher.passow/fdh-builder

TODO

#568213

Short descriptions including Links:   → as Text

 

#568214

GPFS

#568215

Access rights

#568216

Gamma Portal

#568217

Maxwell

#568218

JupyterHub

#568219

conda ?

#568220

explain install from channel instead of fixed environment, but can use environment file from example repository


#568221

distribution

 

#568222

channel  (where to host?)

#568223

environment file (repository with examples)


#568224

Documentation

 

#568225

here VS repository VS sphinx


#568226

Binder

 

#568227

examples with Stefan


#568228

Screencast

 

#568229

login in jupyter hub

#568230

login maxwell display

#568231

use slix

#568232

use hdfview plugin in jupterLab

#568233

create conda env with flashh5



under review


conda create -n flashh5 python=3.10  # 3.10 not necessary, but would prefer 3.8 or higher
source activate flashh5
conda install ipython numpy pandas  #TODO: fix dependcies
conda install -c https://www.desy.de/~cpassow/condarepo/ flashh5

## on jhub
conda install ipykernel
python -m ipykernel install --user --name flashh5 --display-name "flashh5"


## to remove on jhub
## delete from: /home/$USER/.local/share/jupyter/kernels/


moved to repository?
class RunDirectory:
       
   def get_run_table():  # more or less information? RunComment | Number of Files | start & stop time ? 
       ...
   
   def get_run(daq, run_number):  # daq is not needed!

       ...
   
   
class Run:  # constructor optional without RunDirectory or use there self.path
       
   def get_files():
       ...
   
   def get_channels():  # of file #1
       ...

   def get_start_time(): # better as attribute?
       ...
   
   def get_stop_time():  # which?  |  better as attribute?
       ...

   def to_df(daq_map):  # to_df(daq_map, slice) slice=[0:4] -> throw Exception
       ...
   
   def to_series(channel):
       ...
   
   def to_array(channel):
       ...
ideas
run.to_df(daq_map)
run.to_series(daq_adr or daq_map) # on channel only?
run.to_array(daq_adr) # on channel only? 

## interesting?
# run.to_dask(daq_map)
# run.to_xarray(daq_map)