Skip to Content

DEMO - Working with FLASH data

Version 29.1 by cpassow on 2022-04-06 16:47

Experimental data is recorded as HDF files[link] on the GPFS file system[link]. The access rights[link to ACLs] are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[link]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh[link], Maxwell-Display Server[link] or JuyterHub[link]. We recommmend using the JupyterHub for data exploration and the SLURM resources[link] for high performances computing. 

For simplified acccess we provide a conda module flashh5[link] which can be installed in a personal conda environment[link] on the Maxwell Cluster. Example on the usage can be found here [link - repo + binder]


TEST - How to login JupyterHub



TODO

#568334

Short descriptions including Links:   → as Text

 

#568335

GPFS

#568352

Access rights

#568353

Gamma Portal

#568354

Maxwell

#568336

JupyterHub

#568337

conda ?

#568351

explain install from channel instead of fixed environment, but can use environment file from example repository


#568338

distribution

 

#568339

channel  (where to host?)

#568340

environment file (repository with examples)


#568341

Documentation

 

#568342

here VS repository VS sphinx


#568343

Binder

 

#568344

examples with Stefan


#568345

Screencast

 

#568346

login in jupyter hub

#568347

login maxwell display

#568348

use slix

#568349

use hdfview plugin in jupterLab

#568350

create conda env with flashh5



under review


conda create -n flashh5 python=3.10  # 3.10 not necessary, but would prefer 3.8 or higher
source activate flashh5
conda install ipython numpy pandas  #TODO: fix dependcies
conda install -c https://www.desy.de/~cpassow/condarepo/ flashh5

## on jhub
conda install ipykernel
python -m ipykernel install --user --name flashh5 --display-name "flashh5"


## to remove on jhub
## delete from: /home/$USER/.local/share/jupyter/kernels/


moved to repository?
class RunDirectory:
       
   def get_run_table():  # more or less information? RunComment | Number of Files | start & stop time ? 
       ...
   
   def get_run(daq, run_number):  # daq is not needed!

       ...
   
   
class Run:  # constructor optional without RunDirectory or use there self.path
       
   def get_files():
       ...
   
   def get_channels():  # of file #1
       ...

   def get_start_time(): # better as attribute?
       ...
   
   def get_stop_time():  # which?  |  better as attribute?
       ...

   def to_df(daq_map):  # to_df(daq_map, slice) slice=[0:4] -> throw Exception
       ...
   
   def to_series(channel):
       ...
   
   def to_array(channel):
       ...
ideas
run.to_df(daq_map)
run.to_series(daq_adr or daq_map) # on channel only?
run.to_array(daq_adr) # on channel only? 

## interesting?
# run.to_dask(daq_map)
# run.to_xarray(daq_map)