Skip to Content
Last modified by makuadm on 2026-01-07 06:21
From version 41.3
edited by sndueste
on 2025-02-05 14:41
Change comment: There is no comment for this version
To version 33.1
edited by sndueste
on 2023-09-28 16:51
Change comment: There is no comment for this version

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,0 @@
1 -FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).WebHome
Content
... ... @@ -1,26 +1,28 @@
1 -Experimental data is recorded as HDF files[[~[link~]>>doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] on the GPFS file system[[~[link~]>>doc:ASAP3.ASAP3 Data Storage for PETRA III]]. The access rights are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[[~[link~]>>url:https://gamma-portal.desy.de/||shape="rect"]]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh, Maxwell-Display Server[[~[link~]>>url:https://confluence.desy.de/display/MXW/Maxwell+Cluster||shape="rect"]] or JuyterHub[[~[link~]>>url:https://confluence.desy.de/display/MXW/JupyterHub+on+Maxwell||shape="rect"]]. We recommend using the JupyterHub for data exploration and the SLURM resources for high performances computing - see FAB for easy usage of the infrastructure.
1 +Experimental data is recorded as HDF files[link] on the GPFS file system[link]. The access rights[link to ACLs] are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[link]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh[link], Maxwell-Display Server[link] or JuyterHub[link]. We recommmend using the JupyterHub for data exploration and the SLURM resources[link] for high performances computing.
2 2  
3 -{{info title="How to login JupyterHub"}}
4 -=== ===
3 +For simplified acccess we provide a conda module flashh5[link] which can be installed in a personal conda environment[link] on the Maxwell Cluster. Example on the usage can be found here [link - repo + binder]
5 5  
6 -{{view-file att--filename="tmp.mp4" height="150"/}}
7 -{{/info}}
5 +\\
8 8  
9 -{{info}}
10 -=== There are different options that help you to work with the FLASH HDF5 data in Python ===
7 +{{expand title="How to login JupyterHub"}}
11 11  
12 -* The currently developed option for large data sets: [[the FAB package>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]] ... see below
13 -* and for smaller projects:  (% class="Object" %)[[https:~~/~~/gitlab.desy.de/christopher.passow/flash-daq-hdf>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]]
14 14  
15 -(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]]
16 -{{/info}}
10 +{{view-file att--filename="tmp.mp4" height="250"/}}
11 +{{/expand}}
17 17  
13 +\\
18 18  
19 -[[~[~[image:attach:image2023-9-29_11-1-37.png~]~]>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]]
15 +link to Repos and Fab
20 20  
21 -{{expand title="older ideas ..."}}
22 -(% class="Object" %)(object oriented) [[https:~~/~~/gitlab.desy.de/christopher.passow/fdh-builder>>url:https://gitlab.desy.de/christopher.passow/fdh-builder.git||shape="rect"]]
17 +\\
23 23  
19 +\\
20 +
21 +\\
22 +
23 +{{expand}}
24 +\\
25 +
24 24  ----
25 25  
26 26  === TODO ===
... ... @@ -28,7 +28,7 @@
28 28  (% class="task-list" %)
29 29  (((
30 30  {{task reference="/Tasks/Task_18" status="InProgress"}}
31 -Short descriptions including Links:   → as Text
33 +Short descriptions including Links:   → as Text\\
32 32  
33 33  (% class="task-list" %)
34 34  (((
... ... @@ -65,6 +65,7 @@
65 65  {{/task}}
66 66  )))
67 67  
70 +\\
68 68  
69 69  (% class="task-list" %)
70 70  (((
... ... @@ -86,6 +86,7 @@
86 86  {{/task}}
87 87  )))
88 88  
92 +\\
89 89  
90 90  (% class="task-list" %)
91 91  (((
... ... @@ -103,6 +103,7 @@
103 103  {{/task}}
104 104  )))
105 105  
110 +\\
106 106  
107 107  (% class="task-list" %)
108 108  (((
... ... @@ -120,6 +120,7 @@
120 120  {{/task}}
121 121  )))
122 122  
128 +\\
123 123  
124 124  (% class="task-list" %)
125 125  (((
... ... @@ -153,12 +153,15 @@
153 153  {{/task}}
154 154  )))
155 155  
162 +\\
156 156  
164 +\\
157 157  
158 158  ----
159 159  
160 160  ==== under review ====
161 161  
170 +\\
162 162  
163 163  {{code language="bash"}}
164 164  conda create -n flashh5 python=3.10 # 3.10 not necessary, but would prefer 3.8 or higher
... ... @@ -175,8 +175,9 @@
175 175  ## delete from: /home/$USER/.local/share/jupyter/kernels/
176 176  {{/code}}
177 177  
178 -{{code language="py" title="
179 -moved to repository?"}}
187 +\\
188 +
189 +{{code language="py" title="moved to repository?"}}
180 180  class RunDirectory:
181 181  
182 182   def get_run_table(): # more or less information? RunComment | Number of Files | start & stop time ?
... ... @@ -221,6 +221,9 @@
221 221  # run.to_xarray(daq_map)
222 222  {{/code}}
223 223  
234 +\\
224 224  
225 -
236 +\\
226 226  {{/expand}}
238 +
239 +\\
image2023-9-29_11-0-13.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.XWikiGuest
Size
... ... @@ -1,1 +1,0 @@
1 -123.1 KB
Content
image2023-9-29_11-1-37.png
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.XWikiGuest
Size
... ... @@ -1,1 +1,0 @@
1 -90.4 KB
Content
tmp.mp4
Author
... ... @@ -1,1 +1,0 @@
1 -XWiki.XWikiGuest
Size
... ... @@ -1,1 +1,0 @@
1 -1.1 MB
Content