Skip to Content
Last modified by makuadm on 2026-01-07 06:21
From version 41.2
edited by sndueste
on 2025-02-05 11:21
Change comment: Update document after refactoring.
To version 44.1
edited by makuadm
on 2026-01-06 20:47
Change comment: Task [xwiki:FLASHUSER.Offline data analysis (DAQ).DEMO - Working with FLASH data.Tasks.Task_33.WebHome] has been updated!

Summary

Details

Page properties
Parent
... ... @@ -1,1 +1,1 @@
1 -FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).WebHome
1 +FLASHUSER.Offline data analysis (DAQ).WebHome
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.sndueste
1 +XWiki.makuadm
Content
... ... @@ -1,13 +1,11 @@
1 -Experimental data is recorded as HDF files[[~[link~]>>doc:FLASHUSER.Data Acquisition and controls.Data Access at FLASH (DAQ, gpfs,\.\.\.).Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] on the GPFS file system[[~[link~]>>doc:ASAP3.ASAP3 Data Storage for PETRA III]]. The access rights are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[[~[link~]>>url:https://gamma-portal.desy.de/||shape="rect"]]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh, Maxwell-Display Server[[~[link~]>>url:https://confluence.desy.de/display/MXW/Maxwell+Cluster||shape="rect"]] or JuyterHub[[~[link~]>>url:https://confluence.desy.de/display/MXW/JupyterHub+on+Maxwell||shape="rect"]]. We recommend using the JupyterHub for data exploration and the SLURM resources for high performances computing - see FAB for easy usage of the infrastructure.
1 +Experimental data is recorded as HDF files[[~[link~]>>doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).The FLASH HDF5 structure.WebHome]] on the GPFS file system[[~[link~]>>https://docs.desy.de/asap3/]]. The access rights are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[[~[link~]>>url:https://gamma-portal.desy.de/||shape="rect"]]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh, Maxwell-Display Server[[~[link~]>>url:https://confluence.desy.de/display/MXW/Maxwell+Cluster||shape="rect"]] or JuyterHub[[~[link~]>>url:https://confluence.desy.de/display/MXW/JupyterHub+on+Maxwell||shape="rect"]]. We recommend using the JupyterHub for data exploration and the SLURM resources for high performances computing - see FAB for easy usage of the infrastructure.
2 2  
3 3  {{info title="How to login JupyterHub"}}
4 -=== ===
4 +=== ===
5 5  
6 6  {{view-file att--filename="tmp.mp4" height="150"/}}
7 7  {{/info}}
8 8  
9 -\\
10 -
11 11  {{info}}
12 12  === There are different options that help you to work with the FLASH HDF5 data in Python ===
13 13  
... ... @@ -14,15 +14,12 @@
14 14  * The currently developed option for large data sets: [[the FAB package>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]] ... see below
15 15  * and for smaller projects:  (% class="Object" %)[[https:~~/~~/gitlab.desy.de/christopher.passow/flash-daq-hdf>>url:https://gitlab.desy.de/christopher.passow/flash-daq-hdf||shape="rect"]]
16 16  
17 -(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FLASHUSER.Data Acquisition and controls.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]]
15 +(% class="Object" %)See also the collection of Demo data and sample scripts : [[doc:FS-FLASH USER tmp.Data Acquisition and controls 1.DAQ and controls overview.Offline data analysis (DAQ).Collection of HDF5 sample data from different beamlines.WebHome]]
18 18  {{/info}}
19 19  
20 -\\
21 21  
22 22  [[~[~[image:attach:image2023-9-29_11-1-37.png~]~]>>url:https://hasfcpkg.desy.de/fab/fab.html||shape="rect"]]
23 23  
24 -\\
25 -
26 26  {{expand title="older ideas ..."}}
27 27  (% class="Object" %)(object oriented) [[https:~~/~~/gitlab.desy.de/christopher.passow/fdh-builder>>url:https://gitlab.desy.de/christopher.passow/fdh-builder.git||shape="rect"]]
28 28  
... ... @@ -32,45 +32,12 @@
32 32  
33 33  (% class="task-list" %)
34 34  (((
35 -{{task reference="/Tasks/Task_18" status="InProgress"}}
36 -Short descriptions including Links:   → as Text\\
37 -
38 -(% class="task-list" %)
39 -(((
40 -
41 -
42 -{{task reference="/Tasks/Task_19" status="InProgress"}}
43 -GPFS
30 +{{task reference="/Tasks/Task_18" status="InProgress" completeDate="" createDate="2026-01-06" reporter=""}}
31 +Short descriptions including Links:   → as Text
32 +\\
44 44  {{/task}}
45 -
46 -{{task reference="/Tasks/Task_40" status="InProgress"}}
47 -Access rights
48 -{{/task}}
49 -
50 -{{task reference="/Tasks/Task_41" status="InProgress"}}
51 -Gamma Portal
52 -{{/task}}
53 -
54 -{{task reference="/Tasks/Task_42" status="InProgress"}}
55 -Maxwell
56 -{{/task}}
57 -
58 -{{task reference="/Tasks/Task_20" status="InProgress"}}
59 -JupyterHub
60 -{{/task}}
61 -
62 -{{task reference="/Tasks/Task_21" status="InProgress"}}
63 -conda ?
64 -{{/task}}
65 -
66 -{{task reference="/Tasks/Task_36" status="InProgress"}}
67 -explain install from channel instead of fixed environment, but can use environment file from example repository
68 -{{/task}}
69 69  )))
70 -{{/task}}
71 -)))
72 72  
73 -\\
74 74  
75 75  (% class="task-list" %)
76 76  (((
... ... @@ -92,7 +92,6 @@
92 92  {{/task}}
93 93  )))
94 94  
95 -\\
96 96  
97 97  (% class="task-list" %)
98 98  (((
... ... @@ -110,7 +110,6 @@
110 110  {{/task}}
111 111  )))
112 112  
113 -\\
114 114  
115 115  (% class="task-list" %)
116 116  (((
... ... @@ -128,7 +128,6 @@
128 128  {{/task}}
129 129  )))
130 130  
131 -\\
132 132  
133 133  (% class="task-list" %)
134 134  (((
... ... @@ -151,7 +151,7 @@
151 151  use slix
152 152  {{/task}}
153 153  
154 -{{task reference="/Tasks/Task_33" status="InProgress"}}
113 +{{task reference="/Tasks/Task_33" status="InProgress" completeDate="" createDate="2026-01-06" reporter=""}}
155 155  use hdfview plugin in jupterLab
156 156  {{/task}}
157 157  
... ... @@ -162,15 +162,12 @@
162 162  {{/task}}
163 163  )))
164 164  
165 -\\
166 166  
167 -\\
168 168  
169 169  ----
170 170  
171 171  ==== under review ====
172 172  
173 -\\
174 174  
175 175  {{code language="bash"}}
176 176  conda create -n flashh5 python=3.10 # 3.10 not necessary, but would prefer 3.8 or higher
... ... @@ -187,9 +187,8 @@
187 187  ## delete from: /home/$USER/.local/share/jupyter/kernels/
188 188  {{/code}}
189 189  
190 -\\
191 -
192 -{{code language="py" title="moved to repository?"}}
146 +{{code language="py" title="
147 +moved to repository?"}}
193 193  class RunDirectory:
194 194  
195 195   def get_run_table(): # more or less information? RunComment | Number of Files | start & stop time ?
... ... @@ -234,7 +234,6 @@
234 234  # run.to_xarray(daq_map)
235 235  {{/code}}
236 236  
237 -\\
238 238  
239 -\\
193 +
240 240  {{/expand}}