Wiki source code of DEMO - Working with FLASH data

Version 33.1 by sndueste on 2023-09-28 16:51

version	line-number	content
29.1	1	Experimental data is recorded as HDF files[link] on the GPFS file system[link]. The access rights[link to ACLs] are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[link]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh[link], Maxwell-Display Server[link] or JuyterHub[link]. We recommmend using the JupyterHub for data exploration and the SLURM resources[link] for high performances computing.
21.1	2
29.1	3	For simplified acccess we provide a conda module flashh5[link] which can be installed in a personal conda environment[link] on the Maxwell Cluster. Example on the usage can be found here [link - repo + binder]
	4
26.1	5	\\
	6
33.1	7	{{expand title="How to login JupyterHub"}}
26.1	8
	9
	10	{{view-file att--filename="tmp.mp4" height="250"/}}
	11	{{/expand}}
	12
	13	\\
	14
33.1	15	link to Repos and Fab
	16
26.1	17	\\
	18
33.1	19	\\
	20
	21	\\
	22
	23	{{expand}}
	24	\\
	25
21.1	26	----
	27
	28	=== TODO ===
	29
22.1	30	(% class="task-list" %)
	31	(((
	32	{{task reference="/Tasks/Task_18" status="InProgress"}}
	33	Short descriptions including Links: → as Text\\
2.1	34
22.1	35	(% class="task-list" %)
	36	(((
	37
	38
	39	{{task reference="/Tasks/Task_19" status="InProgress"}}
	40	GPFS
	41	{{/task}}
	42
27.1	43	{{task reference="/Tasks/Task_40" status="InProgress"}}
	44	Access rights
	45	{{/task}}
	46
	47	{{task reference="/Tasks/Task_41" status="InProgress"}}
	48	Gamma Portal
	49	{{/task}}
	50
	51	{{task reference="/Tasks/Task_42" status="InProgress"}}
	52	Maxwell
	53	{{/task}}
	54
32.1	55	{{task reference="/Tasks/Task_20" status="InProgress"}}
22.1	56	JupyterHub
	57	{{/task}}
	58
	59	{{task reference="/Tasks/Task_21" status="InProgress"}}
	60	conda ?
	61	{{/task}}
23.1	62
	63	{{task reference="/Tasks/Task_36" status="InProgress"}}
	64	explain install from channel instead of fixed environment, but can use environment file from example repository
	65	{{/task}}
22.1	66	)))
	67	{{/task}}
	68	)))
	69
8.1	70	\\
	71
22.1	72	(% class="task-list" %)
	73	(((
	74	{{task reference="/Tasks/Task_22" status="InProgress"}}
	75	distribution
21.1	76
22.1	77	(% class="task-list" %)
	78	(((
	79
	80
	81	{{task reference="/Tasks/Task_23" status="InProgress"}}
	82	channel (where to host?)
	83	{{/task}}
	84
	85	{{task reference="/Tasks/Task_24" status="InProgress"}}
	86	environment file (repository with examples)
	87	{{/task}}
	88	)))
	89	{{/task}}
	90	)))
	91
21.1	92	\\
	93
22.1	94	(% class="task-list" %)
	95	(((
	96	{{task reference="/Tasks/Task_25" status="InProgress"}}
	97	Documentation
2.1	98
22.1	99	(% class="task-list" %)
	100	(((
	101
	102
	103	{{task reference="/Tasks/Task_26" status="InProgress"}}
	104	here VS repository VS sphinx
	105	{{/task}}
	106	)))
	107	{{/task}}
	108	)))
	109
2.1	110	\\
	111
22.1	112	(% class="task-list" %)
	113	(((
	114	{{task reference="/Tasks/Task_27" status="InProgress"}}
	115	Binder
2.1	116
22.1	117	(% class="task-list" %)
	118	(((
	119
	120
	121	{{task reference="/Tasks/Task_28" status="InProgress"}}
	122	examples with Stefan
	123	{{/task}}
	124	)))
	125	{{/task}}
	126	)))
	127
2.1	128	\\
	129
22.1	130	(% class="task-list" %)
	131	(((
	132	{{task reference="/Tasks/Task_29" status="InProgress"}}
	133	Screencast
15.1	134
22.1	135	(% class="task-list" %)
	136	(((
	137
	138
	139	{{task reference="/Tasks/Task_30" status="InProgress"}}
	140	login in jupyter hub
	141	{{/task}}
	142
	143	{{task reference="/Tasks/Task_31" status="InProgress"}}
	144	login maxwell display
	145	{{/task}}
	146
	147	{{task reference="/Tasks/Task_32" status="InProgress"}}
	148	use slix
	149	{{/task}}
	150
	151	{{task reference="/Tasks/Task_33" status="InProgress"}}
	152	use hdfview plugin in jupterLab
	153	{{/task}}
	154
	155	{{task reference="/Tasks/Task_34" status="InProgress"}}
	156	create conda env with flashh5
	157	{{/task}}
	158	)))
	159	{{/task}}
	160	)))
	161
2.1	162	\\
	163
15.1	164	\\
	165
21.1	166	----
2.1	167
21.1	168	==== under review ====
4.1	169
	170	\\
	171
8.1	172	{{code language="bash"}}
9.1	173	conda create -n flashh5 python=3.10 # 3.10 not necessary, but would prefer 3.8 or higher
8.1	174	source activate flashh5
10.1	175	conda install ipython numpy pandas #TODO: fix dependcies
8.1	176	conda install -c https://www.desy.de/~cpassow/condarepo/ flashh5
	177
	178	## on jhub
11.1	179	conda install ipykernel
8.1	180	python -m ipykernel install --user --name flashh5 --display-name "flashh5"
	181
	182
11.1	183	## to remove on jhub
	184	## delete from: /home/$USER/.local/share/jupyter/kernels/
8.1	185	{{/code}}
	186
	187	\\
	188
6.1	189	{{code language="py" title="moved to repository?"}}
4.1	190	class RunDirectory:
	191
21.1	192	def get_run_table(): # more or less information? RunComment \| Number of Files \| start & stop time ?
4.1	193	...
	194
21.1	195	def get_run(daq, run_number): # daq is not needed!
	196
4.1	197	...
	198
	199
25.1	200	class Run: # constructor optional without RunDirectory or use there self.path
4.1	201
7.1	202	def get_files():
4.1	203	...
	204
18.1	205	def get_channels(): # of file #1
4.1	206	...
	207
8.1	208	def get_start_time(): # better as attribute?
4.1	209	...
	210
8.1	211	def get_stop_time(): # which? \| better as attribute?
4.1	212	...
12.1	213
19.1	214	def to_df(daq_map): # to_df(daq_map, slice) slice=[0:4] -> throw Exception
4.1	215	...
	216
7.1	217	def to_series(channel):
4.1	218	...
	219
7.1	220	def to_array(channel):
4.1	221	...
	222	{{/code}}
	223
6.1	224	{{code language="py" title="ideas"}}
10.1	225	run.to_df(daq_map)
	226	run.to_series(daq_adr or daq_map) # on channel only?
	227	run.to_array(daq_adr) # on channel only?
5.1	228
21.1	229	## interesting?
10.1	230	# run.to_dask(daq_map)
	231	# run.to_xarray(daq_map)
5.1	232	{{/code}}
	233
4.1	234	\\
30.1	235
	236	\\
33.1	237	{{/expand}}
	238
	239	\\

Wiki source code of DEMO - Working with FLASH data

Applications

Navigation