Wiki source code of DEMO - Working with FLASH data

Version 30.1 by cpassow on 2022-04-06 16:53

version	line-number	content
29.1	1	Experimental data is recorded as HDF files[link] on the GPFS file system[link]. The access rights[link to ACLs] are linked to the user's DESY account and can be managed by the PI via the GAMMA portal[link]. The experimental data can be downloaded via the GAMMA portal, but it is advised to use the DESY computing infrastructure. Access point are via ssh[link], Maxwell-Display Server[link] or JuyterHub[link]. We recommmend using the JupyterHub for data exploration and the SLURM resources[link] for high performances computing.
21.1	2
29.1	3	For simplified acccess we provide a conda module flashh5[link] which can be installed in a personal conda environment[link] on the Maxwell Cluster. Example on the usage can be found here [link - repo + binder]
	4
26.1	5	\\
	6
28.1	7	{{expand title="TEST - How to login JupyterHub"}}
26.1	8
	9
	10	{{view-file att--filename="tmp.mp4" height="250"/}}
	11	{{/expand}}
	12
	13	\\
	14
	15	\\
	16
21.1	17	----
	18
	19	=== TODO ===
	20
22.1	21	(% class="task-list" %)
	22	(((
	23	{{task reference="/Tasks/Task_18" status="InProgress"}}
	24	Short descriptions including Links: → as Text\\
2.1	25
22.1	26	(% class="task-list" %)
	27	(((
	28
	29
	30	{{task reference="/Tasks/Task_19" status="InProgress"}}
	31	GPFS
	32	{{/task}}
	33
27.1	34	{{task reference="/Tasks/Task_40" status="InProgress"}}
	35	Access rights
	36	{{/task}}
	37
	38	{{task reference="/Tasks/Task_41" status="InProgress"}}
	39	Gamma Portal
	40	{{/task}}
	41
	42	{{task reference="/Tasks/Task_42" status="InProgress"}}
	43	Maxwell
	44	{{/task}}
	45
22.1	46	{{task reference="/Tasks/Task_20" status="InProgress"}}
	47	JupyterHub
	48	{{/task}}
	49
	50	{{task reference="/Tasks/Task_21" status="InProgress"}}
	51	conda ?
	52	{{/task}}
23.1	53
	54	{{task reference="/Tasks/Task_36" status="InProgress"}}
	55	explain install from channel instead of fixed environment, but can use environment file from example repository
	56	{{/task}}
22.1	57	)))
	58	{{/task}}
	59	)))
	60
8.1	61	\\
	62
22.1	63	(% class="task-list" %)
	64	(((
	65	{{task reference="/Tasks/Task_22" status="InProgress"}}
	66	distribution
21.1	67
22.1	68	(% class="task-list" %)
	69	(((
	70
	71
	72	{{task reference="/Tasks/Task_23" status="InProgress"}}
	73	channel (where to host?)
	74	{{/task}}
	75
	76	{{task reference="/Tasks/Task_24" status="InProgress"}}
	77	environment file (repository with examples)
	78	{{/task}}
	79	)))
	80	{{/task}}
	81	)))
	82
21.1	83	\\
	84
22.1	85	(% class="task-list" %)
	86	(((
	87	{{task reference="/Tasks/Task_25" status="InProgress"}}
	88	Documentation
2.1	89
22.1	90	(% class="task-list" %)
	91	(((
	92
	93
	94	{{task reference="/Tasks/Task_26" status="InProgress"}}
	95	here VS repository VS sphinx
	96	{{/task}}
	97	)))
	98	{{/task}}
	99	)))
	100
2.1	101	\\
	102
22.1	103	(% class="task-list" %)
	104	(((
	105	{{task reference="/Tasks/Task_27" status="InProgress"}}
	106	Binder
2.1	107
22.1	108	(% class="task-list" %)
	109	(((
	110
	111
	112	{{task reference="/Tasks/Task_28" status="InProgress"}}
	113	examples with Stefan
	114	{{/task}}
	115	)))
	116	{{/task}}
	117	)))
	118
2.1	119	\\
	120
22.1	121	(% class="task-list" %)
	122	(((
	123	{{task reference="/Tasks/Task_29" status="InProgress"}}
	124	Screencast
15.1	125
22.1	126	(% class="task-list" %)
	127	(((
	128
	129
	130	{{task reference="/Tasks/Task_30" status="InProgress"}}
	131	login in jupyter hub
	132	{{/task}}
	133
	134	{{task reference="/Tasks/Task_31" status="InProgress"}}
	135	login maxwell display
	136	{{/task}}
	137
	138	{{task reference="/Tasks/Task_32" status="InProgress"}}
	139	use slix
	140	{{/task}}
	141
	142	{{task reference="/Tasks/Task_33" status="InProgress"}}
	143	use hdfview plugin in jupterLab
	144	{{/task}}
	145
	146	{{task reference="/Tasks/Task_34" status="InProgress"}}
	147	create conda env with flashh5
	148	{{/task}}
	149	)))
	150	{{/task}}
	151	)))
	152
2.1	153	\\
	154
15.1	155	\\
	156
21.1	157	----
2.1	158
21.1	159	==== under review ====
4.1	160
	161	\\
	162
8.1	163	{{code language="bash"}}
9.1	164	conda create -n flashh5 python=3.10 # 3.10 not necessary, but would prefer 3.8 or higher
8.1	165	source activate flashh5
10.1	166	conda install ipython numpy pandas #TODO: fix dependcies
8.1	167	conda install -c https://www.desy.de/~cpassow/condarepo/ flashh5
	168
	169	## on jhub
11.1	170	conda install ipykernel
8.1	171	python -m ipykernel install --user --name flashh5 --display-name "flashh5"
	172
	173
11.1	174	## to remove on jhub
	175	## delete from: /home/$USER/.local/share/jupyter/kernels/
8.1	176	{{/code}}
	177
	178	\\
	179
6.1	180	{{code language="py" title="moved to repository?"}}
4.1	181	class RunDirectory:
	182
21.1	183	def get_run_table(): # more or less information? RunComment \| Number of Files \| start & stop time ?
4.1	184	...
	185
21.1	186	def get_run(daq, run_number): # daq is not needed!
	187
4.1	188	...
	189
	190
25.1	191	class Run: # constructor optional without RunDirectory or use there self.path
4.1	192
7.1	193	def get_files():
4.1	194	...
	195
18.1	196	def get_channels(): # of file #1
4.1	197	...
	198
8.1	199	def get_start_time(): # better as attribute?
4.1	200	...
	201
8.1	202	def get_stop_time(): # which? \| better as attribute?
4.1	203	...
12.1	204
19.1	205	def to_df(daq_map): # to_df(daq_map, slice) slice=[0:4] -> throw Exception
4.1	206	...
	207
7.1	208	def to_series(channel):
4.1	209	...
	210
7.1	211	def to_array(channel):
4.1	212	...
	213	{{/code}}
	214
6.1	215	{{code language="py" title="ideas"}}
10.1	216	run.to_df(daq_map)
	217	run.to_series(daq_adr or daq_map) # on channel only?
	218	run.to_array(daq_adr) # on channel only?
5.1	219
21.1	220	## interesting?
10.1	221	# run.to_dask(daq_map)
	222	# run.to_xarray(daq_map)
5.1	223	{{/code}}
	224
4.1	225	\\
30.1	226
	227	\\

Wiki source code of DEMO - Working with FLASH data

Applications

Navigation