Skip to Content

Changes for page 05. Data processing

Last modified by tabermah on 2024-12-11 11:17
From version 19.1
edited by pompidor
on 2021-12-03 11:12
Change comment: There is no comment for this version
To version 27.1
edited by tabermah
on 2023-03-01 14:59
Change comment: There is no comment for this version

Summary

Details

Page properties
Author
... ... @@ -1,1 +1,1 @@
1 -XWiki.pompidor
1 +XWiki.tabermah
Content
... ... @@ -2,26 +2,28 @@
2 2  
3 3  **11.1. Automatic data processing**
4 4  
5 -Unless you un-check the option for the specific acquisition run, autoprocessing will be executed. For standard rotational data autoprocessing is via XDSAPP (Sparta et al. (2016) //J. Apple. Cryst.// **49**, 1085-1092) for strategy calculation we use Mosflm (Leslie //Acta D//**62** (2006) 48-57) and Spotfinder ((% style="color: rgb(0,0,0);text-decoration: none;" %)Zhang et al. (2006). (% style="text-decoration: none;text-align: left;" %)//J. Appl. Cryst.//(% style="color: rgb(0,0,0);text-decoration: none;" %) (% style="text-decoration: none;text-align: left;" %)**39**(% style="color: rgb(0,0,0);text-decoration: none;" %), 112–119)(%%) for producing the heatmap for the grid scans. The results are compiled into HTML-files to give you a convenient way of checking them. The output files are in the processed folders of the corresponding datasets.  A self refreshing index of all results is created in the beamtimes processed folder. During the beamtime you can open it from file::~/~//gpfs/current/processed/index.html in a browser. The HTML-file is also contained in your data backup.
5 +Unless you un-check the option for the specific acquisition run, autoprocessing will be executed. For standard rotational data autoprocessing is via XDSAPP ([[https:~~/~~/www.helmholtz-berlin.de/forschung/oe/ps/macromolecular-crystallography/hzb-mx-software/xdsapp3/index_en.html>>url:https://www.helmholtz-berlin.de/forschung/oe/ps/macromolecular-crystallography/hzb-mx-software/xdsapp3/index_en.html||shape="rect"]], current version 3.1.9) (Sparta et al. (2016) //J. Apple. Cryst.// **49**, 1085-1092) for strategy calculation we use Mosflm (Leslie //Acta D//**62** (2006) 48-57) and Spotfinder ((% style="color: rgb(0,0,0);text-decoration: none;" %)Zhang et al. (2006) (% style="text-decoration: none;text-align: left;" %)//J. Appl. Cryst.//(% style="color: rgb(0,0,0);text-decoration: none;" %) (% style="text-decoration: none;text-align: left;" %)**39**(% style="color: rgb(0,0,0);text-decoration: none;" %), 112–119)(%%) for producing the heatmap for the grid scans. The results are compiled into HTML-files to give you a convenient way of checking them. The output files are in the processed folders of the corresponding datasets.  A self refreshing index of all results is created in the beamtimes processed folder. During the beamtime you can open it from file::~/~//gpfs/current/processed/index.html in a browser. The HTML-file is also contained in your data backup.
6 6  
7 7  \\
8 8  
9 9  **11.2. Manual data processing onsite**
10 10  
11 -Please do not use the control computers of the beamline (haspp11user01-04) nor the haspp11eval01 for data processing (latter is now fully dedicated to autoprocessing.
11 +Please do not use the control computers of the beamline (haspp11user01-04) nor the haspp11eval01 for data processing (latter is now fully dedicated to autoprocessing).
12 12  
13 -Use Maxwell-cluster for data processing as described in the next chapter; on-site, you can directly ssh to username@desy-ps-cpu or use Max-display by connecting to [[https:~~/~~/max-display.desy.de:3443>>url:https://max-display.desy.de:3443||shape="rect"]] via web browser. Access to Maxwell requires a Scientific Account.
13 +Use Maxwell-cluster for data processing as described in the next chapter; on-site, you can directly ssh to username@desy-ps-cpu. Access to Maxwell requires a Scientific Account.
14 14  
15 +Manual data processing after the beamtime or during a remote session is only available for registered [[DESY Scientific Accounts>>doc:FSEC.Science Accounts.WebHome||shape="rect"]].
16 +
15 15  \\
16 16  
17 -**11.3. Manual data processing after your beamtime or during remote session**
19 +//Accessing data from outside the DESY network~://
18 18  
19 -Manual data processing after the beamtime or during a remote session is only available for registered [[DESY Scientific Accounts>>doc:FSEC.Science Accounts.WebHome||shape="rect"]].
21 +Accessing the Maxwell cluster is easiest with Max-display:  connect to [[https:~~/~~/max-fs-display.desy.de:3389/auth/ssh/>>url:https://max-fs-display.desy.de:3389/auth/ssh/||shape="rect"]] via web browser with the credentials of your scientific account.
20 20  
21 -//Accessing data from outside the DESY network~://
22 -In a terminal window:
23 +Accesss via terminal window:
23 23  ssh (% class="Object" %)[[username@desy-ps-ext.desy.de>>mailto:username@desy-ps-ext.desy.de||shape="rect"]](%%)
24 24  ssh username@desy-ps-cpu
26 +
25 25  Give your password when prompted
26 26  Navigate to data by:
27 27  cd /asap3/petra3/gpfs/p11/2020/data/beamtimeID
... ... @@ -42,7 +42,7 @@
42 42  sbatch nxds.sh
43 43  
44 44  //List of software you can use is at~://
45 -(% class="Object" %)[[https:~~/~~/confluence.desy.de/display/IS/Photon+Science>>doc:IS.Photon Science.WebHome||shape="rect"]](%%)
47 +(% class="Object" %)[[https:~~/~~/confluence.desy.de/display/MXW/Photon+Science>>doc:MXW.Photon Science.WebHome||shape="rect"]](%%)
46 46  \\//You can store analysis results in scratch_cc (temporary/testing results) or processed (final results).//
47 47  \\//To use crystallographic programs~://
48 48  module avail
... ... @@ -53,3 +53,7 @@
53 53  \\
54 54  
55 55  \\
58 +
59 +(((
60 +\\
61 +)))
Confluence.Code.ConfluencePageClass[0]
Id
... ... @@ -1,1 +1,1 @@
1 -252809700
1 +353491615
URL
... ... @@ -1,1 +1,1 @@
1 -https://confluence.desy.de/spaces/11/pages/252809700/11. Data processing
1 +https://confluence.desy.de/spaces/11/pages/353491615/11. Data processing