Wiki source code of 06. Data Backup
Last modified by tabermah on 2024-12-10 14:54
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | = 6.1. Backup to USB-drive during beamtime = | ||
| 2 | |||
| 3 | If you bring a USB-drive, you may sync the data onto it during the beamtime. We provide a dedicated PC for this purpose, which is running debian. It is advisable to bring a drive which is known to work fast on Linux systems, as some chipsets might be too slow to keep up with the speed at which you take data. Please also note, that ExFAT is currently not supported. FAT32, NTFS, ext4 should all work fine. | ||
| 4 | |||
| 5 | There is a script prepared, which will continuously sync the data onto your hard-drive. Source and destination path have to be adapted beforehand. Your local contact will help you with this. I(% style="color:#172b4d" %)nstructions(%%) can also be found (% style="color:#172b4d" %)in the hutch. | ||
| 6 | |||
| 7 | There are two possibilities as where to copy your data from: | ||
| 8 | |||
| 9 | * The core file system /asap3/petra3/gpfs/p11/<YEAR>/data/<BEAMTIME>. This is the way we recommend as it has several advantages. You will need to log in with a (% style="color:#172b4d" %) (%%)[[DESY Science Account>>https://xwiki.desy.de/xwiki/bin/view/FSEC/FS-IT%20Topics/Science%20Accounts/||rel="nofollow" shape="rect" style="text-decoration: none;"]] which was added to the beamtimes authorized list by the PI(% style="color:#172b4d" %). | ||
| 10 | * (% style="color:#172b4d" %)The beamline file system /gpfs/current/. You may log in as p11user to do this. Any data resulting from manual processing done on the Maxwell resource will not be visible here and thus not be contained in your backup. As soon as the next user group starts and your beamtime is closed, you will have to stop copying. | ||
| 11 | |||
| 12 | = 6.2. Download your data during or after the beamtime via FTP = | ||
| 13 | |||
| 14 | Data are available for download via FTP protocol through the [[Gamma Portal>>https://gamma-portal.desy.de/pls/apex/f?p=260:101:117413319118083:::::||shape="rect"]] as it is described in [[here>>https://docs.desy.de/asap3/gamma-portal/||shape="rect"]]. | ||
| 15 | |||
| 16 | = 6.3. Download your data during or after the beamtime using Globus = | ||
| 17 | |||
| 18 | A third option is to use [[Globus Online>>url:https://www.globus.org/data-transfer]]: this is available exclusively for Users with Science Accounts and an Institute Subscription to the Globus Group: | ||
| 19 | |||
| 20 | * check with your local administrator if your institute is a subscriber of the Globus Group. If so, log in into [[Globus Online>>url:https://app.globus.org/]] by choosing your Organization and Identity Provider | ||
| 21 | * create an Endpoint on your machine by downloading the suitable "Globus Connect" for you operating system | ||
| 22 | * choose DESY in the "Collection Search" and browse to your Beamtime folder | ||
| 23 | * start transferring data in the file manager section - further instructions can be found on [[FS-EC's XWiki page>>https://xwiki.desy.de/xwiki/bin/view/FSEC/FS-IT%20Topics/Globus%20Online/]] | ||
| 24 | |||
| 25 |