SBC staff recommend that you organize your data into sub-directories as you collect and process data at SBC.
In addition, we HIGHLY recommend that you backup ALL of the data from your SBC HOME DIRECTORY ($HOME) to a single disk or storage system. Plan on needing to backup as much as 250GB of data per day of scheduled beam time.
If you must backup to separate disks, you should realize that the size (free space) of your backup disks could influence how you should organize data collection and data processing. It will simplify the data backup process if you create sub-directories under $HOME (for example, DATA1 and DATA2) that correspond to the separate backup disks that you intend to use, and then limit the amount of data that you place in each sub-directory to the available space on the corresponding backup disk.
Standard Linux Commands
One method of data backup supported at SBC is a network copy.
The network copy method is used to transfer data from an SBC Linux Workstation, to a computer at your home institute (typically running Linux), or to a laptop computer that you bring with you to SBC.
Copying data to a computer at your home institute using standard Linux commands such as “rsync” or “scp” requires, at a minimum, SSH access through your home institute firewall to reach your computer. If you run the data transfer commands from an SBC computer then your firewall must permit incoming ssh traffic from SBC.
If your laptop computer is running a Microsoft Windows operating system, you need to install client software onto it that can transfer files through ssh (such as FileZilla or winSCP).
Linux and Macintosh OS X have ssh clients built-in and can run an SBC-provided script called RemoteBackupScript.csh to perform the backup. Instructions for using this script are given in Using Remote Backup Script.
One linux computer is reserved strictly for data backup purposes, and serves both beamlines 19-ID and 19-BM:
rose.gp.sbc.aps.anl.gov
SBC RemoteBackupScript
The RemoteBackupScript.csh is intended to be run on your computer, either a laptop that you bring to the beamline, or one of the computers at your home institute which has been authorized by SBC as an NX Client computer (since NX requires ssh access).
Linux and Macintosh OS X have ssh client commands built-in, and can run the SBC-provided script.
RemoteBackupScript.csh uses the “rsync” command to perform the backup..
Pre-formated Disk
A different recommended method of data backup supported at SBC is to copy data to a user-provided USB disk that is directly attached to an SBC Linux Workstation.
SBC currently supports USB disks which are formatted for either Windows (NTFS or FAT32), or LINUX (ext3 or ext4).
You may then connect your disk directly to an SBC Linux Workstation and run an SBC script called HomerArchiver to perform the backup.
Alternatively, if your disk is not a compatible format, you may attach it to your own laptop and perform a network copy, as described above.