rsync

Rsync is a fast and extraordinarily versatile file copying tool. It synchronizes files and directories between two different locations (or servers). Rsync copies only the differences of files that have actually changed. An important feature of Rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction. It can copy or display directory contents and copy files, optionally using compression and recursion. It is similar to SCP in that both a source and a destination must be specified, one of which may be remote.

Subdirectory Behavior

How can we recursively transfers all files from the directory /src/directory-name on a local machine into /data/tmp/ on a remote machine?

A trailing slash on the source changes the behavior of rsync. The inclusion of the trailing slash avoids creating an additional directory level at the destination. You can think of a trailing / on a source as meaning “copy the contents of this directory” as opposed to “copy the directory by name”, but in both cases the attributes of the containing directory are transferred to the containing directory on the destination.

Trailing Slash

The below command will copy all of the contents of directory-name into tmp, excluding the parent folder.

rsync -ravz  computer-name:/src/directory-name/  user@remote.host:/data/tmp --log-file=hpc-user-rsync.log 

The folder remote.host:/data/tmp will then contain anything it held previously in addition to the subfolders of directory-name.

No Trailing Slash

On the other hand, the below command will copy directory-name as a parent folder into tmp resulting in a new directory /data/tmp/directory-name. The contents of directory-name will appear exactly as they did on the local machine.

rsync -ravz  computer-name:src/directory-name  user@remote.host:/data/tmp --log-file=hpc-user-rsync.log 

Note that including computer-name is optional when referring to the local machine. Log files are optional but recommended.

Additional Options

Flag	Meaning
-r	Recursive mode; loop through contents of all subfolders
-a	Archive mode; will preserve time stamps and other metadata
-v	Increase verbosity
-z	Compress file data during the transfer
--log-file=FILE	Log everything done in specified FILE