Overview

Log In Before Transferring

To make transfers to/from HPC, you will need to have logged into your account at least once. If you have not, you may encounter "directory does not exist" errors. This is because your home directory is not created until you log in for the first time. See our System Access page for information on logging in.

Designated Data Transfer Node for File Transfers

For efficient file transfers to and from the HPC system, utilize the designated data transfer node, hostname: filexfer.hpc.arizona.edu. This node is optimized for handling large data transfers and is equipped with a high-speed 100 Gb interconnect.

Why Use the Data Transfer Node?

• Optimized Performance.

  The dedicated data transfer node ensures efficient transfer speeds, particularly for large datasets.

• Network Stability.

  Utilizing the data transfer node helps prevent network congestion and potential disruptions on other components of the HPC system.

Do not use hpc.arizona.edu

Using the hostname hpc.arizona.edu for transfers will move your data to the HPC bastion host. The bastion host is not connected to the shared storage array (meaning files stored here will not be accessible on login/compute nodes) and has limited storage capacity. Users are restricted to 10 MB of space on this node and may experience login issues if this is exceeded.

Data Transfers By Size

1. Transfers \(\leq\) 64 MB: For small data transfers, the web portal offers the most intuitive method.
2. Transfers \(<\)100 GB: we recommend SFTP, SCP or Rsync using filexfer.hpc.arizona.edu.
3. Transfers \(>\)100 GB, transfers outside the university, and large transfers within HPC: we recommend using Globus (GridFTP).

Best Practices

• Use the file transfer nodes for large data transfers

  Login and compute nodes are not designed for large file transfers and transfers initiated here may result in network problems. The data transfer nodes (DTNs) are specifically set up for moving large amounts of data and are accessible via the hostname filexfer.hpc.arizona.edu.

• Limit file copy sessions

  You share bandwidth with others. Two or three SCP sessions are probably ok; \(>\)10 is not.

• Consolidate files

  If you are transferring many small files, consider collecting them in a tarball first.

Transfer Software Summary

Software	CLI Interface?	GUI Interface?	Access to Cloud Services?				Notes
			Google Drive	AWS	Box	Dropbox
Globus	✅	✅	✅	✅	❌	❌
SFTP	✅	✅	❌	❌	❌	❌
SCP	✅	✅	❌	❌	❌	❌	On Windows, WinSCP is available as a GUI interface
Rsync	✅	✅	❌	❌	❌	❌	Grsync is a GUI interface for Rsync for multiple platforms.
Rclone	✅	❓	✅	✅	✅	✅	Rclone has recently announced they have an experimental GUI.
Cyberduck	✅	✅	✅	✅	✅	✅
iRODS	✅	✅	❌	❌	❌	❌

File Transfers and SSH Keys

Several of the file transfer methods listed below use authentication based on the SSH protocol, including SCP, SFTP, and Rsync. Therefore, adding your SSH Key to the filexfer.hpc.arizona.edu node can allow one to avoid entering passwords when using those methods. See the documentation for adding SSH Keys.