Storage and Transfers

HPC Storage

General Storage

Do you allow users to NFS mount their own storage onto the compute nodes?

No. We NFS mount storage across all compute nodes so that data is available independent of which compute nodes are used. See our page on transferring data for more information.

I can't transfer my data to HPC with an active account. What's wrong?

After creating your HPC Account, your home directory will not be created until you log in for the first time. Without your home directory, you will not be able to transfer your data to HPC. If you are struggling and receiving errors, sign into your account either using the CLI through the bastion or logging into Open OnDemand and then try again.

I accidentally deleted files, can I get them back?

It's unlikely. Backups are not made and anything deleted is permanently erased. If you have deleted an important file, reach out to our consultants as soon as possible and they may be able to retrieve it, however, this is not guaranteed.
To ensure your data are safe, we strongly recommend:

• Making frequent backups, ideally in three places and two formats. Helpful information on moving data can be found on our page Transferring Data.
• Use rm and rm -r with caution as these commands cannot be undone! Consider using rm -i when removing files/directories. The -i flag will prompt you to manually confirm file removals to make really sure they can be deleted.
• You can open a support ticket to request assistance. Files that are deleted may not have been removed from the storage array immediately (though this is not guaranteed), don't wait more than a few days.

My home directory is full, what's using all the space?

If your home directory is full and you can't find what is taking up all the space, it's possible the culprit is a hidden file or directory. Hidden objects are used for storing libraries, cached singularity/apptainer objects, saved R session, Anaconda environments, configuration files, and more. It's important to be careful with hidden, or "dot", files since they often control your environment and modifying them can lead to unintended consequences.

To view the sizes of all the objects (including hidden) in your home, one quick command is du -hs $(ls -A ~), for example:

[netid@junonia ~]$ du -hs $(ls -A ~)
32K     Archives
192M    bin
4.7G    Software
46M     .anaconda
1.9M    .ansys
4.0K    .apptainer
16K     .bash_history
4.0K    .bash_logout
4.0K    .bash_profile
12K     .bashrc
20M     ondemand
      

I'd like to share data I have stored on HPC with an external collaborator, is this possible?

Unfortunately, without active university credentials it is not possible to access HPC compute or storage resources. External collaborates who need ongoing access may apply for Designated Campus Colleague, or DCC, status. This is a process done through HR and will give the applicant active university credentials allowing them to receive HPC sponsorship.
Otherwise, data will need to be moved off HPC and made available on a mutually-accessible platform. This may include (but is not limited to): Google Drive, AWS S3, Box, and CyVerse's Data Store.

Why am I getting the error Received message too long when I try to upload data to HPC?

Check your ~/.bashrc or ~/.bash_profile to see if they are printing any output to the terminal. Transfer software like SCP or SFTP require a "silent" terminal to work successfully. If you find any echo or printf statements, comment them out and retry your transfer.

xdisk

Can someone other than a PI manage a group's xdisk?

Yes, a PI can add a trusted group member as a delegate by following the instructions in our Research and Class Groups page. Once a group member is added as a delegate, they can manage the group's xdisk allocation on behalf of their PI in the user portal.

Who owns our group's /xdisk?

A group's PI owns the xdisk allocation. By default, your PI has exclusive read/write/execute privileges for the root folder /xdisk/<pi_netid>.

Can a PI make their /xdisk writeable for their whole group?

By default, members of a research group only have write access to their subdirectories under /xdisk/<pi_netid>. If they so choose, a PI may allow their group members to write directly to that location by running the following command:

chmod g+w /xdisk/<pi_netid>
      

For more information on Linux file permissions, see our cheat sheet.

Where can group members store their files?

When an xdisk allocation is created, a subdirectory is automatically generated for and owned by each individual group member. Group members can access their individual spaces by:

cd /xdisk/<pi_netid>/<netid>
      

If a user joins the group after the xdisk was created and /xdisk/<pi_netid> is not writeable for group members, contact our consultants and they can create one.

A group member's directory isn't in our /xdisk, how can we add it?

Typically when an xdisk allocation is created, it will automatically generate a directory for each group member. In the unlikely event that it doesn't or, more commonly, a group member is added after the allocation has been created, contact our consultants and they can create one.

Do we need to request an individual allocation within the /xdisk for each user in our group?

No, the full xdisk allocation is available for every member of the group. It's up to group members to communicate with one another on how they want to utilize the space.

Why am I getting xdisk emails?

xdisk is a temporary storage space available to your research group. When it's close to its expiration date, notifications will be sent to all members of your group..

Why am I getting "/xdisk allocations can only be authorized by principal investigators"?

xdisks are managed by your group's PI by default. This means if you want to request an xdisk or modify an existing allocation (e.g., extending the time limit or increasing the storage quota), you will need to consult your PI. Your PI may either perform these actions directly or, if they want to delegate xdisk management to a group member, they may do so by following the instructions under Delegating Group Management Rights.

How can we modify our xdisk allocation?

To modify your allocation's time limit or storage quota, your PI can either do so through the Web Portal under the Storage tab, or via the command line. If your PI would like to delegate management rights to a group member, they may follow the instructions under Delegating Group Management Rights. Once a group member has received management rights, they may manage the allocation through our web portal.

Why am I getting "xdisk: command not found"?

If you're getting errors using xdisk commands in a terminal session, check that you are on a login node. If you are on the bastion host (hostname: gatekeeper), are in an interactive session, or are on the filexfer node, you won't be able to check or modify your xdisk. When you are on a login node, your terminal prompt should show the hostname junonia or wentletrap. You can also check your hostname using the command hostname

Why am I getting errors when trying to extend my allocation?

If you're trying to extend your group's allocation but are seeing something like:

(puma) [netid@junonia ~]$ xdisk -c expire -d 1
invalid request_days: 1
      

for every value you enter, your xdisk has likely reached its maximum time limit. To check, have a delegate or PI go to portal.hpc.arizona.edu, click Manage XDISK, and look at the box next to Duration. If you see 300, your allocation cannot be extended further. If your allocation is at its limit, you will need to back up your data to external storage (e.g., a local machine, lab server, or cloud service). Once your xdisk has been removed (either by expiring or through manual deletion), you can immediately create a new allocation and restore your data. Detailed xdisk information can be found on our HPC High Performance Storage page. You may also want to look at our page on Transferring Data.

Can we keep our xdisk allocation for more than 300 days?

No, once an xdisk has reached its time limit it will expire. It's a good idea to start preparing for this early by making frequent backups and paying attention to xdisk expiration emails.

What happens when our xdisk allocation expires?

Once an xdisk expires, all the associated data are deleted. Deleted data are non-retrievable since HPC is not backed up. It's advised to keep frequent backups of your data on different platforms, for example a local hard drive or a cloud-based service, or (even better) both! Check our Storage documentation for more information on alternative storage offerings.

What's the best way to backup/transfer our data before our xdisk expires?

Before your group's xdisk expires, you'll want to make an external backup of anything you need to keep. External storage options include personal computers, lab servers, external hard drives, or cloud services such as AWS. If you're moving large quantities of data, Globus is a great option. We have instructions in our Globus documentation for setting up and using this software. We strongly recommend making archives (.tar, .zip, files etc.) of large directories prior to transferring them off the system. In general, transfer software struggles with moving many small files and performs much more efficiently moving fewer large files. You will get the better transfer speeds (sometimes by orders of magnitude) if you compress your files prior to transferring them. This can be done on our filexfer node which is designed for large file management operations (hostname: filexfer.hpc.arizona.edu).

Once our xdisk expires, can we request a new one?

Yes, a new xdisk may be requested immediately after the old partition expires. Data, however, may not be transferred directly from the old partition to the new one.

Can a PI have more than one xdisk active at a time?

No, only one xdisk may be active per PI at a given time.

Rental Storage

How do I request rental storage?

Rental storage can be requested by PIs through the user portal. A guide with screenshots can found in our rental storage documentation.

Why can't I see my rental allocation when I log into HPC?

Rental allocations are not mounted on the login or compute nodes. You can access your allocation by logging in to our data transfer nodes: filexfer.hpc.arizona.edu

Can I analyze data stored in /rental directly in my jobs?

No, rental storage is not mounted on the HPC login or compute nodes which means jobs cannot access /rental directly. Data stored in /rental need to be moved to /home, /groups, or /xdisk to be available.

R-DAS Storage

If my VPN connection is dropped, will my connection survive?

Yes, if you reconnect to the VPN, your connection will still be available.

Why am I getting "There was a problem connecting to the server"?

This error is seen if you are not connected to the University VPN.

Can I access R-DAS for HPC usage?

R-DAS is not mounted on the HPC compute nodes or login nodes, and is not meant for running computations. But you can follow the steps in our R-DAS documentation to share data between your R-DAS allocation and your HPC storage (/home, /groups, /xdisk):

Tier 2 AWS Storage

Who can submit a Tier 2 storage request?

A PI must submit their group's storage request. The exception to this is if a group member has been designated xdisk/storage delegate. Delegates may submit a request on behalf of their PI by switching users in the user portal.

How long is the turnaround time to between submitting the storage request and obtaining an S3 account?

In most cases it will be in one business day.

What is the pricing for S3?

You should check the Amazon site. As of March 2022:

• Frequent Access Tier, First 50 TB / Month $0.023 per GB
• Frequent Access Tier, Next 450 TB / Month $0.022 per GB
• Frequent Access Tier, Over 500 TB / Month $0.021 per GB

Can I move my data directly to Glacier if I know it is archival. That way I can skip the S3 expenses?

Not in the first release, but potentially as a future offering.

Is the monthly billing based on average usage?

Yes. The capacity used is recorded daily and the billing is a monthly average.

What is a KFS number?

It is used for accounting purposes and used by your Department's finance specialist. If you are unsure what your KFS number is, contact your department's financial services office.

Can I view my storage in each of S3, Glacier and Deep Glacier?

Yes, you can use the CLI (command line interface) for information about your usage

Can I limit the amount of data that goes into S3 so I can control the expense?

No, but you can track the usage and remove any data that should not be there.

What is the difference between Glacier and Deep Glacier?

Glacier is effectively large, slow disks and Deep Glacier is tape storage.

I use workflows to move data using Google Drive, and to share with others. Will AWS support something similar?

Amazon S3 will likely support what you do. Perhaps our consultants can help to rework your workflows.

Are there charges for data movement?

You will not be charged for data ingress, egress or other operations.

Can I have multiple S3 buckets associated with my account?

Yes

Are there any limits on uploads, e.g. may transfer size/day?

The max file size is 5 TB based on Amazon's official documentation.

What why am I receiving an email: "ALARM: "ua-rt-t2-netid capacity growth alarm" in US West (Oregon)"?

This alert is sent to notify you whenever your storage usage grows by 10% relative to the previous week. This ensures that you're aware of any unintended spikes and the potential resulting costs.

What does this file transfer error mean: "The operation is not valid for the object's access tier"?

This error indicates your data have migrated to an archival storage tier (Glacier or Deep Glacier). They will first need to be restored to the Standard access tier to make them downloadable.