Job Limits
Job Limits
To ensure equitable access and efficient use of our clusters, we impose restrictions on the total compute resources available to individual users, jobs, and HPC groups. These limits vary based on the specific cluster and partition in use.
The table below outlines the maximum resources that can be concurrently used at both the user and group levels.
Open OnDemand jobs have different limits
Note that the below limits apply to interactive terminal jobs and batch jobs. Open OnDemand jobs have further restrictions that are detailed in the Open OnDemand Job Limit section below.
High priority job limits
Buy-in groups are limited to using the resources they purchased. If you are a member of a buy-in group, you can check your specific limits by following the instructions in the Checking Limits and Usage below.
Available hardware
If you're looking for more information on the physical resources available for jobs, see our Compute Resources page for more information.
| Puma | Ocelote | El Gato | |
|---|---|---|---|
| Global Limits | |||
| Maximum Simultaneous User Jobs | 1000 | 1000 | 1000 |
| Maximum Job Walltime | 10 days | 10 days | 10 days |
| Standard Job Resources | |||
| Maximum Memory Per Group | 16996 GB | 10000 GB | 10000 GB |
| Maximum CPUs per Group | 3290 | 1024 | 1024 |
| Maximum GPUs per Group | 4 | 10 | N/A |
| Windfall Job Resources | |||
| Maximum Memory per User | Unlimited | Unlimited | Unlimited |
| Maximum CPUs per User | 6000 | 6000 | 6000 |
| Maximum GPUs per Job | Unlimited | Unlimited | N/A |
| High Priority Job Resources | |||
| Maximum CPUs per Group | Number Purchased | N/A | N/A |
| Maximum GPUs per Group | Number Purchased | N/A | N/A |
Open OnDemand Jobs Limits
Open OnDemand graphical jobs are more limited in size and duration than batch or interactive terminal jobs. Jobs that need more resources or longer runtimes can be submitted as batch jobs.
Single node resources
Note that the maximum CPUs and GPUs listed in the table below are based on the resources available by node. For information, see Compute Resources.
| Puma | Ocelote | El Gato | |
|---|---|---|---|
| Maximum Nodes per Job | 1 | 1 | 1 |
| Maximum CPUs per Job | 94 | 28 (Standard/GPU nodes) 48 (High memory node) |
16 |
| Maximum GPUs per Job | 4 | 2 | N/A |
| Maximum Walltime per Job | 4 days | 4 days | 4 days |
Checking Limits and Usage
You can view job limits for a specific group by using the job-limits <group_name> command in a terminal session. The output will display the current usage of your group and personal jobs, so the results may vary.
Below is an example output from this command for a sample user who is a member of a buy-in group. The buy-in resources available to them are indicated in the High Priority field. If you are not a member of a buy-in group, this field will not appear.
Please note that the output of job-limits is specific to the queried group, meaning the User Limits displayed apply only to jobs submitted by that user using the group's account.
(puma) [<your_netid>@wentletrap ~]$ job-limits <group>
Group Limits: <group>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number
|Running/Limit |Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Standard | -/16998G | -/3290 | -/gres/gpu=4 | -/-
High Priority | -/16998G | -/94 | -/gres/gpu=0 | -/-
--------------------------------------------------------------------------------
User Limits: <your_netid>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number*
|Running/Limit |Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Windfall | -/- | -/6000 | -/- | -/1000
Standard | -/16998G | -/3290 | -/gres/gpu=4 |
High Priority | -/16998G | -/94 | -/gres/gpu=0 |
--------------------------------------------------------------------------------
*Max jobs across all groups and partitions.
Individual Job Limits
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Time
--------------------------------------------------------------------------------
Windfall | 16998G | 6000 | | 10-00:00:00
Standard | 16998G | 3290 | gres/gpu=4 | 10-00:00:00
High Priority | 16998G | 94 | gres/gpu=0 | 10-00:00:00
--------------------------------------------------------------------------------
(ocelote) [<your_netid>@wentletrap ~]$ job-limits <group>
Group Limits: <group>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number
| Running/Limit | Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Standard | -/10T | -/1024 | -/gres/gpu=10 | -/-
--------------------------------------------------------------------------------
User Limits: <your_netid>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number*
| Running/Limit | Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Windfall | -/- | -/6000 | -/- | -/1000
Standard | -/10T | -/1024 | -/gres/gpu=10 |
--------------------------------------------------------------------------------
*Max jobs across all groups and partitions.
Individual Job Limits
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Time
--------------------------------------------------------------------------------
Windfall | 8064G | 6000 | | 10-00:00:00
Standard | 8064G | 1024 | gres/gpu=10 | 10-00:00:00
--------------------------------------------------------------------------------
(elgato) [<your_netid>@wentletrap ~]$ job-limits <group>
Group Limits: <group>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number
| Running/Limit | Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Standard | -/10T | -/1024 | -/- | -/-
--------------------------------------------------------------------------------
User Limits: <your_netid>
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Job Number*
| Running/Limit | Running/Limit | Running/Limit |Submitted/Limit
--------------------------------------------------------------------------------
Windfall | -/- | -/6000 | -/- | -/1000
Standard | -/10T | -/1024 | -/- |
--------------------------------------------------------------------------------
*Max jobs across all groups and partitions.
Individual Job Limits
--------------------------------------------------------------------------------
Job Type | Memory | CPU | GPU | Time
--------------------------------------------------------------------------------
Windfall | 8064G | 6000 | | 10-00:00:00
Standard | 8064G | 1024 | | 10-00:00:00
--------------------------------------------------------------------------------