Node Information
Features
Features are requested under the --constraint option. Because the cluster is consortium of hardware, attributes allow the user to specify which type of node they wish to use (e.g. ib, edr_ib, intel)
Requesting Features/Constraints
To request a feature/constraint, you must add the following to your submit script.
#SBATCH --constraint=<feature_name>
<feature_name> is one of the features defined below.
To reqeust multiple features/constraints, you must add the following line to your submit script using one of the following operators below:
| Operator | Operator Symbol | Example |
|---|---|---|
| AND | & (ampersand) | #SBATCH --constraint="intel&ib" |
| OR | | (pipe) | #SBATCH --constraint="edr_ib|hdr100_ib" |
Feature Table
| Feature | Description |
|---|---|
intel |
Intel CPUs |
amd |
AMD CPUs |
nvidia |
NVIDIA GPU |
amdgpu |
AMD GPU |
ib |
At least EDR (100Gb/s) Infiniband connection |
edr_ib |
EDR (100Gb/s) Infiniband connection |
hdr100_ib |
HDR100 (100Gb/s) Infiniband connection |
noib |
Without Infiniband connection |
avx2 |
AVX2 instruction set |
avx512 |
AVX512 instruction set |
a40 |
NVIDIA A40 GPUs. Must request --gres to be assigned GPU. |
l40 |
NVIDIA L40 GPUs. Must request --gres to be assigned GPU. |
v100 |
NVIDIA V100 GPUs. Must request --gres to be assigned GPU. |
a100 |
NVIDIA A100 GPUs. Must request --gres to be assigned GPU. |
q6000 |
NVIDIA Quadro RTX 6000 GPUs. Must request --gres to be assigned GPU. |
q8000 |
NVIDIA Quadro RTX 8000 GPUs. Must request --gres to be assigned GPU. |
double |
Double precision GPUs. Must request --gres to be assigned GPU. |
single |
Single precision GPUs. Must request --gres to be assigned GPU. |
Memory Limits
We reserve a chunk of memory per node for system services to prevent the node from crashing. This varies with the amount of memory reported by the server.
This also goes for --mem-per-core. You'll have to take the number of cores requested per node and multiply that by your --mem-per-core value and make sure it does not go above the allowed limit.
| Total amount of memory on node | Amount allowed to request |
|---|---|
| 32 GB | 30 GB |
| 64 GB | 61 GB |
| 128 GB | 125 GB |
| 192 GB | 186 GB |
| 256 GB | 251 GB |
| 384 GB | 376 GB |
| 512 GB | 503 GB |
| 768 GB | 754 GB |
| 1 TB | 1007 GB |