Node Information
Features
Features are requested under the --constraints option. Because the cluster is consortium of hardware, attributes allow the user to specify which type of node they wish to use (e.g. ib, edr_ib, intel)
#SBATCH --constraint="intel"
#SBATCH --constraint="intel&ib"
| Feature | Description |
|---|---|
compile |
Code compiled on these nodes will run on all nodes in the cluster |
intel |
Intel CPUs |
amd |
AMD CPUs |
ib |
At least FDR (56Gb/s) Infiniband connection |
edr_ib |
EDR (100Gb/s) Infiniband connection |
hdr_ib |
HDR100 (100Gb/s) Infiniband connection |
noib |
Without Infiniband connection |
avx2 |
AVX2 instruction set |
avx512 |
AVX512 instruction set |
k80 |
NVIDIA K80 GPUs. Must request --gres to be assigned GPU. |
v100 |
NVIDIA V100 GPUs. Must request --gres to be assigned GPU. |
a100 |
NVIDIA A100 GPUs. Must request --gres to be assigned GPU. |
q6000 |
NVIDIA Quadro RTX 6000 GPUs. Must request --gres to be assigned GPU. |
q8000 |
NVIDIA Quadro RTX 8000 GPUs. Must request --gres to be assigned GPU. |
double |
Double precision GPUs. Must request --gres to be assigned GPU. |
Memory Limits
We reserve a chunk of memory per node for system services to prevent the node from crashing. This varies with the amount of memory reported by the server.
This also goes for --mem-per-core. You'll have to take the number of cores requested per node and multiply that by your --mem-per-core value and make sure it does not go above the allowed limit.
| Total amount of memory on node | Amount allowed to request |
|---|---|
| 32 GB | 30 GB |
| 64 GB | 61 GB |
| 128 GB | 125 GB |
| 192 GB | 186 GB |
| 256 GB | 251 GB |
| 384 GB | 376 GB |
| 512 GB | 503 GB |
| 768 GB | 754 GB |
| 1 TB | 1007 GB |