Available GPU types on the cluster, from least to most powerful. Use the
gpu: field in tccp.yaml or check live availability
with gpus in the REPL.
| GPU | VRAM | best for | notes |
|---|---|---|---|
t4 |
16 GB | Inference, light training | Oldest Turing card. Slow FP32 but decent FP16. Good for small models and debugging. |
p100 |
16 GB | Legacy workloads | Pascal generation. No tensor cores. Mainly useful when nothing else is free. |
rtx_6000 |
24 GB | Medium training | Turing with tensor cores. Decent for mid-size models. |
v100 |
32 GB | General training | Volta. First-gen tensor cores. Solid all-rounder, good FP16 throughput. |
a100-40gb |
40 GB | Large model training | Ampere. 3rd-gen tensor cores, TF32, BF16. Fast interconnect. Great price/perf. |
rtx_a5000 |
24 GB | Medium training, multi-GPU | Ampere workstation GPU. 8 per node — good for data-parallel training. |
l40 |
48 GB | Large models, fine-tuning | Ada Lovelace. Good VRAM for big batch sizes or large models. |
rtx_a6000 |
48 GB | Large models | Ampere workstation. 48 GB VRAM, 8 per node. |
rtx_6000ada |
48 GB | Large models, fast training | Ada Lovelace workstation. Newer arch than rtx_a6000. |
a100-80gb |
80 GB | LLMs, very large models | The workhorse. 80 GB handles most LLM fine-tuning. High bandwidth. |
l40s |
48 GB | Fast training, inference | Ada Lovelace server GPU. Better FP8 than L40. Newer generation. |
h100 |
80 GB | LLMs, fastest training | Hopper. 4th-gen tensor cores, FP8, Transformer Engine. Top-tier throughput. |
t4 — cheapest, 16 GB is enoughv100 or a100-40gba100-80gb or l40sa100-80gb or h100, possibly multi-GPUrtx_a5000 (8 per node) or a100-80gb:2l40 / rtx_a6000 / rtx_6000ada (all 48 GB)gpu: none or omit it with type: pythonRough VRAM needs for common tasks:
| task | approx VRAM | suggested GPU |
|---|---|---|
| ResNet/EfficientNet training | 4–8 GB | t4 |
| BERT fine-tuning | 8–12 GB | t4, v100 |
| GPT-2 (124M) training | 8–12 GB | t4, v100 |
| Stable Diffusion fine-tuning | 16–24 GB | v100, rtx_6000 |
| LLaMA 7B LoRA fine-tuning | 16–24 GB | v100, a100-40gb |
| LLaMA 7B full fine-tuning | 32–40 GB | a100-40gb |
| LLaMA 13B LoRA fine-tuning | 32–40 GB | a100-40gb, a100-80gb |
| LLaMA 13B full fine-tuning | 60–80 GB | a100-80gb |
| LLaMA 70B LoRA (QLoRA 4-bit) | 40–48 GB | a100-80gb, l40s |
| Inference (quantized) | model size / quant bits | varies |
Set a GPU at the project level or per-job:
# project-wide default gpu: a100 # per-job override jobs: train: script: train.py gpu: a100-80gb gpu_count: 2
Shorthand with colon syntax:
gpu: a100-80gb:2 # equivalent to gpu: a100-80gb + gpu_count: 2
When you set a GPU, tccp auto-bumps memory to 32G
(64G for 80GB GPUs) and cpus to 4. Override
if you need more.
v