Tight. Expect paging.
Open H100 listings →VRAM calculator
VRAM calculator for cloud GPUs
Weights, KV cache, LoRA and trainer states against H100 80 GB, H200 141 GB or B200. If you cannot name the spill, you do not need the upgrade.
VRAM ≈ weights + KV cache + adapters + trainer states + activations.
- Weights70.0 GB
- KV cache8.4 GB
- Activations0.8 GB
How to use it
Four steps, then confirm the live SKU
VRAM ≈ weights + KV cache + adapters + trainer states + activations.
- 01
Name the model size
Parameters in billions. 70B is 70. A 8B LoRA is not a 70B full fine-tune. Do not mix them.
- 02
Pick precision and workload
BF16 weights are 2 bytes. FP8 is 1. INT4 is 0.5. Inference is weights plus KV. LoRA adds adapters. Full fine-tune adds optimizer states. That is where 80 GB dies.
- 03
Set context and batch
Long sequence blows the KV cache. Batch blows activations. If you were shrinking context to survive, you have a memory problem, not a FLOPs problem.
- 04
Read fit, tight, or spill
Fits means headroom. Tight means expect paging. Spill means the cheap H100 is the expensive one. This is an envelope, not nvidia-smi.
VRAM before TFLOPS
Fine-tunes and long-context serving die on memory first. If Adam states plus activations do not fit, extra TFLOPS are an expensive fan. I built this calculator so you can name the spill before you rent H200 because a landing page said “better.”
A 70B in BF16 is about 140 GB of weights alone. That does not fit an H100. In FP8 it is about 70 GB, plus KV. That is why the default here is FP8, not a FLOPs chart.
What the breakdown means
Weights are parameters times bytes per parameter. KV cache grows with sequence length and batch. LoRA adapters are a small extra. Full fine-tune trainer states are the expensive part: optimizer, gradients, activations.
I use a crude envelope. Your real trainer will differ. If this says spill, believe it. If this says fits, still smoke-test. Do not skip nvidia-smi.
- If the floor is ≤ 80 GB, H100 is the hour to beat.
- If the floor is 80–141 GB, H200 can drop GPU count.
- If you need 180 GB+ without stacking Hopper, look at B200 or MI300X.
What this is not
It is not a profiler. It does not know your framework, your FlashAttention build, or whether you offload to CPU. It will not save a job that pages.
Use it to fail the wrong SKU in thirty seconds. Then rent the card whose memory matches the job.
Catalog from-prices are a snapshot. Estimates are envelopes, not quotes. Confirm the live SKU, region and stock with the provider before you rent.
FAQ
VRAM calculator for cloud GPUs questions
Does Llama 70B fit on an H100?+
Adapters and FP8/INT4 serving often do. Full fine-tune with optimizer states often wants more than 80 GB or more than one GPU. Put 70 in parameters, pick the precision and workload, and read the breakdown.
Why is this different from a FLOPs comparison?+
Because memory is the bill. If you were swapping or shrinking context to survive on H100, H200 can win even at a higher hourly rate. If you already fit, stay.
Is this exact?+
No. It is an envelope: weights + KV + adapters + trainer + activations. Confirm with nvidia-smi on a five-minute smoke test before a 48-hour run.
H100 or H200 for long context?+
Long context is a KV cache problem. If this calculator shows spill on 80 GB and fit on 141 GB, H200 can drop GPU count. That is how it wins the bill.