Which GPU should you rent?
Start from the bytes, not the badge. Weights, optimizer states and KV cache decide the GPU. Price comes second.
- LLM inference up to ~30B (quantized): a 24 to 48 GB card is enough. RTX 4090, RTX 5090 or L40S.
- 70B-class inference: 80 to 96 GB on one card. RTX PRO 6000, A100 or H100. Add a larger KV cache and H200 starts to pay.
- LoRA and QLoRA fine-tuning: 24 GB for 7B to 13B, 48 to 96 GB for 30B and up.
- Training and multi-GPU jobs: NVLink matters. H100 SXM, H200 or B200 when the row is in stock.
- Image and video generation: VRAM and bandwidth, not NVLink. Consumer and Ada cards are usually the cheapest hour.
Not sure the model fits? Use the VRAM fit tool, then price the run with the hour cost calculator. If two cards look close, open a duel: RTX PRO 6000 vs RTX 5090, H100 vs H200 or L40S vs A100.
Where to rent: marketplace, managed cloud or hyperscaler
Marketplaces such as Vast.ai have the cheapest hour and the most variance. Managed clouds such as RunPod trade a little price for a product. Specialist clouds like Lambda and CoreWeave sell clusters and support. Hyperscalers sell the account you already have, at a higher hour. Read on-demand vs spot before you chase the cheapest row.
