The best GPU for an LLM is the cheapest one that holds the model, the cache and the optimizer, and that you can actually start today. Not the newest badge. This guide sizes the job first, then points at the cloud hour. Every price lives on the GPU pricing chart and the rent a GPU page.
Step 1: count the bytes
Weights take parameters times bytes per parameter. FP16 is 2 bytes, FP8 or INT8 is 1, 4-bit is about 0.5. A 7B model is about 14 GB at FP16. A 70B model is about 140 GB at FP16 and about 35 to 40 GB at 4-bit. KV cache comes on top and grows with context length and batch size.
Training is hungrier. Full fine-tuning with Adam in mixed precision needs on the order of 16 bytes per parameter for weights, gradients and optimizer states, before activations. LoRA and QLoRA cut that sharply, which is why they run on a single card. The VRAM fit tool does this arithmetic for you.
Best GPU for LLM inference
- Up to about 13B, any precision: a 24 GB card such as RTX 4090 or A10 is enough, and it is the cheapest hour.
- Up to about 30B quantized: RTX 5090 with 32 GB, or L40S with 48 GB and FP8.
- 70B-class at 4-bit with real context: L40S, RTX PRO 6000 with 96 GB, or an 80 GB A100 or H100.
- Long context or big batches on 70B+: H200 with 141 GB, or B200 when the row is in stock.
Decode is memory-bandwidth-bound, so HBM cards pull ahead at small batch. A cheaper GDDR card can still win on cost per token when the model fits and the batch is large. Benchmark your own model before you commit a month. If you are choosing between workstation and datacenter boards, read RTX PRO 6000 vs H100 and L40S vs H100.
Best GPU for fine-tuning
- LoRA or QLoRA on 7B to 13B: a single 24 GB card.
- QLoRA on 30B to 70B: 48 to 96 GB. L40S, RTX PRO 6000 or A100 80 GB.
- Full fine-tune of 7B to 13B: 80 GB plus, often several GPUs, because optimizer states dominate.
- Anything multi-GPU: NVLink matters. H100 SXM, H200 or B200.
Best GPU for AI training and deep learning
For serious training the card is a cluster decision: NVLink or InfiniBand, memory per GPU, and whether the row is on-demand or a quote. H100 is still the default hour most catalogs can sell you. H200 helps when memory is the wall. B200 helps when FP4 and NVLink 5 are real in your stack and the row is live. See H100 vs H200 and B200 vs H200.
For classic deep learning on smaller models, vision and research notebooks, a 24 to 48 GB card is enough and an A100 is a mature option. Do not pay Hopper money for a job that fits a 4090.
Best GPU for Stable Diffusion and image generation
Diffusion is VRAM and bandwidth, not NVLink. SDXL-class models run on 12 to 16 GB. Flux-class models are happier with 24 GB or more unless you quantize. A 4090 or 5090 is usually the cheapest hour, and an L40S is the datacenter option. For stacked pipelines with several large models loaded at once, step up to 48 or 96 GB. See RTX 5090 vs RTX 4090.
Cheapest GPU for AI: how to shop it
Cheapest means cheapest that fits. Rank by the lowest on-demand hour among cards with enough VRAM, then check stock and billing. Marketplaces are the cheapest hour, and they are the least predictable. Read on-demand vs spot before you trust a discount, and use the hour cost calculator to price the whole job instead of one hour.
FAQ: best GPU for LLM work
What GPU do I need to run a 70B model?
At 4-bit, one 48 to 96 GB card is enough for weights and a modest cache. At 16-bit you need about 140 GB, so two 80 GB GPUs or one H200 plus headroom.
Is a 4090 good enough for LLM fine-tuning?
For LoRA or QLoRA on 7B to 13B, yes. It is the cheapest hour that works. For larger models or full fine-tunes, you need more VRAM.
Is H100 overkill for inference?
Often. If the model fits 48 GB and you do not need HBM bandwidth, an L40S or PRO 6000 can win on cost per token. If you serve large models with big caches, H100 and H200 earn their hour.
Which is the best cloud GPU for AI on a budget?
The cheapest card that holds your model. Start with the VRAM, then sort the pricing chart by hour.
Some outbound links in these articles may be affiliate links. Rankings on GPUBeacon stay independent.