PerGB

Is a used RTX 3090 still worth it for AI?

Not at today's prices: from $1,475 ($61.46 per GB of VRAM) it's number 17 of 52 PCIe cards with 24 GB or more by price per GB. Cheaper per GB now: Tesla P40 ($11.25/GB), Instinct MI50 32 GB ($19.80/GB), Radeon Pro W6800 ($27.50/GB).

For running models, VRAM decides what fits and memory bandwidth decides how fast it writes; the RTX 3090 has both, CUDA support from every AI tool, and NVLink on most cards. Against it: 350 W, and no FP8 (newer cards run some workloads faster). Two of them hold a 70B model at Q4.

24 GB and more, cheapest per GB

CardVRAMLowestPer GBBandwidthSoftware
Tesla P4024 GB$270.00$11.25/GB346 GB/sCUDA
Instinct MI50 32 GB32 GB$633.68$19.80/GB1,024 GB/s—
Radeon Pro W680032 GB$879.99$27.50/GB512 GB/sROCm
Tesla V100 PCIe 32 GB32 GB$975.00$30.47/GB900 GB/sCUDA
Tesla V100S PCIe32 GB$1,099$34.34/GB1,134 GB/sCUDA
A1664 GB$2,200$34.37/GB800 GB/sCUDA
Arc Pro B60 24 GB24 GB$849.00$35.38/GB456 GB/s—
Radeon RX 7900 XTX24 GB$900.00$37.50/GB960 GB/sROCm
Titan RTX24 GB$910.00$37.92/GB672 GB/sCUDA
Instinct MI10032 GB$1,240$38.75/GB1,229 GB/sROCm
Quadro RTX 600024 GB$979.98$40.83/GB672 GB/sCUDA
Quadro RTX 800048 GB$2,000$41.67/GB672 GB/sCUDA

Live eBay prices, shipping included. Every GPU, the best GPUs for local AI.

Prices and picks update every time the site does (live listings checked within the last few hours).