What can I run?
Pick a local AI model and see the cheapest GPUs that hold it, one card or a set of the same card, with what they cost on eBay right now, the memory to spare and about how fast they run it. Decision models are in the list too, timed per question. Used and server cards are included.
The cheapest setups right now
Live on eBay| Setup | VRAM | Speed (est.) | Price now | Power | Software |
|---|---|---|---|---|---|
| 2 × Tesla P40 | 48 GB0.0 GB spare | 4.0–6.1 tokens/s | $540.00the 2 cheapest listings | 500 W | CUDA 8.0 to 12.9 (not CUDA 13) |
| 4 × Tesla P100 PCIe 16 GB | 64 GB14.0 GB spare | 8.6–13 tokens/s | $721.00the 4 cheapest listings | 1,000 W | CUDA 8.0 to 12.9 (not CUDA 13) |
| 4 × Radeon VII | 64 GB14.0 GB spare | 8.4–14 tokens/s | $1,023the 4 cheapest listings | 1,200 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 4 × Instinct MI50 16 GB | 64 GB14.0 GB spare | 8.4–14 tokens/s | $1,087the 4 cheapest listings | 1,200 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 4 × Tesla V100 PCIe 16 GB | 64 GB14.0 GB spare | 11–16 tokens/s | $1,327the 4 cheapest listings | 1,000 W | CUDA 9.0 to 12.9 (not CUDA 13) |
| 2 × Instinct MI50 32 GB | 64 GB16.0 GB spare | 8.4–14 tokens/s | $1,329the 2 cheapest listings | 600 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 4 × Radeon Pro VII | 64 GB14.0 GB spare | 8.4–14 tokens/s | $1,630the 4 cheapest listings | 1,000 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 4 × Radeon RX 6800 | 64 GB14.0 GB spare | 4.2–7.2 tokens/s | $1,658the 4 cheapest listings | 1,000 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 4 × Radeon RX 7600 XT | 64 GB14.0 GB spare | 2.4–4.0 tokens/s | $1,660the 4 cheapest listings | 760 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 2 × Arc Pro B60 24 GB | 48 GB0.0 GB spare | 3.7–6.4 tokens/s | $1,699the 2 cheapest listings | 400 W | No CUDA (oneAPI / Vulkan) |
| 2 × Radeon Pro W6800 | 64 GB16.0 GB spare | 4.2–7.2 tokens/s | $1,760the 2 cheapest listings | 500 W | ROCm (on AMD's support list) |
| 4 × Quadro RTX 5000 | 64 GB14.0 GB spare | 5.2–7.9 tokens/s | $1,839the 4 cheapest listings | 920 W | CUDA 10.0 and newer |
| 2 × Titan RTX | 48 GB0.0 GB spare | 7.9–12 tokens/s | $1,840the 2 cheapest listings | 560 W | CUDA 10.0 and newer |
| 4 × A2 | 64 GB14.0 GB spare | 2.3–3.5 tokens/s | $1,858the 4 cheapest listings | 240 W | CUDA 11.1 and newer |
| 4 × Arc A770 16 GB | 64 GB14.0 GB spare | 4.6–7.9 tokens/s | $1,872the 4 cheapest listings | 900 W | No CUDA (oneAPI / Vulkan) |
Per question: scaled from the model's reported speed on an RTX PRO 6000 by each card's compute and memory bandwidth, within about half either way. Licenses, scores and sources: decision models.
A set of cards is priced at that many of the cheapest live listings, shipping included.
By model
- Llama 3.1 8B 6 GB at Q4; now Tesla P100 PCIe 12 GB, $128.00
- Qwen3.5 9B 7 GB at Q4; now Tesla P100 PCIe 12 GB, $128.00
- Gemma 4 12B 8 GB at Q4; now Tesla P100 PCIe 12 GB, $128.00
- Qwen3 14B 11 GB at Q4; now Tesla P100 PCIe 12 GB, $128.00
- gpt-oss 20B 14 GB at Q4; now Tesla P100 PCIe 16 GB, $170.00
- Mistral Small 3 24B 16 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Gemma 4 26B-A4B 16 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Gemma 3 27B 21 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Qwen3.8 27B 18 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Qwen3 30B-A3B 20 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Gemma 4 31B 20 GB at Q4; now 2 × Tesla P100 PCIe 12 GB, $257.00
- Qwen3 32B 23 GB at Q4; now Tesla P40, $270.00
- Qwen3.6 35B-A3B 22 GB at Q4; now Tesla P40, $270.00
- Llama 3.3 70B 46 GB at Q4; now 2 × Tesla P40, $540.00
- Llama 4 Scout 109B 68 GB at Q4; now 3 × Tesla P40, $810.00
- gpt-oss 120B 72 GB at Q4; now 4 × Tesla P40, $1,083
- Qwen3.5 122B-A10B 77 GB at Q4; now 4 × Tesla P40, $1,083
- Qwen3.8 Flash-Next 180B 110 GB at Q4; now 4 × Instinct MI50 32 GB, $2,726
- Qwen3 235B-A22B 145 GB at Q4; now 3 × A16, $6,789
- GLM-5.3 Flash 320B 195 GB at Q4; now 4 × A16, $9,171
- Qwen3.5 397B-A17B 245 GB at Q4; now 4 × A16, $9,171
- Llama 3.1 405B 250 GB at Q4; now 4 × A16, $9,171
- DeepSeek R1 / V3 671B 408 GB at Q4; now 3 × H200 NVL, $155,299
- DeepSeek V4.1 Flash 763B 463 GB at Q4; now 4 × H200 NVL, $219,319
- Kimi K3 2.8T 1686 GB at Q4
Decision models (OpenJev, Clef, Decider): they score options instead of writing text, so they're compared by the time per question.
How it works
The memory a model needs is its weights at the quantisation you pick, plus the attention cache for the context and a little for working buffers. Q4 is what most people run at home, close to full quality at a third of the size. Each card keeps about 1 GB for itself, so two 24 GB cards hold 46 GB.
Writing each token reads every active weight once, so the speed is about the memory bandwidth divided by the model's size, times what the software gets out of the card (less off CUDA). Cards sharing a model layer by layer take turns, so a set runs about as fast as one card would. Treat it as a rough figure for one person chatting; a long prompt takes longer to read in.
Power is the cards' rated power. Over about 1,440 W with the rest of the PC, a set needs more than one 15 A circuit (or a 240 V one), and older cards often need a power cable adapter, which each card's page mentions.