PerGB

What can I run?

Pick a local AI model and see the cheapest GPUs that hold it, one card or a set of the same card, with what they cost on eBay right now, the memory to spare and about how fast they run it. Decision models are in the list too, timed per question. Used and server cards are included.

The cheapest setups right now

Live on eBay
SetupVRAMSpeed (est.)Price nowPowerSoftware
2 × Tesla P4048 GB0.0 GB spare4.0–6.1 tokens/s$540.00the 2 cheapest listings500 WCUDA 8.0 to 12.9 (not CUDA 13)
4 × Tesla P100 PCIe 16 GB64 GB14.0 GB spare8.6–13 tokens/s$721.00the 4 cheapest listings1,000 WCUDA 8.0 to 12.9 (not CUDA 13)
4 × Radeon VII64 GB14.0 GB spare8.4–14 tokens/s$1,023the 4 cheapest listings1,200 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Instinct MI50 16 GB64 GB14.0 GB spare8.4–14 tokens/s$1,087the 4 cheapest listings1,200 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Tesla V100 PCIe 16 GB64 GB14.0 GB spare11–16 tokens/s$1,327the 4 cheapest listings1,000 WCUDA 9.0 to 12.9 (not CUDA 13)
2 × Instinct MI50 32 GB64 GB16.0 GB spare8.4–14 tokens/s$1,329the 2 cheapest listings600 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Radeon Pro VII64 GB14.0 GB spare8.4–14 tokens/s$1,630the 4 cheapest listings1,000 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Radeon RX 680064 GB14.0 GB spare4.2–7.2 tokens/s$1,658the 4 cheapest listings1,000 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Radeon RX 7600 XT64 GB14.0 GB spare2.4–4.0 tokens/s$1,660the 4 cheapest listings760 WNot on ROCm's support list (older ROCm or Vulkan)
2 × Arc Pro B60 24 GB48 GB0.0 GB spare3.7–6.4 tokens/s$1,699the 2 cheapest listings400 WNo CUDA (oneAPI / Vulkan)
2 × Radeon Pro W680064 GB16.0 GB spare4.2–7.2 tokens/s$1,760the 2 cheapest listings500 WROCm (on AMD's support list)
4 × Quadro RTX 500064 GB14.0 GB spare5.2–7.9 tokens/s$1,839the 4 cheapest listings920 WCUDA 10.0 and newer
2 × Titan RTX48 GB0.0 GB spare7.9–12 tokens/s$1,840the 2 cheapest listings560 WCUDA 10.0 and newer
4 × A264 GB14.0 GB spare2.3–3.5 tokens/s$1,858the 4 cheapest listings240 WCUDA 11.1 and newer
4 × Arc A770 16 GB64 GB14.0 GB spare4.6–7.9 tokens/s$1,872the 4 cheapest listings900 WNo CUDA (oneAPI / Vulkan)

A set of cards is priced at that many of the cheapest live listings, shipping included.

By model

Decision models (OpenJev, Clef, Decider): they score options instead of writing text, so they're compared by the time per question.

How it works

The memory a model needs is its weights at the quantisation you pick, plus the attention cache for the context and a little for working buffers. Q4 is what most people run at home, close to full quality at a third of the size. Each card keeps about 1 GB for itself, so two 24 GB cards hold 46 GB.

Writing each token reads every active weight once, so the speed is about the memory bandwidth divided by the model's size, times what the software gets out of the card (less off CUDA). Cards sharing a model layer by layer take turns, so a set runs about as fast as one card would. Treat it as a rough figure for one person chatting; a long prompt takes longer to read in.

Power is the cards' rated power. Over about 1,440 W with the rest of the PC, a set needs more than one 15 A circuit (or a 240 V one), and older cards often need a power cable adapter, which each card's page mentions.