Cheapest GPUs to run gpt-oss 20B locally
gpt-oss 20B needs about 14 GB of VRAM at Q4 with an 8K-token context (23 GB at Q8). A mixture of experts: 3.6B of its 21B parameters work on each token. The cheapest way to hold it right now: Tesla P100 PCIe 16 GB for $170.00, about 101–151 tokens/s.
The cheapest setups right now
Live on eBay| Setup | VRAM | Speed (est.) | Price now | Power | Software |
|---|---|---|---|---|---|
| Tesla P100 PCIe 16 GB | 16 GB1.4 GB spare | 101–151 tokens/s | $170.00 | 250 W | CUDA 8.0 to 12.9 (not CUDA 13) |
| Radeon VII | 16 GB1.4 GB spare | 99–169 tokens/s | $249.00 | 300 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 2 × Tesla P100 PCIe 12 GB | 24 GB8.4 GB spare | 75–113 tokens/s | $257.00the 2 cheapest listings | 500 W | CUDA 8.0 to 12.9 (not CUDA 13) |
| Instinct MI50 16 GB | 16 GB1.4 GB spare | 99–169 tokens/s | $266.00 | 300 W | Not on ROCm's support list (older ROCm or Vulkan) |
| Tesla P40 | 24 GB9.4 GB spare | 48–71 tokens/s | $270.00 | 250 W | CUDA 8.0 to 12.9 (not CUDA 13) |
| Tesla V100 PCIe 16 GB | 16 GB1.4 GB spare | 124–186 tokens/s | $325.00 | 250 W | CUDA 9.0 to 12.9 (not CUDA 13) |
| 2 × GeForce RTX 2070 Super | 16 GB0.4 GB spare | 62–92 tokens/s | $349.99the 2 cheapest listings | 430 W | CUDA 10.0 and newer |
| Radeon Pro VII | 16 GB1.4 GB spare | 99–169 tokens/s | $355.00 | 250 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 2 × Quadro RTX 4000 | 16 GB0.4 GB spare | 57–86 tokens/s | $359.99the 2 cheapest listings | 320 W | CUDA 10.0 and newer |
| 2 × GeForce RTX 2070 | 16 GB0.4 GB spare | 62–92 tokens/s | $365.50the 2 cheapest listings | 350 W | CUDA 10.0 and newer |
| 2 × GeForce RTX 2080 | 16 GB0.4 GB spare | 62–92 tokens/s | $375.00the 2 cheapest listings | 430 W | CUDA 10.0 and newer |
| Radeon RX 6800 | 16 GB1.4 GB spare | 49–84 tokens/s | $380.00 | 250 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 2 × GeForce RTX 2060 Super | 16 GB0.4 GB spare | 62–92 tokens/s | $387.51the 2 cheapest listings | 350 W | CUDA 10.0 and newer |
| Radeon RX 7600 XT | 16 GB1.4 GB spare | 28–48 tokens/s | $399.99 | 190 W | Not on ROCm's support list (older ROCm or Vulkan) |
| 2 × GeForce RTX 2060 8 GB | 16 GB0.4 GB spare | 46–69 tokens/s | $405.68the 2 cheapest listings | 320 W | CUDA 10.0 and newer |
Per question: scaled from the model's reported speed on an RTX PRO 6000 by each card's compute and memory bandwidth, within about half either way. Licenses, scores and sources: decision models.
A set of cards is priced at that many of the cheapest live listings, shipping included. Speed: an estimate from memory bandwidth for one user chatting, within a third either way.
How it works
The memory a model needs is its weights at the quantisation you pick, plus the attention cache for the context and a little for working buffers. Q4 is what most people run at home, close to full quality at a third of the size. Each card keeps about 1 GB for itself, so two 24 GB cards hold 46 GB.
Writing each token reads every active weight once, so the speed is about the memory bandwidth divided by the model's size, times what the software gets out of the card (less off CUDA). Cards sharing a model layer by layer take turns, so a set runs about as fast as one card would. Treat it as a rough figure for one person chatting; a long prompt takes longer to read in.
Power is the cards' rated power. Over about 1,440 W with the rest of the PC, a set needs more than one 15 A circuit (or a 240 V one), and older cards often need a power cable adapter, which each card's page mentions.