PerGB

Cheapest GPUs to run Qwen3 235B-A22B locally

Qwen3 235B-A22B needs about 145 GB of VRAM at Q4 with an 8K-token context (252 GB at Q8). A mixture of experts: 22B active. The cheapest way to hold it right now: 3 × A16 for $6,789, about 18–27 tokens/s.

The cheapest setups right now

Live on eBay
SetupVRAMSpeed (est.)Price nowPowerSoftware
3 × A16192 GB44.5 GB spare18–27 tokens/s$6,789the 3 cheapest listings750 WCUDA 11.1 and newer
4 × Quadro RTX 8000192 GB43.5 GB spare15–23 tokens/s$8,075the 4 cheapest listings1,040 WCUDA 10.0 and newer
4 × Arc Pro B60 48 GB192 GB43.5 GB spare7.2–12 tokens/s$8,214the 4 cheapest listings800 WNo CUDA (oneAPI / Vulkan)
3 × Instinct MI210192 GB44.5 GB spare26–44 tokens/s$11,398the 3 cheapest listings900 WROCm (on AMD's support list)
4 × A40192 GB43.5 GB spare16–23 tokens/s$13,944the 4 cheapest listings1,200 WCUDA 11.1 and newer
4 × Radeon Pro W7900192 GB43.5 GB spare14–23 tokens/s$16,799the 4 cheapest listings1,180 WROCm (on AMD's support list)
4 × RTX A6000192 GB43.5 GB spare17–26 tokens/s$16,848the 4 cheapest listings1,200 WCUDA 11.1 and newer
4 × GeForce RTX 4090 D 48 GB192 GB43.5 GB spare23–34 tokens/s$20,389the 4 cheapest listings1,700 Wover a 15 A circuitCUDA 11.8 and newer
4 × A100 PCIe 40 GB160 GB11.5 GB spare35–52 tokens/s$23,238the 4 cheapest listings1,200 WCUDA 11.0 and newer
2 × A100 PCIe 80 GB160 GB13.5 GB spare44–65 tokens/s$23,499the 2 cheapest listings600 WCUDA 11.0 and newer
4 × GeForce RTX 4090 48 GB192 GB43.5 GB spare23–34 tokens/s$23,868the 4 cheapest listings1,800 Wover a 15 A circuitCUDA 11.8 and newer
4 × L40192 GB43.5 GB spare19–29 tokens/s$26,597the 4 cheapest listings1,200 WCUDA 11.8 and newer
4 × RTX PRO 5000 Blackwell 48 GB192 GB43.5 GB spare30–45 tokens/s$27,688the 4 cheapest listings1,200 WCUDA 12.8 and newer
3 × RTX PRO 5000 Blackwell 72 GB216 GB68.5 GB spare30–45 tokens/s$27,744the 3 cheapest listings900 WCUDA 12.8 and newer
2 × RTX PRO 6000 Blackwell192 GB45.5 GB spare40–60 tokens/s$28,180the 2 cheapest listings1,200 WCUDA 12.8 and newer

A set of cards is priced at that many of the cheapest live listings, shipping included. Speed: an estimate from memory bandwidth for one user chatting, within a third either way.

How it works

The memory a model needs is its weights at the quantisation you pick, plus the attention cache for the context and a little for working buffers. Q4 is what most people run at home, close to full quality at a third of the size. Each card keeps about 1 GB for itself, so two 24 GB cards hold 46 GB.

Writing each token reads every active weight once, so the speed is about the memory bandwidth divided by the model's size, times what the software gets out of the card (less off CUDA). Cards sharing a model layer by layer take turns, so a set runs about as fast as one card would. Treat it as a rough figure for one person chatting; a long prompt takes longer to read in.

Power is the cards' rated power. Over about 1,440 W with the rest of the PC, a set needs more than one 15 A circuit (or a 240 V one), and older cards often need a power cable adapter, which each card's page mentions.