PerGB

Cheapest GPUs to run Qwen3.8 Flash-Next 180B locally

Qwen3.8 Flash-Next 180B needs about 110 GB of VRAM at Q4 with an 8K-token context (192 GB at Q8). A mixture of experts: 6B active, plus a 51B n-gram embedding. The cheapest way to hold it right now: 4 × Instinct MI50 32 GB for $2,726, about 59–101 tokens/s.

The cheapest setups right now

Live on eBay
SetupVRAMSpeed (est.)Price nowPowerSoftware
4 × Instinct MI50 32 GB128 GB14.2 GB spare59–101 tokens/s$2,726the 4 cheapest listings1,200 WNot on ROCm's support list (older ROCm or Vulkan)
4 × Radeon Pro W6800128 GB14.2 GB spare30–51 tokens/s$3,540the 4 cheapest listings1,000 WROCm (on AMD's support list)
4 × Tesla V100 PCIe 32 GB128 GB14.2 GB spare74–111 tokens/s$3,909the 4 cheapest listings1,000 WCUDA 9.0 to 12.9 (not CUDA 13)
2 × A16128 GB16.2 GB spare66–99 tokens/s$4,439the 2 cheapest listings500 WCUDA 11.1 and newer
4 × Tesla V100S PCIe128 GB14.2 GB spare94–140 tokens/s$4,606the 4 cheapest listings1,000 WCUDA 9.0 to 12.9 (not CUDA 13)
4 × Instinct MI100128 GB14.2 GB spare71–122 tokens/s$5,523the 4 cheapest listings1,200 WROCm (on AMD's support list)
3 × Quadro RTX 8000144 GB31.2 GB spare55–83 tokens/s$6,000the 3 cheapest listings780 WCUDA 10.0 and newer
3 × Arc Pro B60 48 GB144 GB31.2 GB spare26–45 tokens/s$6,159the 3 cheapest listings600 WNo CUDA (oneAPI / Vulkan)
4 × Quadro GV100128 GB14.2 GB spare72–108 tokens/s$6,298the 4 cheapest listings1,000 WCUDA 9.0 to 12.9 (not CUDA 13)
4 × Radeon Pro W7800 32 GB128 GB14.2 GB spare33–57 tokens/s$7,501the 4 cheapest listings1,040 WROCm (on AMD's support list)
2 × Instinct MI210128 GB16.2 GB spare95–162 tokens/s$7,598the 2 cheapest listings600 WROCm (on AMD's support list)
4 × Radeon AI PRO R9700128 GB14.2 GB spare37–63 tokens/s$8,938the 4 cheapest listings1,200 WROCm (on AMD's support list)
3 × A40144 GB31.2 GB spare57–86 tokens/s$10,025the 3 cheapest listings900 WCUDA 11.1 and newer
4 × GeForce RTX 4080 32 GB128 GB14.2 GB spare59–89 tokens/s$10,200the 4 cheapest listings1,280 Wover a 15 A circuitCUDA 11.8 and newer
4 × GeForce RTX 4080 Super 32 GB128 GB14.2 GB spare61–91 tokens/s$10,700the 4 cheapest listings1,280 Wover a 15 A circuitCUDA 11.8 and newer

A set of cards is priced at that many of the cheapest live listings, shipping included. Speed: an estimate from memory bandwidth for one user chatting, within a third either way.

How it works

The memory a model needs is its weights at the quantisation you pick, plus the attention cache for the context and a little for working buffers. Q4 is what most people run at home, close to full quality at a third of the size. Each card keeps about 1 GB for itself, so two 24 GB cards hold 46 GB.

Writing each token reads every active weight once, so the speed is about the memory bandwidth divided by the model's size, times what the software gets out of the card (less off CUDA). Cards sharing a model layer by layer take turns, so a set runs about as fast as one card would. Treat it as a rough figure for one person chatting; a long prompt takes longer to read in.

Power is the cards' rated power. Over about 1,440 W with the rest of the PC, a set needs more than one 15 A circuit (or a 240 V one), and older cards often need a power cable adapter, which each card's page mentions.