PerGB

Does the CPU matter for AI if you're using GPUs?

Not much, as long as the whole model fits in VRAM. The GPU does the work and almost any CPU keeps up. The CPU starts to matter in three cases: its PCIe lanes decide how many GPUs you can connect, its memory bandwidth sets the speed once a model spills into system RAM, and it carries half the load when a big mixture-of-experts model is split between GPU and CPU.

Does the CPU matter for your setup?

Your setupDoes the CPU matter?
One GPU, the model fits in its VRAMBarely. Any 6-core chip from the last several years keeps up.
Several GPUs, llama.cpp / Ollama / LM Studio splitting by layersA little. x4 per card is plenty and even x1 works, though models load slower.
Several GPUs with tensor parallelism (vLLM, SGLang, ExLlamaV2), or serving many usersYes. The number of PCIe lanes and their generation set how fast prompts are read.
A model bigger than your VRAM (offloading layers)A lot. System memory bandwidth, which comes from the CPU's memory channels, sets the speed.
DeepSeek, Qwen3 235B and other huge MoE models on GPU + CPUMore than anywhere else. Memory channels and capacity count, and AVX-512 or AMX help.
Fine-tuning and trainingYes. Lanes sync the cards, cores feed the data, and RAM holds offloaded optimizer state.
Image and video generation (Stable Diffusion, Flux)Barely. The model lives on the card.

What the CPU does while a GPU runs a model

Running a language model has two phases. Reading the prompt (prefill) is heavy maths over every token you send. Writing the answer (generation) produces one token at a time, and each token reads every weight it uses once, so memory bandwidth sets its speed more than compute does. With the model in VRAM, the GPU runs both phases.

The CPU turns text into tokens, launches the GPU's work, picks the next token and runs the server (Ollama, llama.cpp's server, vLLM's scheduler). That keeps about one core busy. An RTX 3090 reads its memory at 936 GB/s and a desktop CPU reads system memory at about 90, but as long as the card holds everything, none of the CPU's work is on the slow path.

You notice a fast core in two cases. One is a small model at very high speed: an 8B model at 100+ tokens/s spends a few milliseconds per token, and launching the GPU's work takes a slice of that. The other is vLLM or SGLang serving many users at once, where the Python scheduler can keep a core busy. A current mid-range chip handles both, and an old quad-core costs you a few percent.

PCIe lanes decide how many GPUs you can run

Every GPU talks to the system over PCIe lanes. They come from the CPU, or from the chipset, which is slower and shared. Desktop chips have 20-28 lanes, enough for one card at x16 and an NVMe drive. On most desktop boards the second x16-shaped slot is wired x4 through the chipset, and better boards split the CPU's 16 lanes x8/x8. Threadripper PRO, Xeon W and EPYC have 64-128 lanes, enough for four to eight cards at full width.

How much link speed matters depends on how the model is split

With NVIDIA's standard driver, consumer GeForce cards generally can't send data straight to each other over PCIe. Transfers between them go through system memory, which is one more reason tensor parallelism on GeForce cards benefits from wide links.

The cheapest CPU for each number of GPUs

To runLanesCheapest CPU listedCheapest with PCIe 4.0 or newer
1 GPU at x16, plus an NVMe drive20+AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0
2 GPUs at x1640+AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0
4 GPUs at x1664+AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0
7-8 GPUs at x16128+AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0

Lanes only help if the board wires them to slots. Count the slots that are electrically x16 (the manual tells you; the slot's shape doesn't) and check their spacing, since most big cards take three slots and anything past two needs risers or an open frame. The build page's GPU host mode ranks complete platforms per dollar on those numbers: lanes, x16 slots and the most memory.

When the model doesn't fit in VRAM, the CPU sets the speed

llama.cpp, Ollama and LM Studio run the layers that don't fit on the GPU from system memory, on the CPU. For each token, the GPU's share runs at the GPU's bandwidth and the rest at the CPU's. The two times add up, so the slow part dominates. Llama 3.3 70B at Q4 is about 43 GB; one 24 GB card holds about 20 GB of it with an 8K context, which leaves 23 GB in system memory:

SetupIn system memoryWriting speed (tokens/s)
Two 24 GB cards (RTX 3090), all in VRAM0 GBabout 14
One RTX 3090 + Desktop, 2 × DDR5-600023 GBabout 1.9
One RTX 3090 + EPYC 7002 / 7003, 8 × DDR4-320023 GBabout 3.7
One RTX 3090 + EPYC 9004, 12 × DDR5-480023 GBabout 7.2
CPU only: EPYC 9004, 12 × DDR5-480043 GBabout 5.1

Putting half the model on a fast card is barely quicker than running it on the CPU alone, because once you offload, your speed is the speed of the CPU's memory channels. If a dense model is only a little too big, a second card helps far more than any CPU upgrade (the cheapest 24 GB card listed now: Tesla P40 from $270.00). Bandwidth for each platform is in why memory channels matter.

Big mixture-of-experts models split between GPU and CPU

DeepSeek R1 / V3, Qwen3 235B, gpt-oss 120B and Llama 4 have hundreds of GB of weights, but each token reads only the few experts it is routed to. The usual way to run them on one or two GPUs keeps attention, the shared layers and the context cache on the GPU and puts the expert weights in system memory (llama.cpp's --cpu-moe / --n-cpu-moe, or -ot with a tensor pattern; ktransformers does the same). The GPU keeps prompts and long contexts fast. Writing speed depends on system memory, because each token's experts are read from RAM. Qwen3 235B-A22B at Q4 is about 142 GB and reads about 13 GB per token:

SetupSystem memory bandwidthWriting speed (tokens/s)
One 24 GB GPU + Desktop, 2 × DDR5-600096 GB/sabout 2.3
One 24 GB GPU + EPYC 7002 / 7003, 8 × DDR4-3200204.8 GB/sabout 4.8
One 24 GB GPU + EPYC 9004, 12 × DDR5-4800460.8 GB/sabout 11

It needs about 145 GB of memory in all, so 192 GB of system RAM. A desktop tops out at 192-256 GB, and four sticks on two channels usually run slower. A server platform has both the capacity and the channels. These are the best-value platforms that hold it right now, priced as chip, board and memory (the speeds are for the CPU alone):

CPUBoardMemoryBandwidthTotal
AMD EPYC 9124Other SP5 boards12 × 16 GB DDR5-4800 RDIMM461 GB/s$2,825
AMD EPYC 9015Gigabyte MZ33-AR112 × 16 GB DDR5-6000 RDIMM576 GB/s$3,634
AMD EPYC 9115Gigabyte MZ33-AR112 × 16 GB DDR5-6000 RDIMM576 GB/s$3,725
AMD EPYC 9D64Other SP5 boards12 × 16 GB DDR5-4800 RDIMM461 GB/s$3,263
AMD EPYC 9334Supermicro H13SSL12 × 16 GB DDR5-4800 RDIMM461 GB/s$3,365

DeepSeek's 671B needs about 400 GB; see the cheapest way to run it.

AVX-512, AMX and NPUs: do the CPU's AI features matter?

Only when the CPU does the computing. In a GPU rig that means reading the prompt through layers kept in system memory.

How much system RAM do you need with GPUs?

Which CPU should you buy for your GPUs?

The cheapest platforms to host GPUs now, with 128 GB of memory

ForCPUBoardMemoryTotal
1-2 GPUs (desktop)Intel Core i3-13100F20 lanes · PCIe 4.0H610 (LGA1700)1 x16 slots4 × 32 GB DDR4-3200 UDIMM$660.98
4 GPUs at x16, PCIe 4.0+AMD EPYC 7252128 lanes · PCIe 4.0Gigabyte MZ32-AR05 x16 slots8 × 16 GB DDR4-3200 RDIMM$1,134
6+ GPUs at x16, PCIe 4.0+AMD EPYC 7252128 lanes · PCIe 4.0ASRock Rack ROMED8-2T7 x16 slots8 × 16 GB DDR4-3200 RDIMM$2,603

Prices are live eBay listings for the chip, board and memory. Speeds are estimates from memory bandwidth (llama.cpp gets about 35-60% of it on a CPU and 50-75% on a CUDA card), and real speeds vary with the software. All platforms: build page, GPU host mode; GPUs: the best GPUs for local AI.

Common questions

Will my CPU bottleneck my GPU for AI?

Not for inference while the model fits in VRAM. At very high token rates an old CPU costs a few percent at most. Once layers are offloaded to system memory, the CPU becomes the limit, because its memory bandwidth sets the speed.

How many PCIe lanes do I need for multiple GPUs?

For llama.cpp or Ollama splitting a model by layers, x4 per card is enough, and x1 works but loads slower. For tensor parallelism or training, give each card x8-x16 at PCIe 4.0. Four cards at x16 take 64 lanes, which means Threadripper PRO, Xeon W or EPYC.

Does PCIe 3.0 or x4 slow down LLM inference?

Barely, when the model is split by layers, apart from slower loading. With tensor parallelism it slows prompt reading and serving many users, since the cards exchange data for every layer of every token.

Do I need a server CPU for a multi-GPU AI build?

From three cards up, in practice yes. A desktop chip has the lanes for one card at x16 or two at x8. A used EPYC 7002 / 7003 is the cheap route to 128 lanes and several x16 slots.

Does the CPU matter for Stable Diffusion or Flux?

Barely. The model runs on the GPU, and the CPU only loads it and prepares the inputs.

Is a CPU with an NPU useful for running LLMs?

Not next to a GPU. NPUs are built for small models in laptops at low power and are far slower than a graphics card for language models.

Prices and picks update every time the site does (live listings checked within the last few hours).