Does the CPU matter for AI if you're using GPUs?
Not much, as long as the whole model fits in VRAM. The GPU does the work and almost any CPU keeps up. The CPU starts to matter in three cases: its PCIe lanes decide how many GPUs you can connect, its memory bandwidth sets the speed once a model spills into system RAM, and it carries half the load when a big mixture-of-experts model is split between GPU and CPU.
Does the CPU matter for your setup?
| Your setup | Does the CPU matter? |
|---|---|
| One GPU, the model fits in its VRAM | Barely. Any 6-core chip from the last several years keeps up. |
| Several GPUs, llama.cpp / Ollama / LM Studio splitting by layers | A little. x4 per card is plenty and even x1 works, though models load slower. |
| Several GPUs with tensor parallelism (vLLM, SGLang, ExLlamaV2), or serving many users | Yes. The number of PCIe lanes and their generation set how fast prompts are read. |
| A model bigger than your VRAM (offloading layers) | A lot. System memory bandwidth, which comes from the CPU's memory channels, sets the speed. |
| DeepSeek, Qwen3 235B and other huge MoE models on GPU + CPU | More than anywhere else. Memory channels and capacity count, and AVX-512 or AMX help. |
| Fine-tuning and training | Yes. Lanes sync the cards, cores feed the data, and RAM holds offloaded optimizer state. |
| Image and video generation (Stable Diffusion, Flux) | Barely. The model lives on the card. |
What the CPU does while a GPU runs a model
Running a language model has two phases. Reading the prompt (prefill) is heavy maths over every token you send. Writing the answer (generation) produces one token at a time, and each token reads every weight it uses once, so memory bandwidth sets its speed more than compute does. With the model in VRAM, the GPU runs both phases.
The CPU turns text into tokens, launches the GPU's work, picks the next token and runs the server (Ollama, llama.cpp's server, vLLM's scheduler). That keeps about one core busy. An RTX 3090 reads its memory at 936 GB/s and a desktop CPU reads system memory at about 90, but as long as the card holds everything, none of the CPU's work is on the slow path.
You notice a fast core in two cases. One is a small model at very high speed: an 8B model at 100+ tokens/s spends a few milliseconds per token, and launching the GPU's work takes a slice of that. The other is vLLM or SGLang serving many users at once, where the Python scheduler can keep a core busy. A current mid-range chip handles both, and an old quad-core costs you a few percent.
PCIe lanes decide how many GPUs you can run
Every GPU talks to the system over PCIe lanes. They come from the CPU, or from the chipset, which is slower and shared. Desktop chips have 20-28 lanes, enough for one card at x16 and an NVMe drive. On most desktop boards the second x16-shaped slot is wired x4 through the chipset, and better boards split the CPU's 16 lanes x8/x8. Threadripper PRO, Xeon W and EPYC have 64-128 lanes, enough for four to eight cards at full width.
How much link speed matters depends on how the model is split
- llama.cpp, Ollama and LM Studio split a model by layers by default. Each card holds a run of layers, and only one small vector per token crosses between cards, 16 KB for a 70B model. Even a PCIe 3.0 x1 link (about 1 GB/s) passes thousands of those a second. You see the difference when loading: filling a 24 GB card takes about 25 seconds over 3.0 x1, 7 over 3.0 x4 and under 2 over 4.0 x16, and the SSD is often slower than the link anyway. That's why mining risers work for inference.
- Tensor parallelism (vLLM, SGLang, ExLlamaV2 / TabbyAPI, llama.cpp's row split) shares every layer across all the cards, and they exchange results twice per layer for every token, about 2.6 MB per token for a 70B model. Reading a 4,000-token prompt moves about 10 GB between the cards, roughly 3 seconds over PCIe 3.0 x4 and 0.4 over 4.0 x16, and serving several users multiplies that. In this setup x8 or x16 at PCIe 4.0 per card pays off. NVLink bridges (on RTX 3090 pairs) bypass the slots.
- Training and fine-tuning sync the gradients every step, as much data as the weights being trained, so lanes and PCIe generation matter most here. LoRA fine-tunes move far less.
With NVIDIA's standard driver, consumer GeForce cards generally can't send data straight to each other over PCIe. Transfers between them go through system memory, which is one more reason tensor parallelism on GeForce cards benefits from wide links.
The cheapest CPU for each number of GPUs
| To run | Lanes | Cheapest CPU listed | Cheapest with PCIe 4.0 or newer |
|---|---|---|---|
| 1 GPU at x16, plus an NVMe drive | 20+ | AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0 | AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0 |
| 2 GPUs at x16 | 40+ | AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0 | AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0 |
| 4 GPUs at x16 | 64+ | AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0 | AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0 |
| 7-8 GPUs at x16 | 128+ | AMD EPYC 7351P$25.00 · 128 lanes · PCIe 3.0 | AMD EPYC 7252$44.99 · 128 lanes · PCIe 4.0 |
Lanes only help if the board wires them to slots. Count the slots that are electrically x16 (the manual tells you; the slot's shape doesn't) and check their spacing, since most big cards take three slots and anything past two needs risers or an open frame. The build page's GPU host mode ranks complete platforms per dollar on those numbers: lanes, x16 slots and the most memory.
When the model doesn't fit in VRAM, the CPU sets the speed
llama.cpp, Ollama and LM Studio run the layers that don't fit on the GPU from system memory, on the CPU. For each token, the GPU's share runs at the GPU's bandwidth and the rest at the CPU's. The two times add up, so the slow part dominates. Llama 3.3 70B at Q4 is about 43 GB; one 24 GB card holds about 20 GB of it with an 8K context, which leaves 23 GB in system memory:
| Setup | In system memory | Writing speed (tokens/s) |
|---|---|---|
| Two 24 GB cards (RTX 3090), all in VRAM | 0 GB | about 14 |
| One RTX 3090 + Desktop, 2 × DDR5-6000 | 23 GB | about 1.9 |
| One RTX 3090 + EPYC 7002 / 7003, 8 × DDR4-3200 | 23 GB | about 3.7 |
| One RTX 3090 + EPYC 9004, 12 × DDR5-4800 | 23 GB | about 7.2 |
| CPU only: EPYC 9004, 12 × DDR5-4800 | 43 GB | about 5.1 |
Putting half the model on a fast card is barely quicker than running it on the CPU alone, because once you offload, your speed is the speed of the CPU's memory channels. If a dense model is only a little too big, a second card helps far more than any CPU upgrade (the cheapest 24 GB card listed now: Tesla P40 from $270.00). Bandwidth for each platform is in why memory channels matter.
Big mixture-of-experts models split between GPU and CPU
DeepSeek R1 / V3, Qwen3 235B, gpt-oss 120B and Llama 4 have hundreds of GB of weights, but each token reads only the few experts it is routed to. The usual way to run them on one or two GPUs keeps attention, the shared layers and the context cache on the GPU and puts the expert weights in system memory (llama.cpp's --cpu-moe / --n-cpu-moe, or -ot with a tensor pattern; ktransformers does the same). The GPU keeps prompts and long contexts fast. Writing speed depends on system memory, because each token's experts are read from RAM. Qwen3 235B-A22B at Q4 is about 142 GB and reads about 13 GB per token:
| Setup | System memory bandwidth | Writing speed (tokens/s) |
|---|---|---|
| One 24 GB GPU + Desktop, 2 × DDR5-6000 | 96 GB/s | about 2.3 |
| One 24 GB GPU + EPYC 7002 / 7003, 8 × DDR4-3200 | 204.8 GB/s | about 4.8 |
| One 24 GB GPU + EPYC 9004, 12 × DDR5-4800 | 460.8 GB/s | about 11 |
It needs about 145 GB of memory in all, so 192 GB of system RAM. A desktop tops out at 192-256 GB, and four sticks on two channels usually run slower. A server platform has both the capacity and the channels. These are the best-value platforms that hold it right now, priced as chip, board and memory (the speeds are for the CPU alone):
| CPU | Board | Memory | Bandwidth | Total |
|---|---|---|---|---|
| AMD EPYC 9124 | Other SP5 boards | 12 × 16 GB DDR5-4800 RDIMM | 461 GB/s | $2,825 |
| AMD EPYC 9015 | Gigabyte MZ33-AR1 | 12 × 16 GB DDR5-6000 RDIMM | 576 GB/s | $3,634 |
| AMD EPYC 9115 | Gigabyte MZ33-AR1 | 12 × 16 GB DDR5-6000 RDIMM | 576 GB/s | $3,725 |
| AMD EPYC 9D64 | Other SP5 boards | 12 × 16 GB DDR5-4800 RDIMM | 461 GB/s | $3,263 |
| AMD EPYC 9334 | Supermicro H13SSL | 12 × 16 GB DDR5-4800 RDIMM | 461 GB/s | $3,365 |
DeepSeek's 671B needs about 400 GB; see the cheapest way to run it.
AVX-512, AMX and NPUs: do the CPU's AI features matter?
Only when the CPU does the computing. In a GPU rig that means reading the prompt through layers kept in system memory.
- AVX-512 (Ryzen 7000 and newer, EPYC 9004 and newer, Xeon Scalable) speeds up llama.cpp's maths on the CPU. Intel's desktop Core chips from the 12th generation on don't have it.
- AMX is a matrix unit in 4th-generation Xeon Scalable (Sapphire Rapids) and newer. In tools that use it, such as ktransformers, it makes prompt reading on the CPU several times faster (the cheapest chip with AMX listed now: Intel Xeon Gold 6450C, $99.99).
- Neither one speeds up writing, because memory bandwidth is the limit there.
- NPUs (Ryzen AI, Core Ultra) run small models in laptops at low power and add nothing to a GPU rig.
How much system RAM do you need with GPUs?
- If everything runs on the GPUs, have at least as much RAM as the biggest model file you load, and ideally as much as your total VRAM. llama.cpp maps the file, and with enough RAM to cache it a reload takes seconds.
- If you offload layers or keep MoE experts in RAM, you need room for the part kept off the GPU plus the system: 192-256 GB for Qwen3 235B at Q4, 512 GB for DeepSeek.
- Fill every channel. Speed comes from the number of filled channels, and bigger sticks don't add any. Servers take registered memory (RDIMM or UDIMM).
- A 40 GB model loads in about 6-12 seconds from an NVMe drive, over a minute from a SATA SSD and several minutes from a hard drive.
Which CPU should you buy for your GPUs?
- For one GPU, keep the desktop you have or get any recent Ryzen 5 / Core i5, and spend the money on VRAM.
- For two GPUs, a desktop board that splits x8/x8 is fine for llama.cpp-style splitting. Tensor parallelism or a third card calls for a workstation or server platform.
- For three to eight GPUs, a used EPYC 7002 / 7003 gives you 128 lanes of PCIe 4.0, boards with up to seven x16 slots, and eight memory channels for when a model spills over. Which board for it.
- For huge MoE models on one or two GPUs, buy the most memory channels you can afford: EPYC 9004 / 9005 with 12 channels of DDR5, or a Xeon with AMX if you'll use ktransformers.
The cheapest platforms to host GPUs now, with 128 GB of memory
| For | CPU | Board | Memory | Total |
|---|---|---|---|---|
| 1-2 GPUs (desktop) | Intel Core i3-13100F20 lanes · PCIe 4.0 | H610 (LGA1700)1 x16 slots | 4 × 32 GB DDR4-3200 UDIMM | $660.98 |
| 4 GPUs at x16, PCIe 4.0+ | AMD EPYC 7252128 lanes · PCIe 4.0 | Gigabyte MZ32-AR05 x16 slots | 8 × 16 GB DDR4-3200 RDIMM | $1,134 |
| 6+ GPUs at x16, PCIe 4.0+ | AMD EPYC 7252128 lanes · PCIe 4.0 | ASRock Rack ROMED8-2T7 x16 slots | 8 × 16 GB DDR4-3200 RDIMM | $2,603 |
Prices are live eBay listings for the chip, board and memory. Speeds are estimates from memory bandwidth (llama.cpp gets about 35-60% of it on a CPU and 50-75% on a CUDA card), and real speeds vary with the software. All platforms: build page, GPU host mode; GPUs: the best GPUs for local AI.
Common questions
Will my CPU bottleneck my GPU for AI?
Not for inference while the model fits in VRAM. At very high token rates an old CPU costs a few percent at most. Once layers are offloaded to system memory, the CPU becomes the limit, because its memory bandwidth sets the speed.
How many PCIe lanes do I need for multiple GPUs?
For llama.cpp or Ollama splitting a model by layers, x4 per card is enough, and x1 works but loads slower. For tensor parallelism or training, give each card x8-x16 at PCIe 4.0. Four cards at x16 take 64 lanes, which means Threadripper PRO, Xeon W or EPYC.
Does PCIe 3.0 or x4 slow down LLM inference?
Barely, when the model is split by layers, apart from slower loading. With tensor parallelism it slows prompt reading and serving many users, since the cards exchange data for every layer of every token.
Do I need a server CPU for a multi-GPU AI build?
From three cards up, in practice yes. A desktop chip has the lanes for one card at x16 or two at x8. A used EPYC 7002 / 7003 is the cheap route to 128 lanes and several x16 slots.
Does the CPU matter for Stable Diffusion or Flux?
Barely. The model runs on the GPU, and the CPU only loads it and prepares the inputs.
Is a CPU with an NPU useful for running LLMs?
Not next to a GPU. NPUs are built for small models in laptops at low power and are far slower than a graphics card for language models.
Prices and picks update every time the site does (live listings checked within the last few hours).