./gear/ram-vs-vram-local-llms — buyers' notes

RAM vs VRAM: how much memory do local LLMs actually need?

Weights cost about 0.6 bytes per parameter at q4 and 1.1 at q8, plus a gigabyte or more for context. On Macs, unified memory serves both roles; on PCs, VRAM holds the model while system RAM is only fallback. Offloading layers to system RAM slows inference sharply.

The memory math for local models: weights per parameter at q4/q8, KV-cache and context costs, unified vs discrete VRAM, and why offloading to system RAM hurts.

[ the math, once, forever ]

A model's memory footprint is weights + KV cache + runtime overhead. Weights are simple arithmetic: parameters × bytes per parameter. At q4 that's roughly 0.6 bytes/param in practice (4.5 effective bits plus metadata), at q8 about 1.1, at fp16 a flat 2. The KV cache grows with context length and model depth — budget 1–4GB for a 8–32k context window on mid-size models, more on long-context 70B runs. Overhead (runtime, buffers, OS) is the tax nobody lists: assume 3–6GB on a desktop OS doing anything else at all.

Every row above assumes you leave headroom for context and OS. "It loads" and "it runs with a useful context window" are different sentences.

[ unified memory vs VRAM ]

On a Mac there's one pool: system RAM is GPU memory, shared at 400–800GB/s depending on chip. No offloading decision exists — the model is either in the pool or it isn't. On a PC there are two pools: VRAM at 1TB/s+ on the card, and DDR5 system RAM at ~100GB/s across the bus. That 10× bandwidth gap is why "just offload the extra layers" turns a snappy model into a typing-simulator. Partial offload works for batch jobs where latency doesn't matter; for interactive use it's the most regretted dollar saved in local AI.

[ what to actually buy ]

For the Apple-specific budgets and config picks, the original notes stand: how much RAM for 7B, 13B, and 30B models. If this math is deciding between platforms, Mac Studio vs GPU PC takes it from here, and the under-$3,000 builds turn it into a shopping list. If you already own a gaming PC, skip all of it and start there: turn an old gaming PC into a local AI box.

CHECK PRICES see options at the retailer →

AFFILIATE DISCLOSURE: links in this post are affiliate links — if you buy through them I earn a small commission at no extra cost to you. RobotNurse is independent; I only recommend hardware I would actually deploy. No vendor paid for placement.