What hardware runs a local local RAG workflow?

A chat model plus an embedder for retrieval over your own documents. The smallest cataloged machine that runs Llama 3.3 + Nomic Embed Text is a NVIDIA GeForce RTX 5090 · 32GB VRAM. Ranked below, smallest memory first — each model at its best quant, one model loaded at a time.

machine	Llama 3.3	Nomic Embed Text
NVIDIA GeForce RTX 5090 · 32GB VRAM	70B @ Q2_K · ~30–45 tok/s	137M @ F16 · ~2670–4440 tok/s
AMD Radeon Pro W7900 · 48GB VRAM	70B @ Q3_K_M · ~10–16 tok/s	137M @ F16 · ~1220–2030 tok/s
Apple M3 Max · 48GB	70B @ Q2_K · ~7–11 tok/s	137M @ F16 · ~650–1080 tok/s
Apple M4 Max · 48GB	70B @ Q2_K · ~9–14 tok/s	137M @ F16 · ~810–1350 tok/s
Apple M4 Pro · 48GB	70B @ Q2_K · ~5–9 tok/s	137M @ F16 · ~500–830 tok/s
Apple M1 Max · 64GB	70B @ Q4_K_M · ~4–7 tok/s	137M @ F16 · ~650–1080 tok/s
Apple M1 Ultra · 64GB	70B @ Q4_K_M · ~7–12 tok/s	137M @ F16 · ~1130–1880 tok/s
Apple M2 Max · 64GB	70B @ Q4_K_M · ~4–7 tok/s	137M @ F16 · ~650–1080 tok/s
Apple M2 Ultra · 64GB	70B @ Q4_K_M · ~7–12 tok/s	137M @ F16 · ~1130–1880 tok/s
Apple M2 Max · 96GB	70B @ Q5_K_M · ~4–6 tok/s	137M @ F16 · ~650–1080 tok/s
Apple M2 Ultra · 128GB	70B @ Q8_0 · ~4–7 tok/s	137M @ F16 · ~1130–1880 tok/s
Apple M3 Max · 128GB	70B @ Q8_0 · ~2–4 tok/s	137M @ F16 · ~650–1080 tok/s

Fits computed by the same engine as the CLI — reproduce this table with:

$npx runlocal-sh advise llama-3.3, nomic-embed-text

Already have a machine? Check your exact rig →