Can I run Mistral Nemo?

12B with 128K context, built with NVIDIA. Great mid-size all-rounder.

chatgeneralagents · sizes: 12B · Apache-2.0

What runs it

machine	best fit	verdict	speed
Apple M1 · 8GB	12B Q4_K_M	❌ Won't fit	—
Apple M1 Pro · 16GB	12B Q8_0	🐢 CPU-only (slow)	0.9–2 tok/s
Apple M2 · 16GB	12B Q8_0	🐢 CPU-only (slow)	0.5–0.9 tok/s
Apple M3 Pro · 18GB	12B Q4_K_M	✅ Runs (tight)	10–16 tok/s
Apple M2 Pro · 32GB	12B Q8_0	✅ Runs comfortably	8–13 tok/s
Apple M4 Pro · 48GB	12B Q8_0	✅ Runs comfortably	10–17 tok/s
Apple M3 Max · 36GB	12B Q8_0	✅ Runs comfortably	13–20 tok/s
Apple M2 Max · 64GB	12B Q8_0	✅ Runs comfortably	13–20 tok/s
Apple M4 Max · 128GB	12B Q8_0	✅ Runs comfortably	17–30 tok/s
Apple M2 Ultra · 192GB	12B Q8_0	✅ Runs comfortably	18–30 tok/s
NVIDIA GeForce RTX 3060 · 12GB VRAM	12B Q5_K_M	✅ Runs (tight)	15–25 tok/s
NVIDIA GeForce RTX 4070 · 12GB VRAM	12B Q5_K_M	✅ Runs (tight)	20–35 tok/s
NVIDIA GeForce RTX 4090 · 24GB VRAM	12B Q8_0	✅ Runs comfortably	30–50 tok/s
NVIDIA GeForce RTX 5080 · 16GB VRAM	12B Q8_0	✅ Runs (tight)	30–45 tok/s
NVIDIA GeForce RTX 5090 · 32GB VRAM	12B Q8_0	✅ Runs comfortably	55–90 tok/s
AMD Radeon RX 7900 XTX · 24GB VRAM	12B Q8_0	✅ Runs comfortably	30–45 tok/s