Can I run Falcon3 7B?

chatcodingreasoning · sizes: 7B

What runs it

machinebest fitverdictspeed
Apple M1 · 8GB 7B Q6_K 🐢 CPU-only (slow) 0.7–1 tok/s
Apple M1 Pro · 16GB 7B Q6_K ✅ Runs (tight) 15–25 tok/s
Apple M2 · 16GB 7B Q6_K ✅ Runs (tight) 8–14 tok/s
Apple M3 Pro · 18GB 7B Q8_0 ✅ Runs (tight) 9–16 tok/s
Apple M2 Pro · 32GB 7B Q8_0 ✅ Runs comfortably 13–20 tok/s
Apple M4 Pro · 48GB 7B Q8_0 ✅ Runs comfortably 16–25 tok/s
Apple M3 Max · 36GB 7B Q8_0 ✅ Runs comfortably 20–35 tok/s
Apple M2 Max · 64GB 7B Q8_0 ✅ Runs comfortably 20–35 tok/s
Apple M4 Max · 128GB 7B Q8_0 ✅ Runs comfortably 25–45 tok/s
Apple M2 Ultra · 192GB 7B Q8_0 ✅ Runs comfortably 30–45 tok/s
NVIDIA GeForce RTX 3060 · 12GB VRAM 7B Q8_0 ✅ Runs (tight) 17–30 tok/s
NVIDIA GeForce RTX 4070 · 12GB VRAM 7B Q8_0 ✅ Runs (tight) 25–40 tok/s
NVIDIA GeForce RTX 4090 · 24GB VRAM 7B Q8_0 ✅ Runs comfortably 50–80 tok/s
NVIDIA GeForce RTX 5080 · 16GB VRAM 7B Q8_0 ✅ Runs comfortably 45–80 tok/s
NVIDIA GeForce RTX 5090 · 32GB VRAM 7B Q8_0 ✅ Runs comfortably 90–150 tok/s
AMD Radeon RX 7900 XTX · 24GB VRAM 7B Q8_0 ✅ Runs comfortably 45–75 tok/s

Shopping for a machine? What hardware do I need to run Falcon3 7B? →

Check your exact rig: open the checker →

$npx runlocal-sh can-i-run falcon3

Architectural kin

2 models share Falcon3 7B's KV-cache geometry — they size memory the same way, so a rig that fits one tends to fit its same-size kin.

Gemma 3
shares 4kv/256hd
Gemma 2
shares 4kv/256hd

Explore the whole catalog by architecture: open the graph →