A code-generation model plus a small embedder for repo indexing. The smallest cataloged machine that runs Qwen2.5-Coder + Nomic Embed Text is a Apple M1 · 8GB. Ranked below, smallest memory first — each model at its best quant, one model loaded at a time.
| machine | Qwen2.5-Coder | Nomic Embed Text |
|---|---|---|
| Apple M1 · 8GB | 1.5B @ Q8_0 · ~25–40 tok/s | 137M @ F16 · ~75–120 tok/s |
| Apple M2 · 8GB | 1.5B @ Q8_0 · ~35–55 tok/s | 137M @ F16 · ~110–180 tok/s |
| Apple M3 · 8GB | 1.5B @ Q8_0 · ~35–55 tok/s | 137M @ F16 · ~110–180 tok/s |
| AMD Radeon RX 7600 · 8GB VRAM | 7B @ Q6_K · ~17–30 tok/s | 137M @ F16 · ~210–340 tok/s |
| NVIDIA GeForce RTX 3070 · 8GB VRAM | 7B @ Q6_K · ~30–45 tok/s | 137M @ F16 · ~340–560 tok/s |
| NVIDIA GeForce RTX 4060 · 8GB VRAM | 7B @ Q6_K · ~17–30 tok/s | 137M @ F16 · ~210–340 tok/s |
| NVIDIA GeForce RTX 3080 · 10GB VRAM | 14B @ Q3_K_M · ~35–60 tok/s | 137M @ F16 · ~570–960 tok/s |
| AMD Radeon RX 6700 XT · 12GB VRAM | 14B @ Q4_K_M · ~14–25 tok/s | 137M @ F16 · ~280–460 tok/s |
| AMD Radeon RX 7700 XT · 12GB VRAM | 14B @ Q4_K_M · ~16–25 tok/s | 137M @ F16 · ~310–520 tok/s |
| NVIDIA GeForce RTX 3060 · 12GB VRAM | 14B @ Q4_K_M · ~14–25 tok/s | 137M @ F16 · ~270–450 tok/s |
| NVIDIA GeForce RTX 4070 · 12GB VRAM | 14B @ Q4_K_M · ~20–35 tok/s | 137M @ F16 · ~380–630 tok/s |
| AMD Radeon RX 6800 XT · 16GB VRAM | 14B @ Q5_K_M · ~17–30 tok/s | 137M @ F16 · ~370–610 tok/s |
Fits computed by the same engine as the CLI — reproduce this table with:
Already have a machine? Check your exact rig →