Best local AI for NVIDIA GeForce RTX 3060 · 12GB VRAM

The best local AI on a NVIDIA GeForce RTX 3060 · 12GB VRAM right now is Qwen3 (14B @ Q3_K_M, ~18–30 tok/s). Full ranking below — capability × real speed on this hardware, never sponsored.

#modelbest fitverdictspeed
1 Qwen3 14B Q3_K_M ✅ Runs comfortably 18–30 tok/s
2 DeepSeek-R1 (distill) 14B Q3_K_M ✅ Runs (tight) 17–30 tok/s
3 Qwen2.5 14B Q3_K_M ✅ Runs (tight) 17–30 tok/s
4 Llama 3.1 8B Q6_K ✅ Runs comfortably 20–35 tok/s
5 Gemma 3 12B Q3_K_M ✅ Runs (tight) 16–25 tok/s
6 Mistral 7B 7B Q6_K ✅ Runs comfortably 20–35 tok/s
7 Gemma 2 9B Q4_K_M ✅ Runs comfortably 19–30 tok/s
8 Phi-4 14B Q3_K_M ✅ Runs (tight) 16–25 tok/s
9 Mistral Nemo 12B Q4_K_M ✅ Runs (tight) 17–30 tok/s
10 Gemma 4 12B Q4_K_M ✅ Runs (tight) 14–25 tok/s
11 Llama 3.2 3B Q5_K_M ✅ Runs comfortably 50–80 tok/s
12 Yi 1.5 9B Q5_K_M ✅ Runs comfortably 20–35 tok/s
$npx runlocal-sh install qwen3

Rankings use the same GQA-aware fit + bandwidth-calibrated speed math as the CLI — no sponsorships, no affiliate re-sorting, ever.

Different machine? Apple M1 · 8GB · Apple M1 Pro · 16GB · Apple M2 · 16GB · Apple M3 Pro · 18GB · Apple M2 Pro · 32GB · Apple M4 Pro · 48GB

Or check a specific model on your exact rig: open the checker →