Local AI Guide
Recommended specifications for each local LLM model
Compare the Q4 required memory, native context length, quantization conditions of the latest open weight model, and expected speed for each representative device before purchasing.
Start with a model
Find hardware for this model
Also running document search?
Reserve this for embedding and reranking models sharing the same GPU or unified memory. Leave it at none if they run separately on the CPU.
Matching devices 51· 5 shown
Comfortable fit · Q4_K_M · Headroom 4.9GB
₩3,590,000
42.3 ~ 46.9 tok/s
Comfortable fit · Q4_K_M · Headroom 4.9GB
₩4,825,000
38.3 ~ 51.9 tok/s
Comfortable fit · Q4_K_M · Headroom 38.4GB
₩5,091,500
25.3 ~ 28.1 tok/s
Comfortable fit · Q4_K_M · Headroom 27.4GB
₩6,780,000
57.5 ~ 79.8 tok/s
Comfortable fit · Q4_K_M · Headroom 38.4GB
₩6,925,000
13.0 ~ 36.2 tok/s
Price is the range midpoint, with a basic host added for GPU-only prices. Speed is an estimate for the same model and 4K input.
Where to start
Don't just look at the model's total parameters, look at the Q4 required memory and active parameters as well. Even for the same model, load availability and perceived speed vary depending on the available memory and bandwidth of the equipment.
Bonsai
Bonsai 2 27B local LLM: specs, speed, memory
Compare estimated memory, prefill, token generation speed, quantization and 262,144-token context conditions for Bonsai 2 27B TERNARY_PQ2 across local hardware.
DeepSeek
DeepSeek-V4.1-Flash local LLM: specs, speed, memory
Compare documented DeepSeek-V4.1-Flash runs on Spark, an RTX 5090 with 128GB RAM, and an M3 Ultra with 256GB. Storage size, published decode measurements, and prefill availability are identified separately for each format.
Ling
Ling-3.0-flash-VL local LLM: specs, speed, memory
Compare estimated memory, prefill, decode speed, quantization and 131,072-token context for Ling-3.0-flash-VL in MXFP4.
MiniCPM
MiniCPM5-2B local LLM: specs, speed, memory
Compare estimated memory, prefill, decode speed, quantization and 131,072-token context for MiniCPM5-2B in Q4_K_M.
Qwen
Qwen3.8 27B local LLM: specs, speed, memory
Compare estimated memory, prefill, decode speed, quantization and 262,144-token context for Qwen3.8 27B in Q4_K_M.
Qwen
Qwen3.8-Flash-Next local LLM: specs and speed
Compare estimated memory, prefill, decode speed, quantization and 262,144-token context for Qwen3.8-Flash-Next in Q4_K_M.
Gemma
Gemma 4 12B local LLM: specs, speed, quantization
Compare estimated memory, prefill, decode speed, quantization and 262,144-token context for Gemma 4 12B in Q4_K_M.
Gemma
Gemma 4 26B-A4B (MoE) Local LLM Recommended Specifications·Speed·Quantization
Compare estimated memory, prefill, decode speed, quantization and 262,144-token context for Gemma 4 26B-A4B (MoE) in Q4_K_M.
Qwen
Qwen3.6 35B-A3B (MoE): local LLM speed and memory
Compare estimated memory, prefill, decode speed, quantization and 262,144-token context for Qwen3.6 35B-A3B (MoE) in Q4_K_M.
GLM
GLM-5.3-Flash (MoE) Local LLM Recommended Specifications·Speed·Quantization
Compare estimated memory, prefill, decode speed, quantization and 1,048,576-token context for GLM-5.3-Flash (MoE) in Q4_K_M.
DeepSeek
DeepSeek-V4-Flash-0731 (MoE) Local LLM Recommended Specifications·Speed·Quantization
Compare estimated memory, prefill, decode speed, quantization and 1,048,576-token context for DeepSeek-V4-Flash-0731 (MoE) in Q4_K_M.
Holo4
Holo4 27B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 262,144-token context for Holo4 27B Q4_K_M.
Holo4
Holo4 35B-A3B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 262,144-token context for Holo4 35B-A3B Q4_K_M.
LFM
LFM2.5-VL-3B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 32,768-token context for LFM2.5-VL-3B Q4_K_M.
K2 Horizon
K2 Horizon 0.9B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 131,072-token context for K2 Horizon 0.9B Q4_K_M.
K2 Horizon
K2 Horizon 3.7B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 524,288-token context for K2 Horizon 3.7B Q4_K_M.
K2 Horizon
K2 Horizon 7B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 524,288-token context for K2 Horizon 7B Q4_K_M.
K2 Horizon
K2 Horizon 32B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 524,288-token context for K2 Horizon 32B Q4_K_M.
K2 Horizon
K2 Horizon MoVA 36B-A4B local LLM: specs, speed, memory
Compare estimated memory, hardware-specific prefill and token generation speed, quantization, and a 524,288-token context for K2 Horizon MoVA 36B-A4B Q4_K_M.