Local AI Guide
Popular local LLM speed results
Explore prefill, time to first token and generation speed across 20 device-model configurations, then compare hardware or open serving settings.
Where to start
Choose a device-model pair to start with its settings. You can change the combination on the speed screen.
Q4_K_M · 4K
Gemma 4 12B on Mac mini M4 24GB: chat speed
Try Gemma 4 12B on a Mac mini M4 24GB. Watch the initial wait and the response stream to judge whether the pace suits document summaries or short conversations.
Q4_K_M · 4K
Qwen3.8 27B on Mac mini M4 32GB: response speed
Explore Qwen3.8 27B response speed on a Mac mini M4 32GB. Beyond fitting the model, see whether the wait for a question and a longer response suits you.
Q4_K_M · 4K
Qwen3.8 27B: is a Mac mini M4 Pro 48GB worth it?
Check the speed that matters when buying an M4 Pro 48GB for Qwen3.8 27B. Compare prefill and generation separately to decide whether your work benefits from moving beyond an M4 32GB.
Q4_K_M · 4K
Mac mini M4 Pro 64GB with Qwen3.6 35B-A3B (MoE): prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.6 35B-A3B (MoE) on Mac mini M4 Pro 64GB with recommended settings and the same 4K input.
Q4_K_M · 4K
Qwen3.8 27B on MacBook Pro M5 Pro 64GB
Explore Qwen3.8 27B on a MacBook Pro M5 Pro 64GB without a separate server. Check input processing and response generation, then compare with a desktop GPU configuration.
Q4_K_M · 4K
Qwen3.8 27B on Mac Studio M4 Max 128GB
Check prefill and generation for Qwen3.8 27B on a Mac Studio M4 Max 128GB. Compare whether you need the large memory capacity or a smaller configuration is enough for this model.
Q4_K_M · 4K
Studio M4 Max 128GB with Qwen3.6 35B-A3B (MoE): prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.6 35B-A3B (MoE) on Studio M4 Max 128GB with recommended settings and the same 4K input.
Q4_K_M · 4K
Studio M3 Ultra 512GB with Qwen3.8-Flash-Next: prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.8-Flash-Next on Studio M3 Ultra 512GB with recommended settings and the same 4K input.
Q4_K_M · 4K
Qwen Flash Next on Mac Studio M5 Ultra 512GB
Simulate Qwen3.8 Flash Next on an M5 Ultra 512GB. Separate model capacity from response speed and compare with an M3 Ultra configuration with the same 512GB capacity.
Q4_K_M · 4K
RTX 3090 24GB with Gemma 4 12B: prefill and generation speed
Experience prefill, time to first token, and generation speed for Gemma 4 12B on RTX 3090 24GB with recommended settings and the same 4K input.
Q4_K_M · 4K
Qwen3.8 27B on RTX 3090: do you need an upgrade?
If you own an RTX 3090 24GB, explore Qwen3.8 27B at this response pace first. Compare with an RTX 5090 and distinguish a memory limitation from waiting time.
Q4_K_M · 4K
Qwen3.8 27B on RTX 4090: prefill and generation
Watch Qwen3.8 27B process input and respond on an RTX 4090 24GB. Compare with an RTX 3090 to see how waiting time differs despite the same 24GB memory capacity.
NVFP4 · 4K
Qwen3.8 27B on RTX 5090: recommended settings
Try the recommended Qwen3.8 27B configuration on an RTX 5090 32GB. Inspect quantization and acceleration, then compare prefill and response speed with an RTX 4090.
Q4_K_M · 4K
RTX 5090 32GB with Gemma 4 26B-A4B (MoE): prefill and generation speed
Experience prefill, time to first token, and generation speed for Gemma 4 26B-A4B (MoE) on RTX 5090 32GB with recommended settings and the same 4K input.
Q4_K_M · 4K
RTX PRO 6000 96GB with Qwen3.8-Flash-Next: prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.8-Flash-Next on RTX PRO 6000 96GB with recommended settings and the same 4K input.
NVFP4 · 4K
DGX Spark 128GB with Qwen3.6 35B-A3B (MoE): prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.6 35B-A3B (MoE) on DGX Spark 128GB with recommended settings and the same 4K input.
Q4_K_M · 4K
Qwen Flash Next on two DGX Sparks: what changes?
Explore Qwen3.8 Flash Next on two DGX Sparks. Compare the same model on one unit and separate added model capacity from single-response generation speed.
Q4_K_M · 4K
2× RTX 3090 48GB (NVLink) with Qwen3.6 35B-A3B (MoE): prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.6 35B-A3B (MoE) on 2× RTX 3090 48GB (NVLink) with recommended settings and the same 4K input.
IQ3_S · 4K
2× RTX 5090 64GB (PCIe) with Qwen3.8-Flash-Next: prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.8-Flash-Next on 2× RTX 5090 64GB (PCIe) with recommended settings and the same 4K input.
Q4_K_M · 4K
Ryzen AI Max+ 395 128GB with Qwen3.8 27B: prefill and generation speed
Experience prefill, time to first token, and generation speed for Qwen3.8 27B on Ryzen AI Max+ 395 128GB with recommended settings and the same 4K input.