Local LLM speed and cost calculation criteriaRather than assuming a single number, compare ranges under the same conditions.

Representative comparisons on the equipment detail page are Q4 quantization, 4K context, and single user conditions. We first determine whether the model weights, execution overhead, and KV cache fit within actual available memory.

Speed ranges start with measured benchmark records collected by the site. We compare records whose hardware, model, quantization, runtime, and input and output token counts can be identified, after normalizing them to the same conditions.

When no matching measurement exists, we estimate a range from nearby measured results while accounting for memory bandwidth, active parameters, and runtime overhead. Direct measurements, collected third-party measurements, and measurement-based estimates are not presented as the same evidence level.

We review repeated runs only when their hardware, model, runtime, and test conditions can be verified. User submissions are approved after an administrator checks the conditions and evidence; approval alone does not automatically change calculator results.

We separate prefill, which processes input tokens, from decode, which generates response tokens sequentially. Collected measurements are normalized to the comparison conditions, and values that depart sharply from the repeated-run pattern are excluded from the representative range.

Power is calculated from sustained local inference load ranges rather than instantaneous peaks. TCO compares the purchase price, expected sale price, electricity bill, and actual replacement subscription fee for one and two years.

Actual results may vary depending on runtime version, cooling, quantization file, context length and multi-GPU communication. Therefore, it is recommended that you double-check the speed gauge with the selected model and exact memory configuration just before purchasing.