Q4_K_M · 4K context

MiniCPM5-2B 2.51675648B dense model · estimated Q4_K_M memory about 3GB

A dense open-weight model with 2.51675648B parameters and a native 131,072-token context. Displayed memory and generation speeds are estimates based on quantization and hardware specifications, not measured results.

Estimated Q4_K_M memory: About 3GB
total parameters: 2.51675648B
active parameter: 2.51675648B
native context: 128K

Representative equipment to run this model

RTX 5060 8GB

available 7.25GB · 448GB/s

llama.cpp CUDA · Q4_K_M

Decode

216.5 ~ 246.6 tok/s

First token

0.68 to 1.2 seconds

speed tuning Price, power, and sales composition

2× RTX 5090 64GB (PCIe)

available 61GB · 3584GB/s

llama.cpp CUDA · Q4_K_M · GPU split into 2 pieces

Decode

938.1 ~ 1448 tok/s

First token

0.19 to 0.41 seconds

speed tuning Price, power, and sales composition

MBA M4 16GB

available 13GB · 120GB/s

MLX 4-bit

Decode

53.2 ~ 59.6 tok/s

First token

3.4 to 8.3 seconds

speed tuning Price, power, and sales composition

Running Guide

MiniCPM5-2B Local Serving Run Recipe

Selecting a device changes the quantization, safety context, GPU offload, Flash Attention, KV cache, and batch start values.

Speed tuning for each device

If you want to choose equipment again

You can first narrow down the model configurations based on budget, noise level, operating system and whether it's acceptable used.