Prefill and token generation speed

Qwen3.8 27B on RTX 5090: recommended settings

Try the recommended Qwen3.8 27B configuration on an RTX 5090 32GB. Inspect quantization and acceleration, then compare prefill and response speed with an RTX 4090.

Benchmark-based estimate · 4K input · one user

Last updated 2026-10-30

148.2 ~ 217.8 tok/sDecode speed(baseline: 70.6 ~ 103.7 tok/s)

Model fully loaded (19.03GB / 30.5GB)

First token (TTFT)Context 4,096 input tokens included

0.27 to 0.39 secondsContext 4,096 input tokens included

Prefill speed0.26 to 0.38 seconds (Estimate based on published benchmarks)

10660.8 ~ 15662.2 tok/s0.26 to 0.38 seconds (Estimate based on published benchmarks)

Total time (E2E)117 output tokens included

0.80 to 1.2 seconds117 output tokens included

Press Play to process the input and generate the answer.

Skip prefill

Ready

Change device or model

Device1792 GB/s

Model27B

Expand settings NVFP4 · 4K · NVFP4 + MTP

QuantizationNeed 19.03GB

vLLM 0.28.0 · NVFP4 W4A4 · FP8 KV · RTX 5090-specific ModelOpt checkpoints and single request metrics. MTP stage 2 requires a dedicated GPU setup.

Input context4K tokens

Decode optimizationNVFP4 + MTP

Prefill optimizationBaseline

Sample question:

117 output tokens

Reading this result

Downloading the model file alone does not reproduce this configuration. Model format, runtime and acceleration support must align. Expand the current settings below or open the device-specific recipe.

Required / available memory: 19.0GB / 30.5GB
Runtime: vLLM 0.28.0 · CUDA 13.3.1 · driver 610.43.02 · MTP-2

Compare this model side by side

The comparison uses recommended settings for each device. Quantization can differ, so the gap represents complete serving configurations, not an isolated GPU-only benchmark.

Include prefill Compare generation only

Check configuration

RTX 5090 32GB · Memory 32GB

Qwen3.8 27B NVFP4 configuration. Check that the retailer lists the same memory option.

What changes with a longer document?Expand input length, waiting time, speed and memory

Detailed analysis and all 61 hardware configurations

Popular configurations

Speed simulations by device and model

View all 20

Qwen3.8 27B on RTX 5090: recommended settings

Reading this result

Compare this model side by side

RTX 5090 32GB · Memory 32GB

Speed simulations by device and model

Qwen3.8 27B on Mac mini M4 32GB: response speed

Qwen3.8 27B: is a Mac mini M4 Pro 48GB worth it?

Qwen3.8 27B on MacBook Pro M5 Pro 64GB

Qwen3.8 27B on Mac Studio M4 Max 128GB