Response speed and acceleration
Last updated
Nemotron Bolt vs Qwen3.8-27B: which is faster for agent work?
A short benchmark and a multi-step task can produce different speed rankings.
This guide compares Nemotron Bolt and Qwen3.8-27B on token1 throughput2 and agent-task replay time in NVIDIA's RTX Spark test, then shows how to test them with your own prompts3 and tools.
In NVIDIA's same RTX Spark test, Bolt posted higher TPS and shorter agent-task time than Qwen3.8-27B, while Qwen3.6 and Gemma4 posted higher TPS and shorter task times than Bolt. The table compares all four models' quantization4 and acceleration setups against both measures so you can choose the one closest to your workload. NVIDIA results
Check each model's quantization and acceleration setup in the published results. For your own comparison, use the same document, prompt, and tools for both models. Start with a short prompt, extend to your usual document length, and record answer accuracy, tool calls5, and memory use.
Two metrics describe different kinds of waiting
TPS on AIPerf SPEED-Bench-Coding is the token throughput while generating against a controlled benchmark input. It shows how quickly text is produced, but not the full cycle of selecting and running tools or revising a plan after reading results. Perceived wait also depends on time to first token6 and total output length.
AA-AgentPerf Local replay duration is the time to replay a set of heterogeneous agent trajectories. Because it includes prompts and tool-use steps, it is closer to end-to-end automation time than coding-benchmark TPS, but remains tied to fixed scenarios and a specific runtime7. Your agent may rank models differently if it uses other tools or longer documents.
Where did Bolt beat Qwen3.8 in the published test?
The test used the same RTX Spark system in a 128 GB Surface Laptop Ultra, Windows 11 ARM x64, and llama.cpp. TPS is from AIPerf SPEED-Bench-Coding; replay time is from a fixed AA-AgentPerf Local task set.
| Model and setup | AIPerf coding TPS | AA-AgentPerf Local replay duration (s) |
|---|---|---|
| Nemotron Bolt IQ2_XXS + LiLiCorr | 83.99 | 600.80 |
| Qwen3.8-27B UD-Q4_K_M + DFlash | 28.26 | 1,043.28 |
| Qwen3.6-35B-A3B UD-Q4_K_M + MTP | 96.63 | 453.97 |
| Gemma 4 26B-A4B UD-Q4_K_M + MTP | 100.56 | 480.68 |
Build a small comparison around your workflow
In the table, Bolt had about three times Qwen3.8's generation TPS and finished the fixed agent replay in about 42% less time. Qwen3.6 and Gemma4 had higher TPS and shorter replay durations than Bolt. For continuous token generation, look first at TPS; for multi-step tool automation, focus on replay duration.
Download each model from its official source and match the Qwen3.8-27B quantization and DFlash setup named in the table. Specify whether your Bolt setup includes IQ2_XXS and LiLiCorr. Record file sizes, runtime versions, and memory use for reproducibility. If drafters differ, you are comparing the full acceleration setups rather than only the base models.
Choose one representative task and give both models the same document, question, tools, and output limit. Start with a short exchange to verify operation and answer accuracy, then add real document length and repeated tool calls. Record time to first token, generation speed, end-to-end time, peak VRAM8, and failures. If answers differ, score them with the same rubric to compare speed and quality together.
Which one to choose
For tasks with many steps and decisions based on tool results, prioritize reliable tool calls in your tool format and end-to-end replay time. Bolt's shorter AA-AgentPerf Local result than Qwen3.8-27B is useful for shortlisting, but may not apply if your tools differ. Use a sandbox9 for automation that edits files or retrieves web content.
For repeated short code completions or fixed-input generation, measure TPS and time to first token separately. Qwen3.6 and Gemma4 had higher TPS than Bolt in the published table. Repeat the test with the same input length and GPU10 setup to see which result suits your workload.
Terminology notes
Token — A unit into which a model divides input or output for processing.
Back to the textThroughput — The amount of work processed or generated over time. Comparisons need the unit, such as tokens per second or requests per second.
Back to the textPrompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.
Back to the textQuantization — A way to represent model weights or values with fewer bits, reducing storage and memory use. The coarser representation can also affect accuracy.
Back to the textTool call — A structured request from a model for an external function such as reading a file, searching, or running a command. The agent runtime and its permission settings decide whether the request is actually executed.
Back to the textTime to First Token — The time from sending a request until its first output token arrives.
Back to the textRuntime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textVRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.
Back to the textSandbox — An isolated environment that restricts a program’s access to files, networks, and host capabilities. Its protection depends on mount, network, and execution-permission settings.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the text
Frequently asked questions
- What should you check first in this article?
- This guide compares Nemotron Bolt and Qwen3.8-27B on token throughput and agent-task replay time in NVIDIA's RTX Spark test, then shows how to test them with your own prompts and tools.
- How much memory does it need?
- Download each model from its official source and match the Qwen3.8-27B quantization and DFlash setup named in the table. Specify whether your Bolt setup includes IQ2_XXS and LiLiCorr. Record file sizes, runtime versions, and memory use for reproducibility. If drafters differ, you are comparing the full acceleration setups rather than only the base models.