Response speed and acceleration

Reading the public Arc Pro B70 benchmark: XeStrata and SYCL

Even with the same Arc Pro B70 and IQ3_XXS, compare prefill for reading input and decode for generating an answer separately.

In the XeStrata xe0.1.43.1 release measurement on an Intel Arc Pro B70 32 GB, Qwen3.8-Flash-Next IQ3_XXS processed a 4K prompt1 at 1,511 tok/s prefill2 and generated 256 tokens3 at 81.1 tok/s decode4. The upstream v0.1.43 SYCL path under the same conditions measured 1,259 tok/s and 62.2 tok/s. These are release-reported comparison results; they do not directly represent other GPUs or the completion time of an ordinary chat.

See how Arc Pro B70 handles long input and answer generation. XeStrata xe0.1.43.1 compares Qwen3.8-Flash-Next IQ3_XXS against upstream v0.1.43 SYCL with matching 4K input and 256-token output. Public figures are collected in the table; everyday-chat records and version fixes are read separately.

Requirements and key details
  • 4K generate, IQ3_XXS: XeStrata measured 1,511 tok/s prefill and 81.1 tok/s decode; upstream SYCL measured 1,259 and 62.2 tok/s.
  • Each value is the median of two alternating runs; generate used temperature 0 and produced 256 tokens.
  • Serving, Japanese chat, and coding-agent results use different input and output flows and are measured separately from generate.

The two 4K test figures measure different parts of a request

The XeStrata xe0.1.43.1 release compares Qwen3.8-Flash-Next IQ3_XXS on an Arc Pro B70 32 GB with the upstream v0.1.43 Intel SYCL build. The public generate results below use an approximately 4K-token input and 256-token output; prefill is input processing and decode is subsequent token generation.

IQ3_XXS 4K generate comparison on an Arc Pro B70 32 GB
MeasureXeStrata xe0.1.43.1upstream v0.1.43 SYCL
prefill1,511 tok/s1,259 tok/s
decode81.1 tok/s62.2 tok/s

This table reports the median for temperature-0 generate using the same IQ3_XXS model, an approximately 4K prompt, and 256 output tokens. The release alternated between the two runtimes for two runs; it does not include different context lengths or the ordinary chat path.

With a short prompt, a prefill difference may account for only a small part of the total wait; with a long answer, decode speed matters for longer. For a short question, measure time to first token5. For a longer coding request, also record generation speed and total completion time to see how these figures affect your work.

A blank monitor, desktop tower, and RAM modules on a workbench
Compare prefill as the rate for reading input and decode as the rate for generating answer tokens.

The release fixes an Arc prefill regression

In xe0.1.43, 32K prefill for Qwen3.8-Flash-Next on an Arc Pro B70 fell back from the XMX matrix engine to the slower DP4a path; the release notes say it was about one-third slower. XeStrata xe0.1.43.1 fixes this and raises 32K IQ3_XXS prefill from 1,590 to 2,415 tok/s. This is a fix compared with an earlier XeStrata version, under conditions different from the 4K upstream comparison above.

The release also reports serving and everyday-use measurements separately from generate. For a 32K conversation, serving records prefill, the wait before a follow-up answer, and decode. Japanese chat and coding-agent tests go through serve/server.py and report first-token time and decode. For example, Japanese chat recorded 0.58 seconds to the first token and 62.7 tok/s decode. Each runtime6 generated its own response, with two alternating runs at temperature 0.7, so these figures should remain separate from the single generate test.

A dual-fan graphics card, RAM modules, an NVMe SSD, and an open blank notebook
xe0.1.43.1 restores the XMX prefill path on Intel Arc.

Record results for your own installation

Read the public SYCL comparison in the release table; use xestrata --bench to record your own installation separately. It tests the installed IQ3_XXS for roughly 40 minutes. Stop the running XeStrata server first; add --bench-verbose if you need stage timings, GPU7 load, and temperatures.

Records are saved under records/bench-<time>/ in the program's data directory. Read summary.md and tables.md, then check record.json to confirm the hardware and engine paths. The default is two runs; increase it with an option such as --bench-rounds 3 for more repetitions. A mixed-GPU layer-split configuration measures everyday use only, not generate and serve, so check the scope before comparing tables.

An open PC with two graphics cards beside a box, notebook, and screwdriver
For a comparable local measurement, match the model, prompt, input length, and output token count.

Terminology notes

  1. Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.

    Back to the text
  2. Prefill — The stage where an LLM reads the input prompt and computes representations for its tokens. Longer prompts contain more tokens to process.

    Back to the text
  3. Token — A unit into which a model divides input or output for processing.

    Back to the text
  4. Token decoding — The stage that generates successive output tokens using the input and the response produced so far.

    Back to the text
  5. Time to First Token — The time from sending a request until its first output token arrives.

    Back to the text
  6. Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.

    Back to the text
  7. GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.

    Back to the text

Frequently asked questions

What should you check first in this article?
In the XeStrata xe0.1.43.1 release measurement on an Intel Arc Pro B70 32 GB, Qwen3.8-Flash-Next IQ3_XXS processed a 4K prompt at 1,511 tok/s prefill and generated 256 tokens at 81.1 tok/s decode. The upstream v0.1.43 SYCL path under the same conditions measured 1,259 tok/s and 62.2 tok/s. These are release-reported comparison results; they do not directly represent other GPUs or the completion time of an ordinary chat.