Create a local serving benchmark file

You need Node.js 20 or later and an OpenAI-compatible local server with a model already running.
This tool does not install or download models. It sends a fixed public sample prompt to your running server and records the time to first output and the output duration.

1. Save these two files in the same folder.
   /tools/collect-serving-benchmark.mjs
   /tools/serving-benchmark-metadata.json

2. Replace the null values in the metadata JSON with your actual setup. This is not a file for entering benchmark results in advance.
   hardware.id: The site's hardware ID or a lowercase English identifier.
   hardware.chip: The actual chip name.
   hardware.memoryGB: Total high-speed memory in this configuration, in GB. Do not add separate system RAM to VRAM.
   hardware.deviceCount: Number of GPUs or nodes.
   model.id: Model identifier.
   model.artifact: The publisher/model name of a public model repository. Do not enter a private file path.
   model.revision: Model revision or release version.
   model.quantization: Quantization format of the model file you are actually using.
   runtime.name: llama.cpp / oMLX / LM Studio / SGLang / vLLM / MLX-LM / Ollama / other
   runtime.version: Installed runtime version or build number.
   runtime.os: Operating system and version.
   runtime.driverVersion: The driver version you have checked. Leave not-recorded if you do not know it.
   runtime.kvCache: f16 / bf16 / q8_0 / q4_0 / fp8_e4m3 / fp8_e5m2 / runtime-default
   runtime.acceleration: none / mtp / draft / ngram / mtp+ngram / draft+ngram / specprefill / custom
   workload.outputTokens: Requested output limit; the template uses 128. The result file records the actual token count returned by the server.
   workload.allocatedContextTokens: The context limit actually configured on your server. Replace the template's 4096 if your setting is different.
   workload.cacheState: Leave unknown. This tool cannot inspect or clear the server's internal cache.
   workload.concurrency: Leave 1. Requests are measured one at a time.
   workload.warmupRuns: Number of warm-up runs; the template uses 1. Warm-up runs are not included in the three measured runs.

3. Check the metadata first. This command does not contact the server.
   node collect-serving-benchmark.mjs --metadata serving-benchmark-metadata.json --check-metadata

4. Run the collector with your local server address and served model alias.
   node collect-serving-benchmark.mjs --url http://127.0.0.1:8000/v1 --model SERVED_MODEL --metadata serving-benchmark-metadata.json --output measurement.json --consent

   8000 is an example port. Use your server's actual port, including when using LM Studio.
   Use a loopback address on this computer: 127.0.0.1, localhost or [::1], with an explicit port. External services are not supported.
   SERVED_MODEL is the model alias accepted by the server, not a model file path.
   With the template settings, the tool performs one warm-up run followed by three measured runs. --consent allows it to create a local file containing specifications, settings and numeric measurements.
   Nothing is automatically uploaded to the site or another server. Open and review measurement.json before submitting it through the site's measurement import form.

What the file records
Hardware, model and runtime settings; actual input and output token counts; time to first token; and elapsed time from the first output to the last output.
It does not save prompts, responses, raw logs, token lists, server addresses or local file paths.

Measurement limits
Output duration is measured as the local API stream arrives. It is not the engine's internal GPU execution time.
Time to first token is not converted into prefill throughput. The tool does not estimate or fill in prefill timing or peak memory usage.
It will not create a result file if the server does not return exact token counts, delivers all output in a single chunk, or returns different token counts across the measured runs.
Invalid response content and secret keys are not included in error messages. Existing result files are not overwritten.
An approved submission remains a user measurement. Approval does not mean that the site operator reproduced it on the same hardware.