Executable programs and extensions
Local Jev-like decisions with Qwen: SemIf, Simple Jev and openjev-sglang
A search for Jev now returns everything from tiny Qwen experiments to B200 servers. They do not all build the same thing. Some read the next-token logits of an existing model, some train a new decision head, and some focus on API compatibility. A laptop experiment and a throughput-oriented server should not be compared as if they had the same goal. This guide separates them by how they run.
The simplest baseline reads the next-token scores
Map options to short tokens such as A, B and C, then inspect the logits the model assigns at the next position. A softmax gives relative probabilities within the declared candidates. There is no sentence to decode, and server code assembles the JSON. The model still has to understand each option, the labels must tokenize as expected, and the probabilities may be poorly calibrated. This produces a useful baseline, not the same model as Jev.
SemIf builds an auditable 4B baseline
SemIf is the renamed OpenJev project and reads candidate logits directly from Qwen3.5-4B. When many questions share one state, it can reuse the common prefill and evaluate branches in parallel. The project publishes RTX 3090 experiments with raw records and also includes an MLX backend for Apple Silicon. Its own results show that fast reuse paths can change some decisions relative to fresh scoring, so speed should never be recorded without agreement checks. Before buying hardware, compare fresh and shared modes on ten questions from your own work.
Simple Jev is the approachable API experiment
Simple Jev does not require a separately trained classifier head. It reads logits from open models through Transformers and PyTorch. Its Qwen3.5-0.8B CPU example makes the request and response contract easy to inspect without a large GPU. A successful request says nothing about whether the small model is accurate enough for business automation. Start with a short two- or three-choice task, collect failures, then move to a 4B or another model. Clients that need compatibility can also check the shared implementation behind `/v1/classifier` and `/v1/systemone`.
openjev-sglang targets server throughput
openjev-sglang serves an NVFP4 Qwen3.6-35B-A3B checkpoint through SGLang with radix caching and parallel requests. Its published default uses a single B200, so it is not a consumer-GPU installation recipe. The useful lesson is the server design: preserve a shared prefix so many questions reuse the same prefill. Public claims about processing 64 questions are tied to that accelerator, input length and concurrency setup. For one fast request at home, a 4B direct-scoring baseline is easier to install and validate.
Trained projects change more than the serving path
kev adds LoRA and a small readout head to Qwen2.5-0.5B so one sequence can contain a document and many questions. qwen-rlcd tunes Qwen3.5-0.8B for Choice, Score and Noul decisions. These projects move beyond reading an untouched checkpoint, but their strengths can be narrow because they depend on training data and evaluation scope. Do not choose by download size or one latency number. Check your candidate count and whether small wording changes preserve the decision. The growing ecosystem is useful, but there is not yet one standard local Jev.
