new model · 2026.09.04
Last updated
GLM-5.3 and DeepSeek-V4: open weights, but for a personal PC?
What you need for a team server using large open-weight models
GLM-5.3-Flash and DeepSeek-V4-Flash are large open-weight models whose weights can be downloaded to generate text answers to questions and code on your own server. A team-operated deployment needs memory for the full weights, hardware and a runtime1 supporting parallel execution, and server settings to receive requests.
If you want a team server that does not send code to an external service, distinguish the availability of weights from practical execution on your hardware. Check the model files, accelerators and runtime needed for distributed execution, and whether you can operate requests from multiple users.
Your team manages data and access on its own server
Running a model on an organization-managed server without connecting an external inference2 service lets code and documents be processed internally. Local here includes both personal PCs and team-managed servers. Match memory and runtime so the server can access and compute with the full model weights.
When you manage the server, the team also owns access scope, logs, and permissions. Read each model's license for the planned use and assign responsibility for updates, file storage, and incident response. An occasional personal setup and an always-on multi-user server have different model needs and operational responsibilities.
- Context length, RoPE and sliding-window attention
- MoE vs dense: why low active parameters still need memory
- How much VRAM or unified memory does a local LLM need?
- Run Xing4.0 Locally: Install the Coding Model and Plan GPU Memory
- Qwen3.8-27B release: Can it run on a 32 GB system?
- Qwen3.8-Flash-Next: why 6B active still needs 180B resident
Active parameters and total weights are different quantities
GLM-5.3-Flash is listed at about 320B total with 18B active; DeepSeek-V4-Flash-0731 at about 304B total with 13B active. MoE3 models select some experts for each token4, so the active count is smaller than the total. Other tokens or requests may select different experts, so the runtime needs access to all weights.
With a simplified assumption that all weights are raw 4-bit values, decimal arithmetic gives 160 GB for GLM's 320B and 152 GB for DeepSeek's 304B. These are not measured file sizes or runtime memory. The calculation omits block metadata, mixed precision, KV cache5, and runtime space, so check the actual compressed file and execution conditions.
Check the parallel execution requirements of official checkpoints
Official checkpoints6 may assume a runtime that distributes weights across accelerators and exchanges results between nodes during computation. The official DeepSeek-V4-Flash-0731 example uses 4×GB300 nodes. This is not an installation recipe for one consumer GPU7, so first check the server and parallel-execution requirements in the model card.
Matching the accelerator count alone does not reproduce the same run. Weight partitioning, per-card memory, node interconnect, and the parallel path supported by the runtime must also align. Check that your hardware supports the configuration and that the required software path is available before following the launch command.
A 256 GB unified-memory experiment is a separate compressed-model path
A 256 GB unified-memory system may make it possible to test a compatible compressed conversion, but this is not the official checkpoint's server path. The operating system, apps, and conversation cache also use unified memory8, and being able to store a file does not prove that the chosen runtime can execute that conversion.
First check the experiment file's format and size and whether the same model has been run with that app and runtime version. Test loading and a basic response with a short request, then increase to the code or document length the team will actually use. A very long maximum context does not need to be the default. Increasing input gradually helps locate failures in loading, context, or cache.
Choose the scale that matches the team's operational goal
If the goal is to serve internal requests from multiple users continuously, and data control or throughput9 matters, evaluate an accelerator server together with a maintenance plan. Incident response, updates, model-file storage, and concurrent-request handling are part of the cost. If one person only asks about documents or code occasionally, a smaller model can test the value of a local setup first.
Send the same repository questions to the current and candidate models, then compare code-review usefulness, single-request completion time, and the number of concurrent requests handled. Data control, quality, and operating cost are different criteria and cannot be replaced by one speed figure. Choose a server and model that match the weights actually needed and the team's maintenance capacity.
Terminology notes
Runtime — The software environment that provides facilities needed while a program runs. In local AI, it can also refer to a model execution engine.
Back to the textInference — The process of using a trained model to compute an output for an input. Here, local inference means running the model on the user’s device.
Back to the textMoE — A model architecture that selects some of several expert subnetworks for each input. Total parameters can differ from the number activated for one token.
Back to the textToken — A unit into which a model divides input or output for processing.
Back to the textKV cache — Memory that stores attention keys and values from earlier tokens for reuse during later token generation. Its size depends on context length and batch size.
Back to the textCheckpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.
Back to the textGPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.
Back to the textUnified memory — An architecture where the CPU and GPU share one physical memory pool.
Back to the textThroughput — The amount of work processed or generated over time. Comparisons need the unit, such as tokens per second or requests per second.
Back to the text
Frequently asked questions
- What should you check first in this article?
- GLM-5.3-Flash and DeepSeek-V4-Flash are large open-weight models whose weights can be downloaded to generate text answers to questions and code on your own server. A team-operated deployment needs memory for the full weights, hardware and a runtime supporting parallel execution, and server settings to receive requests.
- How much memory does it need?
- With a simplified assumption that all weights are raw 4-bit values, decimal arithmetic gives 160 GB for GLM's 320B and 152 GB for DeepSeek's 304B. These are not measured file sizes or runtime memory. The calculation omits block metadata, mixed precision, KV cache, and runtime space, so check the actual compressed file and execution conditions.
- Where can you check the conditions and sources?
- Official checkpoints may assume a runtime that distributes weights across accelerators and exchanges results between nodes during computation. The official DeepSeek-V4-Flash-0731 example uses 4×GB300 nodes. This is not an installation recipe for one consumer GPU, so first check the server and parallel-execution requirements in the model card.