Local AI Guide

See the speed before choosing hardware

memory and model

TensorFold 1.0.8 cuts Qwen3.8-27B Mac memory by 17.6 GiB

Removing duplicate loading creates more room to prepare DFlash2 with a long context.

TensorFold 1.0.8 keeps one in-memory copy of Qwen3.8-27B weights, reducing readiness memory with DFlash2 by 17.6 GiB. In the release's 70,000-token context check, M5 Max changed from 47.2 to 29.6 GiB and M3 Ultra from 42.2 to 24.6 GiB. The reduction creates room for this combination's DFlash2 drafter and long context; actual use should be checked with your usual apps and input lengths included.

Contents
Requirements and key details
  • Removing duplicate weights reduced readiness memory by 17.6 GiB with DFlash2 included.
  • At the 70K-context check, M5 Max went from 47.2 to 29.6 GiB and M3 Ultra from 42.2 to 24.6 GiB.
  • The 17.6 GiB reduction comes from eliminating duplicate model weight loading.

The weights now occupy one copy instead of two

With DFlash2, the drafter proposes next tokens1 and Qwen3.8-27B verifies them. TensorFold 1.0.8 keeps one in-memory copy of the Qwen3.8-27B weights on Mac.

The table shows readiness memory with DFlash2 at a 70,000-token context in the TensorFold release check. The version changed under the same stated condition, with a 17.6 GiB reduction on each Mac.

A dark monitor, a compact desktop computer, and a PC case with several visible fans
Set up the server hardware alongside the workstation you use as a client.

Readiness memory fell at 70K context

The figures below show memory reserved to prepare the server and DFlash2 in the TensorFold release check. The version changed from 1.0.7 to 1.0.8; at the same 70,000-token context and generation speed, readiness memory fell by 17.6 GiB on each Mac.

The table records process memory while preparing TensorFold and DFlash2 on those Macs. macOS and your usual apps share unified memory2 too. Run the server on your own Mac and check Activity Monitor's memory pressure and swap growth to assess practical headroom.

Scroll sideways to compare all columns.

TensorFold release readiness memory at 70K context
Hardware1.0.71.0.8Change
M5 Max47.2 GiB29.6 GiB−17.6 GiB
M3 Ultra42.2 GiB24.6 GiB−17.6 GiB

To use the space saved by avoiding duplicate loading, check the server first with --context 4096, then raise the context while your usual apps are open. /memory is a read-only HTTP route on the default API3 server, not a terminal command. Check the process's physical footprint and lifetime peak before increasing request length.

Install 1.0.8 and run it with DFlash2

Upgrade the existing Homebrew install to 1.0.8, then specify the exact model repository and DFlash2 drafter. tensorfold pull verifies large model files by checksum and resumes missing pieces after an interrupted download.

Upgrade to TensorFold 1.0.8
brew upgrade ashhart/tensorfold/tensorfold
tensorfold --version

On an M5 Max or M3 Ultra, download the model and start the server with the commands below. --context 4096 is a starter value for the test request; raise it after checking readiness memory. --drafter attaches DFlash2 to Qwen3.8-27B.

Pull Qwen3.8-27B and DFlash2
tensorfold pull TensorFold/Qwen3.8-27B-MLX-4bit
tensorfold pull z-lab/Qwen3.8-27B-DFlash2
Start the Qwen3.8-27B server
tensorfold serve TensorFold/Qwen3.8-27B-MLX-4bit --name qwen38-local --drafter z-lab/Qwen3.8-27B-DFlash2 --context 4096 --parallel 1 --temperature 0
A laptop with a document window connected by cable to a phone displaying a page
Continue memory checks in the everyday setup where you use the model.

Distinguish readiness memory from everyday use

The 17.6 GiB reduction matters when loading Qwen3.8-27B with DFlash2 or preparing a longer context. Leave room for apps alongside model loading, and check both macOS memory pressure and TensorFold's /memory diagnostics.

Read memory diagnostics from the default API server
curl -fsS http://127.0.0.1:8080/memory
Two compact desktop computers connected to a monitor showing streams of particles
Check server memory and Mac headroom before extending the context.

Terminology notes

  1. Token — A unit into which a model divides input or output for processing.

    Back to the text

  2. Unified memory — An architecture where the CPU and GPU share one physical memory pool.

    Back to the text

  3. API — A defined interface that lets other code call a program’s functions.

    Back to the text

Frequently asked questions

What should you check first in this article?
TensorFold 1.0.8 keeps one in-memory copy of Qwen3.8-27B weights, reducing readiness memory with DFlash2 by 17.6 GiB. In the release's 70,000-token context check, M5 Max changed from 47.2 to 29.6 GiB and M3 Ultra from 42.2 to 24.6 GiB. The reduction creates room for this combination's DFlash2 drafter and long context; actual use should be checked with your usual apps and input lengths included.
How much memory does it need?
The table records process memory while preparing TensorFold and DFlash2 on those Macs. macOS and your usual apps share unified memory too. Run the server on your own Mac and check Activity Monitor's memory pressure and swap growth to assess practical headroom.