mlx-audio: TTS and STT from a single install

mlx-audio is an audio library on top of Apple's MLX². Text in gives you speech, speech in gives you text, and synthesis, recognition, speech-to-speech and music generation all live in one package.

Models do not each bring their own command: one CLI³ and one server stay fixed while the model value changes. Comparing models on the same sentence needs no new code.

It is Mac-only: it runs on Apple Silicon, with no CUDA⁴ or ROCm⁵ path.

It runs on MLX, so it is tuned for Apple Silicon: a Python package whose arithmetic MLX performs.

Install with pip. Saving to mp3 or FLAC also needs ffmpeg; wav alone does not.

One command takes a model, a sentence and a voice. Multiple sentences produce numbered files, and --join_audio merges them.

With --stream the audio arrives while it is generated. Use --play to hear it immediately, and add --save to keep the streamed result.

Supported languages vary by model, and six of the listed ones name Korean. Choosing by name alone can land you on an English-only model.

Cloning splits into models that need a reference voice and models that do not. If cloning is the goal, start from the models that ship it.

In Python you load a model and pass the text. Changing only the model name runs another model on the same code.

Each switch downloads new weights. The first run spends time on the download and later runs use the local cache.

The README publishes no per-model speed figures, so compare output length and memory footprint, then decide by listening.

from mlx_audio.tts.utils import load_model

for name in [
    "mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit",
    "mlx-community/Ming-3.0-tts-tiny-8bit",
]:
    model = load_model(name)
    for chunk in model.generate("같은 문장을 읽어 봐요.", lang_code="Korean"):
        print(name, chunk.audio.shape[0])

Starting the server opens OpenAI-style routes, so an app already wired to that shape only needs a new base URL.

Synthesis takes JSON with the text; transcription takes an audio file as multipart. One server handles both.

mlx_audio.server --host 0.0.0.0 --port 8000

curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model": "mlx-community/Kokoro-82M-bf16", "input": "안녕하세요", "voice": "af_heart"}' \
  --output speech.wav

A web view ships too, but it starts separately from the server rather than with a single command.

The recognition side ships Whisper variants, Qwen3-ASR and Parakeet, in the same package as synthesis with no second install.

The aligner matches sentences to audio at timestamp level, which is where subtitle start times come from.

Diarization⁶ models are listed too, so a meeting recording with several speakers can be split by speaker.

A Swift package lets an iOS or macOS app call it directly without going through the server.

Many models are published at 3, 4, 6 and 8 bits, so changing the suffix on the model name reruns the same sentence at another width.

Lower bits need less memory and shift the sound. Start at 8-bit for dialogue and 4-bit for narration.

There is no baseline for quality or speed; conditions differ per model, so compare the same sentence yourself.

Without ffmpeg only wav is written, so install it first if the pipeline⁷ needs mp3.

Model licences differ per repository, so check the model card before commercial use.