Edit Audio Locally with AuK: Speech Revisions, Cleanup, and TTS

AuK is a 1.5B local model controlled by an instruction. Give it text to synthesize speech, or include an audio file to request a content edit or changes to delivery such as speed, emotion, or voice quality. Noise and reverberation reduction or source isolation use the same instruction-based interface.

Start by deciding whether the task needs source audio. For new narration, provide the text and a voice description; for revising or cleaning a recording, provide the audio and the requested change. AuK is not an ASR³ system that returns only a transcript: its output is an audio file.

AuK released its code and weights on September 9, 2026, with Base and four-step distilled Flash checkpoints⁴. In editing, the instruction describes what to change or preserve while reference audio supplies the speaker and delivery context. For cleanup, the instruction names the source to keep and noise to remove.

Official examples focus on English and Chinese. If Korean pronunciation or a particular delivery style matters, generate a short segment first and check pronunciation, edit boundaries, and remaining background sound before processing a longer recording.

The official README specifies a Python 3.10 environment. Clone the repository, install the core inference⁵ package, and download AuK-Base and Qwen2.5-Omni-3B separately under `ckpts` with the Hugging Face CLI⁶. AuK-Flash is optional for a first Base inference check.

First install a PyTorch⁷ build that matches your GPU⁸. The next table shows the AuK README’s measurement conditions alongside its memory results.

For Apple Silicon, follow the MLX instructions on the official `feat/mlx-apple-silicon` branch. audio.cpp also supports AuK and Flash as experimental GGUF⁹ models. This guide's `--cpu_offload` command and A800 memory table concern CUDA; on Mac, use the installation documentation and model components for the chosen runtime¹⁰.

Use a short audio file that you have permission to process. The official example input is `assets/demo-input-audio/content-edit/content.wav`; for your own file, change the path and specify the phrase to replace. `--gen_seconds` sets the target output duration. If the result is clipped, increase that value first.

A successful run creates `out_content_edit.wav`. Compare the same segment with the source to hear whether the words changed and surrounding speech connects naturally. The second command adapts the official cleanup request into English. Check the output for reduced noise and reverberation, and ensure the voices you wanted to keep remain.

The official README table records `torch.cuda.max_memory_allocated` during BF16¹¹ inference on one NVIDIA A800-SXM4-80GB and compares CPU¹² offload¹³ disabled and enabled. Its two input conditions are text-only generation of 1.5 seconds and generation with five seconds of reference audio.

The table lets you compare each model under the same A800 conditions. Do not treat these figures as minimum memory requirements for another GPU. Check input duration, precision, and memory used by other applications before choosing an offload setting.

AuK code is MIT-licensed. Its required `Qwen/Qwen2.5-Omni-3B` encoder uses `qwen-research`, with non-commercial research and evaluation terms. Before integration into a commercial service, check the conditions for the code and every model component.

AuK fits short spoken edits, voice-over drafts, and recording cleanup. Choose a dedicated ASR system when you need document transcription or speaker timestamps across long recordings; when Korean output matters, start with a tool you can evaluate on Korean speech samples.