Local AI Guide

See the speed before choosing hardware

Local speech recognition & synthesis

Separate voices and sounds in recordings with SAM Audio

Describe a sound, then separate the target from the remaining audio.

Meta's SAM Audio isolates a sound from a complex recording using a text description, visual cue, or time span. The official example runs on a CUDA1 GPU2 and requires requesting access to pretrained checkpoints3 and authenticating. It is a general audio source-separation tool for speech, music, and sound effects, not a dedicated vocal-removal model.

Contents
Requirements and key details
  • Describe the target with text, a time span, or a visual cue.
  • The official example requires Python 3.11+ and a CUDA GPU.
  • Request access to the pretrained checkpoints separately.

Separate a described sound from the rest

SAM Audio is a sound-separation model. In a meeting recording, specify a speaker, background music, or a horn with a natural-language description, time span, or visual cue. It returns the target signal and the remaining signal from the input mix. Meta's SAM Audio overview · Official repository

An audio waveform displayed on a laptop beside a microphone
Name the target sound with a short description.

Prepare CUDA and checkpoint access

Prepare Python 3.11+ and a CUDA-compatible GPU, then install the official repository. First request and receive checkpoint access on the SAM Audio Hugging Face model card. Clone the repository, place meeting.wav in its root, and authenticate with the Hugging Face CLI4.

Install from a clone of the official repository
git clone https://github.com/facebookresearch/sam-audio.git
cd sam-audio
python -m pip install .
Authenticate and prepare the input recording
python -m pip install "huggingface_hub[cli]"
hf auth login
# Enter the token in the interactive prompt
cp /path/to/meeting.wav ./meeting.wav

Describe the target sound precisely

Save the code below as separate.py and run it. The requested speech is saved to target.wav, and the remaining sound to residual.wav. Keep meeting.wav in the repository root; outputs use the processor’s sample rate5.

Text-prompt separation pattern
from sam_audio import SAMAudio, SAMAudioProcessor
import torchaudio
import torch

checkpoint = "facebook/sam-audio-large"
model = SAMAudio.from_pretrained(checkpoint).eval().cuda()
processor = SAMAudioProcessor.from_pretrained(checkpoint)
batch = processor(audios=["meeting.wav"], descriptions=["speech"]).to("cuda")
with torch.inference_mode():
    result = model.separate(batch, predict_spans=False, reranking_candidates=1)
sample_rate = processor.audio_sampling_rate
torchaudio.save("target.wav", result.target[0].cpu().unsqueeze(0), sample_rate)
torchaudio.save("residual.wav", result.residual[0].cpu().unsqueeze(0), sample_rate)
Run the separation script
python separate.py

Review both target and residual signals

Check whether the target contains only the intended speaker or effect, and whether the residual still contains background speech you need. Results depend on the source recording and prompt6. If needed, narrow the description, such as one person's speech, or rerun with the official time-span prompting method.

A desk with headphones, a microphone, and a transcription workspace
Listen to the isolated target and the remaining background separately.

Distinguish general separation from vocal removal

SAM Audio is a general audio-separation tool that targets speech, instruments, and sound effects using text, video, or time spans. For vocal removal, you can try a singing voice prompt on a music mix, but do not assume the quality or workflow matches a dedicated vocal/accompaniment model. Listen for distortion, reverb artifacts, and missing instruments in both outputs.

Check checkpoint terms and access conditions

Checkpoint approval grants access to the model files, but it does not grant rights to separate or share voices in a recording. Notify participants and follow your organization’s retention policy for meeting or counseling audio.

A recorder, notes, and headphones for reviewing speech recognition
Use a time span or video cue to narrow the sound you want.

Terminology notes

  1. CUDA — A software platform for general-purpose computing on NVIDIA GPUs.

    Back to the text

  2. GPU — A processor designed to handle many calculations in parallel. It performs model computations during AI inference.

    Back to the text

  3. Checkpoint — A file containing saved model weights and related state. Versions or tasks in one model family may use different checkpoints.

    Back to the text

  4. CLI — Short for Command-Line Interface: operating a program by entering commands in a terminal.

    Back to the text

  5. Sampling rate — The number of times per second an audio signal is measured when digitized.

    Back to the text

  6. Prompt — The input text sent to a model. For generative models it describes the desired result and strongly shapes the output.

    Back to the text

Frequently asked questions

What should you check first in this article?
Meta's SAM Audio isolates a sound from a complex recording using a text description, visual cue, or time span. The official example runs on a CUDA GPU and requires requesting access to pretrained checkpoints and authenticating. It is a general audio source-separation tool for speech, music, and sound effects, not a dedicated vocal-removal model.
When is it better not to choose this?
SAM Audio is a general audio-separation tool that targets speech, instruments, and sound effects using text, video, or time spans. For vocal removal, you can try a `singing voice` prompt on a music mix, but do not assume the quality or workflow matches a dedicated vocal/accompaniment model. Listen for distortion, reverb artifacts, and missing instruments in both outputs.