Compare local music models for songs, instrumentals and sound effects

A vocal song, accompaniment loop, background bed and sound effect need different inputs and outputs. For lyrics, check lyric conditioning and section development; for a scene effect, check duration limits, text prompting and audio format. Compare documented inputs and outputs rather than ranking everything from one identical prompt.

Local means the code and weights run on your device. Initial downloads need internet, and exposing a server to others is a separate choice. Check checkpoint² size and file licenses, and keep output files organized by project.

ACE-Step 1.5 generates from descriptions and lyrics and documents cover and editing features. More than 50 languages and up to ten minutes are capability descriptions, not guarantees of equal language quality or device runtime.

HeartMuLa 3B accepts lyric and tag files and lists a default maximum duration of 240,000 ms. Its repository reports current inference³ around RTF⁴ 1.0, so it does not support claims of multi-minute songs in seconds. Lazy loading is a memory option, not a speed guarantee.

YuE2 plans a score from lyrics and synthesizes 48 kHz stereo, with score-editing and cover examples. Its official baseline is Linux, Python 3.12 or newer, and a 24 GB BF16⁵-capable NVIDIA GPU⁶.

In the model card's ordinary generation table, an RTX 4090 24GB generated 214.85 seconds of audio in 71.04 seconds, averaged across 32 warm runs⁷, with 11.18 GiB peak VRAM⁸. This was unquantized PyTorch⁹ 2.10, about 0.33 times the audio duration; initial loading was excluded and it processes one song at a time.

The scores below are automatic evaluations published in the YuE WildSongBench documentation and YuE2 model card on September 12, 2026. They cover the same 192 prompts, but are neither human preference results nor a compute-matched comparison.

Standard YuE2 and public model baselines generate two candidates and select the lower-PER¹⁰ result, with four ASR¹¹ passes per candidate. Best-of-8¹² selects by musicality, prompt control and PER, so it should not be compared directly with the standard setting.

For context, its scores (SongBench Avg 6.9632, PER 9.79%) come from that separate selection process. The benchmark uses the YuE2-Vae¹³-legacy decoder; YuE2-Vae is the everyday default for better perceived audio quality.

These are automatic metrics, not universal human judgments of sound quality.

Stable Audio Open 1.0 generates up to 47 seconds of 44.1 kHz stereo from text. Its model card describes sound effects and musical elements as intended uses and specifies English prompts. It is a candidate for effects or an instrumental passage, not a replacement for lyric-led songs. Access requires accepting the Hugging Face license terms.

MusicGen offers 300M, 1.5B and 3.3B sizes with text and melody variants. Its model card says realistic vocals are difficult and performance may be lower outside English. AudioCraft's guide lists 16 GB VRAM or more for medium. It fits background-music ideas and research experiments better.

Beyond the standard 2B DiT¹⁴'s 4/6 GB starting guidance, ACE-Step offers an XL 4B DiT. The current NVIDIA table recommends CPU¹⁵ offload¹⁶ for XL in the 16–20 GB tier, no offload from 20 GB, and a 4B LM from 24 GB. Apple silicon has a dedicated MLX¹⁷ launcher alongside the broader MPS support. YuE2 specifies Linux and 24 GB BF16 NVIDIA; MusicGen medium lists 16 GB VRAM.

Stable Audio Open and HeartMuLa do not state one minimum VRAM for all setups. Do not turn missing data into claims of low-end support or high-end necessity; check the exact checkpoint instructions. Test the Apple-silicon MLX path briefly, without comparing it directly to NVIDIA VRAM tiers.

The current YuE2 README distinguishes Apache 2.0 code/docs from CC BY-NC 4.0 plus additional creator permission for YuE2-3B/VAE weights. Do not carry older YuE license announcements forward. MusicGen's MIT code license does not cover its CC BY-NC 4.0 weights.

Stable Audio Open uses the Stability AI Community License and directs commercial users to separate terms. HeartMuLa says its code and associated weights are Apache 2.0; ACE-Step 1.5 lists MIT. Recheck the exact checkpoint terms.

Training data, reference lyrics/audio, voice imitation and platform policies can remain separate from model licenses. Read the relevant terms before monetizing or releasing music.

For vocal-song experiments, start with ACE-Step; for lyric/tag-file workflows, HeartMuLa; for score control on 24 GB Linux/NVIDIA, YuE2. Stable Audio Open suits effects and musical elements up to 47 seconds; MusicGen suits text- or melody-conditioned background-music experiments. This is a task-based shortlist, not a quality ranking.

For a hands-on comparison, align target duration and genre, and record each model's supported inputs. Compare runtime, memory, Korean lyric intelligibility, sample rate¹⁸ and editing effort. Mark undocumented memory or timing as unspecified.

Start with two models that target the same outcome to limit setup overhead. Use the same short lyrics and genre for songs, or the same sound description and target duration for effects, adapting only to each model's supported input. A cold start may include downloads and initialization, so record it separately from generation and check that the output file is saved.

In your comparison notes, record model and checkpoint, code version, device, settings, output duration, generation time, peak memory and listening notes. Separate published claims from your own measurements; do not fill unmeasured fields with guesses. If the audio is for release, note the license and input-material review too.