Local music generation speed and hardware: how to read RTF correctly

A model that turns text into a short sound effect, one that sings lyrics, and one that extends a melody do different jobs. Their durations and conditioning also differ, so a list of runtimes alone can mislead. Fix the task, output duration, stereo setting and whether vocals or lyrics are required before comparing.

For example, if you need a 30-second instrumental synth-pop loop, first confirm that every candidate can generate text-conditioned backing music. Do not force a lyric-song-only path and a sound-effect path into one ranking. Time only models that support a shared task, and state why unsupported candidates are excluded; that makes a buying decision more useful.

Choose a unit that matches the output. For a song, record the time and actual duration of one song; if a long track is made in segments, include the segment count and stitching time. A larger batch may change how many songs finish per hour without making one song finish sooner. Report single-request latency and throughput⁴ as separate metrics.

The real-time factor (RTF) is elapsed generation time divided by the playback duration of the output. If 60 seconds of audio takes 30 seconds to create, RTF is 0.5, or twice real time.

If it takes 120 seconds, RTF is 2 and slower than real time. Some repositories call the reciprocal—such as ‘34×’—RTF.

Read the definition and units, then recalculate from raw seconds when possible.

The first run may include checkpoint⁵ loading, memory allocation, compilation or kernel setup. Repeated generation with the model resident excludes those costs, so it differs from the wait a user experiences after opening an app. Record the first request (cold start) and requests after warmup separately, and keep a separate end-to-end user wait time.

The table uses figures published in official repositories or model cards, while distinguishing their evidence quality. The older ACE-Step table and ACE-Step 1.5 README claims refer to different model versions; the latter is not a per-device result table reproduced with the profiler.

A blank means this research found no speed figure with enough stated conditions, not that the model is slow.

A music workflow may include a text or lyric planner, audio-token⁶ generation or diffusion, a vocoder or VAE⁷ decoder, and saving or conversion. End-to-end time is the total user wait; per-stage times help locate bottlenecks. ACE-Step 1.5's official profiler can separate LLM⁸ planning, DiT⁹ diffusion, VAE decoding, audio saving and total wall time.

The official YuE2 model card compares three generation modes on an RTX 4090. RTX 4090 values are averages over 32 warm requests; GPU memory is the full-run peak reported by NVML.

Audio seconds are each output's duration and generation seconds are synchronized pipeline¹⁰-call time. RTF in the table is calculated as generation seconds ÷ audio seconds for reference.

The card excludes initial path resolution and saving, so these are not app-start-to-saved-file wait times.

Use the same prompt and lyrics, output duration, sample rate¹¹, channels and quality-related settings. Record checkpoint, code version, backend, precision and step count.

Along with the device name, note the operating system and driver/framework versions. Separate the first run from repeated runs after warmup, measure several times, and report the median and range.

Keep failures, out-of-memory events and truncated outputs in the record.

A small log table is enough to begin. Use columns such as model/revision, device, backend, precision, duration, steps, first-run seconds, seconds for warm runs¹² 1–5, actual output duration, peak memory and status. Copy the same prompt and settings into each model request, and give each output file and log the same experiment ID. Put changed settings on a new row.

Suppose a first run takes 42 seconds, the median of the next five takes 18 seconds, and the output is 30 seconds long. Warm-run conventional RTF is 18÷30=0.60, or about 1.67× real time. Describe the 42-second first-run wait separately. If a source reports 1.67×, show what was divided instead of copying only the label ‘RTF’; readers will not reverse which direction is faster.

After measuring, do not pick a winner from one number; interpret it against the use case. If you launch the app for one song at a time, cold-start wait matters.

For an always-on server, warm-run median and throughput may matter more. Stage timings can show whether long tracks are dominated by DiT or audio-token generation, while planning and loading overhead can loom larger for short outputs.

Mark speed gained by reducing quality settings or duration as a separate comparison.

In a music model, only the planning stage may use an LLM; the audio itself can come from diffusion iterations, autoregressive¹³ audio tokens and decoding. A device with high text-token throughput can still be slow in those stages. For audio generation, look at audio frames¹⁴ or tokens per second, diffusion steps, output duration and end-to-end seconds together.

NVIDIA CUDA¹⁵, Apple MPS and MLX¹⁶ are not interchangeable labels; each project must implement and support the path. ACE-Step 1.5 documents CUDA/MPS and vLLM, PyTorch¹⁷ or MLX paths for its language model, but performance still depends on the exact configuration.

YuE2 documents Linux and a 24GB BF16¹⁸ NVIDIA GPU as its baseline. HeartMuLa's official example defaults to CUDA.

Unified memory¹⁹ in a Mac or the specifications of an RTX 5090 cannot establish support or predict throughput by themselves.

Lyric clarity, prompt adherence, song structure and timbre do not collapse into one universal score. Listen to short samples and judge them for your use case.

Model weights and code can have different licenses, so check the terms for the exact release and its training-data notes before distribution or commercial use. Avoid supplying or reproducing lyrics, recordings or recognizable voices without the relevant rights.