Local AI Guide

See the speed before choosing hardware

Local speech recognition & synthesis

Label Speakers and Inspect Ambient Sounds in Meeting Recordings with LocalAI 4.11

Name speakers in a meeting recording, then review its transcript and surrounding sounds

LocalAI 4.11 is an open-source local AI server. Send it a meeting recording to receive speech segments and a transcript; listen to a speaker segment and save a name for reuse in a later recording. An audio-scene model also returns non-speech events such as coughing or applause with timestamps.

Contents
Requirements and key details
  • Diarization marks who spoke when; ASR turns the speech in those segments into text.
  • Preview a speaker in Studio and explicitly confirm the name before it can be reused in later recordings.
  • Sound events include labels and time spans for non-speech audio such as coughing or applause.
  • Saved voices are cleared on restart and shared among users of the same LocalAI instance.

A meeting recording can yield transcript, speaker, and sound information

LocalAI 4.11 diarization1 marks the start and end of each utterance and assigns temporary labels such as SPEAKER_00 to voices. With ASR2, each segment also contains transcript text.

Sound-event analysis has a different job from speech transcription. It can identify the type and timing of sounds such as a cough or applause in the same recording. That helps when reviewing not only who said what, but also when applause or an alert sound occurred around the meeting.

Speaker names are not imported automatically from an attendee list. Listen to a voice segment and save a name so it can appear for the same voice in a later recording. An internet connection is needed to download model weights during the first installation.

A wooden meeting table with a central microphone surrounded by four empty chairs
Review meeting notes with both the transcript and the timing of each turn.

Use a short recording to review speakers in Studio

Run LocalAI 4.11 or later. In Models → Explore, search for and install parakeet-cpp-nemotron-3-diarization-asr-speakers. This gallery entry combines the models needed for transcription, speaker turns, and speaker recognition. If installation stalls or fails, check progress and errors in Operate → Activity.

After obtaining participants' consent to prepare and register voice profiles, choose the installed model and a short recording in Studio → Diarization. Enable Prepare speakers to remember even for the first test if you want transcript text, then select Diarize. The option requests both text and speaker profiles. First read the anonymous speaker labels and spoken words; save names in the next step.

When results appear, read the transcript sentence by sentence and replay sections where a speaker label changes or voices overlap. Before recording an owner or decision in the minutes, check the label against the recording; leave uncertain turns for review rather than assigning a name.

Preview and save a name, then reuse it in later recordings

When the preceding step returns results, play the person's Preview in Speakers. Choose a segment where their voice is clear, select Name and remember, enter a name, and select Remember. After the server confirms registration, the name appears on that person's turns.

A very short or overlapping preview may be too unclear to name. Prepare another recording with a longer, clearer segment. Compare important statements with the recording even after a name appears.

For the next recording, select the same speaker-recognition-capable model. When it finds a saved voice, it can show the saved name instead of an anonymous label.

A laptop with three colored audio tracks, headphones, and a portable recorder
Use the timeline to locate where overlapping speech starts and ends.

Choose a sound-scene model to include surrounding audio events

The gallery entry for saving speaker names differs from the one that also returns surrounding sound events. To review speech alongside events such as coughing or applause, install parakeet-cpp-multilingual-diarization-speakers-sounds. This offline-recording option combines transcription, speaker turns, saved-voice matching, and sound events.

Once LocalAI is running and the sound-scene model is installed, open a new terminal in the folder containing meeting.wav. For a different file, change the filename in file=@meeting.wav. Send the recording to /v1/audio/diarization with include_text, include_speaker_profiles, and include_sounds enabled and response_format=verbose_json.

Request transcript, speakers, and sound events from a meeting WAV
curl http://localhost:8080/v1/audio/diarization -F model=parakeet-cpp-multilingual-diarization-speakers-sounds -F file=@meeting.wav -F include_text=true -F include_speaker_profiles=true -F include_sounds=true -F response_format=verbose_json

The response's segments contain speech times and text; sounds contains labels and times for non-speech events. Replay labeled intervals to decide which events matter for the meeting record. Both default json and verbose_json can carry sound events; rttm has no field for them.

If the file cannot be read, check the terminal's current folder and filename. For HTTP 501 with include_sounds_unsupported, check whether the selected gallery model supports sound events. Select parakeet-cpp-multilingual-diarization-speakers-sounds when sound analysis is needed, and review model-loading errors in the server.

Check storage scope and server operating state

Saved voices are currently lost when the server restarts. Users of the same LocalAI instance also share enrollment information. Agree with the operator on whose voices are enrolled, who may use them, and how recordings are retained.

Operate → This machine shows VRAM3, RAM4, CPU5, model-disk usage, and processes behind loaded models. If a model does not appear or memory becomes constrained after loading, use this page to inspect current processes and resource use. The first CPU sample may show Measuring… while LocalAI gathers enough data.

Failover routes requests to other model targets according to its configured order and conditions. If a chain includes another LocalAI server or remote target, recordings or request data may be sent there. Check the deployment targets and remove remote paths if audio must stay within your environment.

A small desktop computer beside an audio-track monitor, blank notebook, and headphones
Use the recording timeline to find a turn and compare it with the transcript.

Terminology notes

  1. Speaker diarization — The task of dividing a recording into who-spoke-when segments and labeling them by speaker.

    Back to the text

  2. Automatic Speech Recognition — Technology that recognizes spoken language in audio and converts it to text.

    Back to the text

  3. VRAM — Memory used by a graphics card’s GPU to store model weights and intermediate values.

    Back to the text

  4. System RAM — System memory that temporarily holds data while programs run.

    Back to the text

  5. CPU — The central processor that runs the operating system and general-purpose instructions; in AI workloads it also handles tasks such as preprocessing and data movement.

    Back to the text

Frequently asked questions

What should you check first in this article?
LocalAI 4.11 is an open-source local AI server. Send it a meeting recording to receive speech segments and a transcript; listen to a speaker segment and save a name for reuse in a later recording. An audio-scene model also returns non-speech events such as coughing or applause with timestamps.
How much memory does it need?
Operate → This machine shows VRAM, RAM, CPU, model-disk usage, and processes behind loaded models. If a model does not appear or memory becomes constrained after loading, use this page to inspect current processes and resource use. The first CPU sample may show Measuring… while LocalAI gathers enough data.