Align Korean subtitles and speaker labels with WhisperX

WhisperX transcribes a recording, then uses an alignment model to estimate where each word falls in the audio. Word-level segments make it easier to fix only the captions that drift in a subtitle editor. Alignment matches text that it receives to speech, so a transcription error can still receive a timestamp.

Run transcription and alignment on a short Korean WAV file. Check the sentence text and each word's start and end times, then correct misrecognized words first. Running alignment again with the corrected text makes it easier to find the subtitle segments to fix.

First install ffmpeg and place recording.wav in the working folder. The commands below are for macOS and Debian or Ubuntu Linux. Then install PyTorch¹ and WhisperX in your Python environment.

# macOS
brew install ffmpeg

# Debian / Ubuntu Linux
sudo apt-get update && sudo apt-get install -y ffmpeg

python -m pip install torch torchaudio
python -m pip install whisperx

whisperx recording.wav --output_dir whisperx-output --language ko --model large-v2 --compute_type int8 --device cpu --highlight_words True

Place recording.wav in the current folder and run the command to save whisperx-output/recording.srt and the transcript. Set --language ko for Korean; WhisperX then calculates word timings with its Korean alignment settings.

Speaker diarization² assigns a speaker ID to each recording segment. Accept the access conditions on the pyannote model card used by WhisperX, then prepare a Hugging Face read token³. In macOS or Linux Bash, the command below keeps the token off screen and uses it only in the current shell.

For a conversation with two participants, add --min_speakers 2 --max_speakers 2 to provide the expected speaker count. These values guide the model as it finds segment boundaries. Overlapping speech or brief acknowledgements may be merged or assigned a different ID, so listen again and check the labels.

read -rs HF_TOKEN
printf '\n'
export HF_TOKEN
whisperx recording.wav --output_dir whisperx-output --language ko --model large-v2 --compute_type int8 --device cpu --diarize --min_speakers 2 --max_speakers 2 --highlight_words True --hf_token "$HF_TOKEN"

Open whisperx-output/recording.srt with the WAV file in a subtitle editor and correct the text and timecodes. Checking numbers, names, and overlapping speech first makes it easier to distinguish alignment errors from transcription errors.

A speaker ID is a number that distinguishes speakers, not a person's name. Before adding names to meeting notes, listen to the audio again and keep the WAV and SRT files according to the permissions granted by participants and your organization.