Tag multiple voices

[jinsu35]

I tried using whisper to transcribe a conversation between three people and I thought it would be great if we could tag different voices. So the final result is a list of texts each with the speaker's id in front of it. For example "Alice: Roses are red..." "Bob: But not all roses are red" and so on.

[jongwook]

It is not supported out-of-the-box, but it has been tried in #264 to combine pyannote's diarization with Whisper.

[elmollero]

Buenos días

[Majdoddin]

@jinsu35 @jongwook
www.lexicaps.com seamlessly adds diarization to Whispers transcription. No 3rd party packages.
Feel free to contact me if you need more features.
Announcement: #1537
Repo: https://github.com/Majdoddin/lexicaps

[brainwane]

I noticed someone used whisper.cpp with tinydiarize to do this.

whisper.cpp 1.4.0, large model, small tinydiarize model

[mrienstra]

That model is available here: https://huggingface.co/akashmjn/tinydiarize-whisper.cpp/tree/main

It can be downloaded within whisper.cpp with this command:

bash ./models/download-ggml-model.sh small.en-tdrz

... thanks to @akashmjn's PR (merged Jul 3, 2023): ggerganov/whisper.cpp#1058

[toanhuynhnguyen]

Try WhisperX!