pyannote-audio

Proper diarization still remains a white whale for me, unfortunately. Last I looked into it, the main options required API access to external services, which put me off. I think it was pyannotate.audio[1]. [1]: https://github.com/pyannote/pyannote-audio

Whisper cannot perform speaker diarization at the moment, WhisperX and other solutions that can tend to use pyannot https://github.com/pyannote/pyannote-audio

(not the creator, but I've built something similar for personal use) This is a great library for determining which speaker is speaking during each time in an audio file (this is called speaker diarization); I imagine they used it or something like it. Works really well out of the box! https://github.com/pyannote/pyannote-audio

This is called speaker diarization, basically one of the 3 components of speaker recognition (verification, identification, diarization). You can do this pretty conveniently using pyannote-audio[0]. Coincidentally I did a small presentation on this at a university seminar yesterday :). I could post a Jupyter notebook if you're interested. PS: Bai & Zhang (2020) is a great review on the literature [1] [0] https://github.com/pyannote/pyannote-audio [1] https://arxiv.org/abs/2012.00931

Free startup idea: Use Whisper with pyannote-audio[0]’s speaker diarization. Upload a recording, get back a multi-speaker annotated transcription. Make a JSON API and I’ll be your first customer. [0] https://github.com/pyannote/pyannote-audio