aeneas

That is funny. For audio books I'm currently working on an `epub` command for `tone` which will be able to extract text from `epub` files, e.g.: tone epub --format="markdown" --extract-sentences --one-file-per-chapter output-path/ As a result, you can use https://github.com/readbeyond/aeneas with the generated text / markdown files to create a json mapping file looking like this: { "fragments": [ { "begin": "0.000", "children": [], "end": "7.920", "id": "f000001", "language": "eng", "lines": [ "This is the first sentence of the audio book." ] } } Since aeneas is a bit inaccurate, I'm also working on an improvement with silence detection for these mapping files. If you are looking for something that is "ready to use", you could check out https://github.com/r4victor/syncabook o

I use Aeneas[1], a set of tools to do force alignment. I found it in equal measures an amazing and a hard to navigate resource. Took me a while to set up and configure everything to the point that it was usable. But when it works, it works well. [1] https://github.com/readbeyond/aeneas

here's another solution: https://github.com/readbeyond/aeneas

https://github.com/readbeyond/aeneas can output SMIL e-books where each line of text is associated with the corresponding segment from the audiobook. But I don't know which e-book readers support that format.

You are close enough, so I have to respect my word. I’m not a genius, just a lego builder, I’ve tried a lot of methods, from DL to ML but aeneas project (with some optimizations) gave me the best results. Amazing project and even better personality. Take a look at https://github.com/readbeyond/aeneas Together with espeak-ng, you can get good results for line level alignment for 108 languages.

If you already have the transcript without timestamps (e.g. for an audiobook where you know the source text), you could use https://github.com/readbeyond/aeneas , which infers the timestamps by aligning text-to-speech output with the audio using dynamic time warping. If you don't have the transcript, you'd use a transcription service that also gives you timestamps. E.g. there was a frontpage submission yesterday where someone used AWS Transcription to count the number of words in each minute of a talk: https://news.ycombinator.com/item?id=21635939

Very interesting application. I am not sure if you guys have looked into this, but there is a Python library that can detect timestamps on word level if given the audio and transcript. It's pretty accurate for English: https://github.com/readbeyond/aeneas

Hi, thank you. Having it on conda would be great (https://github.com/readbeyond/aeneas/issues/158 ), so if you feel like it, it would be wonderful! The two points that proved difficult in packaging aeneas (as self-installers and as .deb for Ubuntu/Debian) are: 1. the package must also install/require-as-dep ffmpeg and espeak 2. the package must trigger the compilation of the Python C extensions as described in setup.py Unfortunately I am a Debian/OS X user (I do not even own a Windows machine right now), but I am told that one can use the Win10 IE VirtualBox images.

Thank you. Indeed several users of aeneas adopted it for producing SRT/TTML files, i.e. captions, for videos, both online and offline --- and many of them start with an existing transcript. However, please note that there are limitations on the amount of "non speech" that aeneas can tolerate: for example, long spurious portions of audio or sung passages might affect the quality of the alignment. For details on how aeneas works: https://github.com/readbeyond/aeneas/blob/master/wiki/HOWITW...