yt-fts

I've been trying to solve a problem with implementing semantic search on my YouTube search engine yt-fts (https://github.com/NotJoeMartinez/yt-fts). I've managed to substantially speed up search results by storing subtitle embeddings in Chroma. But a bigger problem has been with how to properly segment the text in a way that accounts for the duration and context of word embeddings while returning precise time stamps. This a blog post exploring what I've tried so far.

You can try using yt-fts[0]. I've had great luck with it. [0] https://github.com/NotJoeMartinez/yt-fts

Thank you! I was able to integrate this into the project[1]. I'm also looking into using your openai-to-sqlite[2] library for semantic search. [1]https://github.com/NotJoeMartinez/yt-fts/pull/25 [2]https://github.com/simonw/openai-to-sqlite

It looks like you're running searches using LIKE: https://github.com/NotJoeMartinez/yt-fts/blob/050981c0519a96... SQLite has a really power full-text search mechanism built in - FTS5. It can handle things like stemming and stop words and relevance ranking. My sqlite-utils Python library includes helper methods for setting that up: https://sqlite-utils.datasette.io/en/stable/python-api.html#...

Probably due the YouTube cookies consent page. I opened an issue: https://github.com/NotJoeMartinez/yt-fts/issues/1 Hope it helps.

This tool itself does works great. This behavior is due to the YouTube cookies consent page. I opened an issue about this specific issue: https://github.com/NotJoeMartinez/yt-fts/issues/1 Glad if you want to help.

yt-fts is a simple python script that uses yt-dlp to scrape all of a youtube channels subtitles and load them into an sqlite database that is searchable from the command line. It allows you to query a channel for specific key word or phrase and will generate time stamped youtube urls to the video containing the keyword.