txtai

As an open source maintainer, I can chime in on this. With TxtAI, I've seen a large uptick in PRs (https://github.com/neuml/txtai/pulls?q=is%3Apr+is%3Aclosed+s...). While the extreme verbosity of Claude messages and commits is very annoying (plus the constant defending itself on why it's a bug), I do think it's a positive that more people are enabled. It does require reviewing the PRs. Some can be tricky just like a human. For example I did merge this PR (https://github.com/neuml/txtai/pull/1136) and it would have completely broke search. But a human could also do that. From an open source standpoint, I say the more the better. You just have to be willing to do the work to review and no not just having AI agents to review what the AI agents are submitting. There still needs to be a human in the loop, if you care about quality.

100% agree on this. I've been working on small local models for years with txtai (https://github.com/neuml/txtai). I've published close to 100 models that can run local for RAG, Agents, Vector Search and more (https://huggingface.co/NeuML/collections).

Local models are great for a lot of things past just software development. We need to move towards solving other real world problems vs just building software. I've been focused on that with TxtAI (https://github.com/neuml/txtai) for 6 years now.

You can make it even simpler and not bother with any of this. With even something as large as 100M vectors, you can just use Torch or GGUF with compression. Even NumPy can take you a long way. Example below. https://github.com/neuml/txtai/blob/master/examples/78_Acces...

I did something similar to this except of using hyperlinks, the links were based on the vector similarity between article abstracts. https://github.com/neuml/txtai/blob/master/examples/58_Advan...

Context and prompt engineering is the most important of AI, hands down. There are plenty of lightweight retrieval options that don't require a separate vector database (I'm the author of txtai [https://github.com/neuml/txtai], which is one of them). It can be as simple this in Python: you pass an index operation a data generator and save the index to a local folder. Then use that for RAG.

I built TxtAI with this philosophy in mind: https://github.com/neuml/txtai

Excellent project. As mentioned in another comment, I've put together an embeddings database using the arxiv dataset (https://huggingface.co/NeuML/txtai-arxiv) recently. For those interested in the literature search space, a couple other projects I've worked on that may be of interest. annotateai (https://github.com/neuml/annotateai) - Annotates papers with LLMs. Supports searching the arxiv database mentioned above. paperai (https://github.com/neuml/paperai) - Semantic search and workflows for medical/scientific papers. Built on txtai (https://github.com/neuml/txtai) paperetl (https://github.com/neuml/paperetl) - ETL processes for medical and scientific papers. Supports full PDF docs.

If you're looking for a lightweight open-source framework designed to handle the patterns mentioned in this article: https://github.com/neuml/txtai Disclaimer: I'm the author of the framework.

I fully agree. Postgres has solved many of the problems that many are re-solving with GenAI related databases. With txtai (https://github.com/neuml/txtai), I've went all in with Postgres + pgvector. Projects can start small with a SQLite backend then switch the persistence to Postgres. With this, you get all the years of battle-tested production experience from Postgres built-in for free.

An alternative you can try is txtai (https://github.com/neuml/txtai). RAG section: https://github.com/neuml/txtai?tab=readme-ov-file#retrieval-... Disclaimer: I'm the primary developer

Thanks for sharing. If you want notebooks that do some of this with local open models: https://github.com/neuml/txtai/tree/master/examples and here: https://gist.github.com/davidmezzetti

VAD is certainly a complex but underappreciated topic. If you like signal processing, FFTs and want to see a similar concept implemented in Python, then check out this code. It's has a fairly well-tuned VAD component built-in. https://github.com/neuml/txtai/blob/master/src/python/txtai/...

txtai (https://github.com/neuml/txtai) has had inline vectorization since 2020. It supports Transformers, llama.cpp and LLM API services. It also has inline integration with LLM models and a built-in RAG pipeline.

Excellent article on BM25! Author of txtai [1] here. txtai implements a performant BM25 index in Python [2] via the arrays package and storing the term frequency vectors in SQLite. With txtai, the hybrid index approach [3] supports both convex combination when BM25 scores are normalized and reciprocal rank fusion (RRF) when they aren't [4]. [1] https://github.com/neuml/txtai [2] https://neuml.hashnode.dev/building-an-efficient-sparse-keyw... [3] https://neuml.hashnode.dev/benefits-of-hybrid-search [4] https://github.com/neuml/txtai/blob/master/src/python/txtai/...