witchcraft

As it happens, I just got my employer's permission to release as open source a Triton back-end for Metal and D3D12 GPUs here: https://github.com/dropbox/neso . As an example of you how can use it to deploy real models there is this project doing ASR and TTS: https://github.com/dropbox/nspeech . Finally, I am also going to be switching the inferencing part of Witchcraft from current Candle on MacOS and OpenVINO on Windows to just Candle with Neso; https://github.com/dropbox/witchcraft

Here is a project with similar goals (I am the main author), but using late interaction semantic search: https://github.com/dropbox/witchcraft

Witchcraft is from-scratch Rust reimplementation of Stanford's XTR-Warp (SIGIR'25, https://arxiv.org/abs/2501.17788 ) multi-vector semantic search engine. Witchcraft runs out of a single SQLite database, is blazing-fast (21ms p.95 end-to-end search latency on NFCorpus on a MacBook Pro), accurate (33% NDCG@10), and easy to deploy in your own apps. The Witchcraft repo also comes with Pickbrain, a sample app and agent skill that you can use to instantly query across all your Claude Code and Codex CLI sessions, effectively giving your agents global long-term memory. Please see the Github page for more details, and feel free to ask questions here, and I will try to answer them.