sonic

Sonic Important📢 Heads up! Sonic is undergoing a big rework and contributions on the master branch are now bug fixes only. All of our efforts go to the v2 branch, where we are introducing long-awaited features. Sonic v2 will support Terabyte-scale ingestion/storage along with BM25 scoring and snippet retrieval (among other things). Stay tuned! Until Sonic v2 is released, don’t waste time diving into the source code of Sonic, as most of it will be completely different in v2. Outside contributions will be limited to short bug fixes until we consider Sonic stable again. Sonic is a fast, lightweight and schema-less search backend. It ingests search texts and identifier tuples that can then be queried against in a microsecond's time. Sonic can be used as a simple alternative to super-heavy and full-featured search backends such as Elasticsearch in some use-cases. It is capable of normalizing natural language search queries, auto-completing a search query and providing the most relevant results for a query. Sonic is an identifier index, rather than a document index; when queried, it returns IDs that can then be used to refer to the matched documents in an external database. A strong attention to performance and code cleanliness has been given when designing Sonic. It aims at being crash-free, super-fast and puts minimum strain on server resources (our measurements have shown that Sonic - when under load - responds to search queries in the μs range, eats ~30MB RAM and has a low CPU footprint; see our benchmarks). Tested at Rust version: rustc 1.98.0 (88d9e12ae 2026-08-18) 🇫🇷 Crafted in Nantes, France. 📰 The Sonic project was initially announced in a post on my personal journal. « Sonic » is the mascot of the Sonic project. I drew it to look like a psychedelic hipster hedgehog. Who uses it? Crisp Scrumpy 👋 You use Sonic and you want to be listed there? Contact me. Demo Sonic is integrated in all Crisp search products on the Crisp platform. It is used to index half a billion objects on a $5/mth 1-vCPU SSD cloud server (as of 2019). Crisp users use it to search in their messages, conversations, contacts, helpdesk articles and more. You can test Sonic live on: Crisp Helpdesk, and get an idea of the speed and relevance of Sonic search results. You can also test search suggestions from there: start typing at least 2 characters for a word, and get suggested a full word (press the tab key to expand suggestion). Both search and suggestions are powered by Sonic. Sonic fuzzy search in helpdesk articles at its best. Lookup for any word or group of terms, get results instantly. Features Search terms are stored in collections, organized in buckets; you may use a single bucket, or a bucket per user on your platform if you need to search in separate indexes. Search results return object identifiers, that can be resolved from an external database if you need to enrich the search results. This makes Sonic a simple word index, that points to identifier results. Sonic doesn't store any direct textual data in its index, but it still holds a word graph for auto-completion and typo corrections. Search query typos are corrected if there are not enough exact-match results for a given word in a search query, Sonic tries to correct the word and tries against alternate words. You're allowed to make mistakes when searching. Insert and remove items in the index; index-altering operations are light and can be committed to the server while it is running. A background tasker handles the job of consolidating the index so that the entries you have pushed or popped are quickly made available for search. Auto-complete any word in real-time via the suggest operation. This helps build a snappy word suggestion feature in your end-user search interface. Full Unicode compatibility on 80+ most spoken languages in the world. Sonic removes useless stop words from any text (eg. 'the' in English), after guessing the text language. This ensures any searched or ingested text is clean before it hits the index; see languages. Simple protocol (Sonic Channel), that let you search your index, manage data ingestion (push in the index, pop from the index, flush a collection, flush a bucket, etc.) and perform administrative actions. Sonic Channel was designed to be lightweight on resources and simple to integrate with; read protocol specification. Easy-to-use libraries, that let you connect to Sonic from your apps; see libraries. How to use it? Installation Sonic is built in Rust. To install it, either download a version from the Sonic releases page, use cargo install or pull the source code from master. 👉 Install from packages: Sonic provides pre-built packages for Debian-based systems (Debian, Ubuntu, etc.). Important: Sonic only provides 64 bits packages targeting Debian 12 for now (codename: bookworm). You might still be able to use them on other Debian versions, as well as Ubuntu (although they rely on a specific glibc version that might not be available on older or newer systems). First, add the Sonic APT repository (eg. for Debian bookworm): echo "deb [signed-by=/usr/share/keyrings/valeriansaliou_sonic.gpg] https://packagecloud.io/valeriansaliou/sonic/debian/ bookworm main" > /etc/apt/sources.list.d/valeriansaliou_sonic.list curl -fsSL https://packagecloud.io/valeriansaliou/sonic/gpgkey | gpg --dearmor -o /usr/share/keyrings/valeriansaliou_sonic.gpg apt-get update Then, install the Sonic package: apt-get install sonic Then, edit the pre-filled Sonic configuration file: nano /etc/sonic.cfg Finally, restart Sonic: service sonic restart 👉 Install from source: If you pulled the source code from Git, you can build it using cargo: cargo build --locked --release You can find the built binaries in the ./target/release directory. Install build-essential, clang, libclang-dev, libc6-dev, g++ and llvm-dev to be able to compile the required RocksDB dependency. Note that the following optional features can be enabled upon building Sonic: