HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
quickwit-oss
tantivy
Rust2019–2026
quickwit-oss / tantivy

tantivy

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust

36Distinct people
54Mentions
0Explicit recommenders
16.4KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:56:33 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year54 dated mentions

Observed mentions: 2019-01-15 01:52:33 UTC to 2026-07-13 07:44:08 UTC.

0369122019-01-01 00:00:00 UTC to 2020-01-01 00:00:00 UTC (exclusive): 8 mentions; 0 submissions, 8 comments20192020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 7 mentions; 1 submissions, 6 comments2021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 8 mentions; 1 submissions, 7 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 10 mentions; 1 submissions, 9 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 4 mentions; 1 submissions, 3 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 12 mentions; 2 submissions, 10 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20192019-01-01 00:00:00 UTC2020-01-01 00:00:00 UTC808
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC716
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC817
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC1019
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC413
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC12210
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC303
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC202

What Hacker News says

4 sources
>Roaring bitmaps and similar data structures get their speed from decoding together consecutive groups of elements, so if you do sequential decoding or decode a large fraction of the list you get excellent performance. EF instead excels at random skipping, so if you visit a small fraction of the list you generally get better performance. This is why it works so well for inverted indexes, as generally the queries are very selective (otherwise why do you need an index?) and if you have good intersection algorithms you can skip a large fraction of documents. There's no sequential decoding in our variant, every access is independent. The roaring bitmap variant is used only for the optional index (1 docid => 0 or 1 value) in the columnar storage (DocValues), not for the inverted index. Since this is used for aggregation, some queries may be a
Read the rest
full scan. The inverted index in tantivy uses bitpacked values of 128 elements with a skip index on top. > I didn't follow the rest of your comment, select is what EF is good at, every other data structure needs a lot more scanning once you land on the right chunk. With BMI2 you can also use the PDEP instruction to accelerate the final select on a 64-bit block The select for the sparse codec is a simple array index access, that is hard to beat (https://github.com/quickwit-oss/tantivy/blob/main/columnar/s...). Compression is not good near the 5k threshold though. PDEP is currently deactivated since ryzen before Zen3 was really slow. Creation speed is also quite important, do you know how "Partitioned Elias-Fano" performs there?
In A compressed indexable bitset

Show HN: Quickwit 0.6 – OSS Elasticsearch alternative on object storage

Hey HN, In 2021, we started building our OSS search engine[0] with the objective to search directly on object storage and substantially lower the infrastructure costs of search at a large scale. Two years after our common crawl demo hit the front page[1], our latest release, Quickwit 0.6, is finally reaching the point where our engine can be a drop-in replacement of Elasticsearch for logs and traces with all your data on object storage. Two years is both very long for a startup and very short when you build a distributed engine. And we decided to do it the hard way: we implemented our own OSS gossip library[2], our own {S3,JSON}-friendly columnar format[3], and of course, we maintain our own search library, tantivy[4], a Lucene equivalent in Rust. From time to time, we also do fun things like language detection[5], always OSS. This is a l
Read the rest
ot of engineering investment and it takes some time to finally reach the end users. I won’t tell you Quickwit is XX times cheaper or faster as we don’t have yet published an unbiased benchmark - I’m working on it and it’s really hard to avoid the common pitfalls. Instead, I will just give two links to show the performance is really good on search and indexing and comments from 2 users we discovered on Twitter/HN: - Search and do analytics on the whole GitHub archive dataset (5.6 billion events ~ 21TB JSON) in less than 2 seconds with a few nodes. See the short demo [6]. - You can achieve an indexing throughput of 1.5GB/s with 40 nodes and it scales linearly [7]. - Nice comment from Arnon Rotem-Gal-Oz, VP, Chief Architect at SAP: « I've just spent the day setting up v.05 on k8s with SSL to Kafka source, VRL transformations, and persistence to Azure -I can see it is shaping up as an alternative to our ES-based log handling. » [8] - Another nice comment seen on HN « it seems to be very easy to run, not very IO intensive, and running fine on a single node with modest hardware with >2 billion log rows. It has a really cool dynamic schema feature too.» [9] Fun fact: at least 4 users are using Garage[10] as the object storage, this OSS project looks really promising and made the HN front page a few months ago[11], we really cherish the OSS for this kind of unexpected combination. Any feedback positive/negative always greatly appreciated here! [0] Quickwit repo: https://github.com/quickwit-oss/quickwit [1] Searching the web under 1000$/month: https://news.ycombinator.com/item?id=27074481 [2] Chitchat gossip library: https://github.com/quickwit-oss/chitchat [3] Columnar format: https://github.com/quickwit-oss/tantivy/tree/main/columnar [4] Tantivy library: https://github.com/quickwit-oss/tantivy/ [5] Whichlang library: https://github.com/quickwit-oss/whichlang [6] GitHub Archive demo in terminal: https://www.youtube.com/watch?v=SNq3bARRlDI [7] Indexing performance: https://twitter.com/fulmicoton/status/1638016949459488768 [8] https://twitter.com/arnonrgo/status/1645429632303235073?s=20 [9] https://news.ycombinator.com/item?id=35742544 [10] Garage object storage: https://garagehq.deuxfleurs.fr/ [11] https://news.ycombinator.com/item?id=33853539
4 points · Open discussion
From my little knowledge of Loki's internals. I think contrary to Loki, Quickwit uses a fully feature search engine library underneath called Tantivy (https://github.com/quickwit-oss/tantivy). Quickwit offers different services (indexer, searcher, ..) that can be ran and scaled independently. It also supports indexing from various sources including file, Rest API, Kafka, Pulsar, Kinesis and more are planned based on community interest. Last but not least, Elasticsearch query API support is being worked on.
In I can't recommend serious use of an all-in-one local Grafana Loki setup

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.