HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
vortex-data
vortex
Rust2024–2026
vortex-data / vortex

vortex

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Systems & InfrastructureData & Databases
Explore on GitHubRead Hacker News discussions
18Distinct people
19Mentions
0Explicit recommenders
3.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 11:56:56 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month19 dated mentions

Observed mentions: 2024-05-02 09:23:14 UTC to 2026-07-30 21:50:52 UTC.

012342024-05-01 00:00:00 UTC to 2024-06-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsMay 20242024-06-01 00:00:00 UTC to 2024-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-07-01 00:00:00 UTC to 2024-08-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-01 00:00:00 UTC to 2024-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-09-01 00:00:00 UTC to 2024-10-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-10-01 00:00:00 UTC to 2024-11-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2024-11-01 00:00:00 UTC to 2024-12-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-12-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-01-01 00:00:00 UTC to 2025-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-02-01 00:00:00 UTC to 2025-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-03-01 00:00:00 UTC to 2025-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-04-01 00:00:00 UTC to 2025-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-05-01 00:00:00 UTC to 2025-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-06-01 00:00:00 UTC to 2025-07-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsJun 20252025-07-01 00:00:00 UTC to 2025-08-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2025-08-01 00:00:00 UTC to 2025-09-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-09-01 00:00:00 UTC to 2025-10-01 00:00:00 UTC (exclusive): 4 mentions; 1 submissions, 3 comments2025-10-01 00:00:00 UTC to 2025-11-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2025-11-01 00:00:00 UTC to 2025-12-01 00:00:00 UTC (exclusive): 4 mentions; 2 submissions, 2 comments2025-12-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-01-01 00:00:00 UTC to 2026-02-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-02-01 00:00:00 UTC to 2026-03-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-03-01 00:00:00 UTC to 2026-04-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-04-01 00:00:00 UTC to 2026-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-05-01 00:00:00 UTC to 2026-06-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-06-01 00:00:00 UTC to 2026-07-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-07-01 00:00:00 UTC to 2026-08-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 commentsJul 2026

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
May 20242024-05-01 00:00:00 UTC2024-06-01 00:00:00 UTC101
Jun 20242024-06-01 00:00:00 UTC2024-07-01 00:00:00 UTC000
Jul 20242024-07-01 00:00:00 UTC2024-08-01 00:00:00 UTC000
Aug 20242024-08-01 00:00:00 UTC2024-09-01 00:00:00 UTC000
Sep 20242024-09-01 00:00:00 UTC2024-10-01 00:00:00 UTC101
Oct 20242024-10-01 00:00:00 UTC2024-11-01 00:00:00 UTC211
Nov 20242024-11-01 00:00:00 UTC2024-12-01 00:00:00 UTC000
Dec 20242024-12-01 00:00:00 UTC2025-01-01 00:00:00 UTC000
Jan 20252025-01-01 00:00:00 UTC2025-02-01 00:00:00 UTC000
Feb 20252025-02-01 00:00:00 UTC2025-03-01 00:00:00 UTC000
Mar 20252025-03-01 00:00:00 UTC2025-04-01 00:00:00 UTC000
Apr 20252025-04-01 00:00:00 UTC2025-05-01 00:00:00 UTC000
May 20252025-05-01 00:00:00 UTC2025-06-01 00:00:00 UTC000
Jun 20252025-06-01 00:00:00 UTC2025-07-01 00:00:00 UTC000
Jul 20252025-07-01 00:00:00 UTC2025-08-01 00:00:00 UTC202
Aug 20252025-08-01 00:00:00 UTC2025-09-01 00:00:00 UTC000
Sep 20252025-09-01 00:00:00 UTC2025-10-01 00:00:00 UTC413
Oct 20252025-10-01 00:00:00 UTC2025-11-01 00:00:00 UTC000
Nov 20252025-11-01 00:00:00 UTC2025-12-01 00:00:00 UTC422
Dec 20252025-12-01 00:00:00 UTC2026-01-01 00:00:00 UTC110
Jan 20262026-01-01 00:00:00 UTC2026-02-01 00:00:00 UTC000
Feb 20262026-02-01 00:00:00 UTC2026-03-01 00:00:00 UTC000
Mar 20262026-03-01 00:00:00 UTC2026-04-01 00:00:00 UTC000
Apr 20262026-04-01 00:00:00 UTC2026-05-01 00:00:00 UTC000
May 20262026-05-01 00:00:00 UTC2026-06-01 00:00:00 UTC000
Jun 20262026-06-01 00:00:00 UTC2026-07-01 00:00:00 UTC202
Jul 20262026-07-01 00:00:00 UTC2026-08-01 00:00:00 UTC211

What Hacker News says

19 sources
westurner·Jul 30, 2026
TigerBeetle has static allocation for their known workload. SIMD is more useful for heavy parallel analytical workloads. There are probably returns from SIMD even for index updates on transactional inserts. SIMD is faster for: vectorized index comparisons, Vectorized Constraint Validation, Multi-Column SIMD Hashing, Masked Bitmaps for Nullable Fields, gather and scatter, string processing, casting to numeric types. IIUC FWIW Cerebras' does not have L2 cache. TIL Vortex is Zero-Copy compatible with Arrow; > vortex-data/vortex: An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation. https://github.com/vortex-data/vortex Vortex also has Segment Profiling and Adaptive Encoding. "What is Vo
Read the rest
rtex? Columnar File Format Explained" https://spice.ai/learn/vortex
In Pgrust v0.2: Now faster than Postgres and Clickhouse Latest
lukekim·Jul 9, 2026Submission

Show HN: Spice 2.0 – Real-Time Analytical Query on Operational Data, Without ETL

Hi HN, we’re Luke and Phillip, and we’re building Spice.ai OSS - a lightweight, portable data and AI engine and powered by Apache DataFusion & Ballista for SQL query, hybrid-search, and LLM-inference across disaggregated-storage used by enterprises like Barracuda Networks and Twilio. With v2.0, you can add a real-time analytics node to your operational database with high-throughput native CDC replication. We first introduced Spice [1] on HN in 2021 and re-launched it on HN [2] in 2024 re-built from the ground up in Rust. In 2025 we launched Spice Cayenne [5], our acceleration and query engine built on the LF+AI Vortex columnar format. Spice includes the concept of a Data Accelerator [3], which is a way to materialize data from disparate sources, such as other databases, in embedded databases like SQLite, DuckDB, and Spice Cayenne. Today w
Read the rest
e're excited to announce Spice 2.0! Our answer to analytics in the era of AI agents: analytical query on operational data - real-time, sandboxed, without ETL. Bolt an analytics node onto your operational database, including Postgres, MySQL, MongoDB, DynamoDB and run sub-second analytical queries on data that's single-digit seconds fresh, with zero query load on production. Agents can't act on yesterday's data. That belief has shaped Spice since we founded it in 2021 and it's why we rebuilt replication from the ground up for 2.0. High-throughput CDC replication, direct from the WAL, binlog, and oplog. No pipelines, no Debezium, no external streaming layer. Native replication bootstraps a 300M-row table in ~9 minutes — ~170x faster ingest than our 1.x Debezium-based path. Start with one table and be querying in minutes. Spice Cayenne, our Vortex-based acceleration engine, is GA: 1.5x faster than DuckDB acceleration with 3x less memory, and 26x faster than Spice 1.x on TPC-DS. On CH-BenCHmark (TPC-C transactions + TPC-H analytics, concurrently), PostgreSQL sustained 266,861 new orders per minute while Spice served the entire analytical workload and production never ran a single analytical query. Petabyte-scale, multi-node distributed query built on Apache Ballista is GA: multi-active schedulers, no single point of failure, and 2.9x faster query from one node to three. No JVM, no ZooKeeper. Just Rust and object storage. Enterprise-ready: a Kubernetes Operator with blue/green deployments and instant rollback, Cedar policy enforced inside the query engine, OIDC, mTLS, and PII masking — all before data ever reaches an agent. You can read about it at https://spice.ai/blog/spice-2-0-is-now-available and in the Spice OSS release notes [4]. This is just the first version, and we’d love to get your feedback! GitHub: https://github.com/spiceai/spiceai [1] https://news.ycombinator.com/item?id=28448887 [2] https://news.ycombinator.com/item?id=39854584 [3] https://github.com/vortex-data/vortex [4] https://spiceai.org/releases/v2.0-stable [5] https://news.ycombinator.com/item?id=46316991
4 points · Open discussion
gnurizen·Jun 15, 2026
Thanks! I think most performance debugging happens during development, what we're bringing to the table is exposure of system behavior in production which often diverges because of changes in the shape of workloads from dev, which are often simplistic and synthetic. So I'd say its late-stage performance debugging and production observability combined that makes this useful. Stay tuned for a follow on post where we show how we used this to optimize an FSST decompression kernel for vortex (https://github.com/vortex-data/vortex).
In Show HN: Continuous Nvidia CUDA PC Sampling Profiler
lukekim·Dec 18, 2025Submission

Show HN: Spice Cayenne – SQL acceleration built on Vortex

Hi HN, we’re Luke and Phillip, and we’re building Spice.ai OSS - a lightweight, portable data and AI engine and powered by Apache DataFusion & Ballista for SQL query, hybrid-search, and LLM-inference across disaggregated-storage used by enterprises like Barracuda Networks and Twilio. We first introduced Spice [1] on HN in 2021 and re-launched it on HN [2] in 2024 re-built from the ground up in Rust. Spice includes the concept of a Data Accelerator [3], which is a way to materialize data from disparate sources, such as other databases, in embedded databases like SQLite and DuckDB. Today we’re excited to announce a new Ducklake-inspired Data Accelerator built on Vortex [3], a highly performant, extensible columnar data format that claims 100x faster random access, 10-20x faster scans, 5x faster writes with a similar compression ratio vs. Ap
Read the rest
ache Parquet. In our tests with Spice, Vortex performs faster than DuckDB with a third of the memory usage, and is much more scalable (multi-file). For real-world deployments, we see the DuckDB Data Accelerator often capping out around 1TB, but Spice Cayenne can do Petabyte-scale. You can read about it at https://spice.ai/blog and in the Spice OSS release notes [4]. This is just the first version, and we’d love to get your feedback! GitHub: https://github.com/spiceai/spiceai [1] https://news.ycombinator.com/item?id=28448887 [2] https://news.ycombinator.com/item?id=39854584 [3] https://github.com/vortex-data/vortex [4] https://spiceai.org/blog/releases/v1.9.0
42 points · Open discussion
pauldix·Sep 11, 2025
I've been following this team's work for a while and what they're doing is super interesting. The file format they created and put into the LF, Vortex, is very welcome innovation in the space: https://github.com/vortex-data/vortex I'm excited to start doing some experimentation with Vortex to see how it can improve our products. Great stuff, congrats to Will and team!
In Spiral

Showing the 15 most recent of 19 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.