HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
pola-rs
polars
Rust2020–2026
pola-rs / polars

polars

Extremely fast Query Engine for DataFrames, written in Rust

40Distinct people
52Mentions
0Explicit recommenders
40.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Extremely fast Query Engine for DataFrames, written in Rust

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 11:20:21 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year52 dated mentions

Observed mentions: 2020-07-08 09:12:40 UTC to 2026-10-06 20:05:15 UTC.

0510152020-01-01 00:00:00 UTC to 2021-01-01 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments20202021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 6 mentions; 1 submissions, 5 comments2022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 9 mentions; 1 submissions, 8 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 10 mentions; 3 submissions, 7 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 14 mentions; 2 submissions, 12 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 10 mentions; 3 submissions, 7 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20202020-01-01 00:00:00 UTC2021-01-01 00:00:00 UTC110
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC615
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC918
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC1037
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC14212
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC202
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC1037

What Hacker News says

10 sources

Show HN: Parquity – find and reproduce Parquet interoperability failures

I built a Python CLI for Parquet interoperability. It runs a given Parquet file through selected readers and shows where it fails with a command. Also uses Hypothesis for fuzzing. Generates tables, runs through a matrix of engines and minimizes the found failures. You can also create a schema beforehand and fuzz within it. I believe it can be useful for Parquet engine maintainers and people with multiple Parquet engines involved in their pipeline. Already found and filed two issues with it, one from scanning Apache's own corpus and the other from fuzzing: - https://github.com/pola-rs/polars/issues/28787 - https://github.com/duckdb/duckdb/issues/24731 repo: https://github.com/sovsparrow/parquity
3 points · Open discussion
Here are a few examples you might find credible: - pola-rs/polars: https://github.com/pola-rs/polars/pull/26823 - ~2.68x median speedup of primitive-to-boolean casting credited to Claude Opus 4.6 - pydantic/monty: https://github.com/pydantic/monty/pull/643 - ~53x speedup (488ms down to 9.2ms) of bytes substring search generated with Claude Code - numpy/numpy: https://github.com/numpy/numpy/pull/31573 - ~21x speedup of Python datetime → datetime64 conversion, with Claude Code used for profiling and implementing the performance improvements - apache/datafusion: https://github.com/apache/datafusion/pull/21182 - up to 49x speedup of LIMIT queries by eliminating unnecessary sorts and pushing limits into file scans, generated with Claude Code - apache/datafusion: https://github.com/apache/datafusion/pull/21651 - ~4.39x speedup of ClickBench Q6
Read the rest
by resolving MIN/MAX directly from Parquet metadata instead of scanning columns, generated with Claude Code - pola-rs/polars: https://github.com/pola-rs/polars/pull/27958 - ~3.2x speedup of Int8 Series sum, developed with assistance from Claude Fable and Claude Opus 4.8 - numpy/numpy: https://github.com/numpy/numpy/pull/31274 - up to ~1.44x speedup of common small NumPy reductions, with the fast path written by Claude Code and manually refined
In Is it all just vapourware?
Would there be an advantage to reusing SIMD-optimized libraries for this? Polars Rust is built on Arrow and packed_simd. pola-rs/polars: https://github.com/pola-rs/polars polars - Rust API docs: https://docs.pola.rs/api/rust/dev/polars/#simd lancedb's data format; Lancedb/lance works with [Pandas, DuckDB, Polars, PyArrow,]; https://github.com/lancedb/lance Narwhals' df interface (Python) https://narwhals-dev.github.io/narwhals/ substrait's portable query plans: https://substrait.io/ , https://github.com/ibis-project/ibis-substrait Arrow RecordBatch, https://news.ycombinator.com/item?id=45495738#45546244 cargo-fuzz, TLA+
In Pgrust v0.2: Now faster than Postgres and Clickhouse Latest
`row_index()` was also recently added. df.with_columns(pl.col.b + pl.row_index().is_between(1, 3)) # shape: (5, 2) # ┌─────┬─────┐ # │ a ┆ b │ # │ --- ┆ --- │ # │ i64 ┆ i64 │ # ╞═════╪═════╡ # │ 1 ┆ 10 │ # │ 2 ┆ 21 │ # │ 3 ┆ 31 │ # │ 4 ┆ 41 │ # │ 5 ┆ 50 │ # └─────┴─────┘ > Polars has an optimization to overwite a single value I believe it is just "syntax sugar" for calling `Series.scatter()`[1] > it doesn't allow slicing I believe you are correct: df_polars[1:3, "b"] += 1 # TypeError: cannot use "slice(1, 3, None)" for indexing You can do: df_polars[list(range(1, 4)), "b"] += 1 Perhaps nobody has requested slice syntax? It seems like it would be easy to add. [1]: https://github.com/pola-rs/polars/blob/9079e20ae59f8c75dcce8...
In Pandas 3.0

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.