arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

Data & Databases

Explore on GitHub Read Hacker News discussions

29Distinct people

33Mentions

0Explicit recommenders

17.3KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 7:02:26 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year33 dated mentions

Observed mentions: 2016-02-17 16:17:28 UTC to 2026-02-17 15:43:16 UTC.

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2016	2016-01-01 00:00:00 UTC	2017-01-01 00:00:00 UTC	3	0	3
2017	2017-01-01 00:00:00 UTC	2018-01-01 00:00:00 UTC	2	0	2
2018	2018-01-01 00:00:00 UTC	2019-01-01 00:00:00 UTC	2	0	2
2019	2019-01-01 00:00:00 UTC	2020-01-01 00:00:00 UTC	3	0	3
2020	2020-01-01 00:00:00 UTC	2021-01-01 00:00:00 UTC	3	0	3
2021	2021-01-01 00:00:00 UTC	2022-01-01 00:00:00 UTC	7	1	6
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	2	0	2
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	5	0	5
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	2	0	2
2025	2025-01-01 00:00:00 UTC	2026-01-01 00:00:00 UTC	3	1	2
2026 (partial)	2026-01-01 00:00:00 UTC	2027-01-01 00:00:00 UTC	1	0	1

From the project README

An excerpt from the repository, fetched Nov 13, 2026.

Read the project’s introduction

Apache Arrow Powering In-Memory Analytics Apache Arrow is a universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics. It contains a set of technologies that enable data systems to efficiently store, process, and move data. Major components of the project include: The Arrow Columnar Format: a standard and efficient in-memory representation of various datatypes, plain or nested The Arrow IPC Format: an efficient serialization of the Arrow format and associated metadata, for communication between processes and heterogeneous environments ADBC (Arrow Database Connectivity) ↗: Arrow-powered API, drivers, and libraries for access to databases and query engines The Arrow Flight RPC protocol: based on the Arrow IPC format, a building block for remote services exchanging Arrow data with application-defined semantics (for example a storage server or a database) C++ libraries C bindings using GLib .NET libraries ↗ Gandiva: an LLVM-based Arrow expression compiler, part of the C++ codebase Go libraries ↗ Java libraries ↗ JavaScript libraries ↗ Julia implementation ↗ Python libraries R libraries Ruby libraries Rust libraries ↗ Swift libraries ↗ The ↗ icon denotes that this component of the project is maintained in a separate repository. Arrow is an Apache Software Foundation project. Learn more at arrow.apache.org. What's in the Arrow libraries? The reference Arrow libraries contain many distinct software components: Columnar vector and table-like containers (similar to data frames) supporting flat or nested types Fast, language agnostic metadata messaging layer (using Google's FlatBuffers library) Reference-counted off-heap buffer memory management, for zero-copy memory sharing and handling memory-mapped files IO interfaces to local and remote filesystems Self-describing binary wire formats (streaming and batch/file-like) for remote procedure calls (RPC) and interprocess communication (IPC) Integration tests for verifying binary compatibility between the implementations (e.g. sending data from Java to C++) Conversions to and from other in-memory data structures Readers and writers for various widely-used file formats (such as Parquet, CSV) Implementation status The official Arrow libraries in this repository are in different stages of implementing the Arrow format and related features. See our current feature matrix on git main. How to Contribute Please read our latest project contribution guide. If you are using AI coding tools, please review our AI-generated code guidance. Getting involved Even if you do not plan to contribute to Apache Arrow itself or Arrow integrations in other projects, we'd be happy to have you involved: Join the mailing list: send an email to dev-subscribe@arrow.apache.org. Share your ideas and use cases for the project. Follow our activity on GitHub issues Learn the format Contribute code to one of the reference implementations Continuous Integration Sponsors We use runs-on for managing the project self-hosted runners. We use AWS for some of the required infrastructure for the project.

Read the full README on GitHub

What Hacker News says

2 sources

lidavidm·Feb 16, 2022

There are several community members actively working on making things more accessible in general. (Flight SQL definitely needs some polish here.) For Arrow itself, you could start with the Cookbook [1] (and feel free to file issues if there are things that are unclear, or that you would like to learn more about [2]) The example there is very long, but much of it is repetitive (that said, it could certainly be improved). But to your specific question, what you would eventually do is write a plugin or proxy for Postgres (or another database) implementing these RPC methods. Then you could submit SQL queries using the Flight SQL client (which calls those RPC methods internally) to get back columnar data. You could wrap this all in a JDBC driver as well (which is being actively worked on [3]), in which case you'd be mostly ignorant of what's g

Read the rest

oing on underneath; the benefit there would (hopefully) be faster data access due to the lower layers being columnar (and zero-copy where possible), and enabling parallelism due to Flight (Postgres might not support this for instance, but other databases could, if they could expose internal parallelism or partitioning). Hope that helps. [1]: https://arrow.apache.org/cookbook/ [2]: https://github.com/apache/arrow-cookbook/ [3]: https://github.com/apache/arrow/pull/12254

In Apache Arrow Flight SQL: Accelerating Database Access

cogman10·Feb 16, 2022

I'm not tuned into Arrow all that much. I've read some of the about and stuff, but the code examples (to my eye) look really complex and complicated. [1] Could someone point me to a more glossy "arrow flight sql for dummies" examples? What I'm gleaning from this (or am I wrong?) is you could use a JDBC driver + arrow jdbc client and write... SQL? Or is it something a lot different? Is this the sort of thing where you could just add a plugin to postgres and be arrowified or something? [1] https://github.com/apache/arrow/blob/release-7.0.0/java/flig...

In Apache Arrow Flight SQL: Accelerating Database Access