HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
elixir-explorer
explorer
Elixir2021–2026
elixir-explorer / explorer

explorer

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir

Science & EngineeringData & Databases
Explore on GitHubRead Hacker News discussions
8Distinct people
14Mentions
0Explicit recommenders
1.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 12:54:37 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year14 dated mentions

Observed mentions: 2021-08-23 13:32:56 UTC to 2026-03-31 22:40:50 UTC.

01232021-01-01 00:00:00 UTC to 2022-01-01 00:00:00 UTC (exclusive): 3 mentions; 2 submissions, 1 comments20212022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments20232024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20212021-01-01 00:00:00 UTC2022-01-01 00:00:00 UTC321
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC211
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC303
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC303
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC101
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC211

What Hacker News says

14 sources
cigrainger·Mar 31, 2026Submission

Show HN: Dux, distributed DuckDB-backed dataframes on the Beam

Hey all! I wrote Explorer[1] a good few years ago now with the dream of fast dataframes with a dplyr-like API in a really powerful, ergonomic language (Elixir). It's proved pretty successful. Explorer is used in production at my company, and it's my go-to for quick data analysis. But maintaining it became a true albatross. Polars is an amazing project, but the development process is fast and a lot is very focused on the Python lib. We found that trying to maintain Explorer against Polars was a maintenance nightmare and eventually hit points where we had to give up features and found it extremely difficult to update to the latest. We also tried distributing Explorer and only got so far. A reasonable alternative to Spark was always what I wanted, and I could (tantalisingly, frustratingly) see the pieces there in dataframes and the BEAM, but
Read the rest
couldn't make it happen. We also always knew that the right direction was to be 'lazy by default', accumulating ops and only executing when the dataframe needs to be realised. But this was very difficult with Polars's Series API and eager/lazy split. Enter DuckDB. A few weeks ago, I made a duckdb backend for Explorer. But in doing so I saw that DuckDB would allow us to realise the lazy-by-default and distributed vision. So I went for it. And here we are. Dux as in ducks as in multiple ducks. Plus an 'x' in the name because, you know, it's Elixir. It's faster than Explorer on a single node. It has a simple, dataframe only API. It distributes arbitrarily on Erlang clusters on the BEAM. Startup is faster than Spark and for many use cases it's simpler and faster. DuckDB functions are all transparently available, as are custom SQL macros. We have a full graph API, as in GraphX/NetworkX. You can install and use any duckdb extensions, including in distribution. And on the maintenance side, it doesn't use a NIF (it depends on the ADBC library[2] and a DuckDB driver) -- the API is primarily about compiling to SQL. DuckDB is incredible for OLAP on out of memory data. Distribution enables fast exploration of in-memory data and real-time applications. The BEAM gives us battle-hardened distribution almost for free. Give it a shot! I'd love feedback and of course PRs are welcome. Oh, I also made a webpage for it[3]. [1] https://github.com/elixir-explorer/explorer [2] https://github.com/livebook-dev/adbc [3] https://dux.now
7 points · Open discussion
thibaut_barrere·Aug 27, 2024
Since you wrote "using something other than Python" and not necessarily only Ruby, definitely look into Livebook and Elixir, and the whole ecosystem around it, including: - https://github.com/elixir-nx/axon Nx-powered Neural Networks - https://github.com/elixir-nx/nx Multi-dimensional arrays (tensors) and numerical definitions for Elixir - https://github.com/elixir-nx/scholar Traditional machine learning on top of Nx - https://github.com/elixir-nx/bumblebee Pre-trained Neural Network models in Axon (+ Models integration) - https://github.com/elixir-explorer/explorer Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir - https://fly.io/blog/rethinking-serverless-with-flame/ (for offloading large work to remote containers) - https://www.youtube.com/watch?v=RABXu7zqnT0 InstructorEx And of
Read the rest
course Livebook (https://livebook.dev) Old but interesting video https://www.youtube.com/watch?v=g3oyh3g1AtQ (Bumblebee: GPT2, Stable Diffusion, and more in Elixir) A talk on the upcoming Elixir conference (https://2024.elixirconf.com/schedule/#schedules) is actually titled "Livebook in the cloud: GPUs and clustered workflows in seconds".
In Ruby in Jupyter Notebook
thibaut_barrere·Aug 7, 2023
To ensure you do not miss this: LiveBook comes with a Vega Lite integration (https://livebook.dev/integrations -> https://livebook.dev/integrations/vega-lite/), which means you get access to a lot of visualisations out of the box, should you need that (https://vega.github.io/vega-lite/). In the same "standing on giant's shoulders" stance, you can use Explorer (see example LiveBook at https://github.com/elixir-explorer/explorer/blob/main/notebo...), which leverages Polars (https://www.pola.rs), a very fast DataFrame library and now a company (https://www.pola.rs/posts/company-announcement/) with 4M$ seed.
In Elixir Livebook is a secret weapon for documentation
josevalim·Apr 14, 2023
José from the Livebook team. I don't think I can make a pitch because I have limited Python/R experience to use as reference. My suggestion is for you to give it a try for a day or two and see what you think. I am pretty sure you will find weak spots and I would be very happy to hear any feedback you may have. You can find my email on my GitHub profile (same username). In general we have grown a lot since the Numerical Elixir effort started two years ago. Here are the main building blocks: * Nx (https://github.com/elixir-nx/nx/tree/main/nx#readme): equivalent to Numpy, deeply inspired by JAX. Runs on both CPU and GPU via Google XLA (also used by JAX/Tensorflow) and supports tensor serving out of the box * Axon (https://github.com/elixir-nx/axon): Nx-powered neural networks * Bumblebee (https://github.com/elixir-nx/bumblebee): Equivalent t
Read the rest
o HuggingFace Transformers. We have implemented several models and that's what powers the Machine Learning integration in Livebook (see the announcement for more info: https://news.livebook.dev/announcing-bumblebee-gpt2-stable-d...) * Explorer (https://github.com/elixir-nx/explorer): Series and DataFrames, as per this thread. * Scholar (https://github.com/elixir-nx/scholar): Nx-based traditional Machine Learning. This one is the most recent effort of them all. We are treading the same path as scikit-learn but quite early on. However, because we are built on Nx, everything is derivable, GPU-ready, distributable, etc. Regarding visualization, we have "smart cells" for VegaLite and MapLibre, similar to how we did "Data Transformations" in the video above. They help you get started with your visualizations and you can jump deep into the code if necessary. I hope this helps!
In Data wrangling in Elixir with Explorer, the power of Rust, the elegance of R
thejosh·Aug 2, 2022
I've wondered whether it's easier to add data analyst stuff to Elixir that Python seems to have, or add features to Python that Erlang (and by extension Elixir) provides out of the box. By what I can see, if you want multiprocessing on Python in an easier way (let's say running async), you have to use something like ray core[0], then if you want multiple machines you need redis(?). Elixir/Erlang supports this out of the box. Explorer[1] is an interesting approach, where it uses Rust via Rustler (Elixir library to call Rust code) and uses Polars as its dataframe library. I think Rustler needs to be reworked for this usecase, as it can be slow to return data. I made initial improvements which drastically improves encoding (https://github.com/elixir-nx/explorer/pull/282 and https://github.com/elixir-nx/explorer/pull/286, tldr 20+ seconds dow
Read the rest
n to 3). [0] https://github.com/ray-project/ray [1] https://github.com/elixir-nx/explorer
In Elixir Livebook now as a desktop app

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.