KAKalanos·Mar 31, 2022
In Dagger: a new way to build CI/CD pipelinesJuliaParallel
Dagger.jl
Julia2015–2022
JuliaParallel / Dagger.jl
Dagger.jl
A framework for out-of-core and parallel execution
6Distinct people
6Mentions
0Explicit recommenders
734GitHub stars
What to know before exploring
What it is
A framework for out-of-core and parallel execution
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 12:54:37 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year6 dated mentions
Observed mentions: 2015-12-29 17:11:37 UTC to 2022-03-31 15:09:39 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
6 sourcesFirstly, I'll say that we already have work started to implement out-of-core directly in Dagger: https://github.com/JuliaParallel/Dagger.jl/pull/289.
With that PR in place, it should be possible to define a "storage device" which is backed by a database. I haven't had a chance to actually try this, since the PR still needs quite some work and testing, but it's definitely something on my radar!
In DTable a new distributed table implementation in Julia using Dagger.jlThe biggest group outside of numerical computing in Julia land are the PL and systems people though? This includes type theorists [1], database folks [2], distributed systems people ([3] to name just one). There are also a fair number of compiler nuts, hence the existence of multiple projects [4][5] in this space. And this is before getting into things that bridge more than one of the domains above, e.g. [7] or [8].
FTR, I think it's fair to question whether numerical computing should have an outsized influence on the direction of the language. I also think it's a pretty fair comparison to point out how standardized and consistent the Rust governance process is compared to Julia's (the Rust RFC system is an exemplar here). That doesn't mean there is a dearth of PL and systems knowledge in the Julia community though.
[1] https://github.com
Read the rest
/AlgebraicJulia/Catlab.jl
[2] https://www.youtube.com/watch?v=tRBl-6uEJJE
[3] https://github.com/JuliaParallel/Dagger.jl/
[4] https://github.com/FluxML/IRTools.jl, https://github.com/FluxML/MacroTools.jl
[5] https://github.com/JuliaCompilerPlugins
[6] https://github.com/0x0f0f0f/Metatheory.jl
[7] https://2020.splashcon.org/details/splash-2020-rebase/13/Non...
[8] https://github.com/JuliaSymbolics/Symbolics.jl
My current thinking is that it would make sense to implement the standard DataFrame abstraction (with all the generic functionality it offers) with a couple kinds of distributed data frame. One approach is to make a distributed stack of local data frames; another is to make a single data frame where each column is a distributed vector. Not sure which is better. Fortunately, Julia makes it really easy to experiment with these kinds of things. Of course, as soon as you have a system like that you need a scheduler for work at which point you want something like Dask or Dagger (https://github.com/JuliaParallel/Dagger.jl).
In JuliaDBVery interesting I didn't realize the difference.
How about this? https://github.com/shashi/ComputeFramework.jl
But I suspect it has the same problem.
Edit: There is also this https://github.com/JuliaDB/DataStreams.jl
In Julia: A Fast Language for Numerical ComputingI think distributed's scheduler has seen much iteration since then and is now based on tornado. I wonder if that changes your assessment.
Also what do you think of Julia's native distributed capabilities and this library here: https://github.com/shashi/ComputeFramework.jl
In Elemental: C++ library for distributed-memory linear algebra and optimization