If you haven't already, I'd seriously suggest having a look at redo (the theory but also apenwarr's implementation at https://github.com/apenwarr/redo).
It's not going in the same direction you are exactly but building a mental model of it in my head was salutary in terms of thinking through the purpose and implementation of build tools and hopefully it'll be similarly useful to you.
In Piper: A proposal for a graphy pipe-based build systemapenwarr
redo
Python2010–2023
apenwarr / redo
redo
Smaller, easier, more powerful, and more reliable than make. An implementation of djb's redo.
33Distinct people
42Mentions
0Explicit recommenders
1.9KGitHub stars
What to know before exploring
What it is
Smaller, easier, more powerful, and more reliable than make. An implementation of djb's redo.
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 3:56:07 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year42 dated mentions
Observed mentions: 2010-12-14 12:59:14 UTC to 2023-04-25 13:22:40 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2010 | 2010-01-01 00:00:00 UTC | 2011-01-01 00:00:00 UTC | 3 | 2 | 1 |
| 2011 | 2011-01-01 00:00:00 UTC | 2012-01-01 00:00:00 UTC | 6 | 0 | 6 |
| 2012 | 2012-01-01 00:00:00 UTC | 2013-01-01 00:00:00 UTC | 2 | 1 | 1 |
| 2013 | 2013-01-01 00:00:00 UTC | 2014-01-01 00:00:00 UTC | 5 | 1 | 4 |
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 7 | 0 | 7 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 6 | 0 | 6 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 4 | 1 | 3 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 3 | 1 | 2 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 2 | 0 | 2 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 1 | 0 | 1 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 0 | 0 | 0 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 1 | 0 | 1 |
From the project README
An excerpt from the repository, fetched Nov 13, 2026.
Read the project’s introduction
redo - a recursive build system
Smaller, easier, more powerful, and more reliable than make.
This is an implementation of Daniel J. Bernstein's redo
build system. He never released his
version, so other people have implemented different variants based on his
published specification.
This version, sometimes called apenwarr/redo, is probably the most advanced
one, including parallel builds, improved logging, extensive automated tests,
and helpful debugging features.
To build and test redo, run:
./do -j10 test
To install it, run something like this:
DESTDIR= PREFIX=/usr/local ./do -j10 install
View the documentation via readthedocs.org
Visit the source code on github
Discussions and support via the
mailing list (archives).
You can subscribe by sending any email message to
redo-list+subscribe@googlegroups.com (note the plus sign). You can
send questions or feedback (with or without subscribing) by sending
messages to redo-list@googlegroups.com.
Read the full README on GitHubWhat Hacker News says
42 sourcesI use something very similar on my website, except with a few twists - MultiMarkdown (instead of Markdown) for HTML, Perl for piping MultiMarkdown produced HTMLs through some regex changes, and redo instead of Make for rebuilds.
If you are looking for Make replacement - I cannot sing enough praises for redo: https://github.com/apenwarr/redo
In My stack is HTML+CSSIf you've tried `redo`, what issues did you find that put it into the "not good" bucket?
[1] https://github.com/apenwarr/redo
In Using Makefile(s) for GoRelevant: https://apenwarr.ca/log/20101214. Code: https://github.com/apenwarr/redo. It's excellent!
In Recursive Make Considered Harmful (1997) [pdf]Probably it's because this python implementation uses st.st_mtime (https://github.com/apenwarr/redo/blob/master/state.py#L314) which is a double and therefore is not precise to enough digits for maximum granularity.
In Mtime comparison considered harmfulRedo: a top-down software build system (2011)
1 points · Open discussionIf you're interested in build systems, you may find djb's redo concept interesting, as implemented in https://github.com/apenwarr/redo/ for example.
In A simple makefile is a unicornThere exists DJB's redo approach [0], which i implemented [1], where dependencies and non-existence dependencies are only recorded after the build. A typical dofile is a shell script, so you do not need to learn another language. Targets also automatically depend on their own build rules (I have seen such a thing only in makefiles authored by DJB).
I wrote a blog post to show how to integrate dependency output for both dependency and non-existence dependency generation [2]. The game “Liberation Circuit” can be built with my redo implementation; you can output a dependency graph usable with Graphviz [4] using “redo-dot”.
There is only one other redo implementation that I would recommend, the one from Jonathan de Boyne Pollard [5], who rightly notices that compilers should output information about non-existence dependencies [6].
I would not
Read the rest
recommend the redo implementation from Avery Pennarun [7], which is often referenced (and introduced me to the concept), mainly because it is not implemented well: It manages to be both larger and slower than my shell script implementation, yet the documentation says this about the sqlite dependency (classic case of premature optimization):
> I don't think we can reach the performance we want with dependency/build/lock information stored in plain text files
[0] http://cr.yp.to/redo.html
[1] http://news.dieweltistgarnichtso.net/bin/redo-sh.html
[2] http://news.dieweltistgarnichtso.net/posts/redo-gcc-automati...
[3] https://github.com/linleyh/liberation-circuit
[4] https://en.wikipedia.org/wiki/Graphviz
[5] http://jdebp.eu./Softwares/redo/
[6] http://jdebp.eu./FGA/introduction-to-redo.html#CompilerDefic...
[7] https://github.com/apenwarr/redo
"Build systems are the bastard stepchild of every software project" -- me a years ago
I've work in embedded software for over a decade, and all projects have used Make.
I have a love-hate relationship with Make. It's powerful and effective at what it does, but its syntax is bad and it lacks good datastructures and some basic functions that are useful when your project reaches several hundred files and multiple outputs. In other words, it does not scale well.
Worth noting that JGC's Gnu Make Standard Library (GMSL) [1] appears to be a solution for some of that, though I haven't applied it to our current project yet.
Everyone ends up adding their own half-broken hacks to work around some of Make's limitations. Most commonly, extracting header file dependency from C files and integrating that into Make's dependency tree.
I've looked at alter
Read the rest
native build systems. For blank-slate candidates, tup [2] seemed like the most interesting for doing native dependency extraction and leveraging Lua for its datastructures and functions (though I initially rejected it due the the silliness of its front page.) djb's redo [3] (implemented by apenwarr [4]) looked like another interesting concept, until you realize that punting on Make's macro syntax to the shell means the tool is only doing half the job: having a good language to specify your targets and dependency is actually most of the problem.
Oh, and while I'm around I'll reiterate my biggest gripe with Make: it has two mechanisms to keep "intermediate" files, .INTERMEDIATE and .PRECIOUS. The first does not take wildcard arguments, the second does but it also keeps any half-generated broken artifact if the build is interrupted, which is a great way to break your build. Please can someone better than me add wildcard support to .INTERMEDIATE.
[1] http://gmsl.sourceforge.net
[2] http://gittup.org/tup/
Also its creator, Mike Shal, now works at Mozilla on their build system
[3] http://cr.yp.to/redo.html
[4] https://github.com/apenwarr/redo
Redo: Smaller, easier, more powerful, and more reliable alternative to make
2 points · Open discussionClosest in spirit to make, and does essentially all you want to do is "redo", concept drafter by djb, implemented independently by apenwarr[0] and by others. apenwarr's version has a super short "do" script (a couple of hundred lines of portable sh IIRC) which uses the same configuration but does a full rebuild each time.
The two weak points are: (1) multiple outputs from a compilation step (yytab.c yytab.h from yacc) is not properly supported, and (2) no windows support. Other than that it's the perfect minimalistic make replacement -- what make should have been.
Additionally, there tup[1]. With some assumptions about the build process that usually hold in non-distributed builds, it is the fastest, simplest make replacement; You just write a list of commands that builds your final outputs, and by tracing the processes it figures out exa
Read the rest
ctly what needs to be done next time -- nothing more, and nothing less.
[0] https://github.com/apenwarr/redo
[1] http://gittup.org/tup/
I can recommend apenwarr's redo implementation [0]. There's occasional activity on the mailing list [1], which leads me to believe people are using it, but not promoting it much.
The apenwarr implementation includes a full Python implementation as well as a minimal version in < 200 lines of sh. The minimal version doesn't support incremental rebuilds -- the out-of-date-ness tracking in the full Python version uses sqlite -- but it's good for understanding the redo concept. Also good for embedded contexts.
I used the full redo implementation for some data processing tasks once, with mixed results. It was a situation where I couldn't declare the dependency graph up front. With redo, each target declares its dependencies locally when it builds, and redo assembles and tracks the dynamic dependency graph. It's pretty neat, but become difficult
Read the rest
to reason about and debug. Could be that I never got comfortable with the new paradigm, or could be that essential tooling was missing, not sure. I still think redo is promising.
Anyway after a decade of messing with shiny new build tools, I finally learned to stop worrying and love the bomb (make). It's weird and warty but surprisingly capable. Worth the learning investment. Oh and jgrahamc's "GNU Make Book" is great. [2]
[0] https://github.com/apenwarr/redo
[1] https://groups.google.com/forum/#!forum/redo-list
[2] https://www.nostarch.com/gnumake
One misfeature I discovered recently and I wished got fixed is .SECONDARY vs .PRECIOUS [1]. Both are used to tell Make to keep intermediate products of chains of rules. However, .SECONDARY does not accept wildcards (%.o patterns for example), only .PRECIOUS does. However, .PRECIOUS also adds the guard that an incomplete intermediate product is kept in case Make is interrupted, which you would only ever want for a trivial stamp file. In any other case, you risk ending up with a runt file that breaks your next build.
So if, like any significant project I know, your build specifies lots of wildcard ("implicit") rules, you're stuck using .PRECIOUS, in which case you can never interrupt the build for risk of leaving it in a broken state.
FWIW, djb's "redo" concept [2] is based centrally around atomicity to avoid such a problem.
[1] https://www
Read the rest
I recently spent a week replacing all Makefiles in a reasonably complex embedded arm project I maintain with apenwarr's redo [1] (revamping our cobbled-together CI).. Which allows recursive make structured .do files, without tbe downsides - global dependencies and build state is cached across invocations in a throw-away .redo directory. It's an absolute joy to be able to stamp things that aren't files (eg docker containers) and just use one syntax (script of your choice) to orchestrate builds.
Cross-compiled kernel, bootloader, custom packages, full distro rootfs to final sdcard image without a single Makefile. I owe that man a beer.
[1] https://github.com/apenwarr/redo
In Recursive Make Considered Harmful (1997) [pdf]apenwarr's implementation of djb redo[1] contains a minimal, shell implementation of redo in its `minimal/` subdirectory.
It seems like the downside to using redo would be that there are .do files sitting throughout the source tree. It'd be nice if there was a way to consolidate several .do files into a single Redofile, similar to a Makefile or Sakefile.
Like the non-minimal redo, sake is also implemented in Python. I wonder how they compare, performance-wise.
[1] https://github.com/apenwarr/redo
In Sake: like make, but with shell scriptsShowing the 15 most recent of 42 sources. Enable JavaScript to search and browse every indexed mention.