MAMathnerd314·Apr 10, 2024
Neither of those sound like you have actually understood the feature. The point of Tup's partial DAG is that it starts from the file changes and only loads relevant dependency information for those files (and its transitive closure). So for example if you have 1 million .c files and one of them is changed, say a.c, then it does 3 lookups: "what depends on a.c?" (a.o), "what depends on a.o?" (the .exe), "what depends on the .exe?" (nothing, it is a target). So dependency chasing is minimal and it takes very little time until the commands for a.o and the .exe are loaded and rerun. In contrast, in your approach, you will still have to go through the million files - maybe Rig won't require an OS 'stat' call and can just check the watcher's list in-memory for each file, but it will still burn a significant number of cycles following dependency
Read the rest
chains even though they are up-to-date.
And for command tracing, the most significant issue is that it records both files read and files output. Even with dynamic dependencies it is difficult to record both of these dependency types properly. For example, Shake has this "multiple outputs" feature but it is actually really hacky and doesn't work. https://github.com/ndmitchell/shake/issues/382 The Pluto "sound and optimal" build paper discusses this briefly - they use a two-level file-command graph specifically to solve this problem. Since you have files-as-targets you will have trouble recording write dependencies, similarly to Shake.