resholve

le, plus of course maps ("associative arrays"). I don't have to worry about portability problems with common utilities, because my scripts will bring along the expected implementations of coreutils, `find`, `grep`, etc. I'm free to use "modern" alternatives to classic utilities (like `rg` instead of GNU grep or `fd` instead of GNU findutils) if they offer better performance or more readable syntax. Polished interactivity is easy to build by just embedding a copy of `fzf`. And while I don't love Bash for working with structured data, it becomes a lot less painful when I can just pull in `jq`. It's obviously got some disadvantages versus an option like Python, but the amount-of-code to functionality ratio is also much more favorable than Python's. I typically use this for scripting build and development tasks in projects that already use Nix (via Devenv— I have some unpublished changes to the scripts module that add resholve and ShellCheck integration) to manage their environments, so there's no special packaging/distribution/setup burden. IME it makes for vastly more readable and maintainable shell scripts than painstakingly limiting oneself to pure POSIX sh, so it can serve quite well as a middle ground option between barebones sh and a "real" programming language as your little script gradually grows in complexity. > if you need external deps - just statically link them and distribute a cross-compiled binary (use zigcc[1] or easy Nim cross-compilation). The deployment story sounds very slick and hard to beat! This is something I might want to try for scripts I want to be easy to distribute on macOS systems that don't have Nix installed. -- 1: https://github.com/abathur/resholve

is a more chaotic and quirky environment: macOS updates break applications or remove APIs more often than Linux updates do, macOS updates nuke configuration files in `/etc`, or blindly steamroll system users (low UIDs), the login shell doesn't manage environment variables or other aspects of the GUI session, etc. These are extra hurdles that Nix has to work around on macOS. But it generally works well and the only real hooks that Nix needs into your environment are some shell scripts that need to be sourced on startup. If you know how to manage your PATH and shell initialization in general, you can do it with Nix. (Additionally, one distributions of Nix itself includes some extra software for managing these hurdles that works pretty well at the cost of a more opinionated and more "invasive" install-- check out "Determinate Nix" if you're interested.) Other than that, it's basically the same as on Linux. Package selection is good, things mostly work the same way. Nix's support for platforms other than Linux and macOS is less mature. In years past, efforts have waxed and waned, but in the past few years there've been some concerted and persistent efforts to make Nix and Nixpkgs more usable on Windows, NetBSD, and FreeBSD. Even native (e.g., not just cross-compilation from other platforms) support for Windows is actually a thing! You can also easily set up "remote builders" to local VMs for Linux and FreeBSD so that you can transparently build packages and environments for those platforms on your Mac, if that's a frequent choice of development platform. Overall I'd say it's pretty good, but I don't personally know much about the current status of FreeBSD. It looks like there are a few hundred FreeBSD-specific packages in Nixpkgs, and there's also a fork of NixOS based on FreeBSD called NixBSD. Most packages include "platforms = [ unix ];" in their metadata, which means they're at least intended to build for not just Linux and macOS but also FreeBSD, but I don't know if Hydra is building binaries for any FreeBSD platforms. If it's not, then many packages may be untested and have small build problems. My guess is that at worst, Nix on FreeBSD is probably slightly behind where Nix on Linux was when I got into it 7 or 8 years ago. At that time, Nix still felt amazing and very worth it, but certainly required a willingness to learn how to package things and fill in gaps yourself to get the most out of it. Consider dropping by NixOS' Matrix community; I'm sure there's a FreeBSD channel. -- 1: https://github.com/abathur/resholve/

do "${pkgs.coreutils}/bin/pwd" --physical # the same thing and in that way my scripts use only their fixed dependencies at fixed versions without modifying the environment at all. This also works nicely across platforms, so when these scripts run on macOS they get the same versions of the GNU coreutils as they do when they run on Linux, just compiled for the appropriate platform and architecture. Same goes for different architectures. So this way your script runs 'natively' (no virtualization) but you still pin all its dependencies. In other contexts (e.g., cloud-init), I use bash a bit more restrictively depending on what versions I'm targeting. But I still use Nix to provide dependencies so that my scripts use the same versions of external tools regardless of distro or platform: nix shell nixpkgs#toybox --command which which # these do nix run nixpkgs#which -- which # the same thing `nix run` and `nix shell` both behave as desired in pipelines and subshells and all that. (To get the same level of determinism as with the method of use outlined earlier, you'd want to either pin the `nixpkgs` ref in your local flake registry or replace it with a reference that pinned the ref down to a commit hash.) There are is a really cool tool[1] by the Nixer abathur for automagically converting naively-written scripts to ones that pin their deps via Nix, as in the first example. I'm not using it yet but I likely will if the scripts I use for our development environments at work get much bigger-- that way I can store them as normal scripts without any special escaping/templating and linters will know how to read them and all that. Anyhow, it's totally safe to install a newer Bash on macOS, and I recommend doing it for personal interactive use and private scripts. Pkgsrc and MacPorts both have Bash 5.x, if you don't have or want Nix. > I need to write up some thoughts on bash being effectively a lisp if you stare at it the right way. You should! It's totally true, since external commands and shell builtins are in a prefix syntax just like Lisp function calls. Some shells accentuate this a bit, like Fish, where subshells are just parens with no dollar signs so nested subcommands look very Lisp-y, and Elvish, whose author (xiaq) has tried to lean into that syntactic convergence designing a new shell (drawing inspiration from Scheme in various places). > I found this gizmo somewhere and it's worth sharing as well... # MAGIC DEBUGGING LINE #trap '(read -p "[$BASH_SOURCE:$LINENO] $BASH_COMMAND? ")' DEBUG Okay that looks really nifty. I will definitely find a use for that literally tomorrow, if not today. -- 1: https://github.com/abathur/resholve

just a top-N formats thing, but there's no reason we can't keep going. One of the big advantages is that it's fairly cross-platform, even if accuracy is suboptimal. The scanning approach isn't just amazing for resholve's specific needs. I'm stepping down this path to try to identify executables that likely exec their command-line arguments--because this is a sign that resholve need to know how to check invocations of that command for other executables. But it's a good bit better at flagging things that have _any_ exec behavior, and that's a decent-enough first step for letting focus on a smaller set. Once we find these, though, we're generally left with human analysis of the help, manpage, and/or source to figure out what exec the scanner is hitting on and whether it's part of the CLI. I've been ~snoozing on this front for a while, but I feel like a decent fraction of the components for building something like an 80% toolkit exist in at least a primitive form. It could probably benefit from some more static source analysis bits. I also vaguely intend to look into whether LLMs can do a reasonably-good job of making it easier to answer these questions from the help/man/source. Unless something out in that space can be reliably automated, it also needs some way to ~pin assertions about the arguments to the relevant code in a way that they'll break if that specific corner of the codebase changes. (I've wondered if semgrep expressions could handle this.) There are also some other bits around that might fit in, though cross-platform is a common-ish problem. We've looked a little at using a FUSE, but that is hard to operationalize on macOS. I also poked at 3 or so less-interactive dynamic approaches a few years back in https://github.com/abathur/commandeer/ and I'm sure a few more have cut their teeth since then. I also wrote a ~shelved-for-now WIP (https://github.com/abathur/faffer) for using some cursed shell metaprogramming to do something like fuzz/tree-shake a shell script for loose dependencies. That isn't directly viable for many languages, but its focus on ~overloading flow-control structures to make it easier for force branch coverage does seem like something that might be feasible with rewriting tools.

ate the system in ways that even it can't protect us from. > If there is no way to write a generic shebang you might as well just accept that you will need to change it. I'm of (at least) two minds about this. It's obviously correct that there's no way to write a generic shebang that will work everywhere. But I think it's also obviously a poor use of human time and attention, at scale, for us to all individually waste time ~localizing every third-party script for every system we ever run it on (perhaps repeatedly, as the system changes). The impossibility of a 100% solution isn't a good reason to shun 90% solutions. (I say this as someone who develops a tool, https://github.com/abathur/resholve, for ~localizing bash/shell scripts--shebang included. In some sense my ego could benefit from end users needing a tool for this, but the task is inherently fraught on stateful systems and is probably best addressed at the package-manager level.) > Fair point. I am also biased from PRs also trying to insert flake.nix into projects. Just as fair. They mean well (and generally just want to make the project easier to distribute or contribute to), but they can be a bit ~deaf to reasonable concerns about adding to the maintenance burden. > My issue is that nix is doing something nonstandard and it feels like projects are having to accommodate nix's oddities instead of nix being the one to accommodate what already exists. Also fair, though I hope you'll keep your eyes open for cases where people trying to package a project with Nix are shaking loose portability/reproducibility/underspecification problems that are already latent in the project and are worth fixing even if you ultimately reject the .nix file. (This won't be everything. There are certainly cases where Nix is the only one needing the patch and should do the work to keep up with it.) Anecdotal, but it's reasonably common to run into projects with hardcoded paths that leave the build/test processes working cleanly on exactly one person's mutable system without anything anywhere saying what that system is or how they bootstrap it. I've seen this include ~snapshotted test outputs that will only pass on the authors system because it's affected by files or even dotfiles that may be in the project directory but not committed, or higher up in the directory tree on the system. This kind of stuff is just as likely to trip up non-Nix contributors. It may also be a big hurdle to the broader OSS community being able to pick up the torch for them if they get burned out, fall ill, change careers, etc. (Sometimes maintainers are standoffish about fixing these, especially if they distribute their own binaries or directly push artifacts out to package repositories and so on. They may not understand why we want to build from source or run their test suite or build scripts or build the manual. I'm sympathetic on some level, but I must admit I'm flummoxed when people seem actively incurious about reports that suggest they might have trouble building the project if they lost access to their device). > Not all instances of it in build scripts are properly rewritten. Any example? There are definitely asterisks (https://github.com/NixOS/nixpkgs/blob/a4d0fe7270cc03eeb1aba4...), and the default behavior here is one driver for people going out to ask for `/usr/bin/env` (because those can get autopatched). But we do still have the tools to patch them. It isn't the end of the world if you tell us to kick rocks. > Also there would be work in creating a migration plan to not breaking people who currently rely on its existence. Yes (though, perhaps ironically, the closest I have to a pitch for Nix is what an incredible lever nix/nixpkgs/NixOS are for experimenting with the feasibility of ideas like this at scale).

wouldn't see much reason to endure the pain of adopting Nix if they still had to support the heterogenous environments. Much like you're doing here, I've been hoping to figure out how to build a Nix-based packaging flow that can generate deployable standalone bundles or containers. It'd be a heavy way to bundle Shell, but I imagine some projects would take the tradeoff for predictability and reduced support load. But since it would need to take place within a Nix build, I'd need to cross-compile for it to work on macOS. Hoping you know if it's a dead-end or not :)