UNunqueued·Sep 24, 2024
There is a pretty good syntax for dealing with nasty filenames, if you must: ANSI-C quoting[1].
If you have to output in a shellscript in this format, use printf %q
from man printf:
%q ARGUMENT is printed in a format that can be reused as shell input, escaping non-printable
characters with the proposed POSIX $'' syntax.
It is just $'<nasty ansi-c escaped chars>'
$ touch $'\nHello\tWorld\n'
$ ls
One thing I do like about a filesystem that fully supports POSIX filenames is that at the end of the day a filesystem is supposed to represent data. I think it is totally sensible to exclude certain characters, but that it should be done higher up in the stack if possible. Or have a flag that is set at mount time. Perhaps even by subvolume/dataset.
One thing I haven't seen mentioned is that POSIX filenames are so permissiv
Read the rest
e that they allow you to have bytes as filenames that are invalid UTF-8. That's why the popular ncdu[2] program does NOT use json as it's file format, although most think it does. It's actually json but with raw POSIX bytes in filename fields, which is outside of the official json spec. That does not stop folks from using json tools to parse ncdu output though.
Another standard that is also very permissive with filenames is git. When I started exploring new ways to encode data into a git repo, it was only natural that I encountered issues with limitations of filesystems that I would check out in.
Try cloning this repo, and see if you are able to check it out:
https://github.com/benibela/nasty-files
It is amazing how many things it breaks.
If you are writing software that deals with git filenames or POSIX filenames (that includes things like parsing a zip file footer), you can not rely on your standard json encoding function, because the input may contain invalid utf-8. So you may need to do extra encoding/filtering.
[1]: https://www.gnu.org/s/bash/manual/html_node/ANSI_002dC-Quoti...
[2]: https://dev.yorhel.nl/ncdu/jsonfmt