usv

So ASCII actually had dedicated characters for this, 0x1C-0x1F. The problem is that they are non-printing. Unicode has rendered analogs, U+241C-U+241F, but they take more bytes to encode, which can significantly increase file size in large USV files. So my ideal would be to use ASV files rendered as USV in editors. https://github.com/SixArm/usv

Some notes from when the USV project tried using control characters: > We tried using the control characters, and also tried configuring various editors to show the control characters by rendering the control picture characters. > First, we encountered many difficulties with editor configurations, attempting to make each editor treat the invisible zero-width characters by rendering with the visible letter-width characters. > Second, we encountered problems with copy/paste functionality, where it often didn't work because the editor implementations and terminal implementations copied visible letter-width characters, not the underlying invisible zero-width characters. > Third, users were unable to distinguish between the rendered control picture characters (e.g. the editor saw ASCII 31 and rendered Unicode Unit Separator) versus the control

I’m working on a PWA which includes a dictionary search[1] feature and only a static web server (so no server side database to optimize the search). I did want searching to work in offline mode anyway. I decided it was best to generate an index file which the users download on first visit. For some reason I found USV[2] to be the best fit for this. USV I think allows seperating with ASCII control characters, but I used the unicode variants (␟, ␞, and ␝). I really liked this as it allowed me to add the glossary as an array in one of the columns. I wrote the parser my self which searches through the text structure, and it was simple enough. The reason I opted not to use a CSV or a TSV was that I didn’t want to deal with escaping surprise commas or tabs I would find in the dictionary data plus the extra dimension was nice. Since the file is

There is https://github.com/SixArm/usv which is exactly that, but with special unicode characters

Somebody went one step further and invented a format that uses the unicode codepoints as separators: https://github.com/SixArm/usv

According to the USV project, these characters don't work out so well in practice. > We tried using the control characters, and also tried configuring various editors to show the control characters by rendering the control picture characters. > First, we encountered many difficulties with editor configurations, attempting to make each editor treat the invisible zero-width characters by rendering with the visible letter-width characters. > Second, we encountered problems with copy/paste functionality, where it often didn't work because the editor implementations and terminal implementations copied visible letter-width characters, not the underlying invisible zero-width characters. >Third, users were unable to distinguish between the rendered control picture characters (e.g. the editor saw ASCII 31 and rendered Unicode Unit Separator) versu

Not long ago there was also a post about "Unicode Separated Value" https://github.com/sixarm/usv https://news.ycombinator.com/item?id=39679378

The reason why USV did not use the proper ASCII codes for field separator and record separator is a bit too pragmatic for me… https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...

I can't remember the last time I, or anyone I've ever worked with for that matter, ever typed up a CSV from scratch. The whole point of USV is that the delimiters can't normally be typed so you don't have to worry about escaping. USV supports displayable delimiters (see https://github.com/SixArm/usv), so for the much more common case of editing an existing CSV in a text editor, you can just copy and paste.

CSV has some limits and difficulties, but has massive benefits in terms of readability, portability, etc. I feel like USV (Unicode Separated Values) neatly improves CSV while maintaining most of its benefits. https://github.com/sixarm/usv

Wouldn't this easily break your file in subtle ways when someone tries to edit it in their editor and the zero width character is not visible? How could you make the difference with a standard CSV file if it looks like a standard CSV file? They explain why they don't use control characters. Editors are not consistent in how they show control/zero-length characters: https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...

Because they're zero-width. If you can't see them when you print your data, it's a machine-only separator, which makes it a bad separator for data that humans need to look at and work with. (Because CSV is a terrible data exchange format in terms of information per byte. But that makes sense, because it's an intentionally human readable data exchange format, not a machine format) Hence https://github.com/SixArm/usv/tree/main/doc/faq#why-choose-u...

Perfect deployment of HL33tibCe7’s aphorism: > For every interesting HN post, there’s at least one smug commenter who thinks he knows better, but actually doesn’t https://github.com/SixArm/usv/tree/main/doc/faq#why-use-cont...