lite3

Also lack of a proper query language will be a problem for long term serious use. A simple hand-rolled program API can only get you so far, until you need more advanced querying. > Unlike XML or JSON, joedb is a binary file format that does not require any parsing. So, joedb files are much smaller, and processing data is much faster. Some time ago I created a JSON-compatible serialization format that is zero-copy (no parsing required): https://github.com/fastserial/lite3 It doesn't do transactions or history versioning, but it is also very fast in memory. Something like jq or JSONPath on a disk-file version of this format could be interesting.

y 120x in some benchmarks as a result: https://github.com/fastserial/lite3

nction was used. There also might be spare bit somewhere to indicate ‘use the alternative hash function’. Reading the code (nice-looking, BTW, for C code, but since it is C code, also full of warnings that other languages can protect you from) I spotted this (https://github.com/fastserial/lite3/blob/acbb97984eca1183ddc...): > The JSON standard requires that the root-level type always be an ‘object' > or 'array'. This also applies to Lite³. I don’t think that is true, and https://www.json.org/json-en.html agrees with that. Single values (numbers, strings, booleans, null) also are valid json.

nd freed memory isn't reclaimed [3]. These are the two actually difficult problems standing in the way of rkyv getting more comprehensive mutation support, so unfortunately I think there's nothing valuable to take away here. The issue with growing the buffer is that you invalidate any pointers into the buffer and have to manually fix them up (e.g. by re-crawling the buffer). There's not a decent approach I'm aware of which manages these pointer fixups in a way that limits complexity and avoids really nasty API surfaces. The issue with memory reclamation is that you effectively have to add an allocator to your buffer, and that allocator has to also be persistable. I actually got this part working in rel while that was still a project worth hacking on [4]. I just ran out of steam and willingness to play the type-tac-toe required to get a safe API out of it. The main problem I ran into there was related to value branding, since you shouldn't be allowed to make references to objects which cross between contiguous buffers. This project is neat, but it does basically just look like an rkyv-converted `BTreeMap<String, JsonValue>`. [1]: https://github.com/rkyv/rkyv/blob/main/rkyv/src/collections/... [2]: https://github.com/fastserial/lite3/blob/main/src/lite3.c#L5... [3]: https://lite3.io/design_and_limitations.html#autotoc_md29 (see note at end of section) [4]: https://github.com/rkyv/rel/blob/main/rel_example/src/main.r...

h paper from 2024 about a conceptual schemaless zero-copy format. Lightning struck. This idea seemed extraordinary: you essentially send a serialized B-tree over the network so that the receiver can perform lookups and mutations directly without parsing. The only problem: nobody had made a (public) implementation yet. I figured sometimes you have to be the first, so for the last 6 months I've been working on this and am proud to say it is now released and open source under MIT license. The ability to modify serialized data directly is something not typically supported by other formats. I also extended the original format with JSON compatibility, such that it is now possible to convert between Lite³ and JSON. Despite this, it achieves performance on-par with modern zero-copy formats like Flatbuffers and Cap'n Proto. Right now the implementation is in C, but I plan to create bindings for other languages as well. So hopefully some fellow developers find it interesting! Official GitHub repository: https://github.com/fastserial/lite3 Official documentation site: https://lite3.io Mailing list: see project README