concise-encoding

ng/blob/master/cbe.kbnf * The KBNF grammar itself: https://github.com/kstenerud/kbnf/blob/master/kbnf.md#the-kbnf-grammar-in-kbnf

erud/go-concise-encoding/blob/master... but I'll be simplifying the format some more). That way, we can run the same tests on all CE implementations instead of everyone coming up with their own. I'll move the test definitions to their own repo when they're done and then you can just submodule it. I'm thinking that they should look more like: c1 { "type" = { "identifier" = "ce test" "version" = 1 } "tests" = [ { "name" = "Some kind of test" "success" = [ // These must successfully convert to each other { "cte" = "c1 [1 2 3 4]" "cbe" = |u8x 81 01 7a 01 02 03 04 7b| // Events are a kind of text shorthand for what the parsing of cte or cbe should produce "events" = ["v 1" "l" "n 1" "n 2" "n 3" "n 4" "e"] } { ... } ] "failure" = [ // These must fail to decode { "cte" = "c1 100 ]" } { "cbe" = |u8x 81 01 64 7b| } ] } ] }

It only uses UTC offsets, which are a bad idea for so many reasons. https://github.com/kstenerud/concise-encoding/blob/master/ce... > - Bin + txt: Again, I'm unclear on what you mean here, but ASN.1 has both binary and text-based encodings Ah cool, didn't know about those. > - Versioned: Also a little unclear to me The intent is to specify the exact document formatting that the decoder can expect. For example we could in theory decide to make CBE version 2 a bit-oriented format instead of byte-oriented in order to save space at the cost of processing time. It would be completely unreadable to a CBE 1 decoder, but since the document starts with 0x83 0x02 instead of 0x83 0x01, a CBE 1 decoder would say "I can't decode this" and a CBE 2 decoder would say "I can decode this". With documents versioned to the spec, we can change even the fundamental structure of the format to deal with ANYTHING that might come up in future. Maybe a new security flaw in CBE 1 is discovered. Maybe a new data type becomes so popular that it would be crazy not to include it, etc. This avoids polluting the simpler encodings with deprecated types (see BSON) and bloating the format.

RES sane defaults for a lot of things in order to be compliant), but no format can protect you completely. A "fork bomb" using cyclic data is unlikely, unless your application code is really naive (if you're using cyclic data, you need to have a well-reasoned purpose for it, and are likely just using pointers internally - which won't blow out your memory unless you're doing something foolish when processing the resulting structs). Actually, this does give me an idea... make cyclic data disallowed by default, just to cover the common case where people don't use it and don't even want to think about it. Re time formats: global coordinates will always start with a slash, so 12:00:00/-130/-172. UTC offsets will always start with + or -, and be 4 digits long, so 12:00:00+0130 or 12:00:00-0130. The validation rules are very specific, and that does complicate the text format a bit, but this drives to the central purpose of it: The text format is for the USER, and is not what you send to other machines or foreign systems. It's for a user to edit or inspect or otherwise interact with the data on the RARE occasions where that is necessary. So the text format doesn't need to be fast or efficient, only unambiguous and easy for a human to read. You certainly shouldn't open an internet connected service that accepts the text format as input (except maybe during development and debugging...) In fact, I would expect a number of CE implementations (such as for embedded systems) to only include CBE support, since you could just use a standalone command-line tool or the like to analyze the data in most cases. Re: subfields. That would make it harder for a human to read. The text format sacrifices some efficiency for human friendliness and better UX. Parser logic re-use isn't really a priority (other than making sure it's not OBVIOUSLY bad for the parser), because text parsing/encoding is supposed to be the 0.0001% use case. It's not super easy to add new types to the text format grammar, but that's fine because human friendliness trumps almost all, and adding new types should be done with EXTREME CARE. I've lost count of all the types I've added and then scrapped over the years. It's really hard to come up with these AND justify them! The ANTLR grammar is actually more of a documentation thing. I've verified it in a toy parser but it's not actually tied to the reference implementation (yet). The reference implementation currently is similar to a parser combinator, with a lot of inspiration from the golang team's JSON parser (I watched a talk by the guy some time ago and was impressed). But at the same time I'm starting to wonder if it might have been better to implement the reference implementation as just an ANTLR parser after all... leave the optimizations and ensuing complications to other implementations and keep the reference implementation readable and understandable. The binary format code is super simple, and about 1/3 the size of the text format code. The major downside of ANTLR of course is the terrible error reporting.

felt that there were enough pain points, missing types, and missing security features (for example versioning) to warrant a fresh start.

e in gRPC. It is similar to what is used in some trading systems. There is also a FlexBuffers variation if you'd want something closer to JSON/BSON. Must say however, I found it cool that you have some Mac/iOS GitHub repos. Definitely going take some time to check them out -- I used to develop iOS apps. [1] https://developers.google.com/protocol-buffers/docs/referenc... [2] https://github.com/kstenerud/concise-encoding/blob/master/cb... [3] https://developers.google.com/protocol-buffers/docs/proto3#s... [4] https://developers.google.com/protocol-buffers/docs/referenc... [5] https://google.github.io/flatbuffers/flatbuffers_white_paper...

wing the specifications and pointing out issues or anything weird or things that seem wrong or don't make sense. - Implementations in other languages. - Ideas for a schema. - Public outreach, championing online. [1] https://concise-encoding.org/ [2] https://github.com/kstenerud/concise-encoding [3] https://github.com/kstenerud/go-concise-encoding