kstenerud

concise-encoding

ANTLR2019–2024

kstenerud / concise-encoding

concise-encoding

The secure data format for a modern world

Explore on GitHub Read Hacker News discussions

5Distinct people

63Mentions

0Explicit recommenders

280GitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

The secure data format for a modern world

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 7:56:55 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by year63 dated mentions

Observed mentions: 2019-05-31 18:05:03 UTC to 2024-08-13 16:16:01 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
2019	2019-01-01 00:00:00 UTC	2020-01-01 00:00:00 UTC	9	0	9
2020	2020-01-01 00:00:00 UTC	2021-01-01 00:00:00 UTC	17	1	16
2021	2021-01-01 00:00:00 UTC	2022-01-01 00:00:00 UTC	13	0	13
2022	2022-01-01 00:00:00 UTC	2023-01-01 00:00:00 UTC	15	2	13
2023	2023-01-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	6	1	5
2024	2024-01-01 00:00:00 UTC	2025-01-01 00:00:00 UTC	3	0	3

What Hacker News says

9 sources

kstenerud·Dec 11, 2019

This is a very sucky situation to be in, and I've been on both sides of it. I wrote KSCrash [1] a decade ago, and it's become the de-facto standard for crash handling in the Apple ecosystem. I didn't intend for this to happen; I just wanted crash handling MY way. But it's my baby, and I need to support it (technically no, but yes I do), even though I get no financial benefit from it. For a year, I was paid by a private company to supercharge it, but that funding dried up, and as a result the Android port stalled because I took a new job to pay the bills, writing Java code. Musashi [2] is a smaller example. It's an emulator, so it doesn't need nearly as much attention. All the same, a number of 68k based anthology releases for popular game systems used it and contributed neither fixes nor funds. I don't mind so much with this one because,

Read the rest

as I said, it's pretty low maintenance. Today, my latest itch is efficient and human-accessible data communications, and so I'm spearheading a new bidirectional, general-purpose, platform agnostic, encryption-capable, transport-agnostic RPC protocol [3], including all of the supporting technology it requires [4] [5] [6] [7] [8] [9]. If it takes off, it will save the entire planet a TON of time, energy, and cost, and I'll probably not see any donations or thanks from those who benefit the most. Please don't take this as complaining. I'm still going to develop and support my babies, because I'm not doing this for the money; I'm doing it to better the state of computing systems (mostly for my own sanity). However, because I'm forced to find separate employment to support my family, it leaves me with FAR less time to focus on these technologies. My estimate would be that I'm running at 1/4 my usual velocity when I require separate employment. For bigger projects that turns a 1 year project into 4 years. However, from a company perspective I also understand. Giving donations is actually a big pain in the ass, especially if it's to a foreign entity. Normal invoices are FAR easier, but then how do you structure it for free access + payment options without pissing people off? It's a tricky situation... [1] https://github.com/kstenerud/KSCrash [2] https://github.com/kstenerud/Musashi [3] https://github.com/kstenerud/streamux [4] https://github.com/kstenerud/concise-encoding [5] https://github.com/kstenerud/compact-float [6] https://github.com/kstenerud/compact-time [7] https://github.com/kstenerud/varpad [8] https://github.com/kstenerud/variable-bit-padding [9] https://github.com/kstenerud/vlq

In FUSE for macOS is no longer open source

kstenerud·Dec 9, 2019

The primary rule of writing documentation is: Revise it, many times, on separate days. You need to keep coming back to it, reading it with a fresh mind that hasn't thought of what you wrote for at least half a day, preferably longer. What seemed perfectly clear yesterday will come across as ambiguous today. You keep revising until the changes get smaller and start to feel cosmetic. Your job is to write concisely, using simple, unambiguous language that's quick to read and easy to digest. It's a skill that must be learned, practiced, and honed. It's also a good idea to explicitly spell out the meanings of "must not" etc, because people (especially non-native speakers) sometimes get confused. Example: https://github.com/kstenerud/concise-encoding/blob/master/cb... It doesn't take up much room, but it saves a whole lot of trouble down the li

Read the rest

ne.

In Must and Must Not: On writing documentation

kstenerud·Nov 27, 2019

100% agreed, which is why Concise Encoding has 1:1 type-compatible binary [1] and text [2] formats, which can be automatically converted when needed with no data loss. Human accessibility is an absolute must for any modern format. [1] https://github.com/kstenerud/concise-encoding/blob/master/cb... [2] https://github.com/kstenerud/concise-encoding/blob/master/ct...

In In mobile, 62.7% of computation energy cost is spent on data movement [pdf]

Reelin·Nov 27, 2019

Text: https://github.com/kstenerud/concise-encoding/blob/master/ct... Binary: https://github.com/kstenerud/concise-encoding/blob/master/cb...

In In mobile, 62.7% of computation energy cost is spent on data movement [pdf]

kstenerud·Nov 27, 2019

It's not just a problem with data movement inside the machine. Data movement between processes and machines is also horribly wasteful. Text encoding/decoding wastes massive amounts of energy, which is why I've been developing a format over the past 2 years with the benefits of binary (smaller, more efficient) as well as text (human readability/editability): https://github.com/kstenerud/concise-encoding

In In mobile, 62.7% of computation energy cost is spent on data movement [pdf]

kstenerud·Nov 19, 2019

I'm working on this problem now. I'm still building the lower layers, which are: - An RPC mechanism (not built yet) - A bi-directional multiplexing stream protocol [1] (mostly there) - A general purpose binary encoding mechanism [2] (I've gotten bogged down with this for the past few months, but I can finally see the light at the end of the tunnel!) The idea is to get these lower layers finished so that I can build all of my data communications related projects on top of them. One of those is a general-purpose file access system that I can build actual products on top of, such as file managers, remote filesystems, streaming services, synchronization, etc. I'm rebuilding everything underneath because the existing protocols are too chatty and bloated, not modular enough, don't work with encryption, or have weird requirements (like NFS requi

Read the rest

ring kernel integration for example, or they're restricted to the local net, etc). [1] https://github.com/kstenerud/streamux/blob/master/streamux-s... [2] https://github.com/kstenerud/concise-encoding

In Ask HN: How to easily transfer files between devices

kstenerud·Nov 13, 2019

I work on my JSON/BSON/etc replacement: https://github.com/kstenerud/concise-encoding I've been at it a year and a half so far. Designing something like this is actually very challenging! And the kicker is that this technology is only a stepping stone towards what I actually want to build :P

In Ask HN: What do you do when have free time?

kstenerud·Aug 4, 2019

I've been working on a general purpose binary and text encoding format for some time now as a personal project. It supports time, decimal floats, binary blobs as first-class citizens, supports comments, and focuses on encoding the most commonly used values more compactly. The binary format [1] is almost done (I've come up with a more compact date representation that I'll be adding this weekend to replace smalltime in the C and go implementations), and the text format isn't far behind. I almost had a better floating point compression format, but it turned out to be too complicated, and only works well for decimal floating point, so I'll probably not use it in CBE. [1] https://github.com/kstenerud/concise-encoding/blob/master/cb...

In CBOR – Concise Binary Object Representation

kstenerud·May 31, 2019

It's a lot of writing and rewriting. I usually start off putting everything I think I want in the spec into a text document without worrying too much about logical consistency, then leave it overnight, add more things, leave it overnight, reshape, etc. It's important to take a long break after hitting a stopping point so that you gain some perspective. It's also good to solicit comments from your network, although that tends to be as difficult as finding code reviewers, so YMMV. You should also write a reference implementation, because actually implementing something will expose a lot of problems in your assumptions, and it will give other implementers something to test against. Generally, it's 3-6 months from inception to release of a smaller spec if you want it to be solid. Actually, you could follow along with a couple of specs I wrote

Read the rest

since I have a git history of them: https://github.com/kstenerud/streamux/commits/master?after=7... https://github.com/kstenerud/concise-encoding/commits/master...

In Undervalued Engineering Skills: Writing Well