upb

Makes sense. If anyone is interested, I think it could be interesting to see if the refcounting approach I developed for handling cyclic references in my library "upb" would work as a Rust library: https://github.com/haberman/upb/blob/master/upb/refcounted.h The basic idea is that you refcount groups of objects instead of refcounting objects independently. You compute the groups dynamically such that no cycle can span groups. This is sort of like an arena, except that no arena is ever explicitly created, and the collection can be more precise than an arena. In my library, objects go through a two-phase lifecycle. First they are mutable, then they are frozen, after which no further mutations can be made. When an object is frozen, its outbound pointers are also frozen. The nice property of this scheme is that you can perfectly compute t

Good point, although it's not like the stock protobuf implementation (rather than protocol) is the best that can be had. For example, this implementation of protobuf deserialization is quite a bit faster: https://github.com/haberman/upb http://blog.reverberate.org/2011/04/upb-status-and-prelimina...

As someone who has been working on parsing/serialization for many years, I can absolutely confirm that generating schema-specific code will beat generic code almost every time. The article discovers an important tradeoff: speed vs. convenience. Generated code is faster but less convenient, because it adds steps to your compile. And you pay this compile-time overhead for every message type you want to manipulate. The pain of this generated code was one of Kenton Varda's motivations for creating Cap'n Proto after working on Protocol Buffers for years. Unlike Protocol Buffers, Cap'n Proto doesn't need to generate parsing/serialization code because its serialization format also works as an in-memory format. I have taken a somewhat different approach to the problem, with my Protocol Buffer / JSON / etc. serialization framework upb (https:/