> didn’t find any standard for separating protobuf messages
The fact that protobufs are not self-delimiting is an endless source of frustration, but I know of 2 standards for doing this:
- SerializeDelimited* is part of the protobuf library: https://github.com/protocolbuffers/protobuf/blob/main/src/go...
- Riegeli is "a file format for storing a sequence of string records, typically serialized protocol buffers. It supports dense compression, fast decoding, seeking, detection and optional skipping of data corruption, filtering of proto message fields for even faster decoding, and parallel encoding": https://github.com/google/riegeli
In Add extra stuff to a “standard” encoding? Sure, why notprotocolbuffers
protobuf
C++2014–2026
protocolbuffers / protobuf
protobuf
Protocol Buffers - Google's data interchange format
89Distinct people
124Mentions
0Explicit recommenders
72.2KGitHub stars
What to know before exploring
What it is
Protocol Buffers - Google's data interchange format
GitHub
Where it is used
No documented product use was verified in the last source check.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 4:01:40 AM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by year124 dated mentions
Observed mentions: 2014-12-15 12:38:35 UTC to 2026-10-09 18:07:50 UTC.
* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| 2014 | 2014-01-01 00:00:00 UTC | 2015-01-01 00:00:00 UTC | 1 | 1 | 0 |
| 2015 | 2015-01-01 00:00:00 UTC | 2016-01-01 00:00:00 UTC | 11 | 2 | 9 |
| 2016 | 2016-01-01 00:00:00 UTC | 2017-01-01 00:00:00 UTC | 11 | 3 | 8 |
| 2017 | 2017-01-01 00:00:00 UTC | 2018-01-01 00:00:00 UTC | 5 | 0 | 5 |
| 2018 | 2018-01-01 00:00:00 UTC | 2019-01-01 00:00:00 UTC | 12 | 0 | 12 |
| 2019 | 2019-01-01 00:00:00 UTC | 2020-01-01 00:00:00 UTC | 13 | 0 | 13 |
| 2020 | 2020-01-01 00:00:00 UTC | 2021-01-01 00:00:00 UTC | 14 | 1 | 13 |
| 2021 | 2021-01-01 00:00:00 UTC | 2022-01-01 00:00:00 UTC | 7 | 0 | 7 |
| 2022 | 2022-01-01 00:00:00 UTC | 2023-01-01 00:00:00 UTC | 6 | 0 | 6 |
| 2023 | 2023-01-01 00:00:00 UTC | 2024-01-01 00:00:00 UTC | 8 | 1 | 7 |
| 2024 | 2024-01-01 00:00:00 UTC | 2025-01-01 00:00:00 UTC | 14 | 2 | 12 |
| 2025 | 2025-01-01 00:00:00 UTC | 2026-01-01 00:00:00 UTC | 15 | 1 | 14 |
| 2026 (partial) | 2026-01-01 00:00:00 UTC | 2027-01-01 00:00:00 UTC | 7 | 0 | 7 |
From the project README
An excerpt from the repository, fetched Nov 13, 2026.
Read the project’s introduction
Protocol Buffers - Google's data interchange format
Copyright 2008 Google LLC
Overview
Protocol Buffers (a.k.a., protobuf) are Google's language-neutral,
platform-neutral, extensible mechanism for serializing structured data. You
can learn more about it in protobuf's documentation.
This README file contains protobuf installation instructions. To install
protobuf, you need to install the protocol compiler (used to compile .proto
files) and the protobuf runtime for your chosen programming language.
Working With Protobuf Source Code
Most users will find working from
supported releases to be
the easiest path.
If you choose to work from the head revision of the main branch your build will
occasionally be broken by source-incompatible changes and insufficiently-tested
(and therefore broken) behavior.
If you are using C++ or otherwise need to build protobuf from source as a part
of your project, you should pin to a release commit on a release branch.
This is because even release branches can experience some instability in between
release commits.
Bazel with Bzlmod
Protobuf supports
Bzlmod with Bazel 8 +.
Users should specify a dependency on protobuf in their MODULE.bazel file as
follows.
bazel_dep(name = "protobuf", version = <VERSION>)
Users can optionally override the repo name, such as for compatibility with
WORKSPACE.
bazel_dep(name = "protobuf", version = <VERSION>, repo_name = "com_google_protobuf")
Bazel with WORKSPACE
Users can also add the following to their legacy
WORKSPACE file.
Note that with the release of 30.x there are a few more load statements to
properly set up rules_java and rules_python.
http_archive(
name = "com_google_protobuf",
strip_prefix = "protobuf-VERSION",
sha256 = ...,
url = ...,
)
load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
protobuf_deps()
load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies")
rules_java_dependencies()
load("@rules_java//java:repositories.bzl", "rules_java_toolchains")
rules_java_toolchains()
load("@rules_python//python:repositories.bzl", "py_repositories")
py_repositories()
Protobuf Compiler Installation
The protobuf compiler is written in C++. If you are using C++, please follow
the C++ Installation Instructions to install protoc along
with the C++ runtime.
For non-C++ users, the simplest way to install the protocol compiler is to
download a pre-built binary from our GitHub release page.
In the downloads section of each release, you can find pre-built binaries in
zip packages: protoc-$VERSION-$PLATFORM.zip. It contains the protoc binary
as well as a set of standard .proto files distributed along with protobuf.
If you are looking for an old version that is not available in the release
page, check out the Maven repository.
These pre-built binaries are only provided for released versions. If you want
to use the github main version at HEAD, or you need to modify protobuf code,
or you are using C++, it's recommended to build your own protoc binary from
source.
If you would like to build protoc binary from source, see the C++ Installation Instructions.
Protobuf Runtime Installation
Protobuf supports several different programming languages. For each programming
language, you can find instructions in the corresponding source directory about
how to install protobuf runtime for that specific language:
Language
Source
C++ (include C++ runtime and protoc)
src
Java
java
Python
python
Objective-C
objectivec
C#
csharp
Ruby
ruby
Go
protocolbuffers/protobuf-go
PHP
php
Dart
dart-lang/protobuf
JavaScript
protocolbuffers/protobuf-javascript
Quick Start
The best way to learn how to use protobuf is to follow the tutorials in our
developer guide.
If you want to learn from code examples, take a look at the examples in the
examples directory.
Documentation
The complete documentation is available at the Protocol Buffers doc site.
Support Policy
Read about our version support policy
to stay current on support timeframes for the language libraries.
Developer Community
To be alerted to upcoming changes in Protocol Buffers and connect with protobuf developers and users,
join the Google Group.
Read the full README on GitHubWhat Hacker News says
8 sourcesIt looks like Protobuf actually has a test suite to ensure compliance with the protocol:
https://github.com/protocolbuffers/protobuf/tree/main/confor...
Seems like a good idea for protocols in general to have an official test suite, as a way to address this problem
In Add extra stuff to a “standard” encoding? Sure, why notI actually went through all projects listed in [1] because I remember this very quirk. It turns out that there are many such libraries that have two variants of encode/decode functions, where the second variant prepends a varint length. In my brief inspection there do exist a few libraries with only the second variant (e.g. Rust quick-protobuf), which is legitimately problematic [2].
But if the project in question was indeed protobuf.js (see loeg's comments), it clearly distinguishes encode/decode vs. encodeDelimited/decodeDelimited. So I believe the project should not be blamed, and the better question would be why so many people chose to add this exact helper. Well, because Google itself also had the same helper [3] [4]! So at this point protobuf should just standardize this simple framing format [5], instead of claiming that protobuf h
Read the rest
as no obligation to define one.
[1] https://github.com/protocolbuffers/protobuf/blob/main/docs/t...
[2] https://github.com/tafia/quick-protobuf/issues/130
[3] https://protobuf.dev/reference/java/api-docs/com/google/prot...
[4] https://github.com/protocolbuffers/protobuf/blob/main/src/go...
[5] Use an explicitly different name though, so that the meaning of "encoding/decoding protobuf messages" doesn't change.
It looks like it is in fact universal. Just glancing at the code here, it looks like the tool searches any arbitrary file for bytes that look like encoded protobuf descriptors, specifically looking for bytes that are plausibly the beginning of a FileDescriptorProto message defined here:
https://github.com/protocolbuffers/protobuf/blob/main/src/go...
This takes advantage of the fact that such descriptors are commonly compiled into programs that use protobuf. The descriptors are usually embedded as constant byte arrays. That said, not all protobuf implementations embed the descriptors and those that do often have an option to inhibit such embedding (at the expense of losing some dynamic introspection features).
In Block YouTube ads on AppleTV by decrypting and stripping ads from ProfobufHow to learn to use protoc in 21 easily infuriating steps
3 points · Open discussionDon't get me wrong, it is a PITA.
But it's good enough and has very wide and mature support for many languages and a good story for backward compatibility which is important when you have files you want to read 2 years from now or when you have different version of your clients and servers co-existing.
There _could_ be a better interchange format in theory (and many have been proposed), but a combination of maturity and network effect (and the fact that many of the alternatives to protobuf do improve some things but do other things worse than protobuf) make it very hard to chose an alternative.
EDIT: ah, about discriminating zero value from unset value (aka options), they're back in proto3 (experimental) https://github.com/protocolbuffers/protobuf/blob/main/docs/i...
In How Turborepo is porting from Go to RustOn optional.. this was a regression in proto that is somewhat helped by https://github.com/protocolbuffers/protobuf/blob/main/docs/f... ; I have no idea whether protobuf for rust has started taking advantage of this.
JSON is awful in every way.
In How Turborepo is porting from Go to Rustrecent versions of proto3 have added back the “optional” keyword that can be used on any field. see: https://github.com/protocolbuffers/protobuf/blob/main/docs/f...
In How Turborepo is porting from Go to Rust