m3

> I can't find any other open source time series database except Mimir/Cortex which allows this much scale (clustering options in their open source version) The following open source time series databases also can scale horizontally to many nodes: - Thanos - https://github.com/thanos-io/thanos/ - M3 - https://github.com/m3db/m3 - Cluster version of VictoriaMetrics - https://docs.victoriametrics.com/Cluster-VictoriaMetrics.htm... (I'm CTO at VictoriaMetrics) > Can we use Prometheus/Mimir as general purpose time series database? This depends on what do you mean under "general purpose time series database". Prometheus/Mimir are optimized for storing (timestamp, value) series where timestamp is a unix timestamp in milliseconds and value is a floating-point number. Each series has a name and can have arbitrary set of additional (label=value)

Excited to be able to share this with everyone today! A lot of hard work has gone into getting M3 to 1.0, so this represents a major milestone. In getting to this point, we've focused quite a bit on stability and resilency, performance improvements (most notably during bootstrapping), and ease of configuration. All of these changes are aimed at ensuring M3 provides you the bulletproof, high performance experience one would expect of a production grade metrics engine. For more information on what went into the release, take a look at the changelog (https://github.com/m3db/m3/releases/tag/v1.0.0). Thank you to the open source community and everyone who has been involved in getting M3 to 1.0! We're very excited for M3's future!

There are other time series databases worth comparison with TimescaleDB: * InfluxDB - https://www.influxdata.com/ * VictoriaMetrics - https://github.com/VictoriaMetrics/VictoriaMetrics/ * M3DB - https://github.com/m3db/m3/

I suggest checking out M3DB[1]. My team & I use it to serve metrics for all of Uber, we have ~1500 hosts across various clusters. It's serving us quite well. [1]: https://github.com/m3db/m3

Yes I've seen that also work, it's a lot of stitching together things yourself and we had to put a lot of caching in front of the inverted index we were using, however definitely plausible. ClickHouse doesn't do any streaming of data between nodes as you scale up and down which was a big thing for us since we had large datasets and needed to rebalance when cluster expanded/shrunk. With regards to trie vs inverted index for Graphite data, I'd actually still be inclined to say inverted index is better based on the amount of queries I saw at Uber with Graphite where people did `servers.*.disk.bytes-used` type queries which is way faster to do using an inverted index since you have a postings list for each part of the dot-separated metric name, rather than traversing a trie with thousands to tens of thousands of entries in index 1 host part

Chronosphere, a startup founded by two of the early M3 engineers, just raised 11 million dollars to build a monitoring platform based around M3DB: https://techcrunch.com/2019/11/05/chronosphere-launches-with... Uber also uses M3DB extensively internally and the project is nowhere near being abandoned or on life support: https://github.com/m3db/m3/commits/master

Some of us (read... one of us, and not me, yet) uses TLA+ when theorizing changes to parts of M3DB, which is a distributed time series database. You can see the specs here, the current TLA+ models the consistency model of data being persisted to disk (flush, snapshot), there was at some point TLA+ for describing the quorum writes/reads along with the background tick event loop but that must be elsewhere now. https://github.com/m3db/m3/tree/master/specs

What's interesting about some of the more modern monitoring systems like M3 and Prometheus is that they have a reverse index on top of the column store entries to very quickly find the relevant metrics for a multi-dimensional query. In fact M3 uses FST index segments, a common Apache Lucene segment which is used by ElasticSearch, for secondary index metric name and dimension full-text search capabilities: https://github.com/m3db/m3/tree/master/src/m3ninx/index/segm...

It is definitely a huge step forward for time series data collection at scale, great props to the authors. With M3TSZ vs vanilla TSZ the major difference is that instead of XORing the value component of each datapoint it is determined if the precision of the float value is within a few significant digits and if so, it is turned into an int representation with delta-of-delta of the int value from value to value used to represent the different changes in value rather than just XORing the float bits. Also tracking how many significant digits and dampening the need to encode a change in that from value to value is also performed. We should write up the specific algorithm differences for the value component encoding. You can see the relative code here in "writeIntVal": https://github.com/m3db/m3/blob/b25e111aab06fb7cde9f7e1be8b6...

We use this https://github.com/m3db/m3/blob/master/src/x/sync/pooled_wor... all over our code base so its definitely stable enough to use in your own projects if you have a need, although it does require some tuning. My guess is that the Go team would not consider this critical / core enough to include in the standard library and I'd be inclined to agree with them.

Fascinating read. Although the idea of using thread pools evokes the pthread management, this post is rather convincing that such "hand-holding" is necessary in applications with intense SLAs. Alas, the magic the Go team has worked with routines doesn't yield a free lunch for everybody. If we accept that pooling is necessary in some cases, I'm curious – is there a common source that these applications use? In trying to answer my own question, I found that M3 has a mature-looking implementation of such an abstract solution. https://github.com/m3db/m3/tree/master/src/x/sync. Elsewhere, I couldn't find anything similar in the usual suspects. CockroachDB has one-off, specific implementations in the places where they've decided pooling is worth it. Looks like Kubernetes uses the stdlib's `sync.Pool` interface in a similar way, but doesn't use

Surprisingly I found other smaller repos render slower (maybe because more was visible), for instance: https://go-city.github.io/#/github.com/m3db/m3