promscale

I work as a Product Manager at Timescale. :) The data tiering feature will be only available in the Timescale cloud offering, we do not have plans to support this feature in oss/self-hosted at the moment. Thanks for sharing your use-case with TimescaleDB. I'm curious to understand how are you ingesting metrics today into TimescaleDB, do you manage your own schema, retention, compression and downsampling? We at Timescale have built a product named Promscale for easy metrics and traces ingestion with automatic schema-management, compression, retention and downsampling capabilities with full SQL support. Have you tried Promscale (https://github.com/timescale/promscale) for your metrics use-case? To learn more about Promscale join us in #promscale channel in our community slack (https://slack.timescale.com/)

They say: > if you want to have a seamless experience between metrics and traces, then current experience of stitching together Prometheus & Jaeger is not great. But I wonder if using Promscale https://github.com/timescale/promscale would make Prometheus & Jaeger not such a big problem as SigNoz imply. Promscale readme: > Promscale is a unified metric and trace observability backend for Prometheus, Jaeger and OpenTelemetry built on PostgreSQL and TimescaleDB. Either way, SigNoz seems interesting indeed. And am glad to see that SigNoz supports OpenTelemetry.

Hi! So the team is over 100 at this point, but engineering effort is spread across multiple products at this point. The core timescaledb repo [0] currently has 10-15 primary engineers, with a few others working on DB hyperfunctions and our function pipelining [1] in a separate extension [2]. I think generally the set of outside folks who contribute to low-level database internals in C is just smaller than other type of projects. We also have our promscale product [3], which is our observability backend powered by SQL & TimescaleDB. And then there is Timescale Cloud [4], which is obviously a large engineering effort, most of which does not happen in public repos. Interested? We're growing the teams aggressively! Fully remote & global. https://www.timescale.com/careers -- [0] https://github.com/timescale/timescaledb [1] https://www.timesc

At first, let's give the definition of `time series`. This is a series of (timestamp, value) pairs ordered by timestamp. The `value` may contain arbitrary data - a floating-point value, a text, a json, a data structure with many columns, etc. Each time series is uniquely identified by its name plus an optional set of {label="value"} labels. For example, temperature{city="London",country="UK"} or log_stream{host="foobar",datacenter="abc",app="nginx"}. ClickHouse is perfectly optimized for storing and querying of such time series, including metrics. That's true that ClickHouse isn't optimized for handling millions of tiny inserts per second. It prefers infrequent batches with big number of rows per each batch. But this isn't the real problem in practice, because: 1) ClickHouse provides Buffer table engine for frequent inserts. 2) It is easy

(NB: Promscale team member) Thanks for the positive feedback! Is there anything in particular you are missing in Promscale to be used as a backend for multiple Prometheus instances? We added support for multi-tenancy a couple of months ago (https://blog.timescale.com/blog/simplified-prometheus-monito...) And thanks to a community contribution by 2nick on github Promscale can be integrated with Thanos :) (https://github.com/timescale/promscale/pull/664)

There's a big difference -- that's how the PostgreSQL community works. 2ndQuadrant (now EDB), EDB, Citus (now Microsoft) all add value to open source Postgres, contribute back to the community by bringing new features, new life, usecases, and of course committing changes upstream where possible. Timescale is actually on the more open side of that balance, with the licensing and the community version feature matrix. Also, in this case, Timescale actually has a pretty forgiving license[0] as long as you are not a add-nothing-aaS-provider, perhaps more than it should be, which I've asked about before[1]. Even before that change was made, running just the community edition as a add-nothing-aaS-provider would have been an improvement on the status quo, given how soundly it thrashed some other solutions in the past (ex. Influx[2]) and what you

> TimescaleDB is purpose-built for time-series workloads, so that you can get orders of magnitude better performance at a fraction of the cost, along with a much better developer experience. This means massive scale (100s billions of rows and millions of inserts per second on a single server), 94%+ native compression, 10-100x faster queries than PostgreSQL, InfluxDB, Cassandra, and MongoDB – all while maintaining the reliability, ease-of-use, SQL interface, and overall goodness of PostgreSQL. WITH the links to back up their analysis/comparisons. I think I was first sold the hardest on Timescale when I read the InfluxDB post. Timescale also releasing their hard work over the last ~3 years with a very permissive, surprisingly business friendly license[0] which was previously discussed on HN[1]. This is pretty huge -- personally one of the t

TimescaleDB is a long-term storage option for Prometheus metrics, has no problem with high-cardinality, and now natively supports PromQL (in addition to SQL) [0] (Disclaimer: I work at Timescale) [0] https://github.com/timescale/timescale-prometheus

We have quite a few in our Slack channel: slack.timescale.com Feel free to ask over there :) (Btw - TimescaleDB is designed to work with Prometheus. You can see more here: https://github.com/timescale/timescale-prometheus)

There are number of tools, which can be used in front of TimescaleDB or PostgreSQL. There are plugins for Telegraf, Kafka, Prometheus as described in https://docs.timescale.com/latest/using-timescaledb/ingestin... Timescale builds connector and entire workflow to run Prometheus on top of TimescaleDB and support Grafana in flexible way: https://github.com/timescale/timescale-prometheus

I was testing out both TimescaleDB/InfluxDB recently, including maybe using with Prometheus/Grafana. I was leaning towards Timescale, but InfluxDB was indeed a lot easier to quickly boot a "batteries included" setup and start working with live data. I eventually spent a while reading about Timescale 1 vs 2, and testing the pg_prometheus[1] adapter and started thinking through integrating its schema to our other needs then realizing it's "sunsetted" and then reading about the new timescale-prometheus[2] adapter and reading through its ongoing design doc[3] with updated schema that I'm less a fan of. I finally wound up mostly-settling on Timescale although I've put the Prometheus extension question on hold, just pulling in metrics data and outputting with ChartJS and some basic queries got me a lot closer to done for now. Our use case may b

(Timescale-Prometheus team lead here) I'd suggest using Timescale-Prometheus to connect Prometheus with TimescaleDB. Repo: https://github.com/timescale/timescale-prometheus Design Doc: https://tsdb.co/prom-design-doc

You can use Prometheus on top of TimescaleDB. Timescale builds connector and entire workflow to run Prometheus on top of TimescaleDB and support Grafana in flexible way. Sorry for the promo :) check for details in https://github.com/timescale/timescale-prometheus