pg_prometheus

I was testing out both TimescaleDB/InfluxDB recently, including maybe using with Prometheus/Grafana. I was leaning towards Timescale, but InfluxDB was indeed a lot easier to quickly boot a "batteries included" setup and start working with live data. I eventually spent a while reading about Timescale 1 vs 2, and testing the pg_prometheus[1] adapter and started thinking through integrating its schema to our other needs then realizing it's "sunsetted" and then reading about the new timescale-prometheus[2] adapter and reading through its ongoing design doc[3] with updated schema that I'm less a fan of. I finally wound up mostly-settling on Timescale although I've put the Prometheus extension question on hold, just pulling in metrics data and outputting with ChartJS and some basic queries got me a lot closer to done for now. Our use case may b

This is not the case. TimescaleDB has full support for storing JSON data (inherited from postgres), including with indexes, so you do not need to fully pre-define your schema for these types of applications. In fact, we added support for TimescaleDB to be a read/write backend for long-term Prometheus metrics. You pull from Prometheus via its remote storage backend, and the data appears automatically in TimescaleDB. But then unlike Influx and native Prometheus, you get to JOIN it against additional metadata for richer questions. For more information: https://blog.timescale.com/sql-nosql-data-storage-for-promet... https://github.com/timescale/pg_prometheus

At a high-level, we've taken a unique approach to the time-series data problem. While every other time-series db has taken a non-relational approach (which also means no or poor SQL support), we've built ours on Postgres. As a result, not only do we support full SQL (JOINs, complex WHEREs/GROUPBYs, window functions, etc), but we also inherit the reliability and vast ecosystem of Postgres. At a more detailed level, comparisons between TimescaleDB and others varies depending on the database. Typically though, if you are most concerned with data compression or simple column scans, some of the other options might be more suitable than ours. But otherwise you should see similar (if not better) insert and query performance, while maintaining all the other benefits of a relational DB / Postgres. And with your relational data living alongside tim

There's already a two-way integration between Prometheus and TimescaleDB: https://github.com/timescale/pg_prometheus

A common approach we find for storing devops-related time-series data (as you'd find with influx), is to not denormalize the "labels" or "tag set" into the main metrics table. This obviously saves significant space just by avoiding denormalization, ignoring the indexing overhead as well. You can see that in our Prometheus extension, btw: https://github.com/timescale/pg_prometheus#user-content-norm... Regarding performance against Influx, it really depends. We're working on releasing more complex benchmarks soon. But overall, they have a performance edge if it's a single column scan that precisely matches their particular architecture (e.g., WHERE clause or GROUP BY by a very specific label set). But, we've found that Timescale actually gets higher query performance across a whole set of pretty natural queries (sometimes ridiculously so,

Well...sorta. Postgres/Timescale have pretty rich support for JSON these days (and its more efficient "binary" JSONB), so there are a whole range of options you can do that make it feel much more schema-less than before. In fact, last month we released a beta version of a Prometheus connector for Timescale/Postgres that allows you to store arbitrary Prometheus metrics without pre-defining all these varied schemas: https://github.com/timescale/pg_prometheus The same approach should work for Telegraf, we just haven't yet tried to generalize this plugin.

If the viz tool can speak to Postgres, it works with Timescale. Tableau, Superset, SQLPad, Mode, Plot.ly, etc. We internally use Grafana through a REST interface to a timescale backend. (And in fact, that's how we visualize the Prometheus data we store in Timescale: https://github.com/timescale/pg_prometheus ) But, Grafana Labs is still working on a native Postgres connector (MySQL released earlier this year). They promise us soon :)