logary

n the Span of OpenTracing) site <- Metrics via Prometheus scrape annotation and prom-client api -> Traces to Jaeger Agent via Logary's F# API api -> Metrics to InfluxDB via Logary's F# API api <- Metrics via Prometheus scrape annotation and Logary.Prometheus api -> Events to Mixpanel via Logary.Targets.Mixpanel api -> Logs to Stackdriver via Logary.Targets.Stackdriver (hosted on GCP) Also, Kubernetes ships logs via FluentD to Stackdriver in GCP, but they are not structured, and the remaining infrastructural services also send traces to Jaeger if they can. Logary Rutta is a stand-alone log router, written in Hopac + F# (like Concurrent ML), and used by some of the largest Swedish software companies for thousands of logs and metrics per second. It's capable of shipping to a large number of targets https://github.com/logary/logary/tree/master/src/targets Since it talks HTTP and UDP with a number of encodings (JSON, plain, binary), it's easy to plug into an existing infrastructure and existing log shippers. It can also connect point-to-point to itself with a high-perf binary encoding. Because you can send any JSON into it, it's very easy to get started with together with mobile apps. Logary for JS currently has support for user logs, and I'm currently testing rudimentary metrics and browser info. Logary for .Net supports the OpenTelemetry spec, structured logging and metrics. Of course you can pick any toolchain you want, but I've had great success (and great fun!) writing and using the above. You can see I don't keep logs on disk; it causes them to fill up; if your network is down, your service is down, and then you know it's the network anyway. Once in Logary, you can choose where you send them. I've done an analytics/ETL pipeline based on Logary with its Stackdriver+BigQuery+GooglePubSub targets and with Flink, with great success as well. Logary is free to use for non-profit and then I have a pricing calculator on the home page, for when you start selling the software you build. Pricing aside, how Logary is structured and how I've used it might give you some hints on how to do it yourself.