envoy

Envoy has proven itself in the industry and we didn't want to reinvent the wheel by doing what envoy had already done for observability, rate-limits, connection management etc. And reason for using proxy-wasm was so we don't take hard dependency on any built version of envoy. There are many other benefits too which are listed here [1]. Regarding support for wasm runtime in envoy. We believe wasm support in envoy is not going anywhere and it will continue to become more and more stable over time. Envoy has heathy community and in case of any security vulnerability we will hope that envoy will ship fix quite fast which it has done in the past. See here for details of security patch rollout [2] [1] https://github.com/proxy-wasm/spec/blob/main/docs/WebAssembl... [2] https://github.com/envoyproxy/envoy/blob/main/SECURITY.md

Apple also contributed[1] MPTCP support to Envoy Proxy. [1]https://github.com/envoyproxy/envoy/pull/18780

> "This ensures that neither proxy can see both the client IP address and the destination. CONNECT & CONNECT-UDP support chaining of proxies." Google are a major contributor to Envoy Proxy and Envoy is used in their stack [0]. CONNECT-UDP support was recently added to Envoy [1, 2, 3] 0. https://twitter.com/gcpweekly/status/1708883000061055151 1. https://github.com/envoyproxy/envoy/pull/27714 2. https://github.com/envoyproxy/envoy/pull/29557 3. https://github.com/envoyproxy/envoy/pull/29651

Envoy is the proxy that does the heavy lifting. Istio is just a glorified configuration system. Even if you choose to use Istio you're still using Envoy. You're spot-on about using iptables rules. There is an example here with a yaml configuration and some iptables commands: https://github.com/envoyproxy/envoy/blob/main/configs/origin... You might be able to re-use some of that. It should be pretty easy to get metrics for outbound/inbound http requests, but I don't remember the exact yaml incantation.

You’re welcome - https://github.com/envoyproxy/envoy

Vector is going to add an OpenTelemetry source and sink too [0]. The PR isn't merged yet but Envoy Proxy is about to get an OpenTelemetry exporter that is written in C++ [1] 0. https://vector.dev/releases/0.21.0/#whats-next 1. https://github.com/envoyproxy/envoy/pull/20281

mdasen has already pointed out that "2x" seems to overstate the difference by quite a bit. YARP also has a spikier max latency, which I'd expect for a GC'ed language. I don't see a way to choose the benchmarking platform in the PowerBI dashboard, so I assume all these numbers were collected on Windows. In that case, it doesn't surprise me that YARP is faster: Envoy uses libevent for cross-platform I/O, but libevent is relatively slow on Windows. It exposes Windows-native I/O completion ports as a BSD-style socket API, and the implementation is both inherently somewhat slow and pushes buffering concerns into the Envoy code. Kestrel, YARP's underlying webserver, uses libuv instead. libuv is more actively developed (because of Node.js) and takes the opposite approach, exposing an IOCP-like API even on POSIX systems. Basically, YARP's I/O mod

Re. "fat client" libraries: > Sure, it only worked for JVM languages, and it had a programming model that you had to build your whole app around, but the operational features it provided were almost exactly those of the service mesh. The thing is, all of our microservices communicate with each other using Kafka. Envoy has an issue open for Kafka protocol support [1], but it's a fundamentally difficult issue because adopting Kafka forces you to build out "fat client" code and building a network intercept that can work with pre-existing Kafka client code is non-trivial. On observability, Kafka produces its own metrics. Granted, Kafka doesn't offer the same level of control. But Kafka does offer incredible request durability guarantees. We don't have "outages" - we have increased processing latency, and Istio/Envoy and other service meshes c

Plans are to remove RapidJSON from Envoy proxy - https://github.com/envoyproxy/envoy/issues/4705

https://github.com/envoyproxy/envoy/pull/5580 and https://github.com/envoyproxy/envoy/pull/5887 It's a start. Looking forward to more contributions from the team.

https://github.com/envoyproxy/envoy/blob/master/LICENSE Permissions: Commercial use, Modification, Distribution, Patent use, Private use

Envoy doesn't support static file serving: https://github.com/envoyproxy/envoy/issues/378 I'm sure that's not the only feature it's missing compared to nginx. Envoy is not very comparable to nginx, in my opinion... but I also wouldn't reach for varnish as an nginx alternative either.

Let me start by saying: The gotchas that we've experienced have been far eclipsed by the benefits. Aside from what what discussed, there weren't many things with Envoy specifically. We did have a few minor issues with the Thrift tooling, e.g. https://github.com/envoyproxy/envoy/commit/a3c744294bca2cbf6... but as the above indicates, they were resolved _very_ quickly. The most important thing when making a transition like this is to have as much monitoring and observability as possible without the new tech. We were able to quickly identify and respond to issues we had with Envoy based on existing application and system instrumentation that weren't directly provided by Envoy, along with the vigilance of our engineering team.