transport-site

party servers _right in the middle_ of your own HTTP call, is usually perfectly acceptable (something I've done more than a couple of times, the latest one powering the reverse proxy behind https://transport.data.gouv.fr - code available at https://github.com/etalab/transport-site/tree/master/apps/un...). As a consequence, what would be a bad pattern in Python or Ruby (holding the incoming HTTP connection) is not a problem with Elixir. > because I want to have state or queues in front of background work so there's a well-defined process for retry, error handling, and back-pressure. Unless you deal with immediate stuff like reverse proxying or cheap "one off async tasks" (like recording a metric), there also are solutions to have more "stateful" background works in Elixir, too. A popular background job queue is https://github.com/sorentwo/oban (roughly similar to Sidekiq at al), which uses Postgres. It handles retries, errors etc. But it's not the only solution, as you have other tools dedicated to processing, such as Broadway (https://github.com/dashbitco/broadway), which handles back-pressure, fault-tolerance, batching etc natively. You also have more simple options, such as flow (https://github.com/dashbitco/flow), gen_stage (https://github.com/elixir-lang/gen_stage), Task.async_stream (https://hexdocs.pm/elixir/1.12/Task.html#async_stream/5) etc. It allows to use the "right tool for the job" quite easily. It is also interesting to note there is no need to "go evented" if you need to fetch data from multiple HTTP servers: it can happen in the exact same process (even: in a background task attached to your HTTP server), as done here https://transport.data.gouv.fr/explore (if you zoom you will see vehicle moving in realtime, and ~80 data sources are being polled every 10 seconds & broadcasted to the visitors via pubsub & websockets).

.). As a result, it's easy to use LiveView to create a real-time dashboard to show these metrics (https://jumpshare.com/s/rdGtTa47CzlcgOz4jsOI - low traffic at time of posting) or generate stats for the data producers which are behind the proxy (EDIT: LiveView source code at https://github.com/etalab/transport-site/blob/master/apps/tr... and https://github.com/etalab/transport-site/blob/master/apps/tr...). It's all quite natural to do because it's really baked into the runtime!

fied) - use tooling during the tests: "assert_schema" (with OpenAPISpex) to ensure our API endpoints responses pass the spec (additionalProperties: false helps ensure we get an exception in case of key field rot, again!) - even more useful: crawl our production most important endpoints and tweak the spec until everything is green (an example of useful use of Task.async_stream in Elixir, by the way) https://github.com/etalab/transport-site/pull/3351/files#dif... It can be super frustrating for users to live with the uncertainty of the response of an API for sure, and I was happy to discover the Elixir tooling (OpenAPISpex in particular) worked so nicely once I understood what I had to do.

ob/master/scripts... - exploring a new tool https://github.com/etalab/transport-site/blob/master/scripts... - gradually iterating to create scripts to generate XML queries https://github.com/etalab/transport-site/tree/master/scripts... (later incorporated in the application) - comparing GTFS (transportation data) files via scripting https://github.com/etalab/transport-site/blob/master/scripts... - learning how to compute the checksum of the internal content of a zip file https://github.com/etalab/transport-site/blob/master/scripts... As mentioned in the article, I can definitely recommend to check out https://github.com/wojtekmach/mix_install_examples which has a long list of examples. One last tip is that you can also use "mix run script.exs" on a file not using "Mix.install", in order to rely on the same dependencies & configuration as the main application (e.g. to run a Ecto query https://github.com/etalab/transport-site/blob/master/scripts...).

tc). - Phoenix, Ecto & Elixir are a good "generalist web framework" for a lot of use - Compared to some stacks (non-evented Ruby & Python, for instance), having a part of your app work as a proxy for domain-specific uses, is not a problem, and does not even require you to split that part from the main app (example at https://github.com/etalab/transport-site/tree/master/apps/un..., where we proxy/cache GTFS-RT protobuf & SIRI XML feeds behind rules, adding rate-limiting & hiding target server authentication schemes) - Similarly, anything real-time is at least initially trivial. For instance, creating this real-time view of moving vehicles (https://transport.data.gouv.fr/explore) with 70+ GTFS-RT feeds has been quite easy (PR at https://github.com/etalab/transport-site/pull/2224). The server side is like "EventMachine done right", and the live updates are broadcasted to each client via web socket. No extra server (e.g. AnyCable) or anything is needed. From a perf/ops POV, the amount of topics you do not even have to think about is quite important, which is very nice. Overall I can see myself sticking to this stack for the next 15 years or so, without much worries (I've been coding since 1984 as a kid, so I have a good idea to know when to keep a framework around or not :P)

iew. If I need to demonstrate some simple stuff, I can use "Mix.install" to create one-off scripts. If I need to do more data-science, I can tap into LiveBook, VegaLite etc. All this with a small team or as a solo developer, is quite great, much more than raw performance, in my eyes.