JOjosevalim·Jun 15, 2015
I was not the who asked but thanks for the reply jerf, indeed I thought you had in mind more complex cases than the GenServer one! It is not that sync is hard in Erlang/Elixir but once you want to involve multiple processes, it may be the case.
For example, in pipeline parallelism, we don't want to be completely sync between the stages, because it would mean you would always have at most N "events" in the pipeline at any moment (where N is the number of stages). If you want to have some sort of bound, you need to devise a message protocol between the stages. That's what the folks behind Reactive Streams are doing with their back-pressure work.
This is exactly the kind of situations I want to make easier in future Elixir versions. I talked about them from the perspective of collections (and not FBP) in my Elixirconf keynote: https://youtu.
Read the rest
be/EaP0y4pdKD0?t=1846. We are experimenting with the addition of a GenRouter that would specify how processes communicate (with back-pressure) eventually supporting custom dispatch rules. We are also looking into projects like Basho's sidejob (https://github.com/basho/sidejob) and Ulf's jobs (https://github.com/uwiger/jobs) to ensure we can also provide sampling based routing.
On top of that, we need to discuss data-based parallelism (where we split/spread the data once instead of passing it around) and how it would all fit with supervision trees. It is a lot of work and we likely won't be able to solve all cases but I am very excited about the possibilities if we get it right.