ray

> "Forget about managing complex server infrastructure for your database needs." So what does this run on then? No docs, it's not possible to find any deployment guides for Ray using serverless solutions like Lambda, Cloud Functions or be it your own Firecracker. Instead, every other post mentions EKS or EC2. The Ray team even rejected Lambda support expressedly as far back as 2020 [0]. Uuuuuugh. No thanks! shiver I'd rather cut complexity for practically the same benefit and either do it single machine or have a thin, manageable layer on top a truly serverless infra like in this talk [1] " Processing Trillions of Records at Okta with Mini Serverless Databases". 0: https://github.com/ray-project/ray/issues/9983 1: https://www.youtube.com/watch?v=TrmJilG4GXk

Not currently, but it is being worked on https://github.com/ray-project/ray/issues/53976.

There's a couple of alternatives to exo it seems https://github.com/b4rtaz/distributed-llama and https://github.com/ray-project/ray

I'm guessing this comment is some kind of "if you know, you know." Likely starting from https://docs.ray.io/en/latest/cluster/vms/user-guides/launch... and then trawling through one of these I guess https://github.com/ray-project/ray/issues?q=is%3Aissue+prem+...

Yeah, mmap, I think this is the relevant line [1]. Fun fact, very early on, we used to create one mmapped file per serialized object, but that very quickly broke down. Then we switched to mmapping one large file at the start and storing all of the serialized objects in that file. But then as objects get allocated and deallocated, you need to manage the memory inside of that mmapped file, and we just repurposed a malloc implementation to handle that. [1] https://github.com/ray-project/ray/blob/21202f6ddc3ceaf74fbc...

> And lastly, part three of the dream: it would be great if accessing storage from our services or batch jobs were as easy as accessing in-memory data. I'm a believer! The analytics/columnar store folk are well on the way. Voltron for instance, > Voltron Data is building open source standards that support a multi-language language future. Central to this vision is the Apache Arrow project. https://voltrondata.com/blog/zero-copy-sharing-using-apache-... Being able to send data around between different data processing systems is definitely shifting from specially tuned narrow systems to a core piece of infrastructure. First for analytics/columnar loads. Eventually more. There's other pieces ideally slotting into place. Dma-p2p between/across nvme, gpu's, and systems is the target. Is what the object/data stores need, increasingly, to be br

[Author] OpenAI uses Ray, the open source version of Anyscale Platform, for training their models (video evidence: https://www.youtube.com/watch?v=CqiL5QQnN64). Ray (https://github.com/ray-project/ray) is available for anyone to use. You can also use Aviary (https://github.com/ray-project/aviary) to serve any of those models yourself.

Training times for GSM8k are mentioned here: https://github.com/ray-project/ray/tree/master/doc/source/te...

I've wondered whether it's easier to add data analyst stuff to Elixir that Python seems to have, or add features to Python that Erlang (and by extension Elixir) provides out of the box. By what I can see, if you want multiprocessing on Python in an easier way (let's say running async), you have to use something like ray core[0], then if you want multiple machines you need redis(?). Elixir/Erlang supports this out of the box. Explorer[1] is an interesting approach, where it uses Rust via Rustler (Elixir library to call Rust code) and uses Polars as its dataframe library. I think Rustler needs to be reworked for this usecase, as it can be slow to return data. I made initial improvements which drastically improves encoding (https://github.com/elixir-nx/explorer/pull/282 and https://github.com/elixir-nx/explorer/pull/286, tldr 20+ seconds dow

Python has actually had concurrency since about 2019: https://docs.python.org/3/library/asyncio.html. Having used it a few times, it seems fairly sane, but tbf my experience with concurrency in other languages is fairly limited. edit: ray https://github.com/ray-project/ray is also pretty easy to use and powerful for actual parallelism