tch-rs

Interesting. The repo was owned by Laurent Mazare[0] a few days ago, I'm wondering if HuggingFace just hired them to work on candle full time? [0]: Also maintainer of https://github.com/LaurentMazare/tch-rs

Very nice! I wanted to do something like this but then I would miss on proper CUDA acceleration and lose performance compared to using torchlib. I wrote a forgettable llama implementation for https://github.com/LaurentMazare/tch-rs (pytorch's torchlib rust binding). Still not ideal but at least you get the same GPU performance you would get on pytorch. ...And then I spotted Candle, a new ML framework by the same author: https://github.com/huggingface/candle It's all in Rust, self contained, a huge undertaking, but it looks very promising. They already have a llama2 example!

I first encountered OCaml's PyTorch bindings because apparently they generate a C wrapper around PyTorch's C++ API, and Rust's PyTorch bindings use OCaml's C wrapper. See: https://github.com/LaurentMazare/tch-rs

If you pefer to see it in code there's a succint gpt implementation here https://github.com/LaurentMazare/tch-rs/blob/main/examples/m...

PyTorch has libtorch as its purely native library. There are also Rust bindings for libtorch: https://github.com/LaurentMazare/tch-rs I used this in the past to make a transformer-based syntax annotator. Fully in Rust, no Python required: https://github.com/tensordot/syntaxdot

This is an implementation of Stable Diffusion in Rust using libtorch bindings - including some nice samples of rusty robots! It follows the lines of Huggingface's (amazing) diffusers library. The main goal is to show how a complex model can be converted and re-used on the Rust side.

> How does it do this? Just a list of known packages and their dependencies, or? It checks a known registry or reads package metadata. > I'm using https://github.com/LaurentMazare/tch-rs/ (rust bindings for pytorch's c++ api). Depending on the hardware I'm building for I either need to download a cpu build of pytorch, or a cuda build of pytorch. How would this work with riff? Riff doesn't track which GPU you use at this time, so it can't track that which build of `pytorch` to use, sorry. It might be something we make it aware of if we have enough evidence it would be helpful. > Is there any hope of a good cross compiling story? If so, that would be awesome (currently I just don't even try if there are significant C dependencies)! Regarding cross compiling: We've spoken about it, and I certainly believe it would be desirable. I imagine it

> While Riff does its best to infer external dependencies from your project's crate dependencies How does it do this? Just a list of known packages and their dependencies, or? I'm using https://github.com/LaurentMazare/tch-rs/ (rust bindings for pytorch's c++ api). Depending on the hardware I'm building for I either need to download a cpu build of pytorch, or a cuda build of pytorch. How would this work with riff? Is there any hope of a good cross compiling story? If so, that would be awesome (currently I just don't even try if there are significant C dependencies)! How thoroughly are dependencies cached? Backed to the pytorch example it's a fairly big (and thus slow) download, I don't want that happening too frequently. --- In general this looks like something that could be really useful. Using projects with significant C dependencies cu

As a starting point, maybe take a look at the 'tch' crate [1] and the 'rust-bert' crate [2] built on top of it? [1]: https://github.com/LaurentMazare/tch-rs -> https://crates.io/crates/tch [2]: https://github.com/guillaume-be/rust-bert -> https://crates.io/crates/rust-bert

I suppose you could, but I believe that there are direct bindings from Rust to Pytorch at https://github.com/LaurentMazare/tch-rs. I haven't used it personally, but I've heard fairly good things about it's correctness and responsive maintainer.

You can use PyTorch Rust bindings [1] or OCaml bindings [2]. [1] https://github.com/LaurentMazare/tch-rs [2] https://github.com/LaurentMazare/ocaml-torch