dm-haiku

Is t5x an encoder/decoder architecture? Some more general options. The Flax ecosystem https://github.com/google/flax?tab=readme-ov-file or dm-haiku https://github.com/google-deepmind/dm-haiku were some of the best developed communities in the Jax AI field Perhaps the “trax” repo? https://github.com/google/trax Some HF examples https://github.com/huggingface/transformers/tree/main/exampl... Sadly it seems much of the work is proprietary these days, but one example could be Grok-1, if you customize the details. https://github.com/xai-org/grok-1/blob/main/run.py

As a researcher in RL & ML in a big industry lab, I would say most of my colleagues are moving to JAX [https://github.com/google/jax], which this article kind of ignores. JAX is XLA-accelerated NumPy, it's cool beyond just machine learning, but only provides low-level linear algebra abstractions. However you can put something like Haiku [https://github.com/deepmind/dm-haiku] or Flax [https://github.com/google/flax] on top of it and get what the cool kids are using :)

Hello from the JAX team! We'd like to take this opportunity to give a shout out to some of the awesome projects folks are building on top of JAX, e.g., * Flax, a neural network library for JAX (https://github.com/google/flax) * Haiku, a neural network library for JAX inspired by Sonnet (https://github.com/deepmind/dm-haiku) * RLax, a library for building reinforcement learning agents (https://github.com/deepmind/rlax) * NumPyro, a probabilistic programming library on top of JAX (https://github.com/pyro-ppl/numpyro) * JAX-MD, a differentiable molecular dynamics package built on top of JAX (https://github.com/google/jax-md)