big_vision

Google has quite a few of those. It's hard to figure out if they're really official. https://github.com/googleapis/googleapis/discussions/865 / https://github.com/google-research/big_vision / https://github.com/googleapis

There are plenty of such projects with similar disclaimers under Google's own /google/ organization. To give a few examples: https://github.com/google/python-fire / https://github.com/google/pytype/blob/main/docs/index.md / https://github.com/google/dopamine / https://github.com/google/go-tika Also plenty of official Google organizations that are not /google/ , but have official projects. https://github.com/googleapis/googleapis/discussions/865 / https://github.com/google-research/big_vision / you can find plenty more I wouldn't read too much into it, since it can mean virtually anything.

jax is very much a working (and in my view better, aside from the lack of community) software support. Especially if you use their images (which they do). > > Tensorflow They have been using jax/flax/etc rather than tensorflow for a while now. They don't really use pytorch from what I see on the outside from their research works. For instance, they released siglip/siglip2 with flax linen: https://github.com/google-research/big_vision TPUs very much have software support, hence why SSI etc use TPUs. P.S. Google gives their tpus for free at: https://sites.research.google/trc/about/, which I've used for the past 6 months now

That's not how they work. The original GPT-4 paper has some detail: https://cdn.openai.com/papers/gpt-4.pdf Or read up on PaliGemma: https://github.com/google-research/big_vision/blob/main/big_...

I think this is based off Google research https://github.com/google-research/big_vision

The the vision training models are available here: https://github.com/google-research/big_vision/tree/main which I am assuming, based on the research paper is what was used for the project.

An open-source implementation of the Better plain ViT baselines for ImageNet-1k research paper in Google's JAX and Flax. An update from some of the same authors of the original paper proposes simplifications to ViT that allows it to train faster and better. Among these simplifications include 2d sinusoidal positional embedding, global average pooling (no CLS token), no dropout, batch sizes of 1024 rather than 4096, and use of RandAugment and MixUp augmentations. They also show that a simple linear at the end is not significantly worse than the original MLP head. Simple ViT Research Paper: https://arxiv.org/abs/2205.01580 Official Github repository: https://github.com/google-research/big_vision Developer updates can be found on: https://twitter.com/EnricoShippole In collaboration with Dr. Phil 'Lucid' Wang: https://github.com/lucidrains