mesh

It seems the main complaint is about confusing shapes / dimensions. Xarray has already be mentioned, but this is a broader concept, called named dimensions, sometimes also named tensors, named axes, labeled tensors, etc, which has often been proposed before, and many implementations exists. https://nlp.seas.harvard.edu/NamedTensor https://namedtensor.github.io/ https://docs.pytorch.org/docs/stable/named_tensor.html https://github.com/JuliaArrays/AxisArrays.jl https://github.com/ofnote/tsalib https://github.com/google-deepmind/tensor_annotations https://github.com/google-deepmind/penzai https://github.com/tensorflow/mesh/ https://github.com/facebookresearch/torchdim https://xarray.dev/ https://returnn.readthedocs.io/en/latest/getting_started/dat... (that's my own development; unfortunately the doc is very much outdated on that)

Mesh-TensorFlow (https://github.com/tensorflow/mesh) solves the problem of networks being too large to fit into a single GPU's memory. I just trained a network with > 1 billion parameters across 4 GPUs. Beside size of data set and compute power, representational capacity (size of embeddings) is a key blocker to learning, and so this library opens up new possibilities for many of us.