determined

Use Determined if you want a nice UI https://github.com/determined-ai/determined#readme

Check out Determined (https://github.com/determined-ai/determined). It supports deploying onto k8s and handles running horovod (and soon other dtrain backends), with most of the complexity abstracted behind a few configuration values. Also, it gives you stuff like experiment tracking / hp search (asha) /scheduling / profiling and etc. Disclaimer: I work for Determined.

Check out Determined https://github.com/determined-ai/determined to help manage this kind of work at scale: Determined leverages Horovod under the hood, automatically manages cloud resources and can get you up on spot instances, T4's, etc. and will work on your local cluster as well. Gives you additional features like experiment management, scheduling, profiling, model registry, advanced hyperparameter tuning, etc. Full disclosure: I'm a founder of the project.

Ah I see - I think we're pretty much on the same page in terms of timetables. Although if you include TPU, I think it's fair to say that custom accelerators are already a moderate success. Updated my profile. I've been working on DL training platforms and distributed training benchmarking for a bit so I've gotten a nice view into the GPU/TPU battle. Shameless plug: you should check out the open-source training platform we are building, Determined[1]. One of the goals is to take our hard-earned expertise on training infrastructure and build a tool where people don't need to have that infrastructure expertise. We don't support TPUs, partially because a lack of demand/TPU availability, and partially because our PyTorch TPU experiments were so unimpressive. [1] GH: https://github.com/determined-ai/determined, Slack: https://join.slack.com/t/d

Congrats on the launch! This looks interesting, however I feel like this space is quite crowded. You mentioned that your most important feature is the fact that you are open-source, but off the top of my head I can think of several projects: * Kubeflow: https://github.com/kubeflow/kubeflow * MLFlow: https://github.com/mlflow/mlflow * Pachyderm: https://github.com/pachyderm/pachyderm * DVC: https://github.com/iterative/dvc * Polyaxon: https://github.com/polyaxon/polyaxon * Sacred: https://github.com/IDSIA/sacred * pytorch-lightning + grid: https://github.com/PyTorchLightning/pytorch-lightning * DeterminedAI: https://github.com/determined-ai/determined * Metaflow: https://github.com/Netflix/metaflow * Aim: https://github.com/aimhubio/aim * And so many more... In addition to this list, several other hosted platform offer experiments trackin

Hi William -- we have absolutely not copied any of Lightning's APIs. In fact, our PyTorch API makes some significantly different design choices than Lightning does -- e.g., we require users to step optimizers and run the backward pass explicitly, which is a bit lower-level but allows for more flexibility when using the API. For instance, here is an example of a GAN using our PyTorch API: https://github.com/determined-ai/determined/blob/master/exam... This is a port of this PyTorch Lightning example: https://github.com/PyTorchLightning/pytorch-lightning/blob/m... Despite the former being a port of the latter, there are significant differences between the two APIs. More broadly, we welcome competition in this space and think there's a lot that we can all learn from one another.

Congratulations to the Grid team on the fundraise and the announcement! Exciting stuff. It seems like there is an emerging consensus that (a) DL development requires access to massive compute, but (b) if you’re only using off-the-shelf PyTorch or TensorFlow, moving your model from your personal development environment to a cluster or cloud setting is too difficult — it is easy to spend most of your time managing infrastructure rather than developing models. At Determined AI, we’ve spent the last few years building an open source DL training platform that tries to make that process a lot simpler (https://github.com/determined-ai/determined), but I think it's fair to say that this is still very much an open space and an important problem. Curious to take a look at Grid AI and see how it compares to other tools in the space -- some other alt