modeldb

This is like an app store of ml models which is pretty cool. There are a couple of tooling around ML that more of compliments the above than being redundant. -- Open Source -- Mlflow : https://github.com/mlflow/mlflow Polyaxon : https://github.com/polyaxon/polyaxon Modelchimp(mine): https://github.com/ModelChimp/modelchimp ModelDB : https://github.com/mitdbg/modeldb Sacred : https://github.com/IDSIA/sacred -- Non open source -- Cometml : https://www.comet.ml/ Weights and Biases : https://www.wandb.com/ MissinglinkAi : https://missinglink.ai/

The model development pipeline is still a far cry from the maturity of the software development pipeline, and will have to get there in order to reduce the hands-on heavy lifting required for model development and deployment. Along with packaging a trained model are things like: Snapshot/versioning of the training and test data used to create the model, versioning of the model, storing versioned models in a model registry, auto-deploying models from the registry to target environments, telemetry from deployed models. Closest I've found is https://github.com/mitdbg/modeldb, and I've spoken to the woman leading the effort. They still have data versioning as an open question, and don't see the need. But there are training set modification, results RCA, and other use cases that drive the need to catalog training/test data with the model that