skypilot

Great to see this effort! Couldn't think of a better duo to experiment with it! >with academic groups to help support research that might help make AI more accessible Jeremy, are you already talking with Sky Computing Lab? I recall they had an interesting project about SkyPilot which seemed helpful https://sky.cs.berkeley.edu/ https://github.com/skypilot-org/skypilot

It sounds like you might be wanting something for GPU batch job management. Some things to check out would be gpu orchestration tools, specifically: Slurm, Run.ai, and Skypilot. Or maybe you're kind of wanting a serverless GPU cloud - check out Runpod, Modal, Baseten, and Replicate. Links: https://slurm.schedmd.com/documentation.html https://www.run.ai/ml-workflow-management https://github.com/skypilot-org/skypilot https://www.runpod.io/serverless-gpu https://modal.com/pricing https://www.baseten.co/pricing/ https://replicate.com/pricing

I have been researching methods and projects built around training and running LLMs locally. I'm interested in what others have been using on this front (including straight up Pytorch/Transformers). Here's what I've gathered so far: Engines/APIs: - vllm: Inference and serving engine for LLMs (none quantized models only?) [1] - ollama: Go project to run, create and share LLMs [2] - llama.cpp: Inference of LLaMA models in C/C++ w/UI (including quantized models) [3] - llama-cpp-python: run OpenAI compatible API bindings for llama.cpp [4] - llm-engine: engine for fine-tuning and serving LLMs [5] - Lamini: hosted? closed source? solution for training LLMs [6] - GPT4All: free, locally run chatbot [7] - SkyPilot: framework for running LLMs, AI, and batch jobs [8] - HuggingFace Transformers: APIs and tools to download and train

Since people are suggesting alternatives, I'd like to shoutout skypilot: https://github.com/skypilot-org/skypilot EDIT: looks like this actually uses it under the hood: https://github.com/run-house/runhouse/blob/main/requirements...

Interesting, happy to chat and provide feedback as I have been working in this field for the last few years. Did you get inspiration by any chance from the following paper : https://arxiv.org/pdf/2205.07147.pdf and their recent implementation https://github.com/skypilot-org/skypilot ?

It seems training the Vicuna on custom dataset could be quite easy as well, according to the following: https://github.com/skypilot-org/skypilot/tree/master/llm/vic...

Their mention of SkyPilot isn't an accident, it seems to be a "find me cheap spot instances" project: https://github.com/skypilot-org/skypilot#readme and as best I can tell that's what the yaml files in their repo are for: https://github.com/lm-sys/FastChat/blob/main/scripts/train-v...

We made a simple guide on how to run LLaMA chatbots on any cloud(s) of your choosing using one CLI command, powered by the open-source SkyPilot project. Highlights - cost reduction - better GPU availability - cloud portability (big three + Lambda supported)