kalavai-client

OK, it may take you a couple more minutes than that, but it really is quick. And open source. I'm struggling to get people to use the tool, and I'd love feedback to get the features right! The accompanying written guide for the video is here: https://kalavai-net.github.io/kalavai-client/self_hosted_llm... This is the repo: https://github.com/kalavai-net/kalavai-client Thank you HN folks!

Alright, I know, the title is on the "click bait" side. But hear me out. Researchers in the UK rely on High Performance Computing systems to do their work. If you combine the top 8 HPCs in the country, they amount to 1,000 GPUs altogether. For context Meta alone used 24,000 just to train one of its models (LLaMa). To make things worse, those HPCs are geting to the end of their life without plans for renewal. With the growing size of AI models (both in vision and language) and the slow pace of hardware improvement, this is only going to get worse. Meta, xAI, OpenAI know this and they are hoarding resources (hundreds of thousands). How are researchers, independent developers and AI/MML hobbyists going to compete in the space? Do we just submit to our techno-feudal lords and pay rent to access crumbs? Is it over? Does anyone else care about

I have a favour to ask. I’ve been working for a while on Kalavai, a project to make distributed AI easy. There are brilliant tools out there to help AI hobbyists and devs on the software layer (shout out to vLLM and llamacpp amongst many others!) but it’s a jungle out there when it comes to procuring and managing the necessary hardware resources and orchestrating them. This has always led me to compromise on the size of the models I end up using (quantized versions, smaller models) to save cost or to play within the limits of my rig. Today I am happy to share the first public version of our Kalavai client (totally free, forever), a CLI that helps you build an AI cluster from your everyday devices. Our first use case is distributed LLM deployment, and we hope to expand this with the help of the community. Now, the favour! I’d love for peop

I’ve been working for a while on Kalavai (https://kalavai.net), a project to make distributed AI easy without a devops certification. There are brilliant tools out there to help AI hobbyists and devs on the software layer (shout out to vLLM and llamacpp amongst many others!) but it’s a jungle out there when it comes to procuring and managing the necessary hardware resources and orchestrating them. This has always led me to compromise on the size of the models I end up using (quantized versions, smaller models) to save cost or to play within the limits of my rig. Today I am happy to share the first public version of our Kalavai client (totally free, forever), a CLI that helps you build an AI cluster from your everyday devices. https://github.com/kalavai-net/kalavai-client Our first use case is distributed LLM deployment, and we hope to ex