sdnext

Disty of SD.Next has made a version in diffusers format. https://huggingface.co/Disty0/Index-anisora-5B-diffusers For the record, the dev branch of SD.Next (https://github.com/vladmandic/sdnext) already supports it.

Been running Linux on the A770 for about 2 years now. Very happy with the driver situation. Was a bit rough very early on, but it's nice and stable now. Recommend at least Linux 6.4, but preferably newer. I use a rolling release distro(Artix) to get up to date kernels. ML stuff can be a pain sometimes because support in pytorch and various other libraries is not as prioritised as CUDA. But I've been able to get llama.cpp working via ollama, which has experimental intel gpu support. Worked fine when I tested it, though I haven't actually used it very much, so don't quote me on it. For image gen, your best bet is to use sdnext(https://github.com/vladmandic/sdnext), which supports Intel on linux officially, and will automagically install the right pytorch version, and do a bunch of trickery to get libraries that insist on CUDA to work in man

> it won't ever be a viable option For production workloads, I generally agree. It's an unsupported hack with a questionable future, I wouldn't do anything money-making with it. However, for tinkering and consumer workloads, it already works pretty well. Enough of cuDNN and cuBLAS work to run PyTorch and in turn, Stable Diffusion with https://github.com/lshqqytiger/ZLUDA - there's even a fairly user-friendly setup process already in https://github.com/vladmandic/automatic . I was able to get a personal non-ML related project working on my AMD card in just a few minutes, which saved me a lot of development time before I then deployed the production workload on NV hardware (this is probably why AMD pulled the plug on the project - it's almost more of a boost to NV than anything else, AMD really need people to be writing code on ROCm to depl

While I currently use SD.Next[1], I have tested ComfyUI locally with my AMD card. The UI can be daunting, but you learn quite a great deal about how a Stable Diffusion pipeline works. In addition some innovations and advances find their way into ComfyUI first. [1] https://github.com/vladmandic/automatic

for people with Intel video cards (all 10 of us!) there's also SD.Next (automatic1111 fork): https://github.com/vladmandic/automatic

Koboldcpp [1], which builds on llamacpp and adds a gui, is a great way to run these models. Most people aren't running these models at full weight, ggml quantization is recommended for cpu+gpu or gptq if you have the gpu vram. GGML 13b models at 4bit (Q4_0) take somewhere around 9gb of ram and q5_K_M take about 11gb. Gpu offloading support has also been added, I've been using 22 layers on my laptop rtx 2070 max q 8gb vram and CLBlast. I get around ~2-3 tokens per second with 13b models. In my experience, running 13b models is worth the extra time it takes to generate a response compared to 7b models. GPTQ is faster, I think, but I can't fit a quantized 13b model so I don't use it. TheBloke [2] has been quantizing models and uploading them to HF and will probably upload a quantized version of this online soon. His discord server also has g