alpaca.cpp

Where's the 30B-in-6GB claim? ^FGB in your GH link finds [0] which is neither by jart nor by ggerganov but by another user who promptly gets told to look at [1] where Justine denies that claim. [0] https://github.com/antimatter15/alpaca.cpp/issues/182 [1] https://news.ycombinator.com/item?id=35400066

maybe https://huggingface.co/chavinlo/Alpaca-65B/tree/main via https://github.com/antimatter15/alpaca.cpp/issues/124#issuec... ? (I have not tried this yet.)

Yeah, this is weird, because there are plenty of open source implementations of the lama model on github - alpaca.cpp in c++ is one, there are many others in PyTorch such as the one used by ChatLLaMa. But without the weights they're not very useful (unless you're going to try to train it yourself - good luck with that unless you've got a lot of compute power available). A quick check on github and I find this one also with an Apache license: https://github.com/chris-alexiuk/alpaca-lora and alpaca.cpp with an MIT license: https://github.com/antimatter15/alpaca.cpp/blob/master/LICEN...

Also check out Alpaca; you can self-host this one, the 7B and 13B variants produce surprisingly good results and are fast enough just running on CPU: https://github.com/antimatter15/alpaca.cpp

Be aware this file is a single ~8GB 4-bit model (ggml-alpaca-13b-q4.bin) instead of the 2x ~4GB models (ggml-model-q4_0.bin, ggml-model-q4_0.bin.1) that most llama.cpp style inference running programs expect. You'll probably have to edit the line, n_parts = LLAMA_N_PARTS.at(hparams.n_embd); in chat.cpp (or main.cpp) to hard code it to treat this 1 file model properly like, n_parts = 1; Or re-write the parameter config subroutine to recognize and handle non-standard weights file. magnet: magnet:?xt=urn:btih:053b3d54d2e77ff020ebddf51dad681f2a651071&dn=ggml-alpaca-13b-q4.bin&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A6969%2Fannounce&tr=udp%3A%2F%2F9.rarbg.com%3A2810%2Fannounce torrent: https://btcache.me/torrent/053B3D54D

Alpaca [1], perhaps. It's based on facebook's model (LLaMA) and its been trained on a conversational style, same as chat gpt. I don't know if it can produce code, though. [1] https://github.com/antimatter15/alpaca.cpp

Interesting issue in that repo.. https://github.com/antimatter15/alpaca.cpp/issues/23

https://github.com/antimatter15/alpaca.cpp has links