LLaMA-Adapter

I'm pretty confident that the landscape is going to look very different by the end of the year, as there are so many people poking around now. I think that significantly smaller models will definitely be good enough for specialized tasks, but an equivalently tuned larger model will always be better, the question is by how much. On Meta's benchmarks [1], there's only a tiny gap between 30B and 65B, for example. For 65B, GPTQ 4-bit should fit LLaMA 65B into 40GiB of memory. Currently the cheapest way to run that at an acceptable speed would be to use 2 x RTX 3090/4090s (~$2500-3000) or maybe a Jetson Orin 64GB (~$2000). I've seen people trying to run it on an M1 Max and it's just a bit too slow to comfortably use (I get a similar speed to when I try it on my 5950X - about 1-2 tokens/s), but it seems like it's within a factor or two of being

There's a difference between buzzwords and jargon. Buzzwords can start out as jargon, but have their technical meaning stripped by users who are just trying to sound persuasive. Examples include words like synergy, vertical, dynamic, cyber strategy, and NFT. That's not what's happening in the parent comment. They're talking about projects like https://github.com/ZrrSkywalker/LLaMA-Adapter https://github.com/microsoft/LoRA https://github.com/tloen/alpaca-lora and specifically the paper: https://arxiv.org/pdf/2106.09685.pdf Lora is just a way to re-train a network for less effort. Before we had to fiddle with all the weights, but with Lora we're only touching 1 in every 10,000 weights. The parent comment says GPT4all doesn't give us a way to train the full size Llama model using the new lora technique. We'll have to build that ourselves. Bu

Keep an eye on these guys: https://github.com/ZrrSkywalker/LLaMA-Adapter/issues/2