llama-models

The article talks about the fact that those model are "available to anyone", even Llama is Open Source from that point of view: https://github.com/meta-llama/llama-models Where Llama is not Open Source is in total respect of certain ethical principles: https://opensource.org/blog/metas-llama-license-is-still-not... So, it’snot 100% Open Source inside the context of the article it’s acceptable. Moreover, even DeepSeek is not OpenSource, it’s "OpenWeight"

Yes? I can't use the licensed model to train content? In the same way I can't use BSL to use the software in the cloud or Meta's model license when I can't be a big company. * BSL - https://github.com/clockworklabs/SpacetimeDB/blob/master/LIC... * Meta - https://github.com/meta-llama/llama-models/blob/main/models/...

I'd pump the brakes a bit on the 10M context expectations. Its just another linear attention mechanism with rope scaling [1]. They're doing something similar to what cohere did recently, using a global attention mask and a local chunked attention mask. Notably the max sequence length in training was 256k, but the native short context is still just 8k. I'd expect the retrieval performance to be all over the place here. Looking forward to seeing some topic modeling benchmarks run against it (ill be doing so with some of my local/private datasets). [1] https://github.com/meta-llama/llama-models/blob/eececc27d275... EDIT: should be fair/complete and note they do claim perfect NIAH text retrieval performance across all 10M tokens for the Scout model on their blog post: https://ai.meta.com/blog/llama-4-multimodal-intelligence/. There are some s