NeuralFlow

Know it's a loose analogy. But we really don't know how AI works, really, when we peer into the memory. And earlier today someone posted how to Visualize internals of Mistral 7B. And we don't really know how brain neural net works. And here is a study peering into it. And in both posts, the visual output is very similar. They are both neural nets really. https://news.ycombinator.com/item?id=39378773 https://github.com/valine/NeuralFlow

A few days ago I saw a post using NeuralFlow to help explain the repetition problem. https://old.reddit.com/r/LocalLLaMA/comments/1ap8mxh/what_ca... > I’ve done some investigation into this. In a well trained model, if you plot the intermediate output for the last token in the sequence, you see the values update gradually layer to layer. In a model that produces repeating sequences I almost always see a sudden discontinuity at some specific layer. The residual connections are basically flooding the next layer with a distribution of values outside anything else in the dataset. > The discontinuity is pretty classic overfitting. You’ve both trained a specific token to attend primarily to itself and also incentivized that token to be sampled more often. The result is that if that token is ever included at the end of the context the model is

Yeah I apologize, a lot of the information is scattered across threads right now. I should have spent more time compiling everything in one place. This comment chain in particular might have some of what you’re looking for: https://www.reddit.com/r/LocalLLaMA/comments/1ap8mxh/comment... Other relevant threads to put it all in one place: https://www.reddit.com/r/LocalLLaMA/comments/198x01d/openpir... https://www.reddit.com/r/LocalLLaMA/comments/19a5hdx/morehum... https://www.reddit.com/r/LocalLLaMA/comments/1apz94o/neuralf... https://github.com/valine/NeuralFlow/blob/master/README.md The one thing I don’t talk about is the specifics of the instruction generalization which unfortunately I’m not able to share, even though I very much want to.