HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
valine
NeuralFlow
Python2024–2024
valine / NeuralFlow

NeuralFlow

Visualize the intermediate output of Mistral 7B

3Distinct people
4Mentions
0Explicit recommenders
398GitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Visualize the intermediate output of Mistral 7B

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 6:28:55 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by hour4 dated mentions

Observed mentions: 2024-02-15 03:29:20 UTC to 2024-02-15 21:51:03 UTC.

012024-02-15 03:00:00 UTC to 2024-02-15 04:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsFeb 15, 03:002024-02-15 04:00:00 UTC to 2024-02-15 05:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 05:00:00 UTC to 2024-02-15 06:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 06:00:00 UTC to 2024-02-15 07:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 07:00:00 UTC to 2024-02-15 08:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 08:00:00 UTC to 2024-02-15 09:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 09:00:00 UTC to 2024-02-15 10:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 10:00:00 UTC to 2024-02-15 11:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 11:00:00 UTC to 2024-02-15 12:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 12:00:00 UTC to 2024-02-15 13:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsFeb 15, 12:002024-02-15 13:00:00 UTC to 2024-02-15 14:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 14:00:00 UTC to 2024-02-15 15:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-02-15 15:00:00 UTC to 2024-02-15 16:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 16:00:00 UTC to 2024-02-15 17:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 17:00:00 UTC to 2024-02-15 18:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-02-15 18:00:00 UTC to 2024-02-15 19:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 19:00:00 UTC to 2024-02-15 20:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 20:00:00 UTC to 2024-02-15 21:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-02-15 21:00:00 UTC to 2024-02-15 22:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsFeb 15, 21:00

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Feb 15, 03:002024-02-15 03:00:00 UTC2024-02-15 04:00:00 UTC110
Feb 15, 04:002024-02-15 04:00:00 UTC2024-02-15 05:00:00 UTC000
Feb 15, 05:002024-02-15 05:00:00 UTC2024-02-15 06:00:00 UTC000
Feb 15, 06:002024-02-15 06:00:00 UTC2024-02-15 07:00:00 UTC000
Feb 15, 07:002024-02-15 07:00:00 UTC2024-02-15 08:00:00 UTC000
Feb 15, 08:002024-02-15 08:00:00 UTC2024-02-15 09:00:00 UTC000
Feb 15, 09:002024-02-15 09:00:00 UTC2024-02-15 10:00:00 UTC000
Feb 15, 10:002024-02-15 10:00:00 UTC2024-02-15 11:00:00 UTC000
Feb 15, 11:002024-02-15 11:00:00 UTC2024-02-15 12:00:00 UTC000
Feb 15, 12:002024-02-15 12:00:00 UTC2024-02-15 13:00:00 UTC000
Feb 15, 13:002024-02-15 13:00:00 UTC2024-02-15 14:00:00 UTC000
Feb 15, 14:002024-02-15 14:00:00 UTC2024-02-15 15:00:00 UTC101
Feb 15, 15:002024-02-15 15:00:00 UTC2024-02-15 16:00:00 UTC000
Feb 15, 16:002024-02-15 16:00:00 UTC2024-02-15 17:00:00 UTC000
Feb 15, 17:002024-02-15 17:00:00 UTC2024-02-15 18:00:00 UTC101
Feb 15, 18:002024-02-15 18:00:00 UTC2024-02-15 19:00:00 UTC000
Feb 15, 19:002024-02-15 19:00:00 UTC2024-02-15 20:00:00 UTC000
Feb 15, 20:002024-02-15 20:00:00 UTC2024-02-15 21:00:00 UTC000
Feb 15, 21:002024-02-15 21:00:00 UTC2024-02-15 22:00:00 UTC101

What Hacker News says

1 sources
A few days ago I saw a post using NeuralFlow to help explain the repetition problem. https://old.reddit.com/r/LocalLLaMA/comments/1ap8mxh/what_ca... > I’ve done some investigation into this. In a well trained model, if you plot the intermediate output for the last token in the sequence, you see the values update gradually layer to layer. In a model that produces repeating sequences I almost always see a sudden discontinuity at some specific layer. The residual connections are basically flooding the next layer with a distribution of values outside anything else in the dataset. > The discontinuity is pretty classic overfitting. You’ve both trained a specific token to attend primarily to itself and also incentivized that token to be sampled more often. The result is that if that token is ever included at the end of the context the model is
Read the rest
incentivized to repeat it again. ... > Literally just plotting the output of the layer normalized between zero and one. For one token in mistral 7B it’s a 4096 dimension tensor. Because of the residual connections if you plot that graph for every layer you get a really nice visualization. > Edit: Here's my visualization. It’s a simple idea but I've never personally seen it done before. AFAIK this is a somewhat novel way to look at transformer layer output. > Initial output: https://imgur.com/sMwEFEw > Over-fit output: https://imgur.com/a0obyUj > Second edit: Code to generate the visualization: https://github.com/valine/NeuralFlow This is nearly identical to the overfitting example in the repo, only really representing a binary, but it's a good start. Perhaps some transformations can be applied to help further?
In Show HN: NeuralFlow – Visualize the intermediate output of Mistral 7B

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.