HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
ridgerchu
matmulfreellm
Python2024–2024
ridgerchu / matmulfreellm

matmulfreellm

Implementation for MatMul-free LM.

7Distinct people
7Mentions
0Explicit recommenders
3.1KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Implementation for MatMul-free LM.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 1:56:48 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week7 dated mentions

Observed mentions: 2024-06-09 11:00:07 UTC to 2024-09-07 13:21:46 UTC. Weeks start Monday, UTC.

01232024-06-03 00:00:00 UTC to 2024-06-10 00:00:00 UTC (exclusive): 3 mentions; 1 submissions, 2 commentsJun 32024-06-10 00:00:00 UTC to 2024-06-17 00:00:00 UTC (exclusive): 2 mentions; 2 submissions, 0 comments2024-06-17 00:00:00 UTC to 2024-06-24 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-06-24 00:00:00 UTC to 2024-07-01 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2024-07-01 00:00:00 UTC to 2024-07-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-07-08 00:00:00 UTC to 2024-07-15 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-07-15 00:00:00 UTC to 2024-07-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsJul 152024-07-22 00:00:00 UTC to 2024-07-29 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-07-29 00:00:00 UTC to 2024-08-05 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-05 00:00:00 UTC to 2024-08-12 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-12 00:00:00 UTC to 2024-08-19 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-19 00:00:00 UTC to 2024-08-26 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-08-26 00:00:00 UTC to 2024-09-02 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2024-09-02 00:00:00 UTC to 2024-09-09 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsSep 2

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Jun 32024-06-03 00:00:00 UTC2024-06-10 00:00:00 UTC312
Jun 102024-06-10 00:00:00 UTC2024-06-17 00:00:00 UTC220
Jun 172024-06-17 00:00:00 UTC2024-06-24 00:00:00 UTC000
Jun 242024-06-24 00:00:00 UTC2024-07-01 00:00:00 UTC101
Jul 12024-07-01 00:00:00 UTC2024-07-08 00:00:00 UTC000
Jul 82024-07-08 00:00:00 UTC2024-07-15 00:00:00 UTC000
Jul 152024-07-15 00:00:00 UTC2024-07-22 00:00:00 UTC000
Jul 222024-07-22 00:00:00 UTC2024-07-29 00:00:00 UTC000
Jul 292024-07-29 00:00:00 UTC2024-08-05 00:00:00 UTC000
Aug 52024-08-05 00:00:00 UTC2024-08-12 00:00:00 UTC000
Aug 122024-08-12 00:00:00 UTC2024-08-19 00:00:00 UTC000
Aug 192024-08-19 00:00:00 UTC2024-08-26 00:00:00 UTC000
Aug 262024-08-26 00:00:00 UTC2024-09-02 00:00:00 UTC000
Sep 22024-09-02 00:00:00 UTC2024-09-09 00:00:00 UTC101

What Hacker News says

3 sources
The quantization approach is basically identical to the 1.58bit LLM paper: https://arxiv.org/abs/2402.17764 The main addition of the new paper seems to be the implementation of optimized and fused kernels using triton, as seen here: https://github.com/ridgerchu/matmulfreellm/blob/master/mmfre... This is quite useful, as this should make training this type of LLMs much more efficient. So this is a ternary weight LLM using quantization aware training (QAT). The activations are quantized to 8 bits. The matmal is still there, but it is multiplying the 8 bit activations by one bit values. Quantization aware training with low bit weights seems to lead to reduced overfitting by an intrensic tendency to regularize. However, also the model capacity should be reduced compared to a model with the same number of weights and a higher number of bits pe
Read the rest
r weights. It's quite possible that this only becomes apparent after the models have been trained with a significant number of tokens, as LLMs seem to be quite sparse. Edit: In addition to the QAT they also changed the model architecture to use a linear transformer to reduce reliance on multiplications in the attention mechanism. Thanks to logicchains for pointing this out.
In Scalable MatMul-Free Language Modeling

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.