HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
qwopqwop200
GPTQ-for-LLaMa
Python2023–2023
qwopqwop200 / GPTQ-for-LLaMa

GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

15Distinct people
20Mentions
0Explicit recommenders
3.1KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

4 bits quantization of LLaMA using GPTQ

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 7:57:46 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week20 dated mentions

Observed mentions: 2023-03-10 21:44:13 UTC to 2023-07-11 06:17:01 UTC. Weeks start Monday, UTC.

02462023-03-06 00:00:00 UTC to 2023-03-13 00:00:00 UTC (exclusive): 5 mentions; 0 submissions, 5 commentsMar 62023-03-13 00:00:00 UTC to 2023-03-20 00:00:00 UTC (exclusive): 6 mentions; 0 submissions, 6 comments2023-03-20 00:00:00 UTC to 2023-03-27 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2023-03-27 00:00:00 UTC to 2023-04-03 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2023-04-03 00:00:00 UTC to 2023-04-10 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-04-10 00:00:00 UTC to 2023-04-17 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-04-17 00:00:00 UTC to 2023-04-24 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-04-24 00:00:00 UTC to 2023-05-01 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-01 00:00:00 UTC to 2023-05-08 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-08 00:00:00 UTC to 2023-05-15 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 commentsMay 82023-05-15 00:00:00 UTC to 2023-05-22 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-05-22 00:00:00 UTC to 2023-05-29 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments2023-05-29 00:00:00 UTC to 2023-06-05 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-06-05 00:00:00 UTC to 2023-06-12 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 comments2023-06-12 00:00:00 UTC to 2023-06-19 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-06-19 00:00:00 UTC to 2023-06-26 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-06-26 00:00:00 UTC to 2023-07-03 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-03 00:00:00 UTC to 2023-07-10 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2023-07-10 00:00:00 UTC to 2023-07-17 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsJul 10

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Mar 62023-03-06 00:00:00 UTC2023-03-13 00:00:00 UTC505
Mar 132023-03-13 00:00:00 UTC2023-03-20 00:00:00 UTC606
Mar 202023-03-20 00:00:00 UTC2023-03-27 00:00:00 UTC202
Mar 272023-03-27 00:00:00 UTC2023-04-03 00:00:00 UTC303
Apr 32023-04-03 00:00:00 UTC2023-04-10 00:00:00 UTC000
Apr 102023-04-10 00:00:00 UTC2023-04-17 00:00:00 UTC000
Apr 172023-04-17 00:00:00 UTC2023-04-24 00:00:00 UTC000
Apr 242023-04-24 00:00:00 UTC2023-05-01 00:00:00 UTC000
May 12023-05-01 00:00:00 UTC2023-05-08 00:00:00 UTC000
May 82023-05-08 00:00:00 UTC2023-05-15 00:00:00 UTC000
May 152023-05-15 00:00:00 UTC2023-05-22 00:00:00 UTC000
May 222023-05-22 00:00:00 UTC2023-05-29 00:00:00 UTC202
May 292023-05-29 00:00:00 UTC2023-06-05 00:00:00 UTC000
Jun 52023-06-05 00:00:00 UTC2023-06-12 00:00:00 UTC101
Jun 122023-06-12 00:00:00 UTC2023-06-19 00:00:00 UTC000
Jun 192023-06-19 00:00:00 UTC2023-06-26 00:00:00 UTC000
Jun 262023-06-26 00:00:00 UTC2023-07-03 00:00:00 UTC000
Jul 32023-07-03 00:00:00 UTC2023-07-10 00:00:00 UTC000
Jul 102023-07-10 00:00:00 UTC2023-07-17 00:00:00 UTC101

What Hacker News says

5 sources
The 4-bit GPTQ LLaMA models are the current top-performers. This site has done a lot of the heavy lifting: https://github.com/qwopqwop200/GPTQ-for-LLaMa With 30b-4bit on a RTX 4090, I'm seeing numbers like: Output generated in 4.17 seconds (4.03 tokens/s, 21 tokens) Output generated in 4.38 seconds (4.25 tokens/s, 23 tokens) Output generated in 4.57 seconds (4.25 tokens/s, 24 tokens) Output generated in 3.86 seconds (3.40 tokens/s, 17 tokens) The lower size (7b, 13b) are even faster with lower memory use. A 16GB 3080 should be able to run the 13b at 4-bit just fine with reasonable (>1 token/s) latency.
In Llama.cpp: Port of Facebook's LLaMA model in C/C++, with Apple Silicon support
text-generation-webui supports state of the art 4bit GPTQ quantization for LLaMA[0], reducing VRAM overhead by 75% with no output performance loss compared to baseline fp16.[1] LLaMA-13B, rivaling GPT-3 175B, requires only 10GB* of VRAM with 4bit GPTQ quantization. LLaMA-30B fits on a 24GB* consumer video card with no output performance loss, beating GPT-3 175B. Multi-GPU support[2] means LLaMA-65B, rivaling PaLM-540B, runs on 2x3090. *Further improvements in active development will reduce VRAM requirements another 30-40% with no performance loss (ex. flash attention). [0] https://github.com/qwopqwop200/GPTQ-for-LLaMa [1] https://arxiv.org/abs/2210.17323 [2] https://github.com/oobabooga/text-generation-webui/issues/14... For LLaMA set-up instructions, including GPTQ 4bit, refer to this wiki article: https://github.com/oobabooga/text-gener
Read the rest
In LLaMA Text-Generation-Webui

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.