HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
ggml-org
ggml
C++2022–2026
ggml-org / ggml

ggml

Tensor library for machine learning

31Distinct people
37Mentions
1Explicit recommenders
15.7KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

Tensor library for machine learning

GitHub

Where it is used

No documented product use was verified in the last source check.

What people are saying

If you don't care about the details of how those model servers work, then something that abstracts out the whole process like LM Studio or Ollama is all you need. However, if…

manca used a recommendation phrase. One indexed comment matches. This is a phrase match, not a verdict.

Favorable Commenters praise ggml for its performance improvements on legacy hardware and its minimalist design philosophy. One user explicitly links to the repository as a preferred entry point.

This week’s discussion is favorable. This is not a standing endorsement. Read the supporting reply

Also in these discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Favorable

Commenters praise ggml for its performance improvements on legacy hardware and its minimalist design philosophy. One user explicitly links to the repository as a preferred entry point.

Supporting reply 1 · Supporting reply 2 · Supporting reply 3 · Supporting reply 4 · 2 discussions summarized

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 8:59:02 AM UTC.

Ggml v1.0.0 direct repository linkDebated

Users widely praise ggml v1.0.0 and llama.cpp for consistent performance gains on existing hardware and broad compatibility, but a vocal minority criticizes recent architectural changes and build processes as telemetry-heavy or overly complex. This disagreement centers on whether the project remains a simple, open engineering tool or has drifted toward proprietary-style practices that alienate core developers.

68 readable replies · Supporting reply 1 · Supporting reply 2 · Supporting reply 3 · Supporting reply 4
llama.cpp v1.0 official project website

Read the discussion on HN. A full-thread summary is not available yet.

Mentions over time

HN mentions by year37 dated mentions

Observed mentions: 2022-09-20 13:33:46 UTC to 2026-08-17 19:22:51 UTC.

05101520252022-01-01 00:00:00 UTC to 2023-01-01 00:00:00 UTC (exclusive): 3 mentions; 2 submissions, 1 comments20222023-01-01 00:00:00 UTC to 2024-01-01 00:00:00 UTC (exclusive): 23 mentions; 6 submissions, 17 comments2024-01-01 00:00:00 UTC to 2025-01-01 00:00:00 UTC (exclusive): 5 mentions; 1 submissions, 4 comments20242025-01-01 00:00:00 UTC to 2026-01-01 00:00:00 UTC (exclusive): 4 mentions; 0 submissions, 4 comments2026-01-01 00:00:00 UTC to 2027-01-01 00:00:00 UTC (exclusive): 2 mentions; 0 submissions, 2 comments. Incomplete calendar period.2026*

* An incomplete calendar period is marked with an outlined bar. Its count is not directly comparable with a full period.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
20222022-01-01 00:00:00 UTC2023-01-01 00:00:00 UTC321
20232023-01-01 00:00:00 UTC2024-01-01 00:00:00 UTC23617
20242024-01-01 00:00:00 UTC2025-01-01 00:00:00 UTC514
20252025-01-01 00:00:00 UTC2026-01-01 00:00:00 UTC404
2026 (partial)2026-01-01 00:00:00 UTC2027-01-01 00:00:00 UTC202

What Hacker News says

23 sources
https://github.com/ggerganov/ggml TinyGrad is also targeting CPU inference, and IIRC it works ok in Apache TVM. One note is that prompt ingestion is extremely slow on CPU compared to GPU. So short prompts are fine (and tokens can be streamed once the prompt is ingested), but long prompts feel extremely sluggish. Another is that CPUs with more than 128-bit DDR5 memory busses are very expensive, and CPU token generation is basically RAM bandwidth bound.
In Falcon 180B
I did a quick run under profiler and on my AVX2-laptop the slowest part (>50%) was matrix multiplication (sgemm). In current version of GGML if OpenBLAS is enabled, they convert matrices to FP32 before running sgemm. If OpenBLAS is disabled, on AVX2 plaftorm they convert FP16 to FP32 on every FMA operation, which even worse (due to repetition). After that, both ggml_vec_dot_f16 and ggml_vec_dot_f32 took first place in profiler. Source: https://github.com/ggerganov/ggml/blob/master/src/ggml.c#L10... But I agree, that in theory, and only with AVX512 BF16 (not exactly FP16, but similar) will be fast with VDPBF16PS instruction. Implementation is not there yet.
In Stable Diffusion in C/C++
For those getting started, the easiest one click installer I've used is Nomic.ai's gpt4all: https://gpt4all.io/ This runs with a simple GUI on Windows/Mac/Linux, leverages a fork of llama.cpp on the backend and supports GPU acceleration, and LLaMA, Falcon, MPT, and GPT-J models. It also has API/CLI bindings. I just saw a slick new tool https://ollama.ai/ that will let you install a llama2-7b with a single `ollama run llama2` command that has a very simple 1-click installer for Apple Silicon Mac (but need to build from source for anything else atm). It looks like it only supports llamas OOTB but it also seems to use llama.cpp (via Go adapter) on the backend - it seemed to be CPU-only on my MBA, but I didn't poke too much and it's brand new, so we'll see. For anyone on HN, they should probably be looking at https://github.com/ggerganov/llam
Read the rest
a.cpp and https://github.com/ggerganov/ggml directly. If you have a high-end Nvidia consumer card (3090/4090) I'd highly recommend looking into https://github.com/turboderp/exllama For those generally confused, the r/LocalLLaMA wiki is a good place to start: https://www.reddit.com/r/LocalLLaMA/wiki/guide/ I've also been porting my own notes into a single location that tracks models, evals, and has guides focused on local models: https://llm-tracker.info/
In Accessing Llama 2 from the command-line with the LLM-replicate plugin

Showing the 15 most recent of 23 sources. Enable JavaScript to search and browse every indexed mention.

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.