HackerReposA FIELD GUIDE TO OPEN SOURCE
ProSponsor the shelf♧ 0
← Back to top all-time
Neroued
ninfer
C++2026–2026
Neroued / ninfer

ninfer

High-performance single-GPU inference for selected model checkpoints and GPUs.

10Distinct people
12Mentions
0Explicit recommenders
3.3KGitHub stars
REPOSITORY AT A GLANCE

What to know before exploring

What it is

High-performance single-GPU inference for selected model checkpoints and GPUs.

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

No other repository is repeatedly linked in the same discussions.

This week on Hacker News

Nov 9–Nov 16 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/16/2026, 12:01:57 AM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by week12 dated mentions

Observed mentions: 2026-07-23 11:49:57 UTC to 2026-10-04 13:42:44 UTC. Weeks start Monday, UTC.

01232026-07-20 00:00:00 UTC to 2026-07-27 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 commentsJul 202026-07-27 00:00:00 UTC to 2026-08-03 00:00:00 UTC (exclusive): 1 mentions; 1 submissions, 0 comments2026-08-03 00:00:00 UTC to 2026-08-10 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-08-10 00:00:00 UTC to 2026-08-17 00:00:00 UTC (exclusive): 2 mentions; 1 submissions, 1 comments2026-08-17 00:00:00 UTC to 2026-08-24 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2026-08-24 00:00:00 UTC to 2026-08-31 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsAug 242026-08-31 00:00:00 UTC to 2026-09-07 00:00:00 UTC (exclusive): 3 mentions; 0 submissions, 3 comments2026-09-07 00:00:00 UTC to 2026-09-14 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-09-14 00:00:00 UTC to 2026-09-21 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-09-21 00:00:00 UTC to 2026-09-28 00:00:00 UTC (exclusive): 0 mentions; 0 submissions, 0 comments2026-09-28 00:00:00 UTC to 2026-10-05 00:00:00 UTC (exclusive): 1 mentions; 0 submissions, 1 commentsSep 28

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges
UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
PeriodStart (UTC)End (exclusive, UTC)MentionsSubmissionsComments
Jul 202026-07-20 00:00:00 UTC2026-07-27 00:00:00 UTC110
Jul 272026-07-27 00:00:00 UTC2026-08-03 00:00:00 UTC110
Aug 32026-08-03 00:00:00 UTC2026-08-10 00:00:00 UTC000
Aug 102026-08-10 00:00:00 UTC2026-08-17 00:00:00 UTC211
Aug 172026-08-17 00:00:00 UTC2026-08-24 00:00:00 UTC303
Aug 242026-08-24 00:00:00 UTC2026-08-31 00:00:00 UTC101
Aug 312026-08-31 00:00:00 UTC2026-09-07 00:00:00 UTC303
Sep 72026-09-07 00:00:00 UTC2026-09-14 00:00:00 UTC000
Sep 142026-09-14 00:00:00 UTC2026-09-21 00:00:00 UTC000
Sep 212026-09-21 00:00:00 UTC2026-09-28 00:00:00 UTC000
Sep 282026-09-28 00:00:00 UTC2026-10-05 00:00:00 UTC101

What Hacker News says

3 sources
Good 60-70 tg and 2K pp Qwen 3.8 27B FP8 can be had for about 5-6K (2xR9700 + PC) Gives about 3-4 concurrent sessions with full 262K Fast 150+ tg and 2-8K pp Qwen 3.8 27B nvfp4 is about 8K (5090 +PC) Gives really only one concurrent session that flies because kv caching is not perfect for ninfer https://github.com/Neroued/ninfer both are very serviceable, I prefer FP8 on 2xR9700 But, yes it doesn't scale that well but in 5 years the same hardware should still be very capable of running some great MoE models, for example Qwen 3.6 35BA3B on 5090 can fly at 600 tg
In Apple caught off guard by AI demand for Mac Mini and Mac Studio

MAKE YOURSELF AT HOME

A few shortcuts

/
Search the collection
?
Open this guide
Esc
Close dialogs and navigation

Your saved repositories and interests stay in this browser.