DOdonpdonp·Jan 25, 2024
I'd like to see a comparison to nitro https://github.com/janhq/nitro which has been fantastic for running a local LLM.
In Ollama releases Python and JavaScript Librariesjanhq
cortex.cpp
C++2024–2024
janhq / cortex.cpp · Archived
cortex.cpp
Local AI API Platform
2Distinct people
2Mentions
0Explicit recommenders
2.7KGitHub stars
What to know before exploring
What it is
Local AI API Platform
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 11:56:56 AM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by day2 dated mentions
Observed mentions: 2024-01-06 04:13:59 UTC to 2024-01-25 07:31:06 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Jan 6 | 2024-01-06 00:00:00 UTC | 2024-01-07 00:00:00 UTC | 1 | 0 | 1 |
| Jan 7 | 2024-01-07 00:00:00 UTC | 2024-01-08 00:00:00 UTC | 0 | 0 | 0 |
| Jan 8 | 2024-01-08 00:00:00 UTC | 2024-01-09 00:00:00 UTC | 0 | 0 | 0 |
| Jan 9 | 2024-01-09 00:00:00 UTC | 2024-01-10 00:00:00 UTC | 0 | 0 | 0 |
| Jan 10 | 2024-01-10 00:00:00 UTC | 2024-01-11 00:00:00 UTC | 0 | 0 | 0 |
| Jan 11 | 2024-01-11 00:00:00 UTC | 2024-01-12 00:00:00 UTC | 0 | 0 | 0 |
| Jan 12 | 2024-01-12 00:00:00 UTC | 2024-01-13 00:00:00 UTC | 0 | 0 | 0 |
| Jan 13 | 2024-01-13 00:00:00 UTC | 2024-01-14 00:00:00 UTC | 0 | 0 | 0 |
| Jan 14 | 2024-01-14 00:00:00 UTC | 2024-01-15 00:00:00 UTC | 0 | 0 | 0 |
| Jan 15 | 2024-01-15 00:00:00 UTC | 2024-01-16 00:00:00 UTC | 0 | 0 | 0 |
| Jan 16 | 2024-01-16 00:00:00 UTC | 2024-01-17 00:00:00 UTC | 0 | 0 | 0 |
| Jan 17 | 2024-01-17 00:00:00 UTC | 2024-01-18 00:00:00 UTC | 0 | 0 | 0 |
| Jan 18 | 2024-01-18 00:00:00 UTC | 2024-01-19 00:00:00 UTC | 0 | 0 | 0 |
| Jan 19 | 2024-01-19 00:00:00 UTC | 2024-01-20 00:00:00 UTC | 0 | 0 | 0 |
| Jan 20 | 2024-01-20 00:00:00 UTC | 2024-01-21 00:00:00 UTC | 0 | 0 | 0 |
| Jan 21 | 2024-01-21 00:00:00 UTC | 2024-01-22 00:00:00 UTC | 0 | 0 | 0 |
| Jan 22 | 2024-01-22 00:00:00 UTC | 2024-01-23 00:00:00 UTC | 0 | 0 | 0 |
| Jan 23 | 2024-01-23 00:00:00 UTC | 2024-01-24 00:00:00 UTC | 0 | 0 | 0 |
| Jan 24 | 2024-01-24 00:00:00 UTC | 2024-01-25 00:00:00 UTC | 0 | 0 | 0 |
| Jan 25 | 2024-01-25 00:00:00 UTC | 2024-01-26 00:00:00 UTC | 1 | 0 | 1 |
What Hacker News says
2 sourcesLook... I appreciate a cool project, but this is probably not a good idea.
> Built on top of the cutting-edge inference library llama.cpp, modified to be production ready.
It's not. It's literally just llama.cpp -> https://github.com/janhq/nitro/blob/main/.gitmodules
Llama.cpp makes no pretense at being a robust safe network ready library; it's a high performance library.
You've made no changes to llama.cpp here; you're just calling the llama.cpp API directly from your drogon app.
Hm.
...
Look... that's interesting, but, honestly, I know there's this wave of "C++ is back!" stuff going on, but building network applications in C++ is very tricky to do right, and while this is cool, I'm not sure 'llama.cpp is in c++ because it needs to be fast' is a good reason to go 'so lets build a network server in c++ too!'.
I mean, I guess you could arg
Read the rest
ue that since llama.cpp is a C++ application, it's fair for them to offer their own server example with an openai compatible API (which you can read about here: https://github.com/ggerganov/llama.cpp/issues/4216, https://github.com/ggerganov/llama.cpp/blob/master/examples/...).
...but a production ready application?
I wrote a rust binding to llama.cpp and my conclusion was that llama.cpp is pretty bleeding edge software, and bluntly, you should process isolate it from anything you really care about, if you want to avoid undefined behavior after long running inference sequences; because it updates very often, and often breaks. Those breaks are usually UB. It does not have a 'stable' version.
Further more, when you run large models and run out of memory, C++ applications are notoriously unreliable in their 'handle OOM' behaviour.
Soo.... I know there's something fun here, but really... unless you had a really really compelling reason to need to write your server software in c++ (and I see no compelling reason here), I'm curious why you would?
It seems enormously risky.
The quality of this code is 'fun', not 'production ready'.