NInikikiki·Jun 8, 2024Submission
Are high AI computing costs weighing you down?
2 points · Open discussionEmerging-AI
ENOVA
Python2024–2024
Emerging-AI / ENOVA
ENOVA
A deployment, monitoring and autoscaling service towards serverless LLM serving.
2Distinct people
2Mentions
0Explicit recommenders
154GitHub stars
What to know before exploring
What it is
A deployment, monitoring and autoscaling service towards serverless LLM serving.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by day2 dated mentions
Observed mentions: 2024-05-30 15:55:38 UTC to 2024-06-08 13:26:37 UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| May 30 | 2024-05-30 00:00:00 UTC | 2024-05-31 00:00:00 UTC | 1 | 1 | 0 |
| May 31 | 2024-05-31 00:00:00 UTC | 2024-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 1 | 2024-06-01 00:00:00 UTC | 2024-06-02 00:00:00 UTC | 0 | 0 | 0 |
| Jun 2 | 2024-06-02 00:00:00 UTC | 2024-06-03 00:00:00 UTC | 0 | 0 | 0 |
| Jun 3 | 2024-06-03 00:00:00 UTC | 2024-06-04 00:00:00 UTC | 0 | 0 | 0 |
| Jun 4 | 2024-06-04 00:00:00 UTC | 2024-06-05 00:00:00 UTC | 0 | 0 | 0 |
| Jun 5 | 2024-06-05 00:00:00 UTC | 2024-06-06 00:00:00 UTC | 0 | 0 | 0 |
| Jun 6 | 2024-06-06 00:00:00 UTC | 2024-06-07 00:00:00 UTC | 0 | 0 | 0 |
| Jun 7 | 2024-06-07 00:00:00 UTC | 2024-06-08 00:00:00 UTC | 0 | 0 | 0 |
| Jun 8 | 2024-06-08 00:00:00 UTC | 2024-06-09 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
2 sourcesShow HN: I found a way to increase the perforamce of GPU cluster scheduling
I worked on this project one year ago since I started working on developing new programs by using GPT and GPU could. And I found out these applications typically run on GPU clouds in industrial environments, where the cost of LLM requests may be ten times higher than that of traditional queries.
Therefore, previous researchers have focused on optimizing LLM inference to increase throughput and reduce the cost of LLM services. A variety of methods have been proposed to optimize LLM inference, which can be categorized into five general categories: computation, weight or activation quantization, memory management, kernel operation optimization, and batch request processing.
However, there has been little research on stable and automatic scaling of serverless LLM services in multi-GPU clusters. Challenges still exist in providing scheduling
Read the rest
services so that LLM developers no longer need to spend time on providing stable and scalable LLM services.
To address this issue, I have constructed ENOVA, a deployment, monitoring, and automatic scaling service specifically for serverless LLM services. ENOVA thoroughly deconstructs the execution process of LLM services, and based on this, it has designed a configuration recommendation module for automatic deployment on any GPU cluster, as well as a performance monitoring module for automatic scaling. On top of them, ENOVA has implemented a deployment execution engine for multi-GPU cluster scheduling. Experimental results show that ENOVA significantly outperforms other state-of-the-art methods and is suitable for widespread deployment in large-scale online systems.
This is an open source for those who is interested in solving this issue. Let me know what you think of the solution to solve this issue.