DUdunwaldo·Dec 8, 2024Submission
KubeAI – AI Inference Operator for Kubernetes
1 points · Open discussionkubeai-project
kubeai
Go2024–2024
kubeai-project / kubeai
kubeai
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
3Distinct people
3Mentions
0Explicit recommenders
1.3KGitHub stars
What to know before exploring
What it is
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 5:04:57 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by week3 dated mentions
Observed mentions: 2024-08-26 18:29:20 UTC to 2024-12-08 00:50:32 UTC. Weeks start Monday, UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Aug 26 | 2024-08-26 00:00:00 UTC | 2024-09-02 00:00:00 UTC | 1 | 1 | 0 |
| Sep 2 | 2024-09-02 00:00:00 UTC | 2024-09-09 00:00:00 UTC | 0 | 0 | 0 |
| Sep 9 | 2024-09-09 00:00:00 UTC | 2024-09-16 00:00:00 UTC | 0 | 0 | 0 |
| Sep 16 | 2024-09-16 00:00:00 UTC | 2024-09-23 00:00:00 UTC | 0 | 0 | 0 |
| Sep 23 | 2024-09-23 00:00:00 UTC | 2024-09-30 00:00:00 UTC | 1 | 1 | 0 |
| Sep 30 | 2024-09-30 00:00:00 UTC | 2024-10-07 00:00:00 UTC | 0 | 0 | 0 |
| Oct 7 | 2024-10-07 00:00:00 UTC | 2024-10-14 00:00:00 UTC | 0 | 0 | 0 |
| Oct 14 | 2024-10-14 00:00:00 UTC | 2024-10-21 00:00:00 UTC | 0 | 0 | 0 |
| Oct 21 | 2024-10-21 00:00:00 UTC | 2024-10-28 00:00:00 UTC | 0 | 0 | 0 |
| Oct 28 | 2024-10-28 00:00:00 UTC | 2024-11-04 00:00:00 UTC | 0 | 0 | 0 |
| Nov 4 | 2024-11-04 00:00:00 UTC | 2024-11-11 00:00:00 UTC | 0 | 0 | 0 |
| Nov 11 | 2024-11-11 00:00:00 UTC | 2024-11-18 00:00:00 UTC | 0 | 0 | 0 |
| Nov 18 | 2024-11-18 00:00:00 UTC | 2024-11-25 00:00:00 UTC | 0 | 0 | 0 |
| Nov 25 | 2024-11-25 00:00:00 UTC | 2024-12-02 00:00:00 UTC | 0 | 0 | 0 |
| Dec 2 | 2024-12-02 00:00:00 UTC | 2024-12-09 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
3 sourcesInfinity embedding engine added to KubeAI
Just merged and released the [Infinity support PR](https://github.com/substratusai/kubeai/pull/197) in KubeAI, adding Infinity as an embedding engine. So you can get embeddings on your local Kubernetes clusters with an OpenAI compatible API.
Infinity is a high performance and low latency embeddings engine: https://github.com/michaelfeil/infinity
KubeAI is a Kubernetes Operator for running OSS ML serving engines: https://github.com/substratusai/kubeai
How to use this?
Deploy on any K8s cluster by running:
```
helm repo add kubeai https://www.kubeai.org
helm install kubeai kubeai/kubeai --wait --timeout 10m
cat > model-values.yaml << EOF
catalog:
bge-embed-text-cpu:
enabled: true
features: ["TextEmbedding"]
owner: baai
url: "hf://BAAI/bge-small-en-v1.5"
engine: Infinity
resourceProfile: cpu:1
minReplicas: 1
EOF
Read the rest
helm install kubeai-models kubeai/models -f ./model-values.yaml
```
Forward kubeai service to local host:
```
kubectl port-forward svc/kubeai 8000:80
```
Afterwards you could use the OpenAI Python client to get embeddings:
```
from openai import OpenAI
# Assumes port-forward of kubeai service to localhost:8000.
client = OpenAI(api_key="ignored", base_url="http://localhost:8000/openai/v1")
response = client.embeddings.create(
input="Your text goes here.",
model="bge-embed-text-cpu"
)
print(response)
```
What’s next?
- Support for autoscaling based on Infinity reported metrics.
Show HN: KubeAI – Open AI on Kubernetes
The open source AI ecosystem is progressing at a pace that is hard to keep up with. Our team is building KubeAI to make it easy to quickly operationalize OSS models and tools within Kubernetes. The goal is to serve as a drop-in replacement for proprietary platforms like OpenAI while also integrating well with existing cloud infrastructure (i.e. messaging systems, databases, etc). Design decisions are being made with an adherence to simplicity as the guiding principle (no dependency on Istio, etc). We currently have implemented support for scale-from-zero inferencing (built-in vLLM & Ollama operator). Next up, we are are going to focus on end-to-end training-through-serving support for LoRA adaptors next!
Would love to hear feedback and ideas!
4 points · Open discussion