ZYzyoraclub·Jun 2, 2026Submission
Show HN: We built an LLM inference engine in pure Python – no PyTorch, no Triton
2 points · Open discussionZyora-Dev
zse
Python2026–2026
Zyora-Dev / zse
zse
The inference engine the open-source world built for itself.
2Distinct people
3Mentions
0Explicit recommenders
242GitHub stars
What to know before exploring
What it is
The inference engine the open-source world built for itself.
GitHub
Where it is used
No documented product use has been verified yet.
What people are saying
No recommendation phrase was found in the indexed comments.
No classified warning is attached to this page. Read the discussions before treating silence as approval.
Also in these discussions
No other repository is repeatedly linked in the same discussions.
This week on Hacker News
Nov 7–Nov 14 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/14/2026, 1:49:46 PM UTC.
No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.
Mentions over time
HN mentions by week3 dated mentions
Observed mentions: 2026-02-26 01:15:25 UTC to 2026-06-02 14:34:48 UTC. Weeks start Monday, UTC.
Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.
View mention counts and date ranges
| Period | Start (UTC) | End (exclusive, UTC) | Mentions | Submissions | Comments |
|---|---|---|---|---|---|
| Feb 23 | 2026-02-23 00:00:00 UTC | 2026-03-02 00:00:00 UTC | 1 | 1 | 0 |
| Mar 2 | 2026-03-02 00:00:00 UTC | 2026-03-09 00:00:00 UTC | 1 | 1 | 0 |
| Mar 9 | 2026-03-09 00:00:00 UTC | 2026-03-16 00:00:00 UTC | 0 | 0 | 0 |
| Mar 16 | 2026-03-16 00:00:00 UTC | 2026-03-23 00:00:00 UTC | 0 | 0 | 0 |
| Mar 23 | 2026-03-23 00:00:00 UTC | 2026-03-30 00:00:00 UTC | 0 | 0 | 0 |
| Mar 30 | 2026-03-30 00:00:00 UTC | 2026-04-06 00:00:00 UTC | 0 | 0 | 0 |
| Apr 6 | 2026-04-06 00:00:00 UTC | 2026-04-13 00:00:00 UTC | 0 | 0 | 0 |
| Apr 13 | 2026-04-13 00:00:00 UTC | 2026-04-20 00:00:00 UTC | 0 | 0 | 0 |
| Apr 20 | 2026-04-20 00:00:00 UTC | 2026-04-27 00:00:00 UTC | 0 | 0 | 0 |
| Apr 27 | 2026-04-27 00:00:00 UTC | 2026-05-04 00:00:00 UTC | 0 | 0 | 0 |
| May 4 | 2026-05-04 00:00:00 UTC | 2026-05-11 00:00:00 UTC | 0 | 0 | 0 |
| May 11 | 2026-05-11 00:00:00 UTC | 2026-05-18 00:00:00 UTC | 0 | 0 | 0 |
| May 18 | 2026-05-18 00:00:00 UTC | 2026-05-25 00:00:00 UTC | 0 | 0 | 0 |
| May 25 | 2026-05-25 00:00:00 UTC | 2026-06-01 00:00:00 UTC | 0 | 0 | 0 |
| Jun 1 | 2026-06-01 00:00:00 UTC | 2026-06-08 00:00:00 UTC | 1 | 1 | 0 |
What Hacker News says
3 sourcesShow HN: ZSE – Single-file LLM engine with dual INT4 kernels
ZSE is an LLM inference engine built for simplicity and efficiency.( Phase 1 )
Features:
- Single .zse file format (model + tokenizer + config embedded)
- Zero network calls on load - works completely offline
- Dual INT4 kernel backend (ZSE Kernel + ZSE bnb Kernel)
- Intelligent layer auto-selects optimal kernel for your hardware
- Fast cold starts for serverless deployments
Benchmarks (H200, Qwen 2.5):
ZSE Kernel:
7B → 5.67 GB VRAM, 37 tok/s, 5.7s cold start
14B → 10.08 GB VRAM, 21 tok/s, 10.5s cold start
32B → 19.47 GB VRAM, 11 tok/s, 20.4s cold start
72B → 41.54 GB VRAM, 6 tok/s, 51.8s cold start
ZSE bnb Kernel:
7B → 6.57 GB VRAM, 46 tok/s, 6.0s cold start
14B → 11.39 GB VRAM, 28 tok/s, 7.1s cold start
32B → 22.27 GB VRAM, 20 tok/s, 20.8s cold start
72B → 47.05 GB VRAM, 16 tok/s, 53.0s cold start
Usage:
pip install
Read the rest
zllm-zse
zse convert Qwen/Qwen2.5-7B-Instruct -o model.zse
zse serve model.zse --port 8000
https://github.com/Zyora-Dev/zse
Show HN: ZSE – Open-source LLM inference engine with 3.9s cold starts
I've been building ZSE (Z Server Engine) for the past few weeks — an open-source LLM inference engine focused on two things nobody has fully solved together: memory efficiency and fast cold starts.
The problem I was trying to solve:
Running a 32B model normally requires ~64 GB VRAM. Most developers don't have that. And even when quantization helps with memory, cold starts with bitsandbytes NF4 take 2+ minutes on first load and 45–120 seconds on warm restarts — which kills serverless and autoscaling use cases.
What ZSE does differently:
Fits 32B in 19.3 GB VRAM (70% reduction vs FP16) — runs on a single A100-40GB
Fits 7B in 5.2 GB VRAM (63% reduction) — runs on consumer GPUs
Native .zse pre-quantized format with memory-mapped weights: 3.9s cold start for 7B, 21.4s for 32B — vs 45s and 120s with bitsandbytes, ~30s for vLLM
All benchmarks ve
Read the rest
rified on Modal A100-80GB (Feb 2026)
It ships with:
OpenAI-compatible API server (drop-in replacement)
Interactive CLI (zse serve, zse chat, zse convert, zse hardware)
Web dashboard with real-time GPU monitoring
Continuous batching (3.45× throughput)
GGUF support via llama.cpp
CPU fallback — works without a GPU
Rate limiting, audit logging, API key auth
Install:
-----
pip install zllm-zse
zse serve Qwen/Qwen2.5-7B-Instruct
For fast cold starts (one-time conversion):
-----
zse convert Qwen/Qwen2.5-Coder-7B-Instruct -o qwen-7b.zse
zse serve qwen-7b.zse # 3.9s every time
The cold start improvement comes from the .zse format storing pre-quantized weights as memory-mapped safetensors — no quantization step at load time, no weight conversion, just mmap + GPU transfer. On NVMe SSDs this gets under 4 seconds for 7B. On spinning HDDs it'll be slower.
All code is real — no mock implementations. Built at Zyora Labs. Apache 2.0.
Happy to answer questions about the quantization approach, the .zse format design, or the memory efficiency techniques.