HeyWillow

willow

C2023–2024

HeyWillow / willow

willow

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

Security & Privacy AI & Machine Learning Self-hosted Learning & Reference Hardware & Embedded Robotics & IoT Automation & Workflows

Explore on GitHub Read Hacker News discussions

5Distinct people

19Mentions

0Explicit recommenders

3.1KGitHub stars

REPOSITORY AT A GLANCE

What to know before exploring

What it is

Open source, local, and self-hosted Amazon Echo/Google Home competitive Voice Assistant alternative

GitHub

Where it is used

No documented product use has been verified yet.

What people are saying

No recommendation phrase was found in the indexed comments.

No classified warning is attached to this page. Read the discussions before treating silence as approval.

Also in these discussions

toverainc/willow-inference-server · linked in 5 of the same discussions

Mentioned in the same Hacker News threads. That is not proof they are substitutes.

This week on Hacker News

Nov 8–Nov 15 UTC · Direct repository links and separately labeled related project stories. Indexed through 11/15/2026, 2:59:58 PM UTC.

No direct repository links were found in the indexed HN discussions over the past seven days. Older discussions are available below.

Mentions over time

HN mentions by month19 dated mentions

Observed mentions: 2023-05-15 14:13:35 UTC to 2024-01-13 23:27:31 UTC.

Counts come from the repository’s dated source evidence. Index cutoff: 2026-11-02 08:16:59 UTC.

View mention counts and date ranges

UTC ranges include their start and exclude their end. Select a period to filter the source evidence.
Period	Start (UTC)	End (exclusive, UTC)	Mentions	Submissions	Comments
May 2023	2023-05-01 00:00:00 UTC	2023-06-01 00:00:00 UTC	7	2	5
Jun 2023	2023-06-01 00:00:00 UTC	2023-07-01 00:00:00 UTC	7	0	7
Jul 2023	2023-07-01 00:00:00 UTC	2023-08-01 00:00:00 UTC	2	0	2
Aug 2023	2023-08-01 00:00:00 UTC	2023-09-01 00:00:00 UTC	1	0	1
Sep 2023	2023-09-01 00:00:00 UTC	2023-10-01 00:00:00 UTC	0	0	0
Oct 2023	2023-10-01 00:00:00 UTC	2023-11-01 00:00:00 UTC	0	0	0
Nov 2023	2023-11-01 00:00:00 UTC	2023-12-01 00:00:00 UTC	1	0	1
Dec 2023	2023-12-01 00:00:00 UTC	2024-01-01 00:00:00 UTC	0	0	0
Jan 2024	2024-01-01 00:00:00 UTC	2024-02-01 00:00:00 UTC	1	0	1

What Hacker News says

7 sources

mdaniel·Jun 28, 2023

I would suspect teaching Willow (https://github.com/toverainc/willow#readme) to do that would be much more reasonable in a kitchen setup, for the exact reason guhidalg mentioned: who wants to use oil soaked hands to touch something in the kitchen?

In Cooklang – Recipe Markup Language

mdaniel·Jun 27, 2023

I dunno if the audiences for these 2 things overlap, but since the posting of Willow (https://news.ycombinator.com/item?id=35948462 https://github.com/toverainc/willow#readme) I've become aware of references to that ESP32-S3 package

In Running Linux with Kernel 6.3 on Open Source Hardware Board with ESP32-S3

kkielhofner·Jun 26, 2023

Somewhat shameless but relevant plug - we use the ESP32-S3 for Willow[0]. The dual core S3 and "high speed" external PSRAM are game changers. What it is capable of (especially considering the price point) is nothing short of incredible. Wake word activation, audio processing (AGC, AEC, etc), audio streaming, even on device speech recognition for up to 400 commands with Multinet. All remarkably performant, easily besting Alexa/Echo in terms of interactivity and response time (even when using an inference server across the internet for speech recognition). Sure we're down in ESP-IDF land and managing everything we have going on in FreeRTOS is a pain but that's not anything you wouldn't have on any microcontroller. We're also doing a lot considering and generally speaking we "just" throw/pin audio tasks (with varying priority) on core 1 whil

Read the rest

e more-or-less dumping everything else on core 0. Seems to be working well so far! [0] - https://github.com/toverainc/willow

In Arduino Uno R4 WiFi

kkielhofner·Jun 21, 2023

We run into this constantly with Willow[0] and the Willow Inference Server[1]. There seems to be a large gap in understanding with many users. They seem to find it difficult to understand a fundamental reality: GPUs are so physically different and better suited to many/most ML tasks all the CPU tricks in the world cannot bring CPU even close to the performance of GPUs (while maintaining quality/functionality) for many tasks. I find this interesting because everyone seems to take it as obvious that integrated graphics vs discrete graphics for gaming aren't even close. Ditto for these tasks. With Willow Inference Server I'm constantly telling people: a six year old $100 Tesla P4/GTX 1070 walks all over even the best CPUs in the world for our primary task of speech to text/ASR - at dramatically lower cost and power usage. Seriously - a GTX 1

Read the rest

070 is at least 5x faster than a Threadripper 5955WX. Our goal is to provide an open-source commercial voice assistant equivalent user experience and that is and will be fundamentally impossible for the foreseeable future on CPU. Slight tangent but there are users in the space who seem to be under the impression that they can use their Raspberry Pi for voice assistant/speech recognition. It's not even close to a fair fight but with the same implementation and settings a GTX 1070 is roughly 90x (nearly two orders of magnitude) faster[2] than a Raspberry Pi... Yes, all-in a machine with a GTX 1070 uses and order of magnitude more power (3w vs 30x) than a Raspberry Pi but the power cost in even countries with the most expensive power in the world results in a $2-$3/mo cost difference - which I feel, at least, is a reasonable trade-off considering the dramatic difference in usability (Raspberry Pi is essentially useless - waiting 10-30 seconds for a response makes pulling your phone out faster). [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server [2] - https://github.com/toverainc/willow-inference-server/tree/wi...

In vLLM: Easy, Fast, and Cheap LLM Serving with PagedAttention

kkielhofner·Jun 15, 2023

I can't agree with this enough. Willow[0] (disclaimer - I'm the creator) just wouldn't be possible without the robust Espressif ecosystem, libraries, code samples, docs, etc. Let alone all on a finished product (ESP BOX, $50) available worldwide through their distribution channels! We have a lot of background and experience in this and related fields but the fact remains we went from concept to initial release in less than six weeks (with two part time devs - if you include me, which you shouldn't haha). [0] - https://github.com/toverainc/willow/

In ESP32-C3 Wireless Adventure: A Comprehensive Guide to IoT [pdf]

kkielhofner·Jun 7, 2023

Shameless plug, I'm the founder of Willow[0]. In short you can: 1) Run a local Willow Inference Server[1]. Supports CPU or CUDA, just about the fastest implementation of Whisper out there for "real time" speech. 2) Run local command detection on device. We pull your Home Assistant entities on setup and define basic grammar for them but any English commands (up to 400) are supported. They are recognized directly on the $50 ESP BOX device and sent to Home Assistant (or openHAB, or a REST endpoint, etc) for processing. Whether WIS or local our performance target is 500ms from end of speech to command executed. [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server

In GGML – AI at the Edge

kkielhofner·Jun 6, 2023

A 15 year head start in a category they essentially defined plus an entire generation of executives, developers, and users doesn’t hurt either. People complain about the “Nvidia tax” but the hardware is superior (untouchable at datacenter scale) and the “tax” turns into a dividend as soon as your (very expensive) team spends hour after hour (week after week) dealing with issues on other platforms compared to anything based on CUDA often being a Docker pull away with absolutely first class support on any ML framework. Nvidia gets a lot of shade on HN and elsewhere but if you’ve spent any time in this field you completely understand why they have 80-90% market share of GPGPU. With Willow[0] and the Willow Inference Server[1] I'm often asked by users with no experience in the space why we don't target AMD, Coral TPUs (don't even get me start

Read the rest

ed), etc. It's almost impossible to understand "why CUDA" unless you've fought these battles and spent time with "alternatives". I’ve been active in the space for roughly half a decade and when I look back to my early days I’m amazed what a beginner like me was able to do because of CUDA. I still routinely am. What you’re able to actually accomplish with a $1000 Nvidia card and a few lines with transformers and/or a Docker container is incredible. That said I am really looking forward to Apple stepping it up here - I’ve given up on AMD ever getting it together on GPGPU and Intel (with Arc) is even further behind. The space needs some real competition somewhere. [0] - https://github.com/toverainc/willow [1] - https://github.com/toverainc/willow-inference-server

In Nvidia releases new AI chip with 480GB CPU RAM, 96GB GPU RAM