transformers.js

Very cool! For some awesome examples of running different types of models in webgpu check out the examples folder here. https://github.com/huggingface/transformers.js

I built a Chrome extension with one feature that transcribes audio to text in the browser using huggingface/transformers.js running the OpenAI Whisper model with WebGPU. It works perfect! Here is a list of examples of all the things you can do in the browser with webgpu for free. [0] The last thing in the world I want to do is listen or watch presidential social media posts, but, on the other hand, sometimes enormously stupid things are said which move the SP500 up or down $60 in a session. So this feature queries for new posts every minute, does ORC image to text and transcribe video audio to text locally, sends the post with text for analysis, all in the background inside a Chrome extension before notify me of anything economically significant. [0] https://github.com/huggingface/transformers.js/tree/main/exa... [1] https://github.com/ad

Another very cool attribute of embeddings and embedding search is that they are resource cheap enough that you can perform them client side. ONNX models can be loaded and executed with transformer.js https://github.com/huggingface/transformers.js/ You can even build and statically host indices like hnsw for embeddings. I put together a little open source demo for this here https://jasonjmcghee.github.io/portable-hnsw/ (it's a prototype / hacked together approximation of hnsw, but you could implement the real thing) Long story short, represent indices as queryable parquet files and use duckdb to query them. Depending on how you host, it's either free or nearly free. I used Github Pages so it's free. R2 with cloudflare would only cost the size what you store (very cheap- no egress fees).