ultravox

This looks like so much fun! I have recently gotten into working with electronics, so it seems like a nice little project to undertake. I noticed that it is dependent on openAIs realtime API, so it got me wondering what open alternatives there are as I would love a more realtime alexa-like device in my home that doesnt contact the cloud. I have only played with software, but the existing solutions have never felt realtime to me. I could only find <https://github.com/fixie-ai/ultravox> that would seem to really work as realtime. It seems to be some model that wires up llama and whisper somehow, rather than treating them as separate steps which is common with other projects. What other options are available for this kind of real-time behaviour?

There’s Ultravox as well (from one of the creators of WebRTC): https://github.com/fixie-ai/ultravox Their model builds a speech-to-speech layer into Llama. Last I checked they have the audio-in part working and they’re working on the audio-out piece.