talk

Yes, I was planning to do this back then, but other stuff came up. There are many different ways in which this simple example can be improved: - better detection of when speech ends (currently basic adaptive threshold) - use small LLM for quick response with something generic while big LLM computes - TTS streaming in chunks or sentences One of the better OSS versions of such chatbot I think is https://github.com/yacineMTB/talk. Though probably many other similar projects also exist by now.

Also curious to hear about your setup. Using whisper too? When I was experimenting with it there was still a lot of annoyance about hallucinations and I was hard coding some "if last phrase is 'thanks for watching', ignore last phrase" I was just googling a bit to see what's out there now for whisper/llama combos and came across this: https://github.com/yacineMTB/talk There's a demo linked on the github page that seems relatively fast at responding conversationally, but still maybe 1-2 seconds at times. Impressive it's entirely offline.

Here’s a project attempting to do just this! https://github.com/yacineMTB/talk