DeepFilterNet

Regarding the less than ideal audio conditions, there are also already models that have impressive noise cancellation. Like this https://github.com/Rikorose/DeepFilterNet one. If you put them in serial, maybe you get better results?

Hey HN, Introducing `dfm`, a simple CLI for background noise removal from audio and video files. It leverages the DeepFilterNet model and FFmpeg to seamlessly enhance your recordings. Huge credit goes to Hendrik Schröter et al. for their original work on DeepFilterNet, which powers this tool. https://github.com/Rikorose/DeepFilterNet I built this for my own needs to more easily process a bunch of video files. Hope others might find it useful as well. .

I tried the Audacity noise-removal plugin recently and it's complete crap. I fed a high-quality audio stream from a Rode mic into a few different options to see which could remove the noise of my server rack. iMovie made the voice sound like a robot and Audacity barely did anything. The only thing that worked was DeepFilterNet and it's free, open-source and cargo installable. There's no reason to lock yourself into an intel-only solution. Just use DeepFilterNet. The results of this on my noisy server room were insanely good. Almost no voice dropout with 100% fan noise removal. https://github.com/Rikorose/DeepFilterNet EDIT: Even more interesting, it looks like OpenVino is just DeepFilterNet glued to Whisper.cpp and tied to Intel hardware. https://github.com/intel/openvino-plugins-ai-audacity/tree/m...

Very cool! Would be curious to see how this compares to https://github.com/Rikorose/DeepFilterNet written in Rust. Or this Samsung Research paper https://research.samsung.com/blog/FSPEN-AN-ULTRA-LIGHTWEIGHT...

DeepFilterNet doesn't use a second microphone but it does do an absurdly good job of removing not-speech from inputs in realtime. Check out the demo video linked in the README. iirc they demonstrate removing guitar sounds and even a vacuum cleaner. It does take some technical elbow grease to integrate but I've used it in calls and while gaming on Linux via Pipewire to great effect. https://github.com/Rikorose/DeepFilterNet

It looks like the library in Rust is using `tract-onnx` to do the inference: https://github.com/Rikorose/DeepFilterNet/blob/2a84d2a1750a5... I am wondering whether using Python for research, training in big data center, and Rust at edge for efficient inference would be a trend in the future. We do have a larger community of C++ right now for inference (e.g. ggml). But Rust crate as component to build applications of AI is joy to use.

Integrate with Pipewire: https://github.com/Rikorose/DeepFilterNet/blob/main/ladspa/R... Youtube demo: https://youtu.be/EO7n96YwnyE Paper Explanation: https://youtu.be/it90gBqkY6k