bergamot-translator

You can use bergamot ( https://github.com/browsermt/bergamot-translator ) with Mozilla's models ( https://github.com/mozilla/firefox-translations-models ). Not the easiest, but easy enough (requires building). I used these two projects to build an on-device translator for Android.

Not sure it qualifies, but I've started building an Android app that wraps bergamot[0] (the firefox translation models) to have on-device translation without reliance on google. Bergamot is already used inside firefox, but I wanted translation also outside the browser. [0]: bergamot https://github.com/browsermt/bergamot-translator

It is running a full transformer type neural network on your cpu, it needs all the speed it can get. It’s because the translation engine requires at least SSSE3.1 instructions[1]. These are translated to wasm SIMD instructions[2] which are only enabled by browsers if the underlying hardware is there to execute these at least somewhat efficient. [1] https://github.com/browsermt/bergamot-translator/issues/418 [2] https://emscripten.org/docs/porting/simd.html

I believe this is called the Bergamot project, more can be found here: https://browser.mt/ The GitHub repo for it is here: https://github.com/browsermt/bergamot-translator The repo contains some details about how to run it in WASM which is quite interesting for embedding it in pages. I've been playing around with using WASM to capture speech to text (https://github.com/ccoreilly/vosk-browser) and automatically translating it using Bergamot. Results have been, ok. I don't think the tech is quite there yet and the speech to text obviously struggles with multiple speakers.

You can find the engine used here [1], the API built around it here [2] and its WASM port here [3] and the WebAssembly matrix multiplication optimizations are here [4] [1] https://marian-nmt.github.io/ [2] https://github.com/browsermt/bergamot-translator [3] https://github.com/browsermt/bergamot-translator/tree/main/w... [4] https://github.com/mozilla/gecko-dev/tree/master/third_party...

Yes, the extension is being developed here [1] and the engine[2]/wasm[3] wrapper here. The models and training pipeline are in the urls your posted and the evaluation of those models are hosted here [4] [1] https://github.com/mozilla/firefox-translations [2] https://github.com/browsermt/bergamot-translator [3] https://github.com/browsermt/bergamot-translator/tree/main/w... [4] https://github.com/mozilla/firefox-translations-models/tree/...

The training pipeline is also on Github! [1] I was experimenting with running the wasm version of bergamot-translator (the translation engine used by the addon) in node [2]. However, if you want more performance, using the Python library [3] or the native C++ interface [4] gets you further because the wasm build is limited to a single thread and thus a blocking interface, and can't use all the processor specific optimisations that are in the native builds. EDIT: Another option is using translateLocally [5], which is a Qt desktop app based on bergamot-translator. It has a native messaging API that is designed as a much faster alternative to the wasm build for browser extensions, but it can also be used from Python [6]. [1] https://github.com/mozilla/firefox-translations-training [2] https://gist.github.com/jelmervdl/a4c8b6b92ad88a885e1cbd5

A fork of marian-dev[1] is the underlying machine-translation engine: - https://github.com/browsermt/marian-dev Development of higher-level code wrapping around marian-dev make suitable for the browser-extension happens at: - https://github.com/browsermt/bergamot-translator Some of the WebAssembly optimizations are available in bergamot-translator/marian-dev. Rest are in Firefox source-code. A start point could be https://bugzilla.mozilla.org/show_bug.cgi?id=1720747. Extension code is open-source, and linked already in other comments: - https://github.com/mozilla/firefox-translations [1] https://github.com/marian-nmt/marian-dev