TTS

TTS: Text-to-Speech for all. TTS is a library for advanced Text-to-Speech generation. It's built on the latest research, was designed to achieve the best trade-off among ease-of-training, speed and quality. TTS comes with pretrained models, tools for measuring dataset quality and already used in 20+ languages for products and research projects. 📢 English Voice Samples and SoundCloud playlist 👨‍🍳 TTS training recipes 📄 Text-to-Speech paper collection 💬 Where to ask questions Please use our dedicated channels for questions and discussion. Help is much more valuable if it's shared publicly, so that more people can benefit from it. Type Platforms 🚨 Bug Reports GitHub Issue Tracker ❔ FAQ TTS/Wiki 🎁 Feature Requests & Ideas GitHub Issue Tracker 👩‍💻 Usage Questions Discourse Forum 🗯 General Discussion Discourse Forum and Matrix Channel 🔗 Links and Resources Type Links 💾 Installation TTS/README.md 👩🏾‍🏫 Tutorials and Examples TTS/Wiki 🚀 Released Models TTS/Wiki 💻 Docker Image Repository by @synesthesiam 🖥️ Demo Server TTS/server 🤖 Running TTS on Terminal TTS/README.md ✨ How to contribute TTS/README.md 🥇 TTS Performance "Mozilla*" and "Judy*" are our models. Details... Features High performance Deep Learning models for Text2Speech tasks. Text2Spec models (Tacotron, Tacotron2, Glow-TTS, SpeedySpeech). Speaker Encoder to compute speaker embeddings efficiently. Vocoder models (MelGAN, Multiband-MelGAN, GAN-TTS, ParallelWaveGAN, WaveGrad, WaveRNN) Fast and efficient model training. Detailed training logs on console and Tensorboard. Support for multi-speaker TTS. Efficient Multi-GPUs training. Ability to convert PyTorch models to Tensorflow 2.0 and TFLite for inference. Released models in PyTorch, Tensorflow and TFLite. Tools to curate Text2Speech datasets underdataset_analysis. Demo server for model testing. Notebooks for extensive model benchmarking. Modular (but not too much) code base enabling easy testing for new ideas. Implemented Models Text-to-Spectrogram Tacotron: paper Tacotron2: paper Glow-TTS: paper Speedy-Speech: paper Attention Methods Guided Attention: paper Forward Backward Decoding: paper Graves Attention: paper Double Decoder Consistency: blog Speaker Encoder GE2E: paper Angular Loss: paper Vocoders MelGAN: paper MultiBandMelGAN: paper ParallelWaveGAN: paper GAN-TTS discriminators: paper WaveRNN: origin WaveGrad: paper You can also help us implement more models. Some TTS related work can be found here. Install TTS TTS supports python >= 3.6, <3.9. If you are only interested in synthesizing speech with the released TTS models, installing from PyPI is the easiest option. pip install TTS If you plan to code or train models, clone TTS and install it locally. git clone https://github.com/mozilla/TTS pip install -e . Directory Structure |- notebooks/ (Jupyter Notebooks for model evaluation, parameter selection and data analysis.) |- utils/ (common utilities.) |- TTS |- bin/ (folder for all the executables.) |- train*.py (train your target model.) |- distribute.py (train your TTS model using Multiple GPUs.) |- compute_statistics.py (compute dataset statistics for normalization.) |- convert*.py (convert target torch model to TF.) |- tts/ (text to speech models) |- layers/ (model layer definitions) |- models/ (model definitions) |- tf/ (Tensorflow 2 utilities and model implementations) |- utils/ (model specific utilities.) |- speaker_encoder/ (Speaker Encoder models.) |- (same) |- vocoder/ (Vocoder models.) |- (same) Sample Model Output Below you see Tacotron model state after 16K iterations with batch-size 32 with LJSpeech dataset. "Recent research at Harvard has shown meditating for as little as 8 weeks can actually increase the grey matter in the parts of the brain responsible for emotional regulation and learning." Audio examples: soundcloud Datasets and Data-Loading TTS provides a generic dataloader easy to use for your custom dataset. You just need to write a simple function to format the dataset. Check datasets/preprocess.py to see some examples. After that, you need to set dataset fields in config.json. Some of the public datasets that we successfully applied TTS: LJ Speech Nancy TWEB M-AI-Labs LibriTTS Spanish - thx! @carlfm01 Example: Synthesizing Speech on Terminal Using the Released Models. After the installation, TTS provides a CLI interface for synthesizing speech using pre-trained models. You can either use your own model or the release models under the TTS project. Listing released TTS models. tts --list_models Run a tts and a vocoder model from the released model list. (Simply copy and paste the full model names from the list as arguments for the command below.) tts --text "Text for TTS" \ --model_name "<type>/<language>/<dataset>/<model_name>" \ --vocoder_name "<type>/<language>/<dataset>/<model_name>" \ --out_path folder/to/save/output/ Run your own TTS model (Using Griffin-Lim Vocoder) tts --text "Text for TTS" \ --model_path path/to/model.pth.tar \ --config_path path/to/config.json \ --out_path output/path/speech.wav Run your own TTS and Vocoder models tts --text "Text for TTS" \ --model_path path/to/config.json \ --config_path path/to/model.pth.tar \ --out_path output/path/speech.wav \ --vocoder_path path/to/vocoder.pth.tar \ --vocoder_config_path path/to/vocoder_config.json Note: You can use ./TTS/bin/synthesize.py if you prefer running tts from the TTS project folder. Example: Training and Fine-tuning LJ-Speech Dataset Here you can find a CoLab notebook for a hands-on example, training LJSpeech. Or you can manually follow the guideline below. To start with, split metadata.csv into train and validation subsets respectively metadata_train.csv