On GitHub, huggingface/parler-tts has picked up 5.6k stars, Python. Inference and training library for high-quality TTS models.
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
Parler-TTS is a lightweight text-to-speech (TTS) model that can generate high-quality, natural sounding speech in the style of a given speaker (gender, pitch, speaking style, etc). It is a reproduction of work from the paper Natural language guidance of high-fidelity text-to-speech with synthetic annotations by Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively.
Contrarily to other TTS models, Parler-TTS is a fully open-source release. All of the datasets, pre-processing, training code and weights are released publicly under permissive license, enabling the community to build on our work and develop their own powerful TTS models.
This repository contains the inference and training code for Parler-TTS. It is designed to accompany the Data-Speech repository for dataset annotation.
[!IMPORTANT]
08/08/2024: We are proud to release two new Parler-TTS checkpoints:
Parler-TTS Mini, an 880M parameter model.
Parler-TTS Large, a 2.3B parameter model.
These checkpoints have been trained on 45k hours of audiobook data.
In addition, the code is optimized for much faster generation: we've added SDPA and Flash Attention 2 compatibility, as well as the ability to compile the model.
📖 Quick Index
Installation
Usage
🎲 Using a random voice
🎯 Using a specific speaker
Training
Demo
Model weights and datasets
Optimizing inference
Installation
Parler-TTS has light-weight dependencies and can be installed in one line:
Parler-TTS has been trained to generate speech with features that can be controlled with a simple text prompt, for example:
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained("parler-tts/parler-tts-mini-v1").to(device)
tokenizer = AutoTokenizer.from_pretrained("parler-tts/parler-tts-mini-v1")
prompt = "Hey, how are you doing today?"
description = "A female speaker delivers a slightly expressive and animated speech with a moderate speed and pitch. The recording is of very high quality, with the speaker's voice sounding clear and very close up."
input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)
🎯 Using a specific speaker
To ensure speaker consistency across generations, this checkpoint was also trained on 34 speakers, characterized by name. The full list of available speakers includes:
Laura, Gary, Jon, Lea, Karen, Rick, Brenda, David, Eileen, Jordan, Mike, Yann, Joy, James, Eric, Lauren, Rose, Will, Jason, Aaron, Naomie, Alisa, Patrick, Jerry, Tina, Jenna, Bill, Tom, Carol, Barbara, Rebecca, Anna, Bruce, Emily.
To take advantage of this, simply adapt your text description to specify which speaker to use: Jon's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise.
You can replace "Jon" with any of the names from the list above to utilize different speaker characteristics. Each speaker has unique vocal qualities that can be leveraged to suit your specific needs. For more detailed information on speaker performance with voice consistency, please refer inference guide.
import torch
from parler_tts import ParlerTTSForConditionalGeneration
from transformers import AutoTokenizer
import soundfile as sf
device = "cuda:0" if torch.cuda.is_available() else "cpu"
model = ParlerTTSForConditionalGeneration.from_pretrained("parler-tts/parler-tts-mini-v1").to(device)
tokenizer = AutoTokenizer.from_pretrained("parler-tts/parler-tts-mini-v1")
prompt = "Hey, how are you doing today?"
description = "Jon's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise."
input_ids = tokenizer(description, return_tensors="pt").input_ids.to(device)
prompt_input_ids = tokenizer(prompt, return_tensors="pt").input_ids.to(device)
generation = model.generate(input_ids=input_ids, prompt_input_ids=prompt_input_ids)
audio_arr = generation.cpu().numpy().squeeze()
sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)
Tips:
Include the term "very clear audio" to generate the highest quality audio, and "very noisy audio" for high levels of background noise
Punctuation can be used to control the prosody of the generations, e.g. use commas to add small breaks in speech
The remaining speech features (gender, speaking rate, pitch and reverberation) can be controlled directly through the prompt
✨ Optimizing Inference Speed
We've set up an inference guide to make generation faster. Think SDPA, torch.compile and streaming!
The training folder contains all the information to train or fine-tune your own Parler-TTS model. It consists of:
1. An introduction to the Parler-TTS architecture
2. The first steps to get started
3. A training guide
[!IMPORTANT]
TL;DR: After having followed the installation steps, you can reproduce the Parler-TTS Mini v1 training recipe with the following command line:
[!IMPORTANT]
You can also follow this fine-tuning guide on a mono-speaker dataset example.
Acknowledgements
This library builds on top of a number of open-source giants, to whom we'd like to extend our warmest thanks for providing these tools!
Special thanks to:
Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively, for publishing such a promising and clear research paper: Natural language guidance of high-fidelity text-to-speech with synthetic annotations.
the many libraries used, namely 🤗 datasets, 🤗 accelerate, jiwer, wandb, and 🤗 transformers.
Descript for the DAC codec model
Hugging Face 🤗 for providing compute resources and time to explore!
Citation
If you found this repository useful, please consider citing this work and also the original Stability AI paper:
@misc{lacombe-etal-2024-parler-tts,
author = {Yoach Lacombe and Vaibhav Srivastav and Sanchit Gandhi},
title = {Parler-TTS},
year = {2024},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/huggingface/parler-tts}}
}
@misc{lyth2024natural,
title={Natural language guidance of high-fidelity text-to-speech with synthetic annotations},
author={Dan Lyth and Simon King},
year={2024},
eprint={2402.01912},
archivePrefix={arXiv},
primaryClass={cs.SD}
}
Contribution
Contributions are welcome, as the project offers many possibilities for improvement and exploration.
Namely, we're looking at ways to improve both quality and speed:
Datasets:
Train on more data
Add more features such as accents
Training:
Add PEFT compatibility to do Lora fine-tuning.
Add possibility to train without description column.
TopGit's last sync did not record any GitHub topics for huggingface/parler-tts. GitHub topics appear in the right sidebar of a repository page; that's the authoritative place to check.
How active is development on huggingface/parler-tts?
The most recent commit recorded on huggingface/parler-tts was 1.9 years ago, based on the GitHub push timestamp. The repository has 591 forks — one of the better signals of community interest.
How many stars does huggingface/parler-tts have?
huggingface/parler-tts has 5.6k GitHub stars — refresh the page for the live number, or check github.com/huggingface/parler-tts. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
What language is huggingface/parler-tts written in?
huggingface/parler-tts is written primarily in Python. GitHub's language field is based on the largest share of bytes in the default branch.
Where do I read more about huggingface/parler-tts?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/huggingface/parler-tts is the definitive source.
Read full README in the tab above.
Curious whether parler-tts is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about parler-tts.