SYSTRAN/faster-whisper — 26.7k★ trên GitHub (Python). Faster Whisper transcription with CTranslate2
Tóm tắt dựng từ metadata GitHub của chính dự án — chưa có bài review TopGit. Trang sẽ tự động cập nhật khi bài review đầy đủ được xuất bản.
VÌ SAO CHƯA CÓ REVIEW
TopGit viết bài đầy đủ cho repo có nhiều sao nhất và được yêu cầu nhiều nhất. Trang này là snapshot trong thời gian chờ — xem README gốc ở tab READ ME.
faster-whisper is a reimplementation of OpenAI's Whisper model using CTranslate2, which is a fast inference engine for Transformer models.
This implementation is up to 4 times faster than openai/whisper for the same accuracy while using less memory. The efficiency can be further improved with 8-bit quantization on both CPU and GPU.
Benchmark
Whisper
For reference, here's the time and memory usage that are required to transcribe 13 minutes of audio using different implementations:
openai/whisper@v20240930
whisper.cpp@v1.7.2
transformers@v4.46.3
faster-whisper@v1.1.0
Large-v2 model on GPU
Implementation
Precision
Beam size
Time
VRAM Usage
openai/whisper
fp16
5
2m23s
4708MB
whisper.cpp (Flash Attention)
fp16
5
1m05s
4127MB
transformers (SDPA)1
fp16
5
1m52s
4960MB
faster-whisper
fp16
5
1m03s
4525MB
faster-whisper (batch_size=8)
fp16
5
17s
6090MB
faster-whisper
int8
5
59s
2926MB
faster-whisper (batch_size=8)
int8
5
16s
4500MB
distil-whisper-large-v3 model on GPU
Implementation
Precision
Beam size
Time
YT Commons WER
transformers (SDPA) (batch_size=16)
fp16
5
46m12s
14.801
faster-whisper (batch_size=16)
fp16
5
25m50s
13.527
GPU Benchmarks are Executed with CUDA 12.4 on a NVIDIA RTX 3070 Ti 8GB.
Small model on CPU
Implementation
Precision
Beam size
Time
RAM Usage
openai/whisper
fp32
5
6m58s
2335MB
whisper.cpp
fp32
5
2m05s
1049MB
whisper.cpp (OpenVINO)
fp32
5
1m45s
1642MB
faster-whisper
fp32
5
2m37s
2257MB
faster-whisper (batch_size=8)
fp32
5
1m06s
4230MB
faster-whisper
int8
5
1m42s
1477MB
faster-whisper (batch_size=8)
int8
5
51s
3608MB
Executed with 8 threads on an Intel Core i7-12700K.
Requirements
Python 3.9 or greater
Unlike openai-whisper, FFmpeg does not need to be installed on the system. The audio is decoded with the Python library PyAV which bundles the FFmpeg libraries in its package.
GPU
GPU execution requires the following NVIDIA libraries to be installed:
cuBLAS for CUDA 12
cuDNN 9 for CUDA 12
Note: The latest versions of ctranslate2 only support CUDA 12 and cuDNN 9. For CUDA 11 and cuDNN 8, the current workaround is downgrading to the 3.24.0 version of ctranslate2, for CUDA 12 and cuDNN 8, downgrade to the 4.4.0 version of ctranslate2, (This can be done with pip install --force-reinstall ctranslate2==4.4.0 or specifying the version in a requirements.txt).
There are multiple ways to install the NVIDIA libraries mentioned above. The recommended way is described in the official NVIDIA documentation, but we also suggest other installation methods below.
Other installation methods (click to expand)
Note: For all these methods below, keep in mind the above note regarding CUDA versions. Depending on your setup, you may need to install the CUDA 11 versions of libraries that correspond to the CUDA 12 libraries listed in the instructions below.
Use Docker
The libraries (cuBLAS, cuDNN) are installed in this official NVIDIA CUDA Docker images: nvidia/cuda:12.3.2-cudnn9-runtime-ubuntu22.04.
Install with pip (Linux only)
On Linux these libraries can be installed with pip. Note that LD_LIBRARY_PATH must be set before launching Python.
Download the libraries from Purfview's repository (Windows & Linux)
Purfview's whisper-standalone-win provides the required NVIDIA libraries for Windows & Linux in a single archive. Decompress the archive and place the libraries in a directory included in the PATH.
from faster_whisper import WhisperModel
model_size = "large-v3"
# Run on GPU with FP16
model = WhisperModel(model_size, device="cuda", compute_type="float16")
# or run on GPU with INT8
# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
# or run on CPU with INT8
# model = WhisperModel(model_size, device="cpu", compute_type="int8")
segments, info = model.transcribe("audio.mp3", beam_size=5)
print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
Warning:segments is a generator so the transcription only starts when you iterate over it. The transcription can be run to completion by gathering the segments in a list or a for loop:
segments, _ = model.transcribe("audio.mp3")
segments = list(segments) # The transcription will actually run here.
Batched Transcription
The following code snippet illustrates how to run batched transcription on an example audio file. BatchedInferencePipeline.transcribe is a drop-in replacement for WhisperModel.transcribe
from faster_whisper import WhisperModel, BatchedInferencePipeline
model = WhisperModel("turbo", device="cuda", compute_type="float16")
batched_model = BatchedInferencePipeline(model=model)
segments, info = batched_model.transcribe("audio.mp3", batch_size=16)
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
Faster Distil-Whisper
The Distil-Whisper checkpoints are compatible with the Faster-Whisper package. In particular, the latest distil-large-v3
checkpoint is intrinsically designed to work with the Faster-Whisper transcription algorithm. The following code snippet
demonstrates how to run inference with distil-large-v3 on a specified audio file:
from faster_whisper import WhisperModel
model_size = "distil-large-v3"
model = WhisperModel(model_size, device="cuda", compute_type="float16")
segments, info = model.transcribe("audio.mp3", beam_size=5, language="en", condition_on_previous_text=False)
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))
For more information about the distil-large-v3 model, refer to the original model card.
Word-level timestamps
segments, _ = model.transcribe("audio.mp3", word_timestamps=True)
for segment in segments:
for word in segment.words:
print("[%.2fs -> %.2fs] %s" % (word.start, word.end, word.word))
VAD filter
The library integrates the Silero VAD model to filter out parts of the audio without speech:
The default behavior is conservative and only removes silence longer than 2 seconds. See the available VAD parameters and default values in the source code. They can be customized with the dictionary argument vad_parameters:
See more model and transcription options in the WhisperModel class implementation.
Community integrations
Here is a non exhaustive list of open-source projects using faster-whisper. Feel free to add your project to the list!
speaches is an OpenAI compatible server using faster-whisper. It's easily deployable with Docker, works with OpenAI SDKs/CLI, supports streaming, and live transcription.
WhisperX is an award-winning Python library that offers speaker diarization and accurate word-level timestamps using wav2vec2 alignment
whisper-ctranslate2 is a command line client based on faster-whisper and compatible with the original client from openai/whisper.
whisper-diarize is a speaker diarization tool that is based on faster-whisper and NVIDIA NeMo.
whisper-standalone-win Standalone CLI executables of faster-whisper for Windows, Linux & macOS.
asr-sd-pipeline provides a scalable, modular, end to end multi-speaker speech to text solution implemented using AzureML pipelines.
Open-Lyrics is a Python library that transcribes voice files using faster-whisper, and translates/polishes the resulting text into .lrc files in the desired language using OpenAI-GPT.
wscribe is a flexible transcript generation tool supporting faster-whisper, it can export word level transcript and the exported transcript then can be edited with wscribe-editor
aTrain is a graphical user interface implementation of faster-whisper developed at the BANDAS-Center at the University of Graz for transcription and diarization in Windows (Windows Store App) and Linux.
Whisper-Streaming implements real-time mode for offline Whisper-like speech-to-text models with faster-whisper as the most recommended back-end. It implements a streaming policy with self-adaptive latency based on the actual source complexity, and demonstrates the state of the art.
WhisperLive is a nearly-live implementation of OpenAI's Whisper which uses faster-whisper as the backend to transcribe audio in real-time.
Faster-Whisper-Transcriber is a simple but reliable voice transcriber that provides a user-friendly interface.
Open-dubbing is open dubbing is an AI dubbing system which uses machine learning models to automatically translate and synchronize audio dialogue into different languages.
Whisper-FastAPI whisper-fastapi is a very simple script that provides an API backend compatible with OpenAI, HomeAssistant, and Konele (Android voice typing) formats.
Model conversion
When loading a model from its size such as WhisperModel("large-v3"), the corresponding CTranslate2 model is automatically downloaded from the Hugging Face Hub.
We also provide a script to convert any Whisper models compatible with the Transformers library. They could be the original OpenAI models or user fine-tuned models.
For example the command below converts the original "large-v3" Whisper model and saves the weights in FP16:
The option --model accepts a model name on the Hub or a path to a model directory.
If the option --copy_files tokenizer.json is not used, the tokenizer configuration is automatically downloaded when the model is loaded later.
Models can also be converted from the code. See the conversion API.
Load a converted model
Directly load the model from a local directory:
model = faster_whisper.WhisperModel("whisper-large-v3-ct2")
Upload your model to the Hugging Face Hub and load it from its name:
model = faster_whisper.WhisperModel("username/whisper-large-v3-ct2")
Comparing performance against other implementations
If you are comparing the performance against other Whisper implementations, you should make sure to run the comparison with similar settings. In particular:
Verify that the same transcription options are used, especially the same beam size. For example in openai/whisper, model.transcribe uses a default beam size of 1 but here we use a default beam size of 5.
Transcription speed is closely affected by the number of words in the transcript, so ensure that other implementations have a similar WER (Word Error Rate) to this one.
When running on CPU, make sure to set the same number of threads. Many frameworks will read the environment variable OMP_NUM_THREADS, which can be set when running your script:
Trang TopGit này là một snapshot — tab "Readme" hiển thị nguyên văn README của repo (đã bỏ link, giữ ảnh). Repo GitHub ở github.com/SYSTRAN/faster-whisper là nguồn chính thức.
SYSTRAN/faster-whisper có bao nhiêu sao?
SYSTRAN/faster-whisper có 26.7k sao GitHub — tải lại trang để xem số mới nhất, hoặc xem trực tiếp github.com/SYSTRAN/faster-whisper. TopGit phản chiếu số sao của GitHub nhưng không cam kết đến từng phút.
SYSTRAN/faster-whisper có những chủ đề gì?
GitHub topics của SYSTRAN/faster-whisper: "deep-learning", "inference", "openai", "quantization", "speech-recognition", "speech-to-text", "transformer", "whisper". TopGit xếp repo vào nhóm AI Tools.
SYSTRAN/faster-whisper có website riêng không?
TopGit chưa ghi nhận URL trang chủ cho SYSTRAN/faster-whisper. Phần README ở tab phía trên thường có link demo, hoặc xem mô tả GitHub của repo.
SYSTRAN/faster-whisper còn đang phát triển không?
Commit gần nhất trên SYSTRAN/faster-whisper là 9 ngày trước (theo timestamp GitHub). Repo có 2.2k fork — một chỉ báo về mức độ quan tâm của cộng đồng.
SYSTRAN/faster-whisper dùng license gì?
SYSTRAN/faster-whisper phát hành theo license MIT. Nên mở file LICENSE trên GitHub để xác nhận — license metadata đôi khi lệch với thực tế dự án.
SYSTRAN/faster-whisper viết bằng ngôn ngữ gì?
SYSTRAN/faster-whisper chủ yếu viết bằng Python. Trường "language" của GitHub dựa trên phần lớn byte ở nhánh mặc định.
Đọc đầy đủ README ở tab phía trên.
Muốn nghe thêm một ý kiến về faster-whisper?
Hỏi một AI đọc được trang này — một cú bấm là có ngay nhận định về faster-whisper.