VibeVoice: ASR Speech Recognition for Long Audio
VibeVoice is Microsoft's open-source speech AI family, and the ASR side is the part still fully usable today: a single model that transcribes long recordings in one pass with speaker labels and timestamps. Reach for it if you need long-form, multi-speaker transcripts and can run a 7B model or the BitNet CPU build. Skip it if you came for the TTS demo — Microsoft pulled that code from the repo after misuse.
What is VibeVoice?
VibeVoice is Microsoft's open-source voice AI family covering both automatic speech recognition (ASR) and text-to-speech (TTS), though only the ASR side ships working code in this repo today. VibeVoice-ASR takes up to 60 minutes of audio in one pass and returns a structured transcript marking who spoke, when, and what was said, with customizable hotwords to catch names or jargon.
Key features of VibeVoice ASR
- ✓Single-pass processing of long recordings — up to an hour — instead of chunking that resets speaker tracking every few seconds.
- ✓Structured output that marks Who (speaker), When (timestamp), and What (content) for each turn.
- ✓Customizable hotwords let you feed in names, acronyms, or domain terms before transcribing.
- ✓Natively multilingual — the README lists support for over 50 languages.
- ✓VibeVoice-ASR-BitNet quantizes the model from 4.62 GB down to 1.58 GB for CPU-only inference, with a real-time factor under 1 on 3+ threads.
- ✓Runs continuous speech tokenizers at a 7.5 Hz frame rate through a next-token diffusion setup built on a small LLM, and also ships as a standard Hugging Face Transformers model.
- ✓vLLM inference support for faster serving, documented separately in docs/vibevoice-vllm-asr.md.
- ✓Finetuning code is included if you need to adapt it to your own audio domain.
What is VibeVoice used for?
- •Transcribing hour-long meetings or podcasts where you need to know who said what and when.
- •Building searchable call-center or interview archives with speaker-labeled transcripts.
- •Running ASR on CPU-only edge hardware via the BitNet build when a GPU isn't available.
- •Feeding domain terms and product names into transcription up front to cut down on misrecognition.
Getting started with VibeVoice
The README excerpt here doesn't include a pip install snippet. It points instead to the Hugging Face Transformers integration (VibeVoice-ASR is available as a standard Transformers model), the docs/vibevoice-asr.md guide, and finetuning-asr/README.md for training setup. For the CPU-only build, Microsoft ships a separate project, VibeASR.cpp, rather than Python bindings in this repo.
Running VibeVoice ASR on your audio
For a quick check without installing anything, the README links a hosted Playground at aka.ms/vibevoice-asr. For local runs, VibeVoice-ASR is integrated into the Hugging Face Transformers library, so it loads through the standard Transformers pipeline rather than a custom script — exact code isn't given in the README excerpt here. Need faster serving? docs/vibevoice-vllm-asr.md documents running it through vLLM. The CPU-only route is VibeVoice-ASR-BitNet, run through the separate VibeASR.cpp project instead of this repo's Python code.
Strengths
- ✓One pass on up to an hour of audio beats chunk-and-stitch pipelines for keeping speaker identity straight.
- ✓The BitNet CPU build is a real answer for teams without GPU budget, not just a toy demo.
- ✓Structured Who/When/What output saves you from wiring up your own speaker-tracking glue code.
- ✓MIT license keeps usage terms simple, at least for the code itself.
VibeVoice limitations and responsible use
- △VibeVoice-TTS code was pulled from this repository in September 2025 after Microsoft found it used in ways inconsistent with its stated intent. The code path is gone.
- △According to Microsoft's README, VibeVoice should not be applied to commercial or real-world applications without further testing — it is intended for research and development purposes only.
- △The model inherits biases and errors from its base model (Qwen2.5 1.5B), and the README warns about deepfake and disinformation risk from convincing synthetic speech.
- △Install steps for local inference aren't spelled out beyond pointing at Transformers, vLLM, or the separate VibeASR.cpp project — expect to hunt through linked docs.
Alternatives to VibeVoice
Frequently asked questions
VibeVoice is released under the MIT license, and the ASR code still in the repository is free to use and modify. Microsoft pulled the VibeVoice-TTS code in September 2025, and the README frames the whole project as research and development rather than a finished commercial product.
VibeVoice-ASR and Whisper both do speech-to-text, but VibeVoice-ASR processes up to 60 minutes of audio in a single pass and includes built-in speaker diarization and timestamps, while Whisper is typically run in shorter chunks and needs a separate diarization tool for speaker labels. The README doesn't publish head-to-head accuracy numbers against Whisper.
VibeVoice-ASR-BitNet is the CPU-only route: a quantized build compressed from 4.62 GB to 1.58 GB that Microsoft says runs with a real-time factor under 1 on three or more CPU threads, no GPU required. It ships as a separate project called VibeASR.cpp rather than through the main Python codebase.
VibeVoice-ASR is natively multilingual, and the README states support for over 50 languages, with a full breakdown in docs/vibevoice-asr.md under language distribution. VibeVoice-TTS and the Realtime model support a smaller, separate set of languages, so check each model's own docs before assuming coverage carries over.
Microsoft removed the VibeVoice-TTS code from the repository in September 2025 after finding it used in ways inconsistent with the project's stated research intent. The README cites Microsoft's responsible-AI principles and flags the risk of high-quality synthetic speech being misused for deepfakes or disinformation as the reason.
VibeVoice-ASR was integrated into the Hugging Face Transformers library as of a 2026 release, so you can load it through the standard Transformers pipeline instead of a custom runtime. Exact loading code isn't included in the README excerpt available here, so check docs/vibevoice-asr.md for the current snippet.
Who should try it — and who should skip
Reach for VibeVoice-ASR if you're transcribing long recordings — meetings, podcasts, interviews — and want speaker labels and timestamps without stitching together your own diarization pipeline, and you're fine deploying a 7B model or its BitNet CPU variant on infrastructure you control. Skip it if you came for the TTS demo (that code is gone), if you need a production-ready commercial deployment (Microsoft explicitly says not yet), or if you'd rather not chase install instructions across docs/vibevoice-asr.md, the Transformers integration, and vLLM docs to get running.
Related repositories
Curious whether VibeVoice is right for you?
Let ChatGPT, Claude, or Perplexity look into it — click below and see what AI actually says about VibeVoice.
