SmartSub: AI Video Subtitling and Dubbing
SmartSub handles the full video-to-dubbed-subtitled-output pipeline in one desktop app, and the entire workflow can run offline at zero cost using local Whisper or FunASR models. The breadth of integrations — 8 transcription engines, 20 translation services, multiple TTS engines including voice cloning — is unusually complete for a free tool. The catch is that this flexibility comes with real setup friction: GPU acceleration packages, model downloads, and per-service configuration add steps that casual users may find intimidating compared to a simple web service. Reach for this if you need offline-capable batch subtitle and dubbing work; skip it if you just want to transcribe one video occasionally and don't want to manage model files.
What is SmartSub?
SmartSub is an open-source desktop application that generates subtitles from video or audio, translates those subtitles, synthesizes AI narration (with optional voice cloning), and burns or embeds the final subtitles into video output. The entire workflow — transcription, translation, TTS, and encoding — can run entirely offline on the user's machine using whisper.cpp, faster-whisper, FunASR, Qwen3-ASR, FireRedASR, or Parakeet models via the sherpa-onnx runtime, combined with local Kokoro or VITS voices. GPU acceleration is supported on NVIDIA (CUDA), AMD/Intel (Vulkan), and Apple Silicon (Core ML/Metal). The tool supports Windows, macOS, and Linux, and offers an integrated online video downloader using yt-dlp and lux.
Core Features for Video Subtitling
- ✓Offline subtitle generation using 8 transcription engines: whisper.cpp (built-in), faster-whisper, FunASR (SenseVoice, Paraformer-zh), Qwen3-ASR, FireRedASR, NVIDIA Parakeet, local Whisper CLI, and 8 cloud providers — all controllable per task
- ✓20 translation services including free Bing/Google interfaces, DeepLX, Ollama (local LLM), DeepSeek, Gemini, Qwen, and any OpenAI-compatible endpoint, with per-service custom parameter support and import/export of configurations
- ✓Local TTS with voice cloning: Kokoro (103 multilingual voices), VITS Chinese (174 voices), and ZipVoice zero-shot cloning, plus cloud options (Edge TTS, Azure Speech, ElevenLabs, Volcano Engine Doubao)
- ✓GPU acceleration across NVIDIA CUDA (11.8.0/12.2.0/12.4.0/13.0.2), AMD/Intel Vulkan, and Apple Silicon Core ML/Metal, with automatic CPU fallback on failure
- ✓Integrated online video downloader using yt-dlp (for YouTube and 1800+ sites) and lux (for Bilibili, Douyin, Xiaohongshu), with cookie import support to access premium content
How SmartSub Helps with Video Content
- •Transcribe and translate foreign-language videos or public lectures into bilingual subtitles for personal viewing
- •Download a Bilibili or YouTube video directly within the app and apply auto-generated or AI-transcribed subtitles
- •Create multilingual versions of video content for international audiences by translating subtitles and dubbing with local TTS
- •Clone your own voice from a short reference recording and use it to narrate entire videos in any supported language
- •Batch-convert podcast episodes, online course recordings, or meeting recordings into SRT subtitle files for archiving or editing
- •Export final videos with hard-burned hard subtitles (visible in any player) or soft subtitles (switchable tracks in MKV containers)
Why Choose SmartSub?
- ✓Complete offline pipeline with no API key required: local Whisper/FunASR transcription, free Bing/Google translation, local Kokoro/VITS TTS, and local ffmpeg burning
- ✓Batch processing with adjustable concurrency for handling large numbers of videos or audio files
- ✓Wide GPU acceleration coverage: CUDA on NVIDIA, Vulkan on AMD/Intel, Core ML/Metal on Apple Silicon — with automatic CPU fallback
- ✓Voice cloning via ZipVoice (local zero-shot) and cloud services (ElevenLabs, Volcano Engine), with automatic quality checks on reference audio
- ✓Built-in online video downloader with dual-engine support (yt-dlp and lux) eliminates the need for a separate download tool
- ✓Active community with QQ support group and MIT license allowing self-hosting and modification
Known Limitations and Considerations
- △Setup complexity is high compared to web-based alternatives — users must download and manage model files, GPU acceleration packages, and per-service credentials
- △macOS Silicon only: the Intel macOS build does not support GPU acceleration, limiting speed on older Macs
- △FunASR, Qwen3-ASR, FireRedASR, and Parakeet engines all run through sherpa-onnx, meaning the app bundles a native library dependency that occasionally causes compatibility issues
- △Edge TTS is marked as a reverse-engineered interface with no guaranteed availability — the README explicitly warns that service could stop working without notice
- △AI subtitle refinement and advanced features depend on having capable local hardware or paying for cloud LLM services; lower-end hardware will see slow transcription and synthesis times
- △No mobile support — this is a desktop-only application, which limits use cases for creators who need to work on the go
Getting Started with SmartSub
SmartSub provides pre-built installers for Windows (x64), macOS (ARM64 for Apple Silicon, x64 for Intel), and Linux (x64). Download from the GitHub Releases page or via Quark网盘 (a Chinese cloud storage link). macOS users can install via Homebrew with `brew tap buxuku/tap` followed by `brew install --cask smartsub` — this automatically detects chip architecture. After installing, the app guides you to download your first transcription model. GPU acceleration packages (such as specific CUDA versions) can be downloaded from within the app's Engine and Model settings page without manual CUDA Toolkit installation. For source build: clone the repo, run `yarn install`, then `yarn dev` to start development mode.
Other Video Subtitling Tools
Common Questions
Yes — the core transcription engines (whisper.cpp, faster-whisper, FunASR, Qwen3-ASR, FireRedASR, Parakeet), built-in free translation (Bing/Google interface), local Kokoro/VITS TTS, ZipVoice voice cloning, and ffmpeg video encoding are all free with no API key required. Paid cloud services (OpenAI, ElevenLabs, Azure, Tencent Cloud, etc.) are optional add-ons.
SmartSub runs on Windows (x64), macOS (ARM64 for Apple Silicon with Core ML/Metal acceleration, x64 for Intel without GPU), and Linux (x64). Pre-built installers are available on the GitHub Releases page, and macOS users can install via Homebrew.
Yes — the app supports CUDA on NVIDIA GPUs (11.8.0/12.2.0/12.4.0/13.0.2), Vulkan on AMD and Intel GPUs, and Core ML/Metal on Apple Silicon. GPU acceleration packages are downloaded from within the app, and the system automatically falls back to CPU if GPU loading fails.
SmartSub can run its full pipeline offline: transcription via local whisper.cpp, faster-whisper, FunASR, Qwen3-ASR, FireRedASR, or Parakeet models; translation via Ollama local LLM or the built-in free Bing/Google interface; TTS via local Kokoro or VITS engines; and video encoding via the bundled ffmpeg. No internet connection is required for any core step.
Run this terminal command: `sudo xattr -dr com.apple.quarantine /Applications/SmartSub.app` — then relaunch the app. This removes the macOS quarantine attribute that blocks apps downloaded from outside the App Store.
SmartSub is released under the MIT license, allowing free use, modification, and redistribution for personal and commercial projects.
The problem it solves
Most subtitle tools force you to chain together separate services — one for transcription, another for translation, another for TTS, another for video encoding — each with its own API setup, cost structure, and file-format quirks. Free online tools typically cap batch size or quality, while professional solutions carry subscription fees and require cloud upload of your media. There's no single open-source desktop app that ties the entire pipeline together with local model support, GPU acceleration, and a zero-cost baseline.
Who should try it — and who should skip
SmartSub is worth installing if you regularly handle multilingual video content, need everything to stay on your own machine for privacy or cost reasons, and have the hardware (especially an NVIDIA GPU or Apple Silicon Mac) to run local models at acceptable speed. Content creators producing bilingual videos, educators archiving lecture recordings, and translators who want full control over the subtitle pipeline will get the most value. You should skip it if you only need to transcribe a single video occasionally and want zero setup, or if you are working on sensitive content that requires enterprise support and SLA guarantees rather than a community-maintained open-source project. Subscribers to commercial AI dubbing services who are satisfied with turnaround time and costs may also find the self-hosting overhead hard to justify.
Related repositories
Still deciding about SmartSub?
One click hands the question to an AI along with this page — see what it says about SmartSub.
