audio.cpp as a unified local ASR + TTS / voice cloning backend? 请支持 audio.cpp 作为本地 ASR + TTS / 声音克隆后端
Hi! I'm the maintainer of audio.cpp.
I noticed pyVideoTrans already supports many different local ASR and TTS engines. This made me wonder if audio.cpp could be useful as a single backend for both ASR and TTS/voice cloning.
audio.cpp currently supports 70+ audio models, including 15 ASR models and 38 TTS/voice clone models, and the list keeps growing through community contributions. Some ASR integrations were contributed directly by the official model teams, including teams from Mistral AI, FunASR, IBM, and Microsoft.
On the TTS side, this includes models such as Qwen3-TTS, IndexTTS 2/2.5, Higgs Audio TTS v3, Fish Audio, PocketTTS, VibeVoice, and many others.
They all run locally through the same C++ runtime, with optimized inference across CUDA, Metal, Vulkan, and HIP.
audio.cpp inference itself has NO Python dependency and provides server APIs as well as CLI support, so pyVideoTrans could potentially access many local ASR, TTS, and voice cloning models through a single backend instead of maintaining separate runtime integrations for each model.
Would this be something you'd be interested in? Happy to answer any questions about audio.cpp.
大家好!我是 audio.cpp 的维护者。
我注意到 pyVideoTrans 已经支持很多不同的本地 ASR 和 TTS 引擎。这让我想到,audio.cpp 是否可以作为一个统一后端,同时支持 ASR、TTS 和声音克隆。
audio.cpp 目前已经支持 70+ 个音频模型,其中包括 15 个 ASR 模型和 38 个 TTS/声音克隆模型,并且还在通过社区贡献持续扩展。其中一些 ASR 集成由官方模型团队直接贡献,包括 Mistral AI、FunASR、IBM 和 Microsoft 的团队。
TTS 方面,目前支持的模型包括 Qwen3-TTS、IndexTTS 2/2.5、Higgs Audio TTS v3、Fish Audio、PocketTTS、VibeVoice 等。
这些模型都可以通过同一个 C++ runtime 在本地运行,并针对 CUDA、Metal、Vulkan 和 HIP 做了性能优化。
audio.cpp 推理本身完全不依赖 Python,同时提供 Server API 和 CLI 支持,因此 pyVideoTrans 可以通过一个统一后端接入多种本地 ASR、TTS 和声音克隆模型,而不需要为每个模型分别维护独立的 runtime 集成。
不知道你们是否会对这种集成方式感兴趣?关于 audio.cpp 的任何问题我都很乐意回答。
Source: jianchang512/pyvideotrans