Add FunASR/SenseVoice as a self-hosted STT option
[!NOTE] License and capability clarification (2026-07-14): FunASR is a toolkit, not a single checkpoint. The FunASR and SenseVoice repository source code is MIT; model weights follow each model card. SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean, and its weights use the linked FunASR Model Open Source License Agreement. Fun-ASR-Nano-2512 is Apache-2.0. Language coverage, punctuation, and performance depend on the selected model and runtime configuration.
Summary
This project builds local voice agents with open-source models. FunASR / SenseVoice would be a strong STT option — significantly faster than Whisper with additional capabilities.
Comparison with current STT options
| Whisper | Moonshine | SenseVoice | |
|---|---|---|---|
| Architecture | Transformer (AR) | Transformer | Non-autoregressive |
| GPU speed | 13x realtime | ~5x realtime* | 170x realtime |
| CPU speed | ❌ | ✅ fast | 17x realtime |
| Latency (5s audio) | ~400ms | ~200ms | <30ms |
| Emotion detection | ❌ | ❌ | ✅ Happy/Sad/Angry |
| Languages | 57 | en only | 50+ |
| License | MIT | MIT | custom model license |
*Moonshine benchmarks vary by hardware
For voice agents, the key advantage is latency — SenseVoice's non-autoregressive architecture processes audio in a single forward pass, giving near-instant results even on CPU.
Integration
from funasr import AutoModel
# STT with emotion (useful for agent personality adaptation)
model = AutoModel(model="iic/SenseVoiceSmall", device="cuda")
result = model.generate(input=audio_chunk)
# Output: "Hello, I need help with my order <|HAPPY|>"Or via OpenAI-compatible API (drop-in for existing pipelines):
funasr-server --device cuda
# POST /v1/audio/transcriptionsThe emotion tags are particularly interesting for voice agents — the agent can adapt its response tone based on detected user emotion.
Happy to contribute an integration PR if interested.
Source: huggingface/speech-to-speech