Feature: Add FunASR/SenseVoice as alternative STT engine

Correction and withdrawal (2026-09-13)

I withdraw the unsupported capability, performance, and integration claims in my original proposal below. The repeated proposals were inappropriate. This edit corrects my own record only; it is not a renewed feature request or a request for maintainer action.

• Released SenseVoiceSmall supports Mandarin Chinese, Cantonese, English, Japanese, and Korean, not 50+ languages.
• Non-autoregressive inference does not guarantee constant or deterministic end-to-end latency independent of audio length. I withdraw the unqualified 5x/25x speedups, fixed 50-100 ms latency, real-time factors, accuracy comparisons, and benchmark tables stated here. They are not verified RealtimeSTT measurements.
• FSMN-VAD and ct-punc are separate models in the FunASR toolkit, not built-in components of the SenseVoiceSmall checkpoint. A pipeline must explicitly select and configure them.
• Chunking audio for an offline SenseVoice model is not native streaming ASR. Paraformer streaming is a separate checkpoint. Toolkit WebSocket services and the OpenAI-compatible HTTP server are distinct deployments; the old snippets below are not validated deployment instructions.
• The installation, architecture, memory, and download-count claims below should not be relied on as verified integration evidence. In particular, the old vLLM installation suggestion is withdrawn; it is not a SenseVoice prerequisite.

Reference: official SenseVoice documentation at a pinned revision. No RealtimeSTT implementation, runtime benchmark, or maintainer acceptance is established by this proposal.

Original title: Feature: Add FunASR/SenseVoice as alternative STT engine

[!NOTE] License and capability clarification (2026-07-14): FunASR is a toolkit, not a single checkpoint. The FunASR and SenseVoice repository source code is MIT; model weights follow each model card. SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean, and its weights use the linked FunASR Model Open Source License Agreement. Fun-ASR-Nano-2512 is Apache-2.0. Language coverage, punctuation, and performance depend on the selected model and runtime configuration.

Motivation

RealtimeSTT provides a robust speech-to-text library. FunASR (16K+ stars) could be an additional engine option alongside Whisper, offering faster inference for realtime scenarios.

Why FunASR for Realtime STT

• Non-autoregressive: SenseVoice processes in a single forward pass — deterministic latency
• 5x faster than Whisper large-v3
• 234M params — smaller model, faster load, less VRAM
• Built-in VAD: FSMN-VAD with configurable thresholds
• Built-in punctuation: ct-punc model
• 50+ languages with auto detection
• Streaming ASR: WebSocket-based real-time recognition support
• OpenAI-compatible API: Standard /v1/audio/transcriptions

Integration

from funasr import AutoModel

# Offline (batch)
model = AutoModel(model="iic/SenseVoiceSmall", vad_model="fsmn-vad")
result = model.generate(input="chunk.wav")

# Or via server
# pip install funasr vllm && funasr-server --device cuda

References

• GitHub: https://github.com/modelscope/FunASR
• Streaming: WebSocket realtime ASR
• pip: pip install funasr