Feature: Add FunASR/SenseVoice as local STT engine
Author: LauraGPTCreated May 31, 2026Updated Jul 15, 2026
Motivation
agenticSeek currently uses Vosk for fully local, real-time microphone transcription. FunASR is a local speech toolkit, and iic/SenseVoiceSmall could be useful as an optional utterance-level backend when its language and accuracy profile fits the user.
This should be additive rather than a direct replacement. The existing Vosk path supports streaming partials, grammar constraints, word timestamps, and TTS echo filtering. SenseVoiceSmall normally consumes a completed utterance, so an integration must preserve the current real-time behavior or make the trade-off explicit.
Proposed scope
- Keep Vosk as the default streaming backend.
- Add a configurable SenseVoice backend, or use SenseVoice as an optional second pass after Vosk detects an utterance boundary.
- Preserve the existing microphone lifecycle and TTS echo-filter behavior.
- Load FunASR lazily and show an actionable dependency/model error when the optional backend is selected.
- Support CPU inference; treat any accelerator support as an optional configuration.
- Document model download size, measured memory usage, supported languages, and the model-specific license.
- Add focused tests for backend selection, missing optional dependencies, final-result normalization, and echo-filter integration.
Capabilities and constraints
- FunASR toolkit source and the SenseVoice repository source are MIT-licensed; model weights follow their individual model cards.
- SenseVoiceSmall supports Chinese, Cantonese, English, Japanese, and Korean. Language coverage and performance vary by checkpoint.
- SenseVoiceSmall can emit language, emotion, and acoustic-event tags; callers should decide whether to retain or strip them.
- A two-clip CPU benchmark against agenticSeek's current Vosk model is included below. It is evidence for this integration discussion, not a universal quality or speed claim.
References
- FunASR: https://github.com/modelscope/FunASR
- SenseVoice: https://github.com/FunAudioLLM/SenseVoice
- SenseVoiceSmall model card: https://huggingface.co/FunAudioLLM/SenseVoiceSmall
- Current agenticSeek STT implementation: https://github.com/Fosowl/agenticSeek/blob/main/sources/speech_to_text.py
Source: Fosowl/agenticSeek