Feature: Voice Channel — local microphone adapter with wake word detection
Author: ar1vit0rCreated Aug 24, 2026Updated Aug 26, 2026
Labelsplatform:allworkstream:connectors
Problem Statement
OpenJarvis currently supports text-based channels (Telegram, Discord, Slack, etc.) but lacks a native voice channel for local microphone interaction. Users who want voice interaction must rely on external tools or build custom integrations.
A built-in voice channel would:
- Enable hands-free interaction with the AI assistant
- Support wake-word detection for always-listening capability
- Provide a complete voice pipeline (wake word → record → STT → agent → TTS)
- Work offline with local models (openWakeWord, faster-whisper)
Proposed Solution
Add a VoiceChannel class that implements the BaseChannel interface with the following components:
Core Architecture:
VoiceChannel
├── Wake Word Detection (openWakeWord)
│ └── Frame-by-frame streaming detection
├── Audio Capture (sounddevice/PipeWire)
│ └── Microphone input with fallback
├── Speech-to-Text (pluggable)
│ └── OpenAI Whisper, Deepgram, faster-whisper
├── Text-to-Speech (pluggable)
│ └── Kokoro, OpenAI, Cartesia
└── Event Bus
└── Publish/subscribe for UI integrationKey Features:
- Wake word detection with configurable keyword and threshold
- Audio capture with sounddevice (PortAudio) and PipeWire fallback
- Pluggable STT/TTS backends via registry pattern
- Event-driven architecture for UI integration
- Voice sample capture for model improvement (opt-in)
- Conversation mode with follow-up wake word detection
Dependencies (all optional):
sounddevice— microphone inputopenwakeword— wake word detectionnumpy— audio processingwebrtcvad— voice activity detection
Acceptance Criteria
-
VoiceChannelimplementsBaseChannelinterface - Wake word detection works with openWakeWord
- Audio capture works with sounddevice (and PipeWire fallback)
- STT/TTS backends are pluggable via registry
- Events are published for UI integration (VOICE_LISTENING_START, VOICE_SPEAKING_START, etc.)
- Voice sample capture is opt-in and non-blocking
- Conversation mode supports follow-up wake word detection
- Unit tests for core functionality
- Documentation with usage examples
Alternatives Considered
- External voice tools — Users could use external STT/TTS services, but this adds complexity and latency
- Plugin system — Could be implemented as a plugin, but voice is a core use case for an AI assistant
- Minimal MVP — Start with just wake word + STT, add TTS later — but a complete pipeline is more useful
Additional Context
I have a working implementation in my personal fork (ar1vit0r/OpenJarvis) with:
- 1,093 lines of production code
- openWakeWord integration
- PipeWire audio capture fallback
- Voice sample capture
- Conversation mode
The implementation has been tested on real hardware and is running in my daily-use deployment.
Contributor: @ar1vit0r (9 merged PRs in OpenJarvis)
Source: open-jarvis/OpenJarvis