Feature: Voice Channel — local microphone adapter with wake word detection

Author: ar1vit0rCreated Aug 24, 2026Updated Aug 26, 2026
Labelsplatform:allworkstream:connectors

Problem Statement

OpenJarvis currently supports text-based channels (Telegram, Discord, Slack, etc.) but lacks a native voice channel for local microphone interaction. Users who want voice interaction must rely on external tools or build custom integrations.

A built-in voice channel would:

  • Enable hands-free interaction with the AI assistant
  • Support wake-word detection for always-listening capability
  • Provide a complete voice pipeline (wake word → record → STT → agent → TTS)
  • Work offline with local models (openWakeWord, faster-whisper)

Proposed Solution

Add a VoiceChannel class that implements the BaseChannel interface with the following components:

Core Architecture:

VoiceChannel
├── Wake Word Detection (openWakeWord)
│   └── Frame-by-frame streaming detection
├── Audio Capture (sounddevice/PipeWire)
│   └── Microphone input with fallback
├── Speech-to-Text (pluggable)
│   └── OpenAI Whisper, Deepgram, faster-whisper
├── Text-to-Speech (pluggable)
│   └── Kokoro, OpenAI, Cartesia
└── Event Bus
    └── Publish/subscribe for UI integration

Key Features:

  • Wake word detection with configurable keyword and threshold
  • Audio capture with sounddevice (PortAudio) and PipeWire fallback
  • Pluggable STT/TTS backends via registry pattern
  • Event-driven architecture for UI integration
  • Voice sample capture for model improvement (opt-in)
  • Conversation mode with follow-up wake word detection

Dependencies (all optional):

  • sounddevice — microphone input
  • openwakeword — wake word detection
  • numpy — audio processing
  • webrtcvad — voice activity detection

Acceptance Criteria

  • VoiceChannel implements BaseChannel interface
  • Wake word detection works with openWakeWord
  • Audio capture works with sounddevice (and PipeWire fallback)
  • STT/TTS backends are pluggable via registry
  • Events are published for UI integration (VOICE_LISTENING_START, VOICE_SPEAKING_START, etc.)
  • Voice sample capture is opt-in and non-blocking
  • Conversation mode supports follow-up wake word detection
  • Unit tests for core functionality
  • Documentation with usage examples

Alternatives Considered

  1. External voice tools — Users could use external STT/TTS services, but this adds complexity and latency
  2. Plugin system — Could be implemented as a plugin, but voice is a core use case for an AI assistant
  3. Minimal MVP — Start with just wake word + STT, add TTS later — but a complete pipeline is more useful

Additional Context

I have a working implementation in my personal fork (ar1vit0r/OpenJarvis) with:

  • 1,093 lines of production code
  • openWakeWord integration
  • PipeWire audio capture fallback
  • Voice sample capture
  • Conversation mode

The implementation has been tested on real hardware and is running in my daily-use deployment.

Contributor: @ar1vit0r (9 merged PRs in OpenJarvis)