#10932·zeroclaw

[Feature]: Surface the voice-note transcript to the user (STT echo)

Author: johnlam1968Created Sep 17, 2026Updated Sep 17, 2026
Labelsenhancementchannelconfigchannel:telegrampriority:p2status:acceptedfollow-uprisk:medium

Summary

Echo the transcription of an inbound voice note back to the sender, as an opt-in per-channel setting, so a speech-recognition error is visible instead of silently shaping the reply.

Problem statement

Today the transcript is folded into the prompt as [Voice] <text> and is never shown to the user. There is no way to tell "the STT misheard me" apart from "the agent misunderstood me" — both present identically as a reply answering something the user did not say.

Why this matters more than it sounds:

  • Speech recognition fails in ways text input cannot: accents, code-switching (e.g. Cantonese/Mandarin/English inside one sentence), proper nouns, numerics.
  • A wrong transcript silently poisons the entire turn. The agent reasons correctly from a false premise, so the output reads as a reasoning error.
  • The user has no recovery path. They can rephrase, but without knowing what was heard they cannot tell what to rephrase for.

Current state per surface:

surface transcript handling user-visible
Telegram folded in as [Voice] <text>; also cached in voice_transcriptions no — the cache is read only by extract_reply_context() (crates/zeroclaw-channels/src/telegram.rs:2857), i.e. when the user replies to the voice message
Mattermost docs: "The transcript is prefixed [Voice] and becomes the message content" no
voice_wake utterance transcribed, fed to the agent no
voice_call transcription_logging: true writes workspace/logs/calls/<ts>_<call_id>.json no — post-call audit file
voice_duplex transcript arrives from the client n/a — client-side STT

Two details suggest this was contemplated and left unfinished:

  1. TelegramChannel::send_voice() accepts caption: Option<&str> (telegram.rs:3675), but its only caller passes None (telegram.rs:3386), and the voice-reply path (synthesize_and_send_voice) never sets a caption field on its multipart form. The parameter is unused throughout the codebase.
  2. voice_transcriptions is written on every inbound voice note but read in exactly one place.

Proposed solution

A per-channel opt-in, default off:

toml
[channels.telegram.<alias>]
send_transcript = true

When enabled, after a voice note transcribes successfully and before the agent turn replies, send the transcript to the same chat/thread as a text message:

️ 我哋今晚去食海鮮,你睇點?

Design constraints worth fixing in v1:

  • Send before the agent's reply, not after. The user should see what was heard while the answer is still generating.
  • Never route it through TTS. The echo is inbound confirmation, not a reply. If it becomes part of the reply text it is spoken aloud, which is worse than no echo.
  • No new plumbing needed. text is already in scope at the echo site — try_parse_voice_message() builds format!("[Voice] {text}") at telegram.rs:2688.
  • Empty/whitespace transcriptions already short-circuit ("Voice transcription returned empty text, skipping") before this point, so no extra guard is required.
  • Open question: whether the echo should be per-message suppressible, or purely a channel setting. A channel setting is probably enough for v1.

voice_wake (the local-microphone/desktop case) has the same gap and arguably needs it more, since there is no chat history for the user to cross-check against. voice_call could reuse its existing transcription_logging flag or gain a parallel "announce" behaviour.

The Channel trait has no pre/post-parse hook, so this must live in each channel implementation, or as a shared helper those implementations call.

Non-goals / out of scope

  • Changing what is sent to the model (the [Voice] {text} content stays as-is).
  • Translating or summarising the transcript.
  • Adding transcript visibility to the gateway web dashboard or ACP/ZeroCode surfaces, which already render full transcripts.

Alternatives considered

  • Prompt the model to open its reply with the transcript. Rejected: the echo becomes part of the reply text and is therefore spoken by TTS.
  • A WASM channel-plugin with provides = "telegram". Possible today, but requires reimplementing long-poll, send, drafts, reactions and approvals to add one message. Too much surface for one feature.
  • Read the audit log. voice_call has one; Telegram does not, and a post-hoc file does not help the user in the moment.

Acceptance criteria

  • send_transcript (or equivalent) is documented and schema-validated, default off.
  • With it enabled, a voice note produces a transcript message before the agent's reply, in the same chat/thread.
  • The echo is never synthesised to speech, even when output_modality = "voice".
  • With it disabled, behaviour is byte-identical to today.
  • Regression test: voice note with transcription configured, echo on → transcript message emitted; echo off → no extra message.

Architecture impact

crates/zeroclaw-channels/src/telegram.rs (primary), optionally mattermost.rs and voice_wake.rs; crates/zeroclaw-config/src/schema.rs for the new field; docs under docs/book/src/channels/.

Risk and rollback

Risk is low: a new default-off boolean with no effect on the message path to the model. Rollback is removing the key or setting it false. The main way it could regress is by accidentally entering the reply text (which would make it speakable) — an acceptance test should assert the TTS input excludes the echo.

Expected routing

Ordinary feature triage

Next decision surface

Ordinary triage is enough. One design point to settle during scoping: per-channel setting only, or also a per-message suppression escape.

Breaking change?

No

Data hygiene checks

  • I removed personal/sensitive data from examples, payloads, and logs.
  • I used neutral, project-focused wording and placeholders.

Prior art

pi-telegram-stt, a third-party extension for the Pi coding agent, ships this as showTranscript: true (default on), added because the author needed STT-error visibility in daily use. Well-precedented, small, and user-visible.

Related but distinct: #10925 (mirror voice replies on Matrix) covers reply modality, not transcript visibility. This request is orthogonal and applies under any output_modality.