FR: Highlight the sentence being read aloud from an AirPods triple-press
What problem would this feature solve?
When I listen to a book with Readest's read-aloud, the phone is in my pocket — I'm walking, driving, doing dishes. That's exactly when a sentence lands hardest, and it's exactly when I can't capture it. By the time I've stopped, pulled the phone out, unlocked it, found the passage and dragged a selection over it, the moment is gone and I've usually lost my place too.
Readest already has the right action — Shift+M, "Highlight Current Sentence"
(helpers/shortcuts.ts). It's just unreachable from a pocket. Snipd solves the same
problem for podcasts with a headphone gesture: triple-press saves what you just heard,
no screen, no interruption.
Describe the solution you'd like.
Let the headset "previous track" gesture — an AirPods triple-press, a headset |<<
button, a car control — save the sentence currently being spoken as a highlight,
instead of skipping back a paragraph. Opt-in, off by default, with an audible
confirmation so you know it worked without looking.
The existing plumbing already reaches from the gesture to the annotation, so this is a routing change rather than new machinery:
| Piece | Where it already lives |
|---|---|
AirPods triple-press → previousTrackCommand |
NativeTTSPlugin.swift registerRemoteCommands |
→ media-session-previous event → JS |
libs/mediaSession.ts |
→ previoustrack handler → backward() |
services/tts/ttsMediaBridge.ts |
Spoken sentence → { cfi, text } |
TTSController.getSpokenSentence() |
tts-highlight-sentence → create-tts-highlight |
hooks/useTTSControl.ts |
| → saved annotation, user's default style | annotator/Annotator.tsx |
So the change is: when the setting is on, the previoustrack handler dispatches
tts-highlight-sentence instead of calling backward(). New view setting
ttsHeadsetPrevAction: 'previous-paragraph' | 'highlight-sentence', defaulting to
today's behaviour, with a "Headset Controls → Previous Track Button" row in the TTS
settings panel.
Android gets it for free — the same media-session-previous path carries a Bluetooth
headset's previous button.
Confirming the capture
A gesture you perform without looking needs an answer you can hear, or you end up
pulling the phone out to check — which is the problem the feature exists to solve. So
the capture plays a short rising two-tone blip into the same output the book is playing
through, plus a haptic on iOS/Android via the @tauri-apps/plugin-haptics path the app
already uses.
Two details that seem worth getting right:
- The cue fires only on a successful capture. If
getSpokenSentence()returns null (between segments, or a source with no resolvable range) nothing is announced — telling a user who can't see the screen that a highlight exists when it doesn't is worse than silence. - Web Audio, not an
<audio>element. The LocalSend transfer cues (services/localsend/sounds.ts) use plain<audio>, but a cue that fires during a TTS session can't:ttsMediaBridge.unblockAudiodocuments that a playing HTMLMediaElement makes WebKit register its own Now Playing client, which fights the native session and shows up as a dead "localhost" lock-screen card. A one-shot oscillator avoids that and needs no asset, so nothing has to be fetched offline or backgrounded.
I'd flag honestly that the Web Audio approach still needs verifying on a device: it
should be audible because WKWebView audio rides the app's already-active .playback
session, and a transient oscillator shouldn't register a Now Playing client the way a
media element does — but "shouldn't" isn't "doesn't". If it turns out WebKit registers
anyway, the fallback is a small native play_cue command on the existing
AVAudioSession in the native-tts plugin. Happy to go straight to the native version if
you'd prefer not to risk the regression at all.
The trade-off, stated plainly
Triple-press is the only gesture available. Single is play/pause, double is next track, and press-and-hold belongs to the system for noise control. So highlighting has to come at the cost of "skip back a paragraph" — hence a setting rather than a rebind.
Two things make that cost smaller than it sounds on iOS:
- The lock-screen card renders the skip-interval commands, not prev/next track (per
the comment in
registerRemoteCommands). SopreviousTrackCommandis reached almost only from hardware, and the on-screen controls are unaffected. skipBackwardis already mapped to sentence-levelbackward(true), so the lock screen keeps a way to go back either way.
The group genuinely affected is car head-unit / steering-wheel |<< buttons, which send
the same command. Worth your opinion on whether the setting should be scoped to
headphone routes only, if that's distinguishable in practice.
Describe workarounds you may have used to acheive the same results.
Stopping, unlocking the phone and selecting the passage by hand — which defeats the
point of listening while my hands are busy. Shift+M works on desktop, but there's no
equivalent on a phone in a pocket.
Additional Context
One open design question I'd like your steer on: which sentence, exactly.
getSpokenSentence() reads getLastRange() — the synthesizer's position, which with
buffered audio runs ahead of what the listener actually just heard. There's already
#getCurrentPlaybackRange() in TTSController for precisely this distinction. Should
the gesture use the playback range, or even capture the previous sentence, on the
theory that you reach for your ear after the line has finished? Getting this wrong by
one sentence is the difference between the feature feeling reliable and feeling broken,
and you'll have better instincts than I do about how far the buffer typically runs.
I have a working patch (7 files, ~300 lines including tests) implementing the routing,
the setting and the cue, with unit tests for the bridge branch and the cue; tsc,
biome lint and biome format are clean. I have not yet verified it on a physical iOS
device with AirPods, which is the one thing that actually matters here, so I'd want to
do that before opening the PR. Happy to do so if this direction is welcome — per
CONTRIBUTING.md I'm raising the issue first.
I searched the tracker before filing: #1407, #1949 and #5085 cover AirPods and lock-screen transport controls, but I didn't find an existing request for a gesture-to-highlight binding.
Source: readest/readest