#6219·readest

FR: Highlight the sentence being read aloud from an AirPods triple-press

Author: ravenintheforrestCreated Sep 14, 2026Updated Sep 14, 2026

What problem would this feature solve?

When I listen to a book with Readest's read-aloud, the phone is in my pocket — I'm walking, driving, doing dishes. That's exactly when a sentence lands hardest, and it's exactly when I can't capture it. By the time I've stopped, pulled the phone out, unlocked it, found the passage and dragged a selection over it, the moment is gone and I've usually lost my place too.

Readest already has the right action — Shift+M, "Highlight Current Sentence" (helpers/shortcuts.ts). It's just unreachable from a pocket. Snipd solves the same problem for podcasts with a headphone gesture: triple-press saves what you just heard, no screen, no interruption.

Describe the solution you'd like.

Let the headset "previous track" gesture — an AirPods triple-press, a headset |<< button, a car control — save the sentence currently being spoken as a highlight, instead of skipping back a paragraph. Opt-in, off by default, with an audible confirmation so you know it worked without looking.

The existing plumbing already reaches from the gesture to the annotation, so this is a routing change rather than new machinery:

Piece Where it already lives
AirPods triple-press → previousTrackCommand NativeTTSPlugin.swift registerRemoteCommands
media-session-previous event → JS libs/mediaSession.ts
previoustrack handler → backward() services/tts/ttsMediaBridge.ts
Spoken sentence → { cfi, text } TTSController.getSpokenSentence()
tts-highlight-sentencecreate-tts-highlight hooks/useTTSControl.ts
→ saved annotation, user's default style annotator/Annotator.tsx

So the change is: when the setting is on, the previoustrack handler dispatches tts-highlight-sentence instead of calling backward(). New view setting ttsHeadsetPrevAction: 'previous-paragraph' | 'highlight-sentence', defaulting to today's behaviour, with a "Headset Controls → Previous Track Button" row in the TTS settings panel.

Android gets it for free — the same media-session-previous path carries a Bluetooth headset's previous button.

Confirming the capture

A gesture you perform without looking needs an answer you can hear, or you end up pulling the phone out to check — which is the problem the feature exists to solve. So the capture plays a short rising two-tone blip into the same output the book is playing through, plus a haptic on iOS/Android via the @tauri-apps/plugin-haptics path the app already uses.

Two details that seem worth getting right:

  • The cue fires only on a successful capture. If getSpokenSentence() returns null (between segments, or a source with no resolvable range) nothing is announced — telling a user who can't see the screen that a highlight exists when it doesn't is worse than silence.
  • Web Audio, not an <audio> element. The LocalSend transfer cues (services/localsend/sounds.ts) use plain <audio>, but a cue that fires during a TTS session can't: ttsMediaBridge.unblockAudio documents that a playing HTMLMediaElement makes WebKit register its own Now Playing client, which fights the native session and shows up as a dead "localhost" lock-screen card. A one-shot oscillator avoids that and needs no asset, so nothing has to be fetched offline or backgrounded.

I'd flag honestly that the Web Audio approach still needs verifying on a device: it should be audible because WKWebView audio rides the app's already-active .playback session, and a transient oscillator shouldn't register a Now Playing client the way a media element does — but "shouldn't" isn't "doesn't". If it turns out WebKit registers anyway, the fallback is a small native play_cue command on the existing AVAudioSession in the native-tts plugin. Happy to go straight to the native version if you'd prefer not to risk the regression at all.

The trade-off, stated plainly

Triple-press is the only gesture available. Single is play/pause, double is next track, and press-and-hold belongs to the system for noise control. So highlighting has to come at the cost of "skip back a paragraph" — hence a setting rather than a rebind.

Two things make that cost smaller than it sounds on iOS:

  • The lock-screen card renders the skip-interval commands, not prev/next track (per the comment in registerRemoteCommands). So previousTrackCommand is reached almost only from hardware, and the on-screen controls are unaffected.
  • skipBackward is already mapped to sentence-level backward(true), so the lock screen keeps a way to go back either way.

The group genuinely affected is car head-unit / steering-wheel |<< buttons, which send the same command. Worth your opinion on whether the setting should be scoped to headphone routes only, if that's distinguishable in practice.

Describe workarounds you may have used to acheive the same results.

Stopping, unlocking the phone and selecting the passage by hand — which defeats the point of listening while my hands are busy. Shift+M works on desktop, but there's no equivalent on a phone in a pocket.

Additional Context

One open design question I'd like your steer on: which sentence, exactly. getSpokenSentence() reads getLastRange() — the synthesizer's position, which with buffered audio runs ahead of what the listener actually just heard. There's already #getCurrentPlaybackRange() in TTSController for precisely this distinction. Should the gesture use the playback range, or even capture the previous sentence, on the theory that you reach for your ear after the line has finished? Getting this wrong by one sentence is the difference between the feature feeling reliable and feeling broken, and you'll have better instincts than I do about how far the buffer typically runs.

I have a working patch (7 files, ~300 lines including tests) implementing the routing, the setting and the cue, with unit tests for the bridge branch and the cue; tsc, biome lint and biome format are clean. I have not yet verified it on a physical iOS device with AirPods, which is the one thing that actually matters here, so I'd want to do that before opening the PR. Happy to do so if this direction is welcome — per CONTRIBUTING.md I'm raising the issue first.

I searched the tracker before filing: #1407, #1949 and #5085 cover AirPods and lock-screen transport controls, but I didn't find an existing request for a gesture-to-highlight binding.