nemotron_asr: expose live-input sessions through the existing realtime STT contract
Problem
Nemotron already supports cache-aware encoding and streaming output from a complete waveform, but cannot currently be selected through the generic /v1/realtime STT handler. The handler requires create_streaming_session(), which the Nemotron Model does not implement.
This was confirmed by source inspection at 41537ec5cf79bcf731a0dd6749cac12ab554a691. It is not a report that Nemotron lacks a streaming encoder.
Existing work
#773 and #774 established cache-aware streaming. #878 added reusable incremental frontend/encoder improvements. Those implementations should be reused and credited, not replaced.
Proposed scope
Add a model-owned session adapter implementing the existing feed, cooperative step, close, done and input_sample_rate contract. Reuse StreamingLogMelSpectrogram and ConformerStreamingState, preserve RNNT predictor state between steps, and emit text deltas before end-of-input. Keep decoding work bounded per step and retain only bounded pending audio/encoder state. Define overload behavior explicitly rather than silently dropping input.
Closing input should flush the centered frontend and encoder tail exactly once, then finish pending decoding. Cancellation must release pending state without publishing a false successful final. Independent sessions must not share decoder state. Preserve existing generate/stream_generate behavior.
Verification plan
Tests should cover arbitrary chunk boundaries, partial output before close, empty/silent/short input, repeated close, cancellation and session reset, persistent RNNT state, frontend parity, and bounded retained state. Add a supported-entrypoint WebSocket regression, then Portuguese and English checkpoint tests comparing final output with existing decoding. Measure partial and commit-to-final latency on Apple Silicon; report samples and p50/p95 without promoting unit tests to runtime proof.
Non-goals
No OpenClaw-specific codec, fixed ports, Nativ lifecycle, new parallel WebSocket protocol, ONNX replacement or batch fallback. Nativ adoption belongs in a separate change after this capability is proven.
Maintainer question
Is a focused Nemotron adapter using this existing session protocol the preferred contribution boundary? Are there pending changes to the session lifecycle or RNNT decoder that this should build on?
Source: Blaizzy/mlx-audio