Slack indexing silently skips older history after the first 900-message page
Problem
Slack indexing can report SUCCESS after fetching only the first history page of a channel. Older messages remain unindexed.
Reproduced on September 11, 2026, with a self-hosted Docker Compose deployment using onyxdotapp/onyx-backend:latest. The running container contains the same faulty logic as current main.
Evidence
An initial historical run, with a poll range starting at the Unix epoch, reported:
- 116 discovered channels visited.
- 27,059 documents and 27,788 chunks indexed.
- 1,747/1,747 processing batches completed; no recorded indexing errors.
- 20 channels each returned 900 history messages, then zero messages on the next request.
Representative log values (channel details omitted):
First request: len(message_batch)=900 oldest=None latest='1789151380.913839'
Second request: len(message_batch)=0 oldest='1789095238.633369' latest='1789151380.913839'The second request searches after the newest message already retrieved, rather than fetching the older page. Processing-batch completion therefore does not prove complete history coverage.
Cause and reproduction
Slack returns channel history newest first. _get_messages consumes one page and returns whether a cursor exists, but does not retain that cursor. Checkpoint resume then:
- Saves
message_batch[0]["ts"](the newest message). - Uses that timestamp as
oldestfor the next request. - Receives an empty page and marks the channel complete.
To reproduce, index a channel with more than 900 top-level messages and enough human messages to avoid the bot-channel bypass. Compare the first two history requests and check whether older messages were indexed.
The timestamp changes appear to originate in #5427. They remain in the connector. This differs from the incremental thread-reply issue in #14012.
Proposed fix
Keep the original oldest boundary. Resume backward using the last (oldest) message's timestamp as latest, with exclusive boundaries. Alternatively, persist and reuse Slack's pagination cursor.
Add a regression test spanning multiple history pages and checkpoint resume, checking for missing or duplicate messages. Update progress calculations accordingly. Existing affected indexes will need historical backfill or a full re-index after the fix.
Source: onyx-dot-app/onyx