Silent early termination of iterators
Describe the bug
Export terminated unexpectedly with a notification that the export was complete.
To Reproduce
I consistently encountered this issue while exporting JSON from a specific channel. It caused the scraping to stop at the oldest message ID of 3879 instead of going all the way down to 0. Even after ruling out Telegram client rate limits and attempting the export multiple times, the process always consistently terminated right at message 3879.I subsequently modified the logic to only terminate when an empty page (0 items) is returned. After this change, it worked perfectly—it successfully requested the remaining IDs, and the messages could be downloaded correctly using those IDs.
Expected behavior
Therefore please relax the termination rule and add a retry mechanism.
Version
v0.20.4
Which OS are you running tdl on?
Windows
Additional context
messages/iter.go
m.lastBatch = len(msgs.Messages) < m.limit
Source: iyear/tdl