#114·minbpe

train() raises ValueError: max() arg is an empty sequence on empty or whitespace-only input

Author: Mefisto04Created Aug 2, 2026Updated Aug 9, 2026

Both BasicTokenizer.train() and RegexTokenizer.train() call max(stats, key=stats.get) without checking whether stats is empty. When there are no byte pairs to merge, training crashes instead of raising a clear error or stopping gracefully.

Steps to reproduce

python
from minbpe import BasicTokenizer, RegexTokenizer

for cls in (BasicTokenizer, RegexTokenizer):
    t = cls()
    t.train("", 260)       # ValueError: max() arg is an empty sequence
    t.train("   ", 260)    # same for RegexTokenizer

Expected behavior

Raise a clear ValueError("cannot train on empty text") before the merge loop, or stop early when stats is empty (similar to encode's if pair not in self.merges: break).

Actual behavior

ValueError: max() arg is an empty sequence

at basic.py:35 and regex.py:56.

Suggested fix

python
stats = get_stats(ids)
if not stats:
    break  # or: raise ValueError("cannot train on empty text")
pair = max(stats, key=stats.get)

Environment

  • minbpe: main
  • Python: 3.11

Notes

Open PR #54 touches this area but there is no tracking issue. Happy to submit a PR with tests if maintainers want this handled.