Extremely Long Text results in PanicException, which is hard to catch in python code

For some extremely long sequences, the tokenizer can result in a PanicException. Example

import tiktoken
tokenizer = tiktoken.get_encoding("cl100k_base")
text = "^" * 1000000

tokenizer.encode(text)  # this throws a PanicException

The issue is that PanicException is not caught even while catching Exception, and can only be caught by catching a BaseException, which is too broad.

Would it be possible to raise a better exception for such a scenario (maybe something similar to what was done here ?)

The workaround that I currently have is catching the BaseException, and checking for "PanicException" in the exception message. Not sure if it is the best way to do this. Would be grateful for any guidance :)