WordTokenizer conflict with nltk >= 3.8.2
Author: ansesameCreated Aug 13, 2025Updated Oct 15, 2025
First, thanks for the library. Second, I found a problem with the requirements after the release of nltk == 3.8.2 for WordTokenizer
The aforementioned nltk == 3.8.2 solves a remote code execution vulnerability, disabled to obtain "punkt" and replaced it with "punkt_tab". Issues:
- https://github.com/nltk/nltk/issues/3266#issuecomment-2284001819
- https://github.com/nltk/nltk/issues/3293
This creates a problem with newspaper3k when downloading "punkt" in REQUIERED_CORPORA. Then I suggest modify REQUIRED_CORPORA or updating the requierements.py to avoid using nltk >= 3.8.2
Source: codelucas/newspaper