The English MIMIC lemmatizer fails to initialize with a KeyError
The English MIMIC lemmatizer (package="mimic", processors="tokenize,pos,lemma") fails to initialize with a KeyError when loading the mimic_nocharlm lemma model.
The issue occurs because the model checkpoint does not contain the optional configuration keys:
charlm_forward_file charlm_backward_file
However, the loader assumes these keys are always present and accesses them with dictionary indexing (self.args[...]), which raises a KeyError for the mimic_nocharlm checkpoint.
To Reproduce Install Stanza (tested with versions 1.13.0 and 1.14.0). Download the English MIMIC models. Run:
import stanza
nlp = stanza.Pipeline(
lang="en",
package="mimic",
processors="tokenize,pos,lemma",
use_gpu=False,
)The pipeline fails during initialization with: KeyError: 'charlm_forward_file'
The traceback points to:
stanza/models/lemma/trainer.py
where the loader executes code equivalent to:
self.args["charlm_forward_file"] = args.get(
"charlm_forward_file",
self.args["charlm_forward_file"],
)Expected behavior
The mimic_nocharlm checkpoint should load successfully.
Since this checkpoint is explicitly a no-CharLM model, missing charlm_forward_file and charlm_backward_file entries should be treated as optional and default to None.
Replacing dictionary indexing with .get() resolves the issue without changing the model's behavior.
For example:
self.args["charlm_forward_file"] = args.get(
"charlm_forward_file",
self.args.get("charlm_forward_file"),
)
self.args["charlm_backward_file"] = args.get(
"charlm_backward_file",
self.args.get("charlm_backward_file"),
)before these values are accessed.
After applying this change, the MIMIC lemmatizer loads successfully and produces normal predictions.
Environment: OS: macOS Python version: 3.10.10 Stanza version: 1.13.0 and 1.14.0
Source: stanfordnlp/stanza