对于联合语言(例如韩语)来说,上下文增强是无效的:词根匹配无法查看复合词内部

Is your feature request related to a problem? Please describe.

LemmaContextAwareEnhancer matches recognizer context words against per-token lemma strings. For Korean (spaCy ko_core_news_sm), lemmas are morpheme-joined strings like 급+여계+좌, so whether a context word survives inside that representation depends on where the statistical morpheme splitter happens to place its + boundaries. The outcome is unpredictable even within the same word class. 계좌 (account) does match 계좌+이체 and 본인+계좌, but not 입금계+좌 (deposit account), 급+여계+좌 (salary account) or 회사+계+좌 (company account). 은행 (bank) matches 신한+은행 but not 국민은+행. It even fails for keywords standing completely alone: 주민등록번호 (resident registration number) lemmatizes to 주민등록번+호 and 예금주 (account holder) to 예금+주, so the keyword misses its own standalone occurrence. I measured this end-to-end over 36 realistic Korean banking/PII sentences (benchmark in the comment below). The current enhancer fires on 16/36 (44%): 3/8 standalone, 7/10 with particles, 6/18 compounds. Raw-text substring matching fires on 36/36, with zero false boosts on the no-context controls. For an English analogy: imagine compounds were written solid, like depositaccount, and your matcher only saw a statistical split like deposit+acc+ount. German compounds have the same shape (Konto inside Gehaltskonto). The existing :-split in NlpArtifacts.set_keywords doesn't help here, since ko uses + and splitting can't recover a keyword the segmentation cuts through. This also limits the effect of the Korean context terms recently added across kr_* recognizers (#1822, #1825, #2212, #2214). Describe the solution you'd like An opt-in SubstringContextAwareEnhancer that matches context words as raw-text substrings within a configurable character window (say ±100 chars) around each detection, keeping the same scoring semantics as the lemma enhancer (context_similarity_factor, min_score_with_context_similarity, MAX_SCORE cap). The cost is bounded by O(context words × window); repeated measurements show no overhead distinguishable from run-to-run noise (details in the benchmark comment). We've been running this in a production Korean PII-masking deployment. Some design questions before I write the PR: 1. Configuration: the enhancer is currently replaceable only programmatically (AnalyzerEngine(context_aware_enhancer=...)). Would you accept an AnalyzerEngineProvider yaml key to select and parameterize the enhancer, with the default unchanged (lemma)? 2. Over-matching: the measurements below show the current default already substring-matches against lemma strings, so raw-text matching has the same English over-match profile (id inside video boosts in both), and the existing context_matching_mode="whole_word" suppresses it in both. Would a per-language matching mode (whole-word for space-delimited languages, raw-substring for agglutinative ones) be the preferred shape? 3. New class, or a raw-text fallback…