Proposal: Simplify complete hierarchical traversal (strict + instance relations) for `Synset`
Description:
Currently, to traverse the complete hierarchical footprint of a Synset, developers must manually concatenate strict and instance pointers:
- Upward:
self.hypernyms() + self.instance_hypernyms() - Downward:
self.hyponyms() + self.instance_hyponyms()
This pattern is repeated frequently inside nltk.corpus.reader.wordnet (such as in max_depth, min_depth, and the newly cached _get_hypernym_paths() in #3775). Beyond internal DRYness, this is a standard requirement for downstream NLP tasks, particularly when evaluating Information Content (IC) similarity metrics and tracking inheritance chains:
- Downward: Required to accurately aggregate propagated corpus frequencies (e.g., from SemCor) across all subsumed concepts.
- Upward: Required to trace paths back to root nodes and determine the Least Common Subsumer (LCS) between synsets.
From a linguistic standpoint, treating instance relations separately defies user expectation. Querying the hypernyms of an instance like "Einstein" or "Barack Obama" should naturally return categories like "physicist" or "president" without requiring a special flag. Because instance relations were originally included within standard hypernyms/hyponyms prior to WordNet 2.1, we need a clean, centralized way to aggregate these relations.
Below are two architectural options for discussion.
Option 1: Add an include_instances flag to existing methods
We can add an optional flag to hypernyms() and hyponyms(). Instances apply only to nouns, but a Part of Speech check would be redundant, since a simple dictionary lookup is only O(1):
def hypernyms(self, include_instances=True):
hypernyms = self._related('@')
if include_instances:
return hypernyms + self._related('@i')
return hypernyms
def hyponyms(self, include_instances=True):
hyponyms = self._related('~')
if include_instances:
return hyponyms + self._related('~i')
return hyponyms- Design & Trade-offs: Setting
include_instances=Trueas the default restores original pre-2.1 linguistic intuition and matches general user intent. While this changes the historical output of NLTK's WordNet interface since the 2.1 split, the breakage is mild, as it does not alter return types or raise any errors in existing downstream code; it simply yields a more semantically complete list of results. But if instances are included in hypernyms by default, legacy code that continues to add the two relations would output the instances twice, and this risk would need to be documented. - Alternative We could avoid the breakage by definining
include_instances=Falseas the default instead. - Note: Modifying existing methods requires dropping down to the private
self._related()functions internally to avoid infinite recursion, bypassing the standard public API.
Option 2: Introduce separate convenience methods (all_hypernyms / all_hyponyms)
Alternatively, we can leave hypernyms() and hyponyms() completely untouched and introduce dedicated helpers:
def all_hypernyms(self):
return self.hypernyms() + self.instance_hypernyms()
def all_hyponyms(self):
return self.hyponyms() + self.instance_hyponyms()- Design & Trade-offs: This approach avoids any breaking changes or signature alterations. Because it builds entirely on top of the standard public API functions (
self.hypernyms(),self.hyponyms(), andself.instance_hypernyms()), it remains clean and decoupled, though it adds two new methods to theSynsetinterface.
Source: nltk/nltk