#3859·nltk

Proposal: Simplify complete hierarchical traversal (strict + instance relations) for `Synset`

Author: ekafCreated Sep 4, 2026Updated Sep 5, 2026

Description:

Currently, to traverse the complete hierarchical footprint of a Synset, developers must manually concatenate strict and instance pointers:

  • Upward: self.hypernyms() + self.instance_hypernyms()
  • Downward: self.hyponyms() + self.instance_hyponyms()

This pattern is repeated frequently inside nltk.corpus.reader.wordnet (such as in max_depth, min_depth, and the newly cached _get_hypernym_paths() in #3775). Beyond internal DRYness, this is a standard requirement for downstream NLP tasks, particularly when evaluating Information Content (IC) similarity metrics and tracking inheritance chains:

  • Downward: Required to accurately aggregate propagated corpus frequencies (e.g., from SemCor) across all subsumed concepts.
  • Upward: Required to trace paths back to root nodes and determine the Least Common Subsumer (LCS) between synsets.

From a linguistic standpoint, treating instance relations separately defies user expectation. Querying the hypernyms of an instance like "Einstein" or "Barack Obama" should naturally return categories like "physicist" or "president" without requiring a special flag. Because instance relations were originally included within standard hypernyms/hyponyms prior to WordNet 2.1, we need a clean, centralized way to aggregate these relations.

Below are two architectural options for discussion.

Option 1: Add an include_instances flag to existing methods

We can add an optional flag to hypernyms() and hyponyms(). Instances apply only to nouns, but a Part of Speech check would be redundant, since a simple dictionary lookup is only O(1):

python
def hypernyms(self, include_instances=True):
    hypernyms = self._related('@')
    if include_instances:
        return hypernyms + self._related('@i')
    return hypernyms

def hyponyms(self, include_instances=True):
    hyponyms = self._related('~')
    if include_instances:
        return hyponyms + self._related('~i')
    return hyponyms
  • Design & Trade-offs: Setting include_instances=True as the default restores original pre-2.1 linguistic intuition and matches general user intent. While this changes the historical output of NLTK's WordNet interface since the 2.1 split, the breakage is mild, as it does not alter return types or raise any errors in existing downstream code; it simply yields a more semantically complete list of results. But if instances are included in hypernyms by default, legacy code that continues to add the two relations would output the instances twice, and this risk would need to be documented.
  • Alternative We could avoid the breakage by definining include_instances=False as the default instead.
  • Note: Modifying existing methods requires dropping down to the private self._related() functions internally to avoid infinite recursion, bypassing the standard public API.

Option 2: Introduce separate convenience methods (all_hypernyms / all_hyponyms)

Alternatively, we can leave hypernyms() and hyponyms() completely untouched and introduce dedicated helpers:

python
def all_hypernyms(self):
    return self.hypernyms() + self.instance_hypernyms()

def all_hyponyms(self):
    return self.hyponyms() + self.instance_hyponyms()
  • Design & Trade-offs: This approach avoids any breaking changes or signature alterations. Because it builds entirely on top of the standard public API functions (self.hypernyms(), self.hyponyms(), and self.instance_hypernyms()), it remains clean and decoupled, though it adds two new methods to the Synset interface.