memo_fingerprint treats a shared sub-object as visited-anywhere instead of on-path, so aliased and independently-built equal structures fingerprint differently
Describe the bug
_canonicalize's cycle tracking (_seen: dict[int, int]) records every object id it has ever visited during a whole traversal, not just the ones still on the current recursion path. So a second occurrence of a shared sub-object short-circuits to ("ref", ordinal) even when it isn't a real cycle, just the same object referenced twice.
That means two value-equal structures fingerprint differently depending on whether their sub-objects happen to be the same Python object in memory:
shared = [1, 2]
aliased = [shared, shared]
independent = [[1, 2], [1, 2]]
aliased == independent # True
memo_fingerprint(aliased) == memo_fingerprint(independent) # FalseSame shape with dict values. The failure direction I can find is only false negatives (extra reprocessing when nothing actually changed), not false positives.
To Reproduce
Ran the snippet above against a clean cocoindex==1.0.21 install; checked, and this line still matches main today.
Expected behavior
Two structures that are equal by value should fingerprint the same, regardless of whether their sub-objects happen to be aliased.
CocoIndex Version
1.0.21 (file unchanged on main since).
Additional context
Looks like the fix is tracking "on the current DFS path" (push/pop around recursion) rather than "ever seen", so a shared-but-fully-processed sub-object gets canonicalized again instead of turned into a ref. I checked #2256's _CanonicalizeState.remember() and it keeps the same seen-anywhere semantics, so this would still apply after that lands. Happy to send a PR, either now or as a follow-up once #2256 merges, whichever's easier for you to review.
Source: cocoindex-io/cocoindex