#3219·visidata

dedupe and FreqTable group cells differently

Author: midichefCreated Sep 5, 2026Updated Sep 5, 2026
Labelsbug

Small description When considering whether two cell values are the same, dedupe.py and FreqTable use subtly different criteria. They should use the same criteria.

Steps to reproduce

echo '{"val": 1}\n{"val": 1.0}' |vd -f jsonl -

freq-col (F) will count the values as two separate rows. But dedupe-rows will consider the1.0 row to be the same as 1 and remove it.

Additional context Dedupe uses a set internally. It treats two items as duplicates if storing both in a set would give a set of size 1. That means they are equal and have same hash value.

FreqTable groups items differently, by their string representation instead: https://github.com/saulpw/visidata/blob/ea9eecbc5d4be22dca627d911ea1a6347101cb75/visidata/pivot.py#L233 That grouping method has its own problem: objects of different type can have the same string representation. For example, the empty list [] has a string representation of '[0]', so a cell with an empty list will be put in the same group as a string that has the value '[0]'. I think those two objects should be distinct.

(this issue is extracted from an old edit of an issue I previously opened as #3196)