extract_numbers.py: a CORRECT multi-year deck reports 5 'high severity' inconsistencies — grouping ignores the period

Author: AB2006-personalCreated Sep 1, 2026Updated Sep 1, 2026

Summary

find_inconsistencies in skills/ib-check-deck/scripts/extract_numbers.py groups figures by category alone and flags anything more than 5% from the largest group. A deck carrying FY2023, FY2024 and FY2025E revenue is therefore self-contradictory by construction — which is every deck with a financial history.

Measured on a four-line deck: a correct deck produced 5 "high severity" inconsistencies, against 1 for a deck containing a genuine contradiction. Precision ≈ 1/6. The true finding is indistinguishable from the noise.

Reproduce

clean.md — correct, no contradiction:

markdown
## Slide 2 — Historical Performance
Revenue grew from $100.0 million in FY2023 to $120.0 million in FY2024.
EBITDA was $20.0 million in FY2023 and $25.0 million in FY2024.
## Slide 5 — Projections
Revenue of $145.0 million is forecast for FY2025E.

planted.md — one genuine contradiction (FY2024 revenue stated twice):

markdown
## Slide 2 — Historical Performance
Revenue was $120.0 million in FY2024.
## Slide 7 — Transaction Summary
FY2024 revenue of $135.0 million supports the valuation.
$ python extract_numbers.py clean.md   --check   ->  5 inconsistencies (all severity "high")
$ python extract_numbers.py planted.md --check   ->  1 inconsistency

On the clean deck it reports revenue $100.0million vs $120.0million, vs $145.0million, and ebitda $20.0million vs $25.0million — all correct figures for different periods. (One of the five is 2023t, the separate parsing defect filed alongside this.)

Cause

python
# lines ~192–210
for num in numbers:
    if num.category != 'other':
        by_category[num.category].append(num)   # period is discarded
...
if diff_pct < 0.05:                              # 5% tolerance on VALUE only

Nothing distinguishes "the same metric stated twice inconsistently" from "the same metric across three periods".

Suggested fix

Key the comparison on (category, period), not category alone:

  • parse a period marker near each figure — FY2024, FY24, 2025E, 2023A;
  • attribute a figure to the period that follows it before the next figure, falling back to the preceding one. Nearest-by-distance is wrong on the commonest sentence there is: in "EBITDA was $20.0m in FY2023 and $25.0m in FY2024", 25.0 sits 13 chars after FY2023 and 18 before FY2024, so distance alone assigns both figures to FY2023 and re-creates the false positive;
  • only report when two values share a category and a period;
  • skip figures with no resolvable period rather than pooling them.

With that change the two fixtures above give 0 and 1 respectively.

Happy to open a PR.

Source: anthropics/financial-services