report_audit.py extract: regex-based extraction produces false-positive 'data points' from narrative table cells

Author: allenhangliu-designCreated Jul 19, 2026Updated Jul 19, 2026

Summary

tools/report_audit.py extract walks Markdown tables and pulls out (label, value, unit) triples to build the 15% audit sample. The extraction regex has no word-boundary or context awareness, so it also grabs numbers embedded in ordinary prose/identifiers inside table cells, producing audit sample items that aren't actually verifiable external claims.

Repro

Given a report table like:

markdown
| Metric | Value | Source | Verified |
|---|---|---|---|
| FY2026E non-GAAP EPS guidance | $1.85–$2.25 | company guidance (raised twice in 2026) | — |
markdown
| Strategy | Recommendation |
|---|---|
| If you already hold | ...if you haven't independently checked the RPO gap against the 10-K, that's the single highest-value verification to do next. |

Running extract on a report containing these produces sample items like:

ID 12  FY2026E non-GAAP EPS guidance · Source     2026.00
ID 34  If you already hold · Recommendation         10.00
  • ID 12 comes from the literal word "2026" inside "raised twice in 2026" in the Source column — not a data point at all.
  • ID 34 comes from the "10" in "10-K" (a filing name) inside a free-text recommendation cell.

Neither is a claim that should go into a fetch-and-cross-check sample; asking someone to "verify" that ID 34 = 10 against macrotrends/stockanalysis is meaningless.

Root cause (as far as I can tell from _parse_md_tables / extract_data_points)

  1. _is_valid_label / the column-header skip-list (_SKIP set, and the col_header.upper() in (...) check in extract_data_points) only contains Chinese terms (来源, 说明, 备注, etc.). English report headers like Source, Notes, Verified aren't filtered, so entire narrative columns get scanned for numbers.
  2. The number regex ([\d,,\.]+) has no requirement that the match be a standalone number — it happily matches digits that are part of an alphanumeric token like 10-K, Q4, COVID-19, etc.

Suggested fixes

  • Extend the header skip-list to include common English equivalents: source, sources, notes, note, verified, recommendation, commentary, summary.
  • Require the matched number to be at a word boundary not immediately followed by a letter (e.g. reject matches like 10-K where digits are immediately followed by - + letter forming a known-identifier pattern, or more simply: skip cells where the "number" is immediately adjacent to a letter with no space, since that's almost always an identifier/citation rather than a standalone value).
  • Optionally: let a report opt a specific column out of extraction (e.g. treat any column whose header matches a broader stoplist, or a markdown convention like a column literally titled "Assumption"/"Model Input", as non-factual/non-verifiable, since these are often self-computed scenario outputs rather than externally-sourced claims).

Happy to submit a PR if a maintainer can confirm which fix direction is preferred — the header-skip-list extension is low-risk, but the "assumption vs. fact" distinction is more of a design call.