Full-page OCR narrates a table as a <p> caption instead of transcribing it — every figure is lost (surya-ocr-2)

Summary

With surya-ocr-2 (Qwen3.5-VL) in full-page mode, a page containing a real data table is sometimes returned with the Table block's html/text as a prose caption (<p>…</p>) that describes the table instead of a <table> that transcribes it. When this happens every cell value is gone — numbers, codes, amounts — while the request still returns 200, the page count is correct, and the page still has plenty of other text. There is no error, no empty page, and nothing signals the loss: silent data loss.

We hit this repeatedly on scanned tax invoices / purchase documents with a line-item table (item code, qty, unit rate, taxable value, tax, amount). We've verified it across many such documents; they are confidential customer records so we can't attach the originals, but the behaviour is consistent and the illustration below matches what we see byte-for-byte in shape.

What we get vs. what we expect

Actual — the Table block comes back as a caption (values absent):

<p>Table with columns: Sr No, Item Code, Description, Qty, UOM, Unit Rate,
Assessable Value, Total Assessable Value, Amount. Includes details for
ACME Concrete Mixer 65XL and Output IGST Rounded Off.</p>

Expected — an actual table with the rows (values present):

<table>
  <tr><td>1</td><td>2132903010</td><td>ACME Concrete Mixer 65XL</td><td>1 EA</td>
      <td>500,000.00</td><td>500,000.00</td><td>50,000.00</td></tr>
  ...
  <tr><td>Invoice Amount</td><td>590,000.00</td></tr>
</table>

Root cause (as far as we can localise it)

The block path reads the same table correctly — only the full-page path captions it. From surya/recognition/__init__.py:

• full_page=True → _full_page_ocr → one HIGH_ACCURACY_BBOX_PROMPT per page ("OCR this image to HTML. Each block is a div…") → the table block is summarised.
• full_page=False → per-block BLOCK_PROMPT ("OCR this block image to HTML.") on the table crop → the table is transcribed in full.

So:

• it is not a detection failure (the text lines are found), and
• it is not the guided-table path — we run with SURYA_GUIDED_TABLE_REC=False.

It is the full-page decode choosing to summarise a table instead of transcribing it, and it seems triggered by dense / slightly-noisy scanned invoice tables.

Two things we confirmed that may help triage:

1. It's the model, not a hard-coded template. The caption phrasing ("Table with columns: <headers>. Includes details for <row samples>.") appears nowhere in the surya source — the model generates it. So it is a templated summary the model learned, and the wording varies per table; a detector must key on structure, not on any fixed string.
2. It's deterministic, not sampling jitter. Re-running full-page OCR on the same page (temp 0) returns the identical caption every time.

Reproduce

1. Take any scanned tax-invoice page whose main body is a multi-column line-item table with amounts.
2. Run full-page OCR (RecognitionPredictor(..., full_page=True) / the whole-page /ocr/enhanced-style call).
3. Inspect the Table block's html: intermittently it is a <p> caption like the one above instead of a <table> — all figures lost, 200 OK, no warning.
4. Re-OCR the same page in block mode (full_page=False) → the full table comes back. (On our reference page: full-page 2277 chars, table captioned; block-mode 2915 chars, every value present.)

Environment: datalab-to/surya-ocr-2, surya-ocr 0.22.1, vLLM 0.27.1, H100, SURYA_GUIDED_TABLE_REC=False.

Impact

For document-understanding pipelines this silently drops the table of amounts — usually the most important content on an invoice — and replaces it with a sentence, on a page that otherwise looks complete.

Our workaround (wrapper-level, structural — in case it helps others)

We keep the full-page result but detect the caption structurally (phrasing- independent) and re-read only those pages via the block path, accepting the result only on a strict content gain so a page that already produced a real <table> can never be regressed:

# STRUCTURAL signal — not string-matching: a block the layout labeled "Table"
# that came back as prose (non-empty html) with NO <table> markup, any wording.
def _is_captioned_table(block) -> bool:
    html = (block.html or "").strip()
    return block.label == "Table" and bool(html) and "<table" not in html.lower()

# For a page with such a block:
#   d    = DetectionPredictor()([page])[0]                    # find the text lines
#   line = RecognitionPredictor()([page],                     # block/line path
#              layout_results=[LayoutResult(bboxes=<d.bboxes as Text>)],
#              full_page=False)[0]
#   if char_count(line) > char_count(full_page):              # strict content gain
#       replace the page with `line`                          # else keep full-page

This recovers the full table on every case we've seen. It's a post-hoc patch, though — a core fix (full-page mode transcribing rather than summarising these tables) would be far better, which is why we're reporting it.

Happy to test any fix against our (private) invoice corpus and report back.