#544·surya

[BUG: Output] Text inside Diagram/Figure regions is never OCR'd — full_page returns an empty skipped block (detection+recognition reads it fine)

Author: sgarchavadaCreated Aug 6, 2026Updated Aug 19, 2026

Summary

On surya-ocr 0.22.1 with the datalab-to/surya-ocr-2 vLLM backend, a page whose content is a diagram containing text (org chart / flowchart — boxes with text labels) is returned with no text at all. The page is not an error: layout confidently classifies it as Diagram, which is in SKIP_OCR_LABELS, so the text inside it is never transcribed.

The text is perfectly readable by the same model — DetectionPredictor finds 92 text lines on the page, and recognizing those lines returns the labels correctly. Nothing in the normal call paths routes to that, so the content is silently lost.

Version

  • surya-ocr 0.22.1 (latest release)
  • backend: vLLM serving datalab-to/surya-ocr-2

What happens

Page: a scanned organisation chart (~90 short text labels inside boxes).

python
rec = RecognitionPredictor(mgr)
pg = rec([img], full_page=True)[0]
# -> 1 block: label='Diagram', raw_label='Diagram', skipped=True, html=''
#    total extracted text: 0 chars

Block mode does not help, because layout returns the same single Diagram box, which block mode also skips:

python
lay = LayoutPredictor(mgr)
lres = lay([img])          # -> ['Diagram']
rec([img], layout_results=lres, full_page=False)   # -> 0 chars

The text is readable — it just isn't reached

Feeding text-line detection output into block mode returns the real content:

python
det = DetectionPredictor()
d = det([img])[0]          # -> 92 text lines detected
lay = LayoutResult(
    bboxes=[LayoutBox(polygon=b.polygon, label="Text", raw_label="Text", position=i)
            for i, b in enumerate(d.bboxes)],
    image_bbox=d.image_bbox)
rec([img], layout_results=[lay], full_page=False)
# -> 976 chars: "Board of Director | CEO | Vice Precident | Accounts, Finance &
#     Procurement | HR & Admin | Technical | Manager HR & Admin | General Manager |
#     Company Secretary | CFO | Deputy GM | IT | Finance | Procurement | ..."

Why the existing fallback doesn't catch it

_full_page_ocr(..., fallback_layout=...) falls back to block mode only on failure — parse error, empty raw output, or a detected repetition loop. A page confidently classified as one Diagram is a success, so no fallback fires, and SKIP_OCR_LABELS = {"Figure", "Image", "Diagram", "Blank-Page"} means both full-page and block mode skip it.

Note: relabelling the region is not a workaround

Relabelling the Diagram box to Text and running block mode does not return a transcription — it returns a prose caption of the image, with invented detail (e.g. it expanded "CEO" to "Central Executive Director"). So the region has to be routed through text-line detection, not simply re-typed.

Related, for context: #410 (hallucinated text) and #335 (missing text although detected).

Impact

For document-understanding pipelines this is silent data loss: HTTP-level success, correct page count, empty text. Diagrams in scanned submissions (org charts, escalation matrices, process flows) carry real information, and there is no signal in the output that content was skipped unless the caller inspects skipped / label per block.

Suggested fix

Some way to OCR text inside Figure/Diagram regions — either an opt-in flag (e.g. ocr_skipped_regions=True), or routing skipped regions through text-line detection + recognition, which already produces the correct result today.

One caution from testing: running detection unconditionally on skipped regions is not safe by itself. On a genuinely blank page (the back of a double-sided scan, showing only faint bleed-through) the detector reported 11 "lines" and recognition generated ~600 characters of fluent but entirely invented English. Any such path probably needs a content/ink guard before it recognizes.