[BUG: Output] Text inside Diagram/Figure regions is never OCR'd — full_page returns an empty skipped block (detection+recognition reads it fine)
Summary
On surya-ocr 0.22.1 with the datalab-to/surya-ocr-2 vLLM backend, a page whose
content is a diagram containing text (org chart / flowchart — boxes with text
labels) is returned with no text at all. The page is not an error: layout
confidently classifies it as Diagram, which is in SKIP_OCR_LABELS, so the text
inside it is never transcribed.
The text is perfectly readable by the same model — DetectionPredictor finds 92
text lines on the page, and recognizing those lines returns the labels correctly.
Nothing in the normal call paths routes to that, so the content is silently lost.
Version
surya-ocr0.22.1 (latest release)- backend: vLLM serving
datalab-to/surya-ocr-2
What happens
Page: a scanned organisation chart (~90 short text labels inside boxes).
rec = RecognitionPredictor(mgr)
pg = rec([img], full_page=True)[0]
# -> 1 block: label='Diagram', raw_label='Diagram', skipped=True, html=''
# total extracted text: 0 charsBlock mode does not help, because layout returns the same single Diagram box,
which block mode also skips:
lay = LayoutPredictor(mgr)
lres = lay([img]) # -> ['Diagram']
rec([img], layout_results=lres, full_page=False) # -> 0 charsThe text is readable — it just isn't reached
Feeding text-line detection output into block mode returns the real content:
det = DetectionPredictor()
d = det([img])[0] # -> 92 text lines detected
lay = LayoutResult(
bboxes=[LayoutBox(polygon=b.polygon, label="Text", raw_label="Text", position=i)
for i, b in enumerate(d.bboxes)],
image_bbox=d.image_bbox)
rec([img], layout_results=[lay], full_page=False)
# -> 976 chars: "Board of Director | CEO | Vice Precident | Accounts, Finance &
# Procurement | HR & Admin | Technical | Manager HR & Admin | General Manager |
# Company Secretary | CFO | Deputy GM | IT | Finance | Procurement | ..."Why the existing fallback doesn't catch it
_full_page_ocr(..., fallback_layout=...) falls back to block mode only on
failure — parse error, empty raw output, or a detected repetition loop. A page
confidently classified as one Diagram is a success, so no fallback fires, and
SKIP_OCR_LABELS = {"Figure", "Image", "Diagram", "Blank-Page"} means both
full-page and block mode skip it.
Note: relabelling the region is not a workaround
Relabelling the Diagram box to Text and running block mode does not return a
transcription — it returns a prose caption of the image, with invented detail
(e.g. it expanded "CEO" to "Central Executive Director"). So the region has to be
routed through text-line detection, not simply re-typed.
Related, for context: #410 (hallucinated text) and #335 (missing text although detected).
Impact
For document-understanding pipelines this is silent data loss: HTTP-level success,
correct page count, empty text. Diagrams in scanned submissions (org charts,
escalation matrices, process flows) carry real information, and there is no signal
in the output that content was skipped unless the caller inspects skipped /
label per block.
Suggested fix
Some way to OCR text inside Figure/Diagram regions — either an opt-in flag
(e.g. ocr_skipped_regions=True), or routing skipped regions through text-line
detection + recognition, which already produces the correct result today.
One caution from testing: running detection unconditionally on skipped regions is not safe by itself. On a genuinely blank page (the back of a double-sided scan, showing only faint bleed-through) the detector reported 11 "lines" and recognition generated ~600 characters of fluent but entirely invented English. Any such path probably needs a content/ink guard before it recognizes.
Source: datalab-to/surya