bug: PDF: a text layer that decodes to shifted glyph codes passes the OCR quality gate with quality_score 1.0
Description
A PDF whose embedded fonts use a custom encoding decodes its letters to byte values in the 0x74 to 0x8f range, while digits and punctuation decode normally. Native extraction returns that garbage on every page. With the default OCR config (tesseract, per-page quality gate on) the result is extraction_method = "native", ocr_used = false, quality_score = 1.0, no text_quality warning, and tables are reported. The gate did not fire, the grade calls the text clean, and the caller gets unreadable output with no signal.
force_ocr = true on the same file returns readable text, so OCR is the right route for it.
Expected: a page whose text layer is mostly letters mapped outside the printable ASCII range, or mostly one- and two-character tokens, fails the per-page gate and goes to OCR. The quality_score for such text must not be 1.0. The text_quality grade should fire too: it counts alphanumerics, and bytes 0x74 to 0x8f are not alphanumeric, but the digits and the many short tokens keep the ratio above its threshold.
Steps to reproduce
- Build a PDF with a Type 1 or TrueType font whose custom encoding maps the letters to codes above 0x73 and leaves digits and punctuation in place.
- Extract with the default OCR config and
pages.extract_pages = true. - Read
extraction_method,quality_scoreandprocessing_warnings.
Relevant files and configuration
xberg v1.2.4, release build, 2026-09-18. Config: {"pages":{"extract_pages":true},"include_document_structure":true} with the default tesseract OCR config filled in. Per-page gate statistics were not logged at info level; the xberg::pdf::ocr debug target carries them.
Source: xberg-io/xberg