bug: PDF: the document structure of long mixed native-and-scanned PDFs holds only paragraph nodes, no headings, tables or page breaks
Description
include_document_structure = true on three long PDFs with mixed native and scanned pages returns a document tree that holds nothing but paragraph nodes:
tables in tables |
structure nodes | node types | |
|---|---|---|---|
| A, long, mixed | 14 | 8,146 | paragraph only |
| B, long, mixed | 490 | 34,683 | paragraph only |
| C, short, native | 60 | 1,054 | paragraph only |
The same build returns heading nodes with levels, group, table, page_break and bounding boxes on a synthetic reportlab PDF with 24 pt and 16 pt Helvetica headings. On the real files there is no heading, no table (although hundreds of tables were extracted into tables), no page_break, and no list.
Expected: the tree carries the tables the extractor already found, page breaks between pages, and headings where the font metrics or the tag tree support them. A tree of paragraphs alone does not let a caller audit structure.
Steps to reproduce
- Extract a long PDF with mixed native and scanned pages with
{"pages":{"extract_pages":true},"include_document_structure":true}and the default OCR config. - Count
document.nodesbycontent.node_type. - Compare
counts.tableswith the number oftablenodes.
Relevant files and configuration
xberg v1.2.4, release build, 2026-09-18. Any long public-sector PDF with many tables should reproduce the table gap. extraction_method was mixed for the two large files and native for the short one.
Source: xberg-io/xberg