StringIndexOutOfBoundsException in TableStructureNormalizer aborts whole document (empty TextChunk substring)

Bug

opendataloader_pdf.convert(...) fails the entire document with java.io.IOException: Parallel page processing failed (CLI return code 1) on table-heavy PDFs. Root cause is an unguarded String.substring during table normalization — TextChunk.getTextChunk calls substring(0, 2) on a length-0 string, throwing StringIndexOutOfBoundsException: begin 0, end 2, length 0. The exception escapes the parallel page stream and aborts the whole file, so no output is produced (no per-page recovery). It is deterministic and immediate (a 4-page file fails in < 1s), so it is not memory/timeout related. Affected files consistently log WARNING: Detected background on page N (filled backgrounds + bordered tables).

Stack trace:

SEVERE: Exception during processing file .pdf: Parallel page processing failed java.io.IOException: Parallel page processing failed at org.opendataloader.pdf.processors.DocumentProcessor.processDocument(DocumentProcessor.java:425) at org.opendataloader.pdf.processors.DocumentProcessor.extractContents(DocumentProcessor.java:195) Caused by: java.util.concurrent.ExecutionException: java.lang.StringIndexOutOfBoundsException at org.opendataloader.pdf.processors.DocumentProcessor.processDocument(DocumentProcessor.java:365) Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end 2, length 0 at java.base/java.lang.String.substring(String.java:2709) at org.verapdf.wcag.algorithms.entities.content.TextChunk.getTextChunk(TextChunk.java:282) at org.verapdf.wcag.algorithms.semanticalgorithms.utils.TextChunkUtils.getTextChunkPartForRange(TextChunkUtils.java:167) at org.opendataloader.pdf.processors.TableStructureNormalizer.addTextChunkToColumns(TableStructureNormalizer.java:113) at org.opendataloader.pdf.processors.TableStructureNormalizer.collectColumnSnapshots(TableStructureNormalizer.java:95) at org.opendataloader.pdf.processors.TableStructureNormalizer.normalize(TableStructureNormalizer.java:64) at org.opendataloader.pdf.processors.TableBorderProcessor.normalizeAndProcessTableBorder(TableBorderProcessor.java:169) at org.opendataloader.pdf.processors.TableBorderProcessor.processTableBorders(TableBorderProcessor.java:100) at org.opendataloader.pdf.processors.DocumentProcessor.lambda$processDocument$7(DocumentProcessor.java:358)

Suggested fix: guard TextChunk.getTextChunk / TextChunkUtils.getTextChunkPartForRange against end > length (and empty chunks), and/or isolate a single failing page/table in DocumentProcessor.processDocument so one bad table does not abort the whole document.

Expected: the empty/degenerate text chunk is skipped or clamped, and the document still produces output.

...

Steps to reproduce

1. Take a machine-generated, table-heavy PDF that also contains page background fills (files that emit WARNING: Detected background on page N).
2. Convert it:
   python复制

   import opendataloader_pdf
   opendataloader_pdf.convert(
       input_path=["file.pdf"], output_dir="out", format="json",                                                                                                                                                  
   )

3. Observe the CLI exits with return code 1 and the StringIndexOutOfBoundsException above; no JSON is written.

Also reproduces with default options and with table_method="cluster", reading_order="off", content_safety_off="off-page" — none avoid it.

Note: the triggering files are confidential internal documents (Korean business reports) and cannot be attached. A minimal synthetic PDF (page background + bordered grid with an empty cell) did not reproduce, so the trigger is a more specific text-run / table-column edge case. Happy to run a debug build or extra logging on the real files privately, or to help isolate a shareable minimal reproducer.

...

Version

2.5.0 (also reproduced on 2.5.1 — latest) ...

Java version

openjdk 17.0.20 2026-07-21

...