[M1] Cap page-render DPI to bound single-page peak memory (the real lever for #458)

Author: hnc-jgleeCreated Jun 12, 2026Updated Aug 18, 2026
Labelsenhancementmemoryfollow-upfixed-in-dev

Relates to: #458 (OOM), follow-up to #543 · related to #562 (native-resolution extraction)

Background

PR #543 (#458 hotfix) removed accumulation-driven heap growth: BufferedImage.flush() after each write (M0) and dropping the retained ContrastRatioConsumer reference (M2). Independent re-review (decompile of verapdf's render path + before/after build on a public sample, PDFUA-Ref-2-09_Scanned.pdf) confirmed those help documents with many images, but do not lower the single-page render peak, which is the dominant cost on large / high-DPI scanned pages.

The #458 OOM stack is a single large allocation during full-page rendering, before any extraction or flush runs:

java.lang.OutOfMemoryError: Java heap space
  at java.awt.image.DataBufferInt.<init>
  at java.awt.image.Raster.createPackedRaster
  at org.apache.pdfbox.pdmodel.graphics.color.PDDeviceRGB.toRGBImage
  at org.apache.pdfbox.rendering.PDFRenderer.renderImageWithDPI   ← peak allocation here
  at org.verapdf...ContrastRatioConsumer.renderPage
  at org.verapdf...ContrastRatioConsumer.getRenderPage

flush() (M0) runs only after getPageSubImage() returns — i.e. after this allocation has already happened. So #543 cannot move this peak. M1 is the actual memory lever for #458.

Problem

StaticLayoutContainers.getContrastRatioConsumer(sourcePdfPath, password, enableAntialias, imagePixelSize) already threads an imagePixelSize (Float) parameter into the verapdf ContrastRatioConsumer constructor. Decompilation of wcag-algorithms 1.29.45 confirms verapdf uses this value as a scale factor, not an absolute pixel cap — roughly renderDPI ≈ imagePixelSize * max(pageWidth, pageHeight) / pageBounds. But every call site passes null, so no scaling is ever applied:

  • ImagesUtils — image extraction (both writeImage / writePicture paths) → ..., false, null)
  • HiddenTextProcessor (--filter-hidden-text) → ..., false, null) (HiddenTextProcessor.java:45)

With imagePixelSize == null, verapdf renders the full page at its default DPI. A large scanned page at default DPI is a multi-hundred-MB DataBufferInt, which is exactly the allocation that OOMs in #458.

Proposal

  1. Route a non-null imagePixelSize from the call sites into getContrastRatioConsumer(...) so the render is bounded. Candidate target: ~144 DPI effective — note this is the current implicit default that #562 wants to raise (see Related), so M1 and #562 pull in opposite directions and must be reconciled. The exact value follows from the verapdf scale-factor formula above, not a hard DPI constant.
  2. Translate the cap through verapdf's scale-factor semantics. Since imagePixelSize is a scale factor (confirmed by decompile), the implementation must compute the imagePixelSize that yields the desired effective DPI for a given page size, rather than passing a DPI directly. Validate on a large page that the resulting raster size is bounded as intended before locking a value.
  3. Make the cap configurable via a --max-render-dpi-style flag (forward-compatible with S2 --rendering-strategy, but does not require S2 to land), so users who need full-fidelity extraction can opt out — this is the seam that lets #562 coexist.
  4. Add a local memory regression check at a low -Xmx on a large-page fixture (forward-compatible with S6 CI matrix, but does not require the full S6 matrix).

Acceptance criteria

  • Before implementation: document how imagePixelSize maps to effective DPI for a given page (the verapdf scale-factor formula), and the imagePixelSize value chosen to hit the target cap. Record in the PR description.
  • At least one call site passes a non-null imagePixelSize; default behaviour applies a documented effective-DPI cap.
  • Reproducible memory check on PDFUA-Ref-2-09_Scanned.pdf (or an equivalent large scanned page): - Baseline (main): OOMs or peak heap clearly elevated at a stated -Xmx (e.g. the -Xmx value that currently fails). - After fix: completes at that same -Xmx, with a measured peak-heap reduction. - Measurement method stated (e.g. jcmd <pid> GC.heap_info / Java Flight Recorder), observed reduction reported in the PR.
  • No regression in extracted-image output for normal documents below the cap (byte-compare on the existing public sample).
  • Cap is configurable via a flag, not hard-coded.
  • Quality/accuracy impact of the cap on contrast-ratio (hidden-text) detection is assessed and documented.

Closing plan

Keep #458 open until this lands, then close them together. #543 alone is a scoped hotfix (accumulation + diagnostics) and #458 was intentionally kept open for this lever. Do not close #458 on #543's merge alone — that would force a close-then-reopen if M1 slips. The per-reporter heap estimates walked back in #543 (the "≈2 GB" numbers) depend on this cap landing. Reporters should see one coherent roadmap: "#458 → hotfixed by #543 (M0/M2/M3) + closed by M1 (this issue)."

Related

  • #562 — "Extract images at native resolution instead of downscaling to 144 DPI." This is the inverse pressure: #562 wants higher resolution for image quality, M1 wants a bounded render for memory. Both touch the same imagePixelSize lever. The configurable --max-render-dpi flag (proposal #3) is the reconciliation point — it lets M1 cap by default for safety while letting #562's users opt into full fidelity. M1 and #562 should be designed with awareness of each other, but solve different problems and are not duplicates.

Out-of-scope references (from #543)

S1 (duplicate render under --filter-hidden-text), S2 (--rendering-strategy), S3 (LRU eviction), S4 (explicit render phase), S9 (direct sub-image extraction). M1 is the prerequisite for a meaningful memory ceiling; the S-items refine it.

Source: opendataloader-project/opendataloader-pdf