#6596·ogx

Lift the `pypdf <6.14` cap once file-search recordings are re-recorded (pypdf ≥ 6.14 changed PDF extraction)

Author: mattfCreated Sep 19, 2026Updated Sep 19, 2026
Labelsdependenciestech-debt

What is the technical debt you think should be addressed?

We pin pypdf>=6.13.3,<6.14 in pyproject.toml (resolves to 6.13.3) and keep the file-processor / vector-io provider specs at pypdf>=6.13.0 (added while unblocking dependabot PR #6587, commit fa79d3887). That cap is a workaround for a pypdf ≥ 6.14 behavior change, not a real compatibility constraint.

Mechanism: pypdf ≥ 6.14 changed PDF text extraction for the llama_experts PDF used by the file-search integration test. The extracted text differs → different chunks → different file-search context injected into the LLM request → the request-body SHA256 no longer matches any recorded response, so replay mode 500s with Recording not found for request hash.

Verified boundary: 6.13.3 passes tests/integration/responses/test_tool_responses.py::test_response_non_streaming_file_search[...-llama_experts_pdf]; 6.18.1 and 6.19.0 both fail (reproduced locally in replay mode with dummy keys). Only the PDF doc variant is affected (the txt variant passes), which is consistent with the change living in the PDF extraction path (src/ogx/providers/inline/file_processor/pypdf/pypdf.pyPdfReader). The exact breaking release is not yet pinned down — bisect 6.14–6.18 to find it.

Important: the cap is enforced in three places, not just pyproject.toml. The integration runner's ogx stack list-deps | uv pip install step reads the provider pip_packages specs, so src/ogx/providers/registry/file_processors.py (×2) and src/ogx/providers/registry/vector_io.py (DEFAULT_VECTOR_IO_DEPS, ×1) all had to be restored to >=6.13.0. A lock-only revert is not enough — that install step re-installs 6.19.x whenever the spec floor exceeds the locked version.

Because the cap is a hard ceiling, it will keep blocking every future pypdf dependabot bump until the recordings (or the cap) are addressed.

What is the benefit of addressing this technical debt?

  • Restores the ability to track the latest pypdf without a version ceiling, so dependabot python-deps PRs stop needing manual reverts.
  • Removes a "magic" version cap that isn't self-explanatory without this issue for context.

Other thoughts

  • First, bisect pypdf 6.14–6.18 to identify the exact release that changed extraction, and confirm whether it is a regression or a more-correct extraction.
  • Two viable resolutions:
    1. If the newer extraction is acceptable, re-record the affected recordings against the latest pypdf (--inference-mode record-if-missing, which needs an API key or the GitHub record workflow), then remove the cap.
    2. If the change is a regression, pin to the last pypdf release whose extraction still matches and report the behavior change upstream to pypdf.
  • Audit for other integration tests that process PDFs via file upload / file-search (beyond llama_experts_pdf) that may also need re-recording.
  • When removing the cap, update pyproject.toml (×2) and the three provider-spec pip_packages entries in a single PR, and re-run the file-search suite in replay mode to confirm the recordings match again.