Add optional local OCR fallback for scanned PDFs
Problem
coworker/pdf_support.py extracts embedded PDF text with pypdf or renders pages as images for a vision-capable model. A scanned or image-only PDF in text fallback mode therefore produces the visible message that it has no extractable text, even when local OCR could recover it.
This is especially limiting for users who select a model without native PDF or vision support, and for workflows that need deterministic local extraction before model inference.
Proposed direction
Add an optional, local OCR fallback behind the existing PDF adaptation flow:
- Try embedded-text extraction first.
- Classify the document as image-only/scanned only when the extracted text is below a documented threshold relative to page count.
- If local OCR is configured, invoke it through a small extractor interface and return bounded text.
- Preserve the current visible note when OCR is unavailable or fails; a PDF must never silently disappear.
- Keep the canonical attachment/history unchanged. OCR should affect only the outbound view, consistent with the current PDF fallback design.
Docling could be one optional local backend, but the interface should not require a hosted service or send files to a third party. I would prefer maintainer direction on the backend and packaging approach before implementing it.
Suggested constraints
- Local-only by default; no external OCR endpoint or document upload.
- Explicit opt-in dependency or process so the default package does not become substantially heavier.
- Hard byte, page, duration, and extracted-character limits.
- Cancellation support for long OCR work.
- Cache by document content digest and extraction mode, extending the existing cache behavior.
- Record whether output came from the embedded text layer or OCR.
- Treat OCR text as untrusted document content.
- Keep embedded-text PDFs on the existing fast path.
Acceptance cases
- A PDF with a usable embedded text layer does not invoke OCR.
- A scanned/image-only PDF invokes the configured local OCR backend and returns bounded text.
- An OCR timeout, unavailable backend, or malformed result produces a visible failure note and does not drop the attachment silently.
- Replaying the same attachment reuses the digest-keyed result.
- Switching later to a model with native PDF support still sends the original PDF because stored history was not mutated.
- Page, byte, time, and output limits are enforced by tests.
Why issue-first
The README notes that features may overlap the maintainers' internal roadmap. This issue is intended to confirm whether local OCR fits that roadmap and which optional backend/package boundary the project would accept before a pull request is prepared.
If this direction fits, I am happy to contribute a narrowly scoped implementation and tests.
Source: andrewyng/openworker