[Bug] Hybrid docling-fast fails on Windows when the user profile path contains non-ASCII characters
Bug
I was trying to convert a PDF containing Korean text using OpenDataLoader PDF Hybrid mode on Windows.
The PDF I used for testing had 6 pages.
I attempted the conversion using the docling-fast Hybrid backend:
opendataloader-pdf `
--hybrid docling-fast `
--hybrid-mode full `
--output-dir "C:\pdf-test\output" `
"C:\pdf-test\sample.pdf"However, the conversion failed when the Windows user profile path contained non-ASCII characters such as Korean characters.
For example:
C:\Users\한글사용자\Here, 한글사용자 is a fictional example and does not represent the actual Windows username.
The Hybrid Server returns HTTP 500:
Docling Fast Server request failed with status 500:
{"status":"failure","errors":["PDF conversion failed. Check server logs for details."]}The Hybrid Server log contains:
RuntimeError: filesystem error: Cannot convert character sequence: Illegal byte sequenceThe error occurs while opening the temporary PDF file created by tempfile.NamedTemporaryFile().
As an additional test, I processed the same PDF directly using DoclingPdfParser.
The PDF was successfully processed even when located under a Windows user profile path containing Korean characters:
C:\Users\한글사용자\Desktop\sample-한글.pdfIt was also successfully processed when the temporary PDF was located in the default Windows %TEMP% directory:
C:\Users\한글사용자\AppData\Local\Temp\tmpXXXXXX.pdfIn both cases, the PDF was processed successfully:
pages= 6However, using the default DocumentConverter() results in:
RuntimeError: filesystem error: Cannot convert character sequence: Illegal byte sequenceThe traceback points to the threaded PDF parsing path:
docling_parse\pdf_parser.py
...
_threaded_pdf_rendererWhen DoclingParseDocumentBackend is explicitly selected, the same PDF can be successfully converted from the same TEMP directory:
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.backend.docling_parse_backend import DoclingParseDocumentBackend
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
backend=DoclingParseDocumentBackend,
)
}
)
result = converter.convert(pdf_path)
print("pages=", result.document.num_pages)Result:
pages= 6Steps to reproduce
Prepare a PDF containing Korean text.
Use Windows with a user profile path containing Korean or other non-ASCII characters.
For example:
C:\Users\한글사용자\Install the required packages.
Start the OpenDataLoader Hybrid Server using
docling-fast.Run:
opendataloader-pdf `
--hybrid docling-fast `
--hybrid-mode full `
--output-dir "C:\pdf-test\output" `
"C:\pdf-test\sample.pdf"- The Hybrid Server returns HTTP 500:
Docling Fast Server request failed with status 500:
{"status":"failure","errors":["PDF conversion failed. Check server logs for details."]}- The server log shows:
RuntimeError: filesystem error: Cannot convert character sequence: Illegal byte sequenceAs an additional test, copy the same PDF to Python's default TEMP directory and process it directly using
DoclingPdfParser.DoclingPdfParsersuccessfully processes the PDF.Processing the same TEMP file using the default
DocumentConverter()reproduces the error.Explicitly selecting
DoclingParseDocumentBackendallows the same TEMP file to be processed successfully.
Investigation / Applied fix
The relevant OpenDataLoader file is:
<venv>\Lib\site-packages\opendataloader_pdf\hybrid_server.pyIn the create_converter() function, the PDF converter was originally created as follows:
return DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options
)
}
)With this configuration, DocumentConverter uses its default PDF backend.
In the affected environment, the following threaded PDF parsing path is used:
DocumentConverter
-> ThreadedDoclingParseDocumentBackend
-> DoclingThreadedPdfParser
-> _threaded_pdf_rendererThe following error occurs during this process:
RuntimeError: filesystem error: Cannot convert character sequence: Illegal byte sequenceApplied fix
I added the following import:
from docling.backend.docling_parse_backend import DoclingParseDocumentBackendThen I changed:
PdfFormatOption(
pipeline_options=pipeline_options
)to:
PdfFormatOption(
backend=DoclingParseDocumentBackend,
pipeline_options=pipeline_options,
)The final code is:
from docling.backend.docling_parse_backend import DoclingParseDocumentBackend
return DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
backend=DoclingParseDocumentBackend,
pipeline_options=pipeline_options,
)
}
)After this change, the same test PDF was converted successfully.
The existing PDF pipeline configuration remains unchanged, including:
- OCR
- EasyOCR
ko,enOCR languages- Table Structure Processing
- TableFormer ACCURATE mode
- Formula Enrichment
- Picture Description
- Accelerator configuration
In other words, only the PDF backend was changed; the other PDF pipeline settings remain unchanged.
Suspected cause
Based on the test results, the issue appears to be related to the interaction between Windows non-ASCII filesystem paths and the threaded PDF backend.
The default DocumentConverter() follows this path and fails:
DocumentConverter
-> ThreadedDoclingParseDocumentBackend
-> DoclingThreadedPdfParser
-> _threaded_pdf_renderer
-> filesystem errorWhen DoclingParseDocumentBackend is explicitly selected, the following path is used and the conversion succeeds:
DocumentConverter
-> DoclingParseDocumentBackend
-> DoclingPdfParser
-> successful conversionThe test results indicate that the following are not the cause of the issue:
- The PDF itself
- Korean text processing
- General support for Korean/non-ASCII file paths
- The Windows
%TEMP%directory itself - The OpenDataLoader CLI command
Based on the current test results, the issue appears to be a compatibility problem when the threaded PDF backend handles Windows filesystem paths containing non-ASCII characters.
Version
Python: 3.12
docling: 2.126.0
docling-parse: 7.16.0
OpenDataLoader PDF: Hybrid mode
Hybrid backend: docling-fast
OCR: EasyOCR
OCR languages: ko,enJava version
N/AThis issue occurs in the Python/Docling PDF processing stack and is not related to Java.
The username 한글사용자 used above is a fictional example to describe the non-ASCII path issue.
Source: opendataloader-project/opendataloader-pdf