#459·olmocr

Writing markdown error : 'gbk' codec can't encode character '\u1eca' in position 3419

Author: dawnchenCreated Apr 29, 2026Updated Apr 29, 2026
Labelsbug

Describe the bug

Windows 11 Conda env python=3.11 remote server at DeepInfra

tried "set PYTHONIOENCODING=utf-8" or "chcp 65001" in Anaconda PowerShell Prompt then run olmocr one file got markdown successfully, another one got "INFO - Writing 1 markdown files for 34c7edfd3ce5eeccdcaa602ca4c03ab5c628649d ERROR - Exception occurred while processing work_hash 34c7edfd3ce5eeccdcaa602ca4c03ab5c628649d: 'gbk' codec can't encode character '\u1eca' in position 3419: illegal multibyte sequence" Traceback olmocr\pipeline.py", line 794, in worker md_f.write(natural_text) UnicodeEncodeError: 'gbk' codec can't encode character '\u1eca' in position 3419: illegal multibyte sequence"

Versions

Python 3.11.15

annotated-doc==0.0.4 anyio==4.13.0 beautifulsoup4==4.14.3 bleach==6.3.0 boto3==1.42.97 botocore==1.42.97 cached_path==1.8.10 certifi==2026.4.22 cffi==2.0.0 charset-normalizer==3.4.7 click==8.3.3 colorama==0.4.6 cryptography==47.0.0 filelock==3.29.0 fsspec==2026.3.0 ftfy==6.3.1 google-api-core==2.30.3 google-auth==2.49.2 google-cloud-core==2.5.1 google-cloud-storage==3.10.1 google-crc32c==1.8.0 google-resumable-media==2.8.2 googleapis-common-protos==1.74.0 h11==0.16.0 hf-xet==1.4.3 httpcore==1.0.9 httpx==0.28.1 huggingface_hub==1.12.0 idna==3.13 Jinja2==3.1.6 jmespath==1.1.0 lingua-language-detector==2.1.1 markdown-it-py==4.0.0 markdown2==2.5.5 markdownify==1.2.2 MarkupSafe==3.0.3 mdurl==0.1.2 olmocr==0.4.27 orjson==3.11.8 packaging==26.0 pillow==12.2.0 proto-plus==1.27.2 protobuf==7.34.1 pyasn1==0.6.3 pyasn1_modules==0.4.2 pycparser==3.0 Pygments==2.20.0 pypdf==6.10.2 pypdfium2==5.7.1 python-dateutil==2.9.0.post0 PyYAML==6.0.3 requests==2.33.1 rich==13.9.4 s3transfer==0.16.1 shellingham==1.5.4 six==1.17.0 smart_open==7.6.0 soupsieve==2.8.3 tqdm==4.67.3 typer==0.25.0 typing_extensions==4.15.0 urllib3==2.6.3 wcwidth==0.6.0 webencodings==0.5.1 wrapt==2.1.2 zstandard==0.25.0