[BUG] 华为昇腾NPU 910B3部署mineru使用pipeline模式报错507035(ACL_ERROR_RT_VECTOR_CORE_EXCEPTION)
Search before asking | 提交之前请先搜索
- I have searched the MinerU Readme and found no similar bug report.
- I have searched the MinerU Issues and found no similar bug report.
- I have searched the MinerU Discussions and found no similar bug report.
Consult the online AI assistant for assistance | 在线 AI 助手咨询
- I have consulted the online AI assistant but was unable to obtain a solution to the issue.
Description of the bug | 错误描述
Description of the bug | 错误描述
华为昇腾910B3 4卡*64G, MinerU使用的官方提供镜像(quay.m.daocloud.io/ascend/vllm-ascend:v0.11.0) 4卡持续运⾏48⼩时持续解析PDF文档 随时间推移,解析成功率持续不可逆下滑
MinerU服务端⽇志中报错:
[ERROR] ERR00100 PTA call acl api failed [Error]: The vector core execution is abnormal. EH9999: Inner Error! rtDeviceSynchronizeWithTimeout execute failed, reason=[vector core exception] EH9999[PID: 7] wait for compute device to finish failed, runtime result = 507035.
MinerU 的 batch_image_analyze 在接住异常后,将其包装成⼀句具有误导性的提示:
RuntimeError: NPU is selected as device, but torch_npu is not available. Please ensure that the torch_npu package is installed correctly
⚠️ 此错误信息具有误导性。 torch_npu 实已成功导⼊并执⾏到 C 扩展层,该⽂案仅为兜底 分⽀,并不表示缺少 Python 包。
关键调⽤链如下: mineru-api (uvicorn) └── FastAPI 异步任务层(_process_task) └── run_parse_job(asyncio.to_thread(do_parse)) └── _process_pipeline └── doc_analyze_streaming └── batch_image_analyze ← pipeline_analyze.py:346 └── torch_npu.npu.set_compile_mode(jit_compile=False) └── torch_npu.npu.synchronize() ← utils.py:72 └── torch_npu._C._npu_synchronize() └── [ACL] AclrtSynchronizeDeviceWithTimeout └── RuntimeError: error code is 507035
How to reproduce the bug | 如何复现
针对MinerU 3.4.0⽂档解析接⼝在910B3 48⼩时持续⾼并发压测中暴露出的严重性能退化现象 华为昇腾910B3 4卡*64G,4卡部署服务实例24个实例(每张卡6个实例) 并发线程数24 (一个实例一个线程同步持续调用MinerU文档解析接口:/file_parse )持续 48.0 ⼩时 测试⽂件1,027 个⽂件(PDF / DOCX),平均⼤⼩2121.8 KB
Operating System Mode | 操作系统类型
Linux
Operating System Version| 操作系统版本
Ubuntu 22.04 LTS (GNU/Linux 5.15.0-101-generic aarch64) npu: Ascend 910B3 Python 3.11.13 mineru 3.4.0 pytorch 2.7.1 pytorch_npu 2.7.1 torchvision 0.22.1
Python version | Python 版本
3.11
Software version | 软件版本 (mineru --version)
>=2.5
Backend name | 解析后端
pipeline
Device mode | 设备模式
npu
Source: opendatalab/MinerU