容器内 GPU MMU Fault: ENGINE GRAPHICS GPC1 GPCCLIENT_T1_3，导致推理回退到 CPU，重启容器后问题解决

Author: luoxzhgCreated Aug 26, 2026Updated Aug 26, 2026

Labelsstatus/new-issuetype/bug-report

bug描述 Describe the Bug

paddle版本

root@5d13d5c8e216:/app# paddle --version
<stdin>:3: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
PaddlePaddle 3.3.1, compiled with
    with_avx: ON
    with_gpu: ON
    with_mkl: ON
    with_mkldnn: ON
    with_python: ON

paddle镜像: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.3.1-gpu-cuda11.8-cudnn8.9 GPU 驱动：| NVIDIA-SMI 580.105.08 Driver Version: 580.105.08 CUDA Version: 13.0 | 模型： PP-OCRv5_server_det，PP-OCRv5_server_rec

容器内运行 nvidia-smi 报错：

root@fgg:/data/docparser# docker compose -f compose.yml exec -it ocr-worker-2 bash
root@5d13d5c8e216:/app# nvidia-smi 
Failed to initialize NVML: Unknown Error

重启容器后没有问题

dmesg 命令报错

# dmesg | grep -iE "xid|nvidia|fallen off the bus" | tail -30
[3629619.021460] NVRM: Xid (PCI:0000:00:0e): 31, pid=810735, name=python, channel 0x0000000d, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC1 GPCCLIENT_T1_3 faulted @ 0x7f74_e9d5f000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3795913.936080] NVRM: Xid (PCI:0000:00:07): 31, pid=208744, name=celery, channel 0x000000dd, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x555b_56a00000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3796144.605563] NVRM: Xid (PCI:0000:00:07): 31, pid=4184852, name=celery, channel 0x000000ba, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x5583_3b3ff000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3796144.990284] NVRM: Xid (PCI:0000:00:07): 31, pid=4184827, name=celery, channel 0x000000d6, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x55c8_a3c79000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3796145.840103] NVRM: Xid (PCI:0000:00:07): 31, pid=4184802, name=celery, channel 0x000000da, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x55e5_97168000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3796156.334885] NVRM: Xid (PCI:0000:00:07): 31, pid=4184815, name=celery, channel 0x0000009d, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x55c8_ad6d2000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[3796156.770484] NVRM: Xid (PCI:0000:00:07): 31, pid=4184812, name=celery, channel 0x000000ad, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x55e5_b1b08000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[4315366.013027] NVRM: Xid (PCI:0000:00:08): 31, pid=3627709, name=celery, channel 0x00000058, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x5601_c3d81000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[4368739.111383] NVRM: Xid (PCI:0000:00:09): 31, pid=2224768, name=celery, channel 0x00000038, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x5569_e44f7000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[4373197.276965] NVRM: Xid (PCI:0000:00:0a): 31, pid=2224945, name=celery, channel 0x00000028, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x559b_8a8a7000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ
[4373326.915040] NVRM: Xid (PCI:0000:00:0a): 31, pid=1024717, name=celery, channel 0x00000029, intr 00000000. MMU Fault: ENGINE GRAPHICS GPC2 GPCCLIENT_T1_0 faulted @ 0x5633_f006f000. Fault is of type FAULT_PDE ACCESS_TYPE_VIRT_READ

其他补充信息 Additional Supplementary Information

No response

Source: PaddlePaddle/Paddle

View original on GitHub View discussion on GitHub