RAM caching consumption during training

GPU investigation — 2026-09-06

Fresh reproduction results: small YOLO11n-seg/COCO128-seg runs completed up to 100 epochs, including the supplied augmentations, without a freeze or OOM; post-warm-up memory largely plateaued. The original private-dataset freeze remains unconfirmed. A separate immediate resume serialization crash was reproduced and fixed in PR #26069; that change must not be treated as resolving this issue’s memory symptom.

Current status — 2026-09-05

High-impact report, cause still unverified. The reported configuration is Linux/RTX 4090, segmentation training, Ultralytics 8.4.118, with custom augmentation/integrations. The supplied resumed-run log says cache=False, despite the earlier code showing disk caching. A screenshot of system-wide memory and an eventual freeze does not isolate a growing trainer allocation, dataloader PSS, filesystem cache, or an OOM kill.

Current main 5eda95e83 includes cache-owner fixes (#24670/#24673) and reduced validation prefetch (#26057), but neither is evidence that this report is fixed. Fresh small classification-loader tests here remained stable; they are a different task/workload and do not reproduce or disprove this report.

Smallest next step: rerun the affected segmentation workload on current main with a shareable dataset/config, first workers=0, cache=False, and optional integrations/custom augmentations disabled; capture per-epoch process-tree RSS/PSS, validation boundaries, and kernel OOM logs, then restore the removed inputs one at a time. This is an isolation procedure, not a confirmed workaround or proposed default change. Lower workers/batch can reduce pressure while investigating. No PR has yet been shown to resolve this specific freeze. Keep open; do not infer a cache leak or create another cache implementation from the current evidence.

---

Original report and historical context

Search before asking

• I have searched the Ultralytics YOLO issues and found no similar bug report.

Ultralytics YOLO Component

Train

Bug

hello there,

I have use yolov11-med segmentation model for train on custom dataset and i have noticed that the RAM consumption increase until no memory left and the train craches so any help with that problem ?

Environment

'OS': 'Linux-5.15.0-139-generic-x86_64-with-glibc2.31', 'Environment': 'Linux', 'Python': '3.10.18', 'Install': 'pip', 'Path': '/home/m.tarek_master/python_enviornments/virtual_env/lib/python3.10/site-packages/ultralytics', 'RAM': '62.56 GB', 'Disk': '1570.9/1832.2 GB', 'CPU': '13th Gen Intel Core i9-13900KS', 'CPU count': 32, 'GPU': 'NVIDIA GeForce RTX 4090, 24083MiB', 'GPU count': 1, 'CUDA': '11.8', 'Package Info': {'filelock': '✅ 3.32.2>=3.16.1', 'numpy': '✅ 2.2.6>=1.23.0', 'matplotlib': '✅ 3.10.8>=3.3.0', 'opencv-python': '✅ 5.0.0.93!=4.13.0.90,>=4.7.0', 'pillow': '✅ 12.3.0>=7.1.2', 'pyyaml': '✅ 6.0.3>=5.3.1', 'requests': '✅ 2.34.2>=2.23.0', 'torch': '✅ 2.7.1+cu118!=2.4.0,>=1.8.0; sys_platform == "win32"', 'torchvision': '✅ 0.22.1+cu118>=0.9.0', 'psutil': '✅ 7.2.2>=5.8.0', 'polars': '✅ 1.43.2>=0.20.0', 'nvidia-ml-py': '✅ 13.610.43>=12.0.0', 'ultralytics-thop': '✅ 2.1.6>=2.1.6'}}

Minimal Reproducible Example

N/A

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!