Cache cleanup deletes folders locked by another process

Author: betacatslingCreated Sep 11, 2026Updated Sep 11, 2026

CacheManager.CacheContext.clean_cache() deletes an old cached folder even when another process holds its lock. The lock failure only continues the inner loop over lock files; execution then reaches folder deletion and orphan-lock cleanup.

Reproduced on master 648fe61b2f239315770fa7678d983eb1e8e6e4c3, Python 3.12/Linux. No ClearML server or credentials are needed:

python

import os
import subprocess
import sys
import tempfile
from pathlib import Path
from clearml.storage.cache import CacheManager

with tempfile.TemporaryDirectory() as root:
    os.environ["CLEARML_CACHE_DIR"] = root
    cache = CacheManager.CacheContext("lock-repro", default_cache_file_limit=1)
    folder = Path(cache.get_cache_folder())
    dataset = folder / "dataset"
    dataset.mkdir(parents=True)
    data = dataset / "data.txt"
    data.write_text("in use")
    os.utime(data, (1, 1))
    os.utime(dataset, (1, 1))
    (folder / "recent").write_text("recent")
    cache.lock_cache_folder(str(dataset))
    try:
        subprocess.run([sys.executable, "-c", "from clearml.storage.cache import CacheManager; CacheManager.CacheContext('lock-repro', default_cache_file_limit=1).clean_cache()"], check=True)
        print(data.exists())  # False; expected True while the folder is locked
    finally:
        cache.unlock_cache_folder(str(dataset))

Expected: preserve the folder and its live lock while another process holds it, then allow eviction after unlock. A stale lock alongside the live lock must not change that protection.

AI assistance was used to investigate and locally verify this behavior.

Source: clearml/clearml

View original on GitHub View discussion on GitHub