[Feature Request] Add option to disable cuDNN/MIOpen backend (workaround for AMD RDNA 4 GPU crashes)
Summary
On AMD RDNA4 GPUs (gfx1200, e.g. Radeon RX 9060 XT), running VAE encoding during latent caching (cache_latents) causes severe GPU-level failures: desktop compositor (KDE) rendering glitches, full system hangs, and GPU resets. This is not limited to sd-scripts — the same crash occurs in ComfyUI when its COMFYUI_ENABLE_MIOPEN=1 option is enabled, and does not occur when disabled.
Root cause
Potentially related issue (while the Windows issue is reported as fixed in the nightly build, I am still affected by it, as it continues to crash in my Linux testing environment):
Proposed solution
The identified workaround is to completely disable the cuDNN/MIOpen backend by inserting the following code immediately after import torch in the script:
torch.backends.cudnn.enabled = False(On ROCm, PyTorch routes convolution through the cuDNN-compatible API, which maps to MIOpen — so this flag is MIOpen's effective on/off switch.)
I confirmed manually that patching this into the training script eliminates the crash/hang/glitch completely.
EDIT 1: Sorry about that. It seemed fine during the first few attempts, but upon further testing, the graphical glitches persisted. At the very least, there is definitely an issue occurring around VAE processing in my environment... However, it is also true that enabling this option has reduced the number of system hangs. (To be honest, the behavior is inconsistent, so I'm not entirely sure what's going on... but I hope this information helps someone.)
EDIT 2: After recreating the venv from scratch and reinstalling the torch-related components, it stopped crashing even with MIOpen enabled... The issue is resolved in ComfyUI as well... I wonder why? Perhaps a series of unfortunate, temporary hardware glitches coincided? In any case, since this is considered a useful option for scenarios where crashes occur with MIOpen enabled, I will leave this feature request open.
Environment
- Arch Linux, kernel 7.1.8 (2026/08/13 updated)
- GPU: AMD Radeon RX 9060 XT (gfx1200, RDNA 4, 16GB VRAM)
- ROCm / PyTorch: 2.13.0+rocm7.2
Source: kohya-ss/sd-scripts