### Is there an existing issue for this problem?
- [x] I have searched the existing issues
### Install method
Docker image on unRAID server
### Operating system
Linux (unRAID)
### GPU vendor
AMD (ROCm)
### GPU model
AI PRO R9700
### GPU VRAM
32 GiB
### Version number
v6.14.1-post1
### Browser
Edge Version 150.0.4078.48 (Official build) (64-bit)
### System Information
On ROCm, quantized checkpoints appear to be dequantized to full precision during
loading. The memory saving that quantization is chosen for is lost entirely, and the
resident footprint is the same no matter which quantization is used.
GGUF checkpoints are unaffected — they load at approximately their on-disk size.
Evidence 1: three different Z Image quantizations, one resident size
Three distinct Z Image checkpoints, three different file sizes, all reporting the
same loaded size:
Model ID | Size on disk | Total model size as loaded
-- | -- | --
d9006a28-cc54-49d3-93d9-3e1652d2cdf5 | 6.16 GiB | 11,739.56 MB
00ce5266-f09d-4287-a4ea-d8ae89f56263 | 6.54 GiB | 11,739.56 MB
f50e705c-df8a-496f-9acf-491fba3d885f | 12.57 GiB | 11,739.56 MB
Two independent FP8 Krea-2 checkpoints both land at exactly 24,452.35 MB. The GGUF
build of the same model family stays at its on-disk size.
precision: bfloat16 is set in invokeai.yaml, and ~2x is what FP8 → bf16 expansion
would produce.
Consequence
The model no longer fits in the cache, so it is evicted and re-staged on every
generation:
GGUF Krea-2 (12.69 GiB resident, ~21 GiB working set with the encoder):
stays cached, repeated in 0.00s cache hits, 22 s per 960x1344 image.
FP8 Krea-2 (23.88 GiB resident, ~32 GiB working set): never a single cache
hit, full re-stage every generation, 82 s per image, and OOM-killed outright at
container limits below 26 GiB.
The staging phase is host-RAM bound with the GPU idle
Sampling the container cgroup's memory.current and amdgpu's mem_info_vram_used
every 2 seconds during an FP8 load:
The weights are fully materialised in system RAM, then copied to VRAM in ~6 seconds.
Steady-state host RAM afterwards is 2.5 GiB. So host RAM must be at least
model-sized during loading even when VRAM is abundant, and the expansion above
doubles what that costs.
What you expected to happen
That an FP8 checkpoint would occupy roughly its on-disk size in memory, and that
selecting a smaller quantization would reduce the memory footprint.
How to reproduce the problem
ROCm container image on a system with 32 GB VRAM.
Install two quantizations of the same model — e.g. a 6 GiB and a 12 GiB Z Image
checkpoint, or an FP8 and a Q8_0 GGUF Krea-2.
Load each and compare the Total model size value in the
[MODEL CACHE] Loaded model log line against the file size on disk.
The quantized non-GGUF checkpoints report a resident size unrelated to their file
size.
Possible cause
The ROCm image does not contain rocminfo, so bitsandbytes cannot detect the GPU
architecture and falls back. Logged at every startup:
Could not detect ROCm GPU architecture: [Errno 2] No such file or directory: 'rocminfo'
ROCm GPU architecture detection failed despite ROCm being available.
Could not detect ROCm warp size: [Errno 2] No such file or directory: 'rocminfo'.
Defaulting to 64. (some 4-bit functions may not work!)
If quantized kernels are unavailable because the architecture could not be
identified, dequantizing the weights at load would be a plausible fallback — and
would explain why GGUF (which has its own dequantization path) behaves differently.
This is speculation on my part; I have not instrumented it. But rocminfo being
absent from the official ROCm image looks like a packaging defect regardless.
Host: 32 GB system RAM (31 GiB usable), AMD Ryzen 7 3800X (8C/16T), container limited
to 26 GiB and 6 CPUs. Nothing else memory-heavy running. Components in the Krea-2
workflow: Qwen3-VL 4B encoder (8,464.46 MB), Qwen Image VAE (242.03 MB).
What this is not
Recorded so others don't repeat the dead ends I worked through:
Not a leak. Host RAM returns to 2.5–7.8 GiB after every load; nothing
accumulates across generations.
Not a regression. 6.14.0 and 6.14.1-post1 are identical. Earlier versions
could not be tested — Krea-2 support does not exist in 6.13.x.
Not fixable via max_cache_ram_gb. Tested at 8.0, 12.0 and 14.0.
Not glibc fragmentation. The FAQ's MALLOC_MMAP_THRESHOLD_=1048576 workaround
was in place throughout, with DISABLE_PINNED_MEMORY=1.
Not a GPU fault. No amdgpu ring timeouts, resets or segfaults in dmesg.
All OOM kills are CONSTRAINT_MEMCG, contained to the container cgroup, with no
Python traceback.
Three smaller issues noticed along the way
Happy to split these out.
1. The RAM cache statistics block never updates. Byte-identical across
consecutive generations (hits: 2 / misses: 4 / cached: 1 / cleared: 1) despite each
run loading several models.
2. Model load time is attributed to the denoise node.krea2_denoise was billed
at 65.2 s while the 9 sampling steps took ~20 s (2.26 s/it) and the Loaded model
line reported 3.16 s. The ~45 s of staging is folded into denoise and invisible in
the graph stats — the timing table points at sampling when the cost is in loading.
This made the problem substantially harder to locate.
3. force_tiled_decode: true appeared to have no effect. VRAM still spiked to
31.6 GiB at VAE decode with it enabled. Possibly not wired to the Qwen Image VAE
path.
### What you expected to happen
I expected to be able to run the models.
### How to reproduce the problem
Start the app load any model which is not quantized.
### Additional context
_No response_
### Discord username
C_Dollerup