[Bug]: Garbage/noise images on Intel Mac + AMD eGPU after macOS 15.7.8/15.7.9 — Metal gemv kernel silently breaks F.linear on MPS (root cause + tested workaround

Checklist

• The issue has not been resolved by following the troubleshooting guide
• The issue exists on a clean installation of Fooocus
• The issue exists in the current version of Fooocus
• The issue has not been reported before recently
• The issue has been reported before but has not been fixed yet

What happened?

After updating macOS Sequoia to 15.7.8, Fooocus 2.5.5 on an Intel Mac with an AMD eGPU produces only deterministic pixel garbage for every prompt, every preset and every checkpoint. No error or warning is printed anywhere — generation completes normally, the output is just noise (see attached image). Running with --always-cpu produces correct images, so the models and the install are fine.

I tracked down the root cause. A layer-by-layer bisection of the SDXL UNet (same inputs on MPS vs CPU) showed the first corrupted outputs are time_embed and label_emb (cosine similarity ~0.64 vs CPU) while the conv layers are exact — and those two MLPs are plain nn.Linear running at batch size 1–2. It reduces to pure PyTorch: on the affected system, F.linear and torch.mv silently return wrong values on MPS whenever the input has ≲ 12 rows, while torch.matmul and torch.addmm are always correct. In other words the macOS 15.7.8 update broke the Metal gemv (matrix-vector) kernel, at least for AMD eGPUs. Since SDXL's time/label embeddings run at batch 1–2, their corruption propagates through every ResBlock and the whole image turns to noise — with no exception raised.

Tested workaround / proposed fix: decompose linear/mv into matmul (+ bias add) on MPS — but only when a startup self-test proves the kernel is actually broken, so healthy systems (e.g. Apple Silicon on healthy drivers) keep the fused kernels and see zero change. With this patch, MPS output matches CPU to ~2e-5 and Fooocus generates correctly again on the affected machine.

In modules/patch.py:

def patch_mps_gemv_workaround():
    # macOS 15.7.8/15.7.9 Metal regression (seen on AMD eGPUs): the gemv kernel
    # used by F.linear / torch.mv for inputs with few rows (M <= ~12) silently
    # returns wrong values on MPS, while torch.matmul is unaffected. This
    # corrupts the SDXL time/label embeddings and turns every generation into
    # noise. Self-test at startup and only decompose linear/mv into matmul+add
    # when the kernel is actually broken, so healthy systems keep the fused ops.
    if not (hasattr(torch.backends, 'mps') and torch.backends.mps.is_available()):
        return

    try:
        g = torch.Generator().manual_seed(0)
        x = torch.randn(2, 320, generator=g)
        w = torch.randn(1280, 320, generator=g)
        ref = torch.nn.functional.linear(x, w)
        out = torch.nn.functional.linear(x.to('mps'), w.to('mps')).cpu()
        if torch.allclose(ref, out, rtol=1e-3, atol=1e-3):
            return
    except Exception as e:
        print(f'MPS gemv self-test could not run ({e}); leaving torch untouched.')
        return

    _original_linear = torch.nn.functional.linear
    _original_mv = torch.mv

    def linear_mps_safe(input, weight, bias=None):
        if input.device.type == 'mps':
            out = torch.matmul(input, weight.t())
            if bias is not None:
                out = out + bias
            return out
        return _original_linear(input, weight, bias)

    def mv_mps_safe(input, vec, *, out=None):
        if input.device.type == 'mps' and out is None:
            return torch.matmul(input, vec.unsqueeze(-1)).squeeze(-1)
        if out is None:
            return _original_mv(input, vec)
        return _original_mv(input, vec, out=out)

    torch.nn.functional.linear = linear_mps_safe
    torch.mv = mv_mps_safe
    print('MPS gemv kernel is broken on this system - applied matmul-decomposition workaround.')

called first in patch_all():

def patch_all():
    if ldm_patched.modules.model_management.directml_enabled:
        ldm_patched.modules.model_management.lowvram_available = True
        ldm_patched.modules.model_management.OOM_EXCEPTION = Exception

    patch_mps_gemv_workaround()
    patch_all_precision()
    patch_all_clip()
    ...

I'm happy to open a PR with this. Since Fooocus is in LTS bug-fix mode and this failure is completely silent (no exception — just garbage output), a self-gated defensive patch could save other Mac + eGPU users a very long debugging session. The self-test also means the patch deactivates itself automatically once Apple fixes the driver.

Steps to reproduce the problem

1. Intel Mac with an AMD eGPU (here: Mac mini 2018 + Radeon RX 6900 XT over Thunderbolt 3), running macOS 15.7.8 or 15.7.9
2. Launch Fooocus normally and generate any image with any prompt/preset/checkpoint
3. The output is deterministic pixel garbage; the console shows no error

The bug also reproduces in ~10 lines of pure PyTorch, without Fooocus:

import torch
x = torch.randn(2, 320)          # few rows -> Metal dispatches the gemv kernel
w = torch.randn(1280, 320)

ref = torch.nn.functional.linear(x, w)
out = torch.nn.functional.linear(x.to('mps'), w.to('mps')).cpu()
mm  = (x.to('mps') @ w.to('mps').t()).cpu()

print('F.linear vs CPU maxdiff:', (ref - out).abs().max().item())  # huge (corr ~0.70)
print('matmul   vs CPU maxdiff:', (ref - mm).abs().max().item())   # ~1e-5 (exact)

What should have happened?

Fooocus should generate normal images, as it did for weeks on this exact machine before the macOS 15.7.8 update. At the PyTorch level, F.linear on MPS should return the same values as on CPU (up to float rounding), like torch.matmul still does.

What browsers do you use to access Fooocus?

Google Chrome

Where are you running Fooocus?

Locally

What operating system are you using?

Mac OS 15.7.9

Console logs

The console output is completely normal — that is the point of this bug: the failure is silent. Startup log from the affected machine (nothing unusual appears during generation, no error is ever printed, the progress bar completes normally and the saved image is noise):


Python 3.10 (conda), Fooocus version: 2.5.5
Total VRAM 32768 MB, total RAM 32768 MB
Set vram state to: SHARED
Device: mps
VAE dtype: torch.float32
Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --attention-split
[Fooocus] Loading models ...
(model loading proceeds normally, generation runs at normal speed, output is garbage)


The actual evidence of the bug is the pure-PyTorch reproduction in "Steps to reproduce" (F.linear vs matmul mismatch on MPS), not the logs.

Additional information

Everything I ruled out before finding the root cause:

• Corrupted models: SHA-256 of checkpoints and LoRAs match Hugging Face (juggernautXL_v8Rundiffusion, official sd_xl_base_1.0, sdxl_lcm_lora, offset lora)
• Environment drift: no file in the Python env changed in ~4 months; the only change on the machine was the macOS 15.7.8 update
• Settings: garbage in both Speed and Extreme Speed, with split and quad attention, in fp32 and fp16
• VAE: encode/decode roundtrip on MPS is clean
• CPU generation (--always-cpu) is pixel-perfect → the problem is MPS-only

PyTorch 2.2.2 is the last macOS x86_64 build, so Intel Mac users cannot get a fix from a newer torch — a Fooocus-side workaround is their only option besides CPU rendering.

Possibly related: pytorch/pytorch#187280 (corrupted MPS output on a macOS 27 beta, Apple Silicon).