NV eGPU (TinyGPU, RTX 4090): tinygrad.llm decodes at ~1 % of VRAM bandwidth by default; JITBEAM=2 gives 9× on Q8_0 but a BEAM candidate faulted the device

Measured on an RTX 4090 (AD102) behind an Intel Thunderbolt 5 dock on a Mac mini (macOS 27, TinyGPU org.tinygrad.tinygpu.driver2 1.0.0/3), tinygrad==0.14.0 + tinymesa==25.2.7.2, DEV=NV:NAK, Qwen3-8B from python -m tinygrad.llm --model … --benchmark 20 (decode = steady per-token line; prefill = --serve on a 3.7k-token prompt; kernel time = the sum of the JIT graphs' GPU-side spans from PROFILE=1):

Model file	Decode tok/s	Prefill tok/s (3.7k)	Kernel time / wall per token	Kernels / token	Host→helper RPCs / token
Q4_K_M	3.61 (277 ms)	30	275.0 / 275.2 ms	622	17
Q4_K_M, JITBEAM=2	5.24 (191 ms)	–	189.7 / 190.8 ms	622	17
Q8_0	4.61 (217 ms)	–	–	622	17
Q8_0, JITBEAM=2	42.9 (23.3 ms)	82 (gen 14 tok/s at 3.7k ctx)	22.6 / 23.3 ms	622	17

What it says:

• The eGPU path is not the cost. 17 helper roundtrips per token, and the graphs' GPU-side spans add up to the wall time, so there is no launch/fetch gap to speak of (I had suspected the small-BAR cpu_access → sysmem placement of queues/kernargs; the numbers say no).
• The default kernel schedule is. Per kernel at baseline: p50 20 µs, p90 1.36 ms, max 10.4 ms; three matvec families take 251 of the 275 ms (r_32_32_4_48_2_2_2_32 ×36 at 2.85 ms each is the FFN matvec — 28 MB of Q4_K weights at ~10 GB/s effective on a ~1 TB/s card). JITBEAM=2 lifts Q4_K_M 1.45× and Q8_0 9.3×; the Q8_0 graphs then read 284–408 GB/s, so NAK itself gets a third of the card's bandwidth once the schedule is right. Q4_K's dequant structure seems to be what BEAM cannot reschedule.
• Two rough edges with JITBEAM on this path:
  1. One of four cold searches killed the process with RuntimeError: Device fault detected (raised from ops_nv.py sleep → dev_impl.is_err_state, inside beam_search → _time_program). The device stays in error state for the process; the next open brings the card back (~4 s) and cache.db keeps every finished kernel, so a retry gets through. A candidate faulting the device probably should not end the search — or at least the fault should be recoverable in-process.
  2. Transformer.warmup() generates from a one-token prompt, so it captures the rollout JIT only; the prefill JIT is captured (and, with JITBEAM, searched) on the first real multi-token prompt — a 45 s stall inside the first request here. A warmup that also runs a >32-token prompt would move that to startup.
• After a 3.7k-token prompt, decode is 14 tok/s against 43 on a short prompt: the symbolic-length attention family (r_…_1187_…, the start_pos-shaped kernels) dominates there.

Asks, in order of how much they'd matter to us: (a) would you consider JITBEAM (or shipped schedules) as the default for tinygrad.llm on NV, given a 9× on Q8_0? (b) making a device fault during BEAM timing non-fatal; (c) an NV counterpart of tinygrad/llm/kernels/amd.py for the dequant matvec and attention (the AMD custom kernels exist; NV has none). Happy to run anything on this rig; the per-kernel profile pickle and the DEBUG=2 logs are available.

Repro: pip install tinygrad==0.14.0 tinymesa==25.2.7.2 (plus the tinygrad/llm/kernels copy — the wheel is missing it, filed separately), then DEV=NV:NAK python -m tinygrad.llm --model https://huggingface.co/Qwen/Qwen3-8B-GGUF/resolve/main/Qwen3-8B-Q8_0.gguf --benchmark 20, once without and twice with JITBEAM=2.