#1050·ai-toolkit

MiniMax-H3 ref2va training: clean hang on first step with triton; OOM at load without triton (RTX 5090 32GB, Windows, torch 2.14+cu130)

Author: vibe-p1073rCreated Sep 18, 2026Updated Sep 18, 2026

Environment

  • Windows 11, RTX 5090 32 GB, torch 2.14.0+cu130, python 3.11/3.12 venv
  • ai-toolkit main (cloned 2026-09-18), minimax_h3 extension (arch: minimax_h3_ref2va, partition ref2va_pruned)
  • Weights: Comfy-Org pruned int8 convrot + nvfp4_awq TE from MODELS_PATH (Comfy layout)

Case A — with triton-windows 3.8.0.post28

Training run loads, VRAM fills (~31.9 GiB), then a CLEAN HANG: python CPU time stays 0 s over 25+ s samples, GPU util 0%, no steps logged, no exit. Card was exclusively available (llama server unloaded; only system dwm/LogonUI contexts present) — verified via nvidia-smi compute-apps. First triton use (JIT compile) appears in earlier runs; hang persists after kernels are compiled.

Case B — without triton

Model load path dequantizes shipped convrot8 weights to full precision (~40 GB bf16) and OOMs during .to() staging even with layer_offloading: true and layer_offloading_transformer_percent 0.5 and 0.75 (fail point: module._apply → convert → t.to, cuMemcpyHtoDAsync OOM). So on 32 GB the dequant path cannot fit, and the triton path hangs.

Discriminator

Same weights run inference fine through kohya musubi minimax_h3_generate_video.py (11.7 s/step), so the card/driver handles the int8 forward. We also reproduced an identical first-step hang with musubi's trainer in 3 variants (filed as kohya-ss/musubi-tuner#1123) — pointing at a shared layer (triton-windows convrot kernels in backward? torch 2.14+cu130 on sm_120?) rather than one trainer.

Config: sd_trainer, LoRA linear 16, steps 5, 41-frame bucket 608x1088, cache_text_embeddings true, low_vram true, layer_offloading as above, disable_sampling true.

Happy to share full logs/configs. Any known-good combo for 5090/Windows would be gold.