flash-attention · Issues· 1308 open

[CuTe, SM80] Forward silently ignores `block_sparse_tensors` and returns dense attention

Windows build fails with CUDA 13.4: missing /Zc:preprocessor, /std:c++17 pinned (plus no sm_121 gencode)

[RFC] FA4 — head_dim=256 & head_dim=512

Support head_dim=512 on SM89 (Ada) for Gemma 4 global attention layers

RuntimeError when the number of tokens is too large

[FA4/B200] return_lse=True makes D64 forward about 19% faster

Cannot install on Windows - Aiter path too long

[FA4][SM120] flash_attn_varlen_func / flash_attn_func illegal memory access at >=4 varlen segments under real-workload memory layouts (b26/b29, GeForce Blackwell)

FA4 SM103 backward preprocess predicate shape mismatch for padded head dimensions

splitkv early-exit path ignores unpadded_lse, writing -inf to the wrong LSE slot

FA4 support for RTX 6000 Pro Blackwell

pip install flash-attn hangs on Google Colab A100 with latest environment

Using `torch.library.wrap_triton` introduces measurable CPU overhead in `rotary`

[FA4] max_seqlen_q tensor causes an argument-annotation mismatch in varlen forward

tests/test_flash_attn.py fails collection without einops