[CuTe, SM80] Forward silently ignores `block_sparse_tensors` and returns dense attention

Author: nien-huiCreated Sep 18, 2026Updated Sep 18, 2026

Problem

On SM80-class GPUs, calling flash_attn_func(..., block_sparse_tensors=...) ignores the block sparse configuration entirely. The kernel computes dense attention and returns it without any error or warning, so the caller believes they got a sparse result when they did not.

Reproduction

Environment: RTX A6000 (SM86), torch 2.14.0+cu130, nvidia-cutlass-dsl 4.7.1, main @ 1bda8f9

Configure every Q block to attend only to KV block 0 (all full blocks, no mask_mod needed), then compare against PyTorch dense / sparse references:

python

import torch, math
from flash_attn.cute import flash_attn_func
from flash_attn.cute.block_sparsity import BlockSparseTensorsTorch

torch.manual_seed(0)
B, S, H, D = 1, 512, 2, 64
q, k, v = [torch.randn(B, S, H, D, device="cuda", dtype=torch.float16) for _ in range(3)]
M, N = S // 128, S // 64
bst = BlockSparseTensorsTorch(
    mask_block_cnt=torch.zeros(B, H, M, dtype=torch.int32, device="cuda"),
    mask_block_idx=torch.zeros(B, H, M, N, dtype=torch.int32, device="cuda"),
    full_block_cnt=torch.ones(B, H, M, dtype=torch.int32, device="cuda"),
    full_block_idx=torch.zeros(B, H, M, N, dtype=torch.int32, device="cuda"),
    block_size=(128, 64),
)
out, lse = flash_attn_func(q, k, v, block_sparse_tensors=bst, return_lse=True)

qt, kt, vt = [x.float().transpose(1, 2) for x in (q, k, v)]
keep = torch.zeros(S, S, dtype=torch.bool, device="cuda"); keep[:, :64] = True
dense = torch.nn.functional.scaled_dot_product_attention(qt, kt, vt).transpose(1, 2)
sparse = torch.nn.functional.scaled_dot_product_attention(qt, kt, vt, attn_mask=keep).transpose(1, 2)
print("vs dense :", (out.float() - dense).abs().max().item())
print("vs sparse:", (out.float() - sparse).abs().max().item())

Output:

vs dense : 0.00014
vs sparse: 0.95

Expected behavior

Either honor the block sparse configuration, or raise an explicit error.

Cause

The SM80 branch in interface.py (arch // 10 == 8) only asserts that paged KV and SplitKV are unsupported. Unlike the SM120 branch, it has no assert not use_block_sparsity.
FlashAttentionForwardSm80.__call__ accepts a blocksparse_tensors argument, but the function body never uses it.

Suggested fix

The minimal fix is to add the following to the SM80 branch:

python

assert not use_block_sparsity, "Block sparsity not supported on SM 8.0"

I am happy to submit a PR for this if the maintainers agree. Whether to actually implement block sparsity in the SM80 kernel is a separate discussion.

Source: Dao-AILab/flash-attention

View original on GitHub View discussion on GitHub