Windows build fails with CUDA 13.4: missing /Zc:preprocessor, /std:c++17 pinned (plus no sm_121 gencode)

Author: tuananhlfcCreated Sep 1, 2026Updated Sep 17, 2026

Hi! I tried building 2.8.4 from source on Windows and hit some failures, all in setup.py's Windows branch. They're build configuration only - no kernel, CUTLASS or host-code changes required. I have them all fixed locally and can open a PR. The README says Windows "might work starting v2.3.2 ... but Windows compilation still requires more testing" and invites people to open an issue, so here it is.

Environment: Windows 11 ARM64, MSVC 14.51 (ARM64 host and target), CUDA 13.4, torch 2.15, CPython 3.13 and 3.14, sm_121 (GB10).

The three things that go wrong, all in build configuration rather than the kernels:

1. There's no sm_121 gencode. cuda_archs() defaults to "80;90;100;110;120" and add_cuda_gencodes() has no 121 branch, so nothing gets emitted for this GPU.

2. CUDA 13.x's CCCL headers refuse MSVC's traditional preprocessor. Every .cu stops here:

cccl/cuda/std/__cccl/preprocessor.h(20): fatal error C1189: #error: MSVC/cl.exe with
traditional preprocessor is used ... pass `/Zc:preprocessor` to cl.exe

The Windows branch passes /Zc:__cplusplus already, just not /Zc:preprocessor.

3. /std:c++17 is pinned, and torch 2.15's headers need C++20:

c10/util/StringUtil.h(171): error C7555: use of designated initializers requires at
  least '/std:c++20'
c10/util/intrusive_ptr.h(775): error C2039: 'strong_ordering': is not a member of 'std'

(There's also M_LOG2E in src/softmax.h(76) coming up undefined, which MSVC only provides with _USE_MATH_DEFINES.)

One thing that might matter more than the ARM64 angle: only (1) is actually architecture-specific. (2) and (3) look like they'd hit Windows x64 with the same CUDA and torch versions just as hard.

I'll open a PR with these unless you see a problem with the approach. The build passes correctness checks against an FP32 SDPA reference, dense and varlen, forward and backward.

Source: Dao-AILab/flash-attention