quantize_4bit fails with "invalid configuration argument" (ops.cu:54) for tensors of exactly >= 2^31 elements

Author: TomMoerasCreated Sep 3, 2026Updated Sep 3, 2026

Versions: bitsandbytes 0.49.1, torch 2.11.0+cu128, CUDA 12.8, NVIDIA H200 (141 GB).

Repro (isolated, one process per size, fp32 flattened input, compress_statistics=True — mirroring axolotl 0.18's _simulate_nf4_roundtrip call):

n = 2**31 - 65536   -> OK
n = 2**31           -> Error invalid configuration argument at line 54 in file /src/csrc/ops.cu
n = 2**31 + 65536   -> same failure

A clean int32 boundary in quantize_4bit's kernel-launch arithmetic.

Real-world trigger: axolotl 0.18's merge CLI enables NF4 round-trip simulation for every 4-bit config; any model carrying a >= 2^31-element tensor (e.g. google/gemma-4-E4B-it's embedding) cannot merge. Smaller-vocab models (Mistral-7B, Qwen3.5-4B) pass, which hides the boundary.

Expected: chunked launch or int64 indexing above 2^31 elements, or a clear size-limit error.

Source: bitsandbytes-foundation/bitsandbytes