IMAGE mode changes integer matmul dtype and loses precision
Turning on IMAGE can change an integer matrix multiplication into float32 and change the answer. Multiplying by an identity matrix turns 16777217 into 16777216.0. This runs on ordinary CPU, without the QCOM emulator.
Save this as repro.py and run DEV=CPU IMAGE=1 FLOAT16=0 python repro.py from a tinygrad checkout:
from tinygrad import Tensor, dtypes
expected = [[16777217, 2], [3, 4]]
x = Tensor(expected, dtype=dtypes.int32)
actual = x @ Tensor.eye(2, dtype=dtypes.int32)
print("expected:", expected)
print("actual: ", actual.tolist())
print("dtype: ", actual.dtype)
assert actual.dtype == dtypes.int32
assert actual.tolist() == expectedBefore the fix:
expected: [[16777217, 2], [3, 4]]
actual: [[16777216.0, 2.0], [3.0, 4.0]]
dtype: dtypes.float
AssertionErrorAfter the fix:
expected: [[16777217, 2], [3, 4]]
actual: [[16777217, 2], [3, 4]]
dtype: dtypes.intThe unpatched IMAGE=0 control gives the correct integer result. IMAGE=2 has the same failure as IMAGE=1.
The image implementation casts products to float32 before the sum. Passing dtype=dtypes.int32 to dot still loses the integer bit before accumulation; casting the answer back is too late. The fix keeps matmul on the ordinary dot path whenever either operand is non-floating. Floating-point pairs keep their existing image path.
Reproduced on upstream fec703cbecad430857c6bedd94a9500e871a4bcc; that is also the proposed PR base.
Source: tinygrad/tinygrad