#2656·smolagents

[BUG] AgentImage inverts pixel values when built from a tensor (255 - array * 255)

Author: VANDRANKICreated Aug 18, 2026Updated Sep 14, 2026

Describe the bug

AgentImage inverts the pixel values of any image built from a tensor. Black comes out white and white comes out black.

The conversion appears twice in src/smolagents/agent_types.py, in to_raw() at line 134 and in to_string() at line 156, both using the same expression:

python
array = self._tensor.cpu().detach().numpy()
img = PIL.Image.fromarray((255 - array * 255).astype(np.uint8))

For a normalized image tensor with values in [0, 1], the scaling to byte range should be array * 255. The leading 255 - flips the intensity.

Steps to reproduce

python
import numpy as np

array = np.linspace(0.0, 1.0, 5, dtype=np.float32)

current  = (255 - array * 255).astype(np.uint8)
expected = (array * 255).astype(np.uint8)

print(current)   # [255 191 127  63   0]
print(expected)  # [  0  63 127 191 255]

Expected behavior

0.0 is black and stays 0. 1.0 is white and stays 255.

Actual behavior

0.0 becomes 255 (white) and 1.0 becomes 0 (black). Every tensor-backed AgentImage is returned as a photographic negative, both when read through to_raw() and when written to disk through to_string().

Why this is likely unintentional

  • The two call sites are identical, so a fix in one place without the other would make to_raw() and the saved PNG disagree.
  • The PIL.Image.Image and bytes input paths do no inversion, so the same picture changes polarity depending only on which type it was constructed from.
  • The surrounding code carries the comment # There is likely simpler than load into image into save, which suggests this block has not been revisited.

If the inversion is deliberate for some tensor convention I have missed, it would help to state that in the docstring, because it is surprising from the call site.

Suggested fix

Drop the 255 - in both places:

python
img = PIL.Image.fromarray((array * 255).astype(np.uint8))

Worth handling two related cases while touching this: an integer tensor already in [0, 255] should not be multiplied again, and a float tensor outside [0, 1] should probably be clipped rather than wrapped by the uint8 cast.

Happy to open a PR with the fix and a round-trip test asserting that a known gradient survives to_raw() and to_string() unchanged.