#290·gpt-oss

Distributed chat executes each tool call independently on every model-parallel rank

Author: sylvesterkaczmarekCreated Aug 16, 2026Updated Aug 16, 2026

Summary

gpt_oss.chat broadcasts user input across distributed ranks, but tool/function execution is not rank-gated. When Torch or Triton chat is launched with torchrun, every rank enters the same browser, Python, or apply_patch branch and executes the tool independently.

Impact

A single model tool call can therefore produce N external side effects on N model-parallel ranks:

  • browser searches/API calls are duplicated;
  • Python tool work is duplicated;
  • apply_patch can race multiple processes against the same working tree;
  • non-deterministic tool results can leave ranks with different conversation histories before the next model-parallel generation step.

get_user_input() already establishes the intended rank-0/broadcast pattern for interactive input; tool results need the same treatment.

Proposed resolution

Execute each tool/function call only on rank 0, serialize Harmony Message results with Message.to_dict(), broadcast the result payload, and reconstruct it with Message.from_dict() on every rank. Propagate rank-0 execution failures to every rank as well so nonzero ranks cannot block forever waiting for a result.

Add mocked distributed regression coverage proving nonzero ranks do not execute the tool callback and receive the rank-0 result.