minGPT example does not verify the GPU count across all nodes
Author: MichaelvllCreated Oct 2, 2025Updated Oct 2, 2025
The above line check the GPU count for the current process, which makes 2 node with 1 GPU each node fail to run, i.e. the slurm launcher script to fail:
https://github.com/pytorch/examples/blob/main/distributed/minGPT-ddp/mingpt/slurm/sbatch_run.sh#L19
Source: pytorch/examples