Scalene doesnt work properly with torchrun / torch.distributed.run
Author: deo-abhijitCreated May 9, 2024Updated Sep 1, 2025
Labelsbug
I got error while running scalene with torch.distributed.run .
I am currently following this doc
python -m torch.distributed.run --nproc_per_node=8 --master_port=2333 tools/train.py projects/configs/VAD/VAD_base.py --launcher pytorch --deterministic --work-dir path/to/save/outputs
this command runs perfectly, but when i replace the python -m with scalene, it raises error. I think the main issue is my train_mz.py takes other arguments as input from command line. and scalene is prolly passing them as args to torch.distributed.run.main() function.
although this is just a speculation.
Also there is very similar stackoverflow question on exactly similar lines.
It would be really nice if someone could help me out here. Thanks
Source: plasma-umass/scalene