🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support
grad clipping computes a different "global" norm on every EP group with TE experts
Seeing nan loss when setting packed_sequence_size > 0
暂无评论,来聊聊你的看法吧