[功能请求] 为 HybridModel 训练添加 Engram 存储
作者: jeyaletheia创建于 2026年9月9日更新于 2026年9月18日
标签enhancementcommunity-requestwaiting-on-customer
Is your feature request related to a problem? Please describe.
I independently developed an Engram integration for HybridModel training and would like to contribute it upstream. It adds n-gram memory before selected standard attention layers. The main integration challenge is keeping token context and sparse memory state consistent across pipeline stages, activation recomputation and checkpoints.
内容来源: NVIDIA/Megatron-LM