question about sink attention
Author: hujiaxin0Created Nov 9, 2024Updated Nov 9, 2024
Thank you very much for your open source work abort streaming-llm!
Here I have a question about sink attention.
By debugging this program, I didn't find the process of generating the sink attention mask. Is casual attention still used to generate the first token?
Would you please provide a link to the code for generating the sink attention mask?
Source: mit-han-lab/streaming-llm