The position id for q
Author: ofhweiCreated Oct 30, 2023Updated Oct 31, 2024
Hello,
I wonder if the position id of query is the same with key or is the actual generated context length (this comment is confusing me)? For example, as mentioned in the toy example, the position id for key of M is 9. What's the position id for q of M? 9 or 12 here? And more , why make this choice?
Looking forward to your response. Thank you.
Source: mit-han-lab/streaming-llm