旋转位置编码
Author: xtceh-yhCreated Aug 2, 2023Updated Jul 16, 2024
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
代码中将输入的前半部分进行旋转位置编码,后半部分不做处理,直接拼接。
Expected Behavior
想知道这样做的原因
Steps To Reproduce
正常环境
Environment
- OS:Ubuntu 20.04
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :Anything else?
No response
Source: zai-org/ChatGLM2-6B