旋转位置编码

Author: xtceh-yhCreated Aug 2, 2023Updated Jul 16, 2024

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

代码中将输入的前半部分进行旋转位置编码,后半部分不做处理,直接拼接。 image

Expected Behavior

想知道这样做的原因

Steps To Reproduce

正常环境

Environment

markdown
- OS:Ubuntu 20.04
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :

Anything else?

No response