[Help] lora微调合并后模型推理速度明显慢了好多
Author: daydayup-zynCreated Oct 13, 2023Updated Aug 9, 2024
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
只用lora微调了20条数据集,合并模型后,推理速度相比原来的基座模型慢了好多,这个是什么原因?跟微调过程中的参数有关系吗?
Expected Behavior
No response
Steps To Reproduce
无
Environment
- OS:
- Python:
- Transformers:
- PyTorch:
- CUDA Support (`python -c "import torch; print(torch.cuda.is_available())"`) :Anything else?
No response
Source: zai-org/ChatGLM2-6B