此微调代码是否在使用 LoRA 的 LLaMA-2-7B 单个 A6000 GPU 中不起作用?

Hi, I am trying to work with your RLHF code to Fine-tune and Reinforcement learning for LLaMA. But I keep getting CUDA out of Memory error while fine-tuning LLaMA-2-7B model with single A6000 GPU even though I use PEFT LoRA method. I applied these changes to get rid of CUDA om but error is still occuring.