trl

工具介绍

列车Transformer语言模式加固学习.

核心特点

• •Trainers: Various fine-tuning methods are easily accessible via trainers like SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer and more.
• •Efficient and scalable:
• •Leverages 🤗 Accelerate to scale from single GPU to multi-node clusters using methods like DDP and DeepSpeed.
• •Full integration with 🤗 PEFT enables training on large models with modest hardware via quantization and LoRA/QLoRA.
• •Integrates 🦥 Unsloth for accelerating training using optimized kernels.
• •Command Line Interface (CLI): A simple interface lets you fine-tune with models without needing to write code.