LoRA

工具介绍

Loralib代码,是"LORA:Low-Rank大语言模型的适应"的一款实施

核心特点

• •GPT-2 Medium E2E (1.5 MB)
• •GPT-2 Medium DART (1.5 MB)
• •GPT-2 Medium WebNLG (1.5 MB)
• •GPT-2 Large E2E (2.3 MB)
• •GPT-2 Large DART (2.3 MB)
• •GPT-2 Large WebNLG (2.3 MB)
• •loralib/ contains the source code for the package loralib, which needs to be installed to run the examples we provide;
• •examples/NLG/ contains an example implementation of LoRA in GPT-2 using our package, which can be used to reproduce the result in our paper;
• •examples/NLU/ contains an example implementation of LoRA in RoBERTa and DeBERTa using our package, which produces competitive results on the GLUE benchmark;
• •See how we use loralib in GPT-2, RoBERTa, and DeBERTa v2