一个用于多任务学习的 PyTorch 库
LibMTL is an open-source library built on PyTorch for Multi-Task Learning (MTL). See the latest documentation for detailed introductions and API instructions.
:star: Star us on GitHub — it motivates us a lot!
:bangbang: A comprehensive survey on Gradient-based Multi-Objective Deep Learning is now available on arXiv, along with an awesome list. Check it out!
LibMTL provides a unified code base to implement and a consistent evaluation procedure including data processing, metric objectives, and hyper-parameters on several representative MTL benchmark datasets, which allows quantitative, fair, and consistent comparisons between different MTL algorithms.LibMTL supports many state-of-the-art MTL methods including 8 architectures and 16 optimization strategies. Meanwhile, LibMTL provides a fair comparison of several benchmark datasets covering different fields.LibMTL follows the modular design principles, which allows users to flexibly and conveniently add customized components or make personalized modifications. Therefore, users can easily and fast develop novel optimization strategies and architectures or apply the existing MTL algorithms to new application scenarios with the support of LibMTL.Each module is introduced in Docs.
LibMTL currently supports the following algorithms:
| Optimization Strategies | Venues | Arguments |
|---|---|---|
| Equal Weighting (EW) | - | --weighting EW |
| Gradient Normalization (GradNorm) | ICML 2018 | --weighting GradNorm |
| Uncertainty Weights (UW) | CVPR 2018 | --weighting UW |
| MGDA (official code) | NeurIPS 2018 | --weighting MGDA |
| Dynamic Weight Average (DWA) (official code) | CVPR 2019 | --weighting DWA |
| Geometric Loss Strategy (GLS) | CVPR 2019 Workshop | --weighting GLS |
| Projecting Conflicting Gradient (PCGrad) | NeurIPS 2020 | --weighting PCGrad |
| Gradient sign Dropout (GradDrop) | NeurIPS 2020 | --weighting GradDrop |
| Impartial Multi-Task Learning (IMTL) | ICLR 2021 | --weighting IMTL |
| Gradient Vaccine (GradVac) | ICLR 2021 | --weighting GradVac |
| Conflict-Averse Gradient descent (CAGrad) (official code) | NeurIPS 2021 | --weighting CAGrad |
| MOML | NeurIPS 2021 | --weighting MOML |
| Nash-MTL (official code) | ICML 2022 | --weighting Nash_MTL |
| Random Loss Weighting (RLW) | TMLR 2022 | --weighting RLW |
| Auto-Lambda (official code) | TMLR 2022 | --weighting AutoLambda |
| MoCo | ICLR 2023 | --weighting MoCo |
| Aligned-MTL (official code) | CVPR 2023 | --weighting Aligned_MTL |
| FAMO (official code) | NeurIPS 2023 | --weighting FAMO |
| SDMGrad (official code) | NeurIPS 2023 | --weighting SDMGrad |
| MoDo (official code) | NeurIPS 2023 | --weighting MoDo |
| FORUM | ECAI 2024 | --weighting FORUM |
| STCH (official code) | ICML 2024 | --weighting STCH |
| ExcessMTL (official code) | ICML 2024 | --weighting ExcessMTL |
| FairGrad (official code) | ICML 2024 | --weighting FairGrad |
| DB-MTL | arXiv |
暂无开放 Issues，或尚未同步最近议题。