[Roadmap] LMFlow Roadmap

This document includes the features in LMFlow's roadmap. We welcome any discuss or contribute to the specific features at related Issues/PRs.

Main Features

• Data
  • DPO dataset format #867
  • Conversation template in DPO #883
  • jinja template #931
  • Tools in conversation dataset #884 #892 #931
  • Packing with block diagonal attention
  • Add a tokenize-only script, allowing tokenization separately without gpu environment. (For those who has large datasets but limited gpu hrs)
• Model
  • Backend
    • ️ Accelerate support #936
  • Tokenization
    • Tokenization update, using hf method #931
• Pipeline
  • Train/Finetune/Align
    • DPO (multi-gpu) #867
    • Iterative DPO #867 #883
    • PPO
    • LISA (multi-gpu, qwen2, chatglm)
    • Batch size and learning rate recommendation (arxiv)
    • No trainer version pipelines, allowing users to customize/modify based on their needs
    • Sparse training for moe models #879
  • Inference
    • vllm inference #860 #863
    • Reward model scoring #867
    • Multiple instances inference (vllm, rm, others) #883
    • Inference checkpointing and resume from checkpoints
    • Inference accelerate EAGLE
    • Inferencer for chat/instruction models, and chatbot.py upgrade #917

Usability

• Make some packages/functions (gradio, vllm, ray, etc.) optional, add conditional import. #905
• Inference method auto-downgrading (vllm>ds, etc.), and make vllm package optional. #905
• Merging similar model methods into hf_model_mixin
• Set torch_dtype='bfloat16' when bf16 is specified, etc. (bf16 is in FinetunerArguments but torch_dtype is in ModelArguments, thus cannot handle in __post_init__(). )

Bug fixes

• model.generate() with dsz3 #861
• merge_lora lora with abs path merging
• load_dataset long data fix #878
• src/lmflow/utils/common.py create_copied_dataclass compatibility when python version >= 3.10 (kw_only issue) #903 #905

Issues left over from history

• use_accelerator -> use_accelerate typo fix (with Accelerate support PR) #936
• model_args.use_lora leads to truncation of the sequence, mentioned in #867
• Make ports, addresses, and all other settings in distributed training tidy and clear (with Accelerate support PR)

Documentation

• Approx GPU memory requirement w.r.t model size & pipeline
• Dev handbook, indicating styles, test list, etc.
• Sphinx warning cleanup and strict warning checking

CI/CD

• ️ CPU-only and offline CI (Ruff, pytest, package build, and documentation build)
• GPU CI for core training, alignment, and inference pipelines
• Online model and dataset tests
• Optional backend integration tests (vllm, sglang, etc.)
• GitHub Release automation
• PyPI publishing with GitHub Actions trusted publishing
• Release workflow, including versioning, approval, and rollback