在GPU 5090上使用docker安装llama factory 出现问题

Reminder

• I have read the above rules and searched the existing issues.

System Info

llamafactory version: 0.9.3.dev0 Platform: 24~24.04.1-Ubuntu Python version: 3.11.11 PyTorch version: 2.7.1+cu128 Transformers version: 4.52.3 Datasets version: 3.6.0 Accelerate version: 1.7.0 PEFT version: 0.15.2 TRL version: 0.9.6 GPU type: NVIDIA GeForce RTX 5090 DeepSpeed version: 0.16.9

Reproduction

执行代码llamafactory-cli webui，启动UI界面，点击训练，

执行llamafactory-cli env 指令之后报错。

Traceback (most recent call last):
  File "/opt/conda/bin/llamafactory-cli", line 5, in <module>
    from llamafactory.cli import main
  File "/app/src/llamafactory/__init__.py", line 28, in <module>
    from .extras.env import VERSION
  File "/app/src/llamafactory/extras/env.py", line 23, in <module>
    import peft
  File "/opt/conda/lib/python3.11/site-packages/peft/__init__.py", line 17, in <module>
    from .auto import (
  File "/opt/conda/lib/python3.11/site-packages/peft/auto.py", line 31, in <module>
    from .config import PeftConfig
  File "/opt/conda/lib/python3.11/site-packages/peft/config.py", line 24, in <module>
    from .utils import CONFIG_NAME, PeftType, TaskType
  File "/opt/conda/lib/python3.11/site-packages/peft/utils/__init__.py", line 17, in <module>
    from .other import (
  File "/opt/conda/lib/python3.11/site-packages/peft/utils/other.py", line 34, in <module>
    from .constants import (
  File "/opt/conda/lib/python3.11/site-packages/peft/utils/constants.py", line 16, in <module>
    from transformers import BloomPreTrainedModel
  File "/opt/conda/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 2045, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 2075, in _get_module
    raise e
  File "/opt/conda/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 2073, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/transformers/models/bloom/modeling_bloom.py", line 37, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/opt/conda/lib/python3.11/site-packages/transformers/modeling_utils.py", line 61, in <module>
    from .integrations.flash_attention import flash_attention_forward
  File "/opt/conda/lib/python3.11/site-packages/transformers/integrations/flash_attention.py", line 5, in <module>
    from ..modeling_flash_attention_utils import _flash_attention_forward, flash_attn_supports_top_left_mask
  File "/opt/conda/lib/python3.11/site-packages/transformers/modeling_flash_attention_utils.py", line 36, in <module>
    from flash_attn.bert_padding import index_first_axis, pad_input, unpad_input  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/flash_attn/__init__.py", line 3, in <module>
    from flash_attn.flash_attn_interface import (
  File "/opt/conda/lib/python3.11/site-packages/flash_attn/flash_attn_interface.py", line 15, in <module>
    import flash_attn_2_cuda as flash_attn_gpu
ImportError: /opt/conda/lib/python3.11/site-packages/flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationESs

之前我之前使用docker安装之后，在训练过程中报错。

  File "/opt/conda/lib/python3.11/site-packages/peft/tuners/tuners_utils.py", line 362, in _cast_adapter_dtype
    param.data = param.data.to(torch.float32)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

我就重新安装了nvcc 12.8，并且安装了torch 2.7.1+cu128。在安装torch的过程中显示包冲突

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
flashinfer-python 0.2.2.post1+cu124torch2.6 requires torch==2.6.*, but you have torch 2.7.1+cu128 which is incompatible.

之后我训练就报一开始的错误，如何解决。

Others

No response