[Bug] DPO training crashes with Qwen3.5-35B-A3B (MoE VLM): TypeError: expected Tensor as element 0 in argument 0, but got list in collator

Reminder

• I have read the above rules and searched the existing issues.

System Info

System Info LlamaFactory version: latest main branch Model: Qwen3.5-35B-A3B (Qwen3_5MoeForConditionalGeneration) Training stage: DPO Framework: DeepSpeed ZeRO-3, torchrun 8 GPUs Python: 3.11 Transformers: 5.2.0

Reproduction Run DPO training on Qwen3.5-35B-A3B (multimodal MoE model) with LlamaFactory. The training crashes at the first batch during data collation.

Error Log File "/root/LlamaFactory-main/src/llamafactory/data/collator.py", line 559, in call combined_input_ids = torch.stack([f["input_ids"] for f in concatenated_features]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: expected Tensor as element 0 in argument 0, but got list

All 8 ranks fail with the same error. The crash occurs in PairwiseDataCollatorWithPadding.call → MultiModalDataCollatorForSeq2Seq.call during rope position_ids computation.

Root Cause Analysis Qwen3.5-35B-A3B uses a hybrid architecture (linear_attention + full_attention layers with MoE), which differs from standard VLMs like Qwen2-VL. The current MultiModalDataCollatorForSeq2Seq._compute_rope_position_ids does not properly handle this architecture, causing it to return a list instead of a Tensor for input_ids in the concatenated DPO features. Key observations: PairwiseDataCollatorWithPadding passes raw list-type input_ids (chosen/rejected) to the parent collator The parent collator's rope computation logic is designed for standard VLM architectures and fails on the hybrid attention pattern of Qwen3.5-MoE This issue is specific to DPO + multimodal MoE VLM combination; SFT or text-only DPO may not be affected Expected Behavior DPO training should work correctly with Qwen3.5-35B-A3B, with proper rope position_ids computation for the hybrid attention architecture. Suggested Fix Update MultiModalDataCollatorForSeq2Seq._compute_rope_position_ids (or add a dedicated code path) to handle models with layer_types containing mixed linear_attention / full_attention patterns, ensuring all outputs are Tensors before torch.stack. Additional Context Config shows layer_types alternating between linear_attention (×3) and full_attention (×1) across 40 layers Model also has MTP (mtp_num_hidden_layers: 1) and MRoPE (mrope_interleaved: true) Warning observed: The fast path is not available because one of the required library is not installed. Falling back to torch implementation. (flash-linear-attention / causal-conv1d missing)

Reproduction

llamafactory-cli train
--stage dpo
--do_train True
--model_name_or_path /root/model/Qwen3.5-35B-A3B
--preprocessing_num_workers 16
--finetuning_type lora
--template qwen3_5_nothink
--flash_attn auto
--dataset_dir data
--dataset DPO_cityzhili
--cutoff_len 1024
--learning_rate 5e-06
--num_train_epochs 3.0
--max_samples 100000
--per_device_train_batch_size 1
--gradient_accumulation_steps 2
--lr_scheduler_type cosine
--max_grad_norm 1.0
--logging_steps 5
--save_steps 100
--warmup_steps 25
--packing False
--enable_thinking False
--report_to none
--output_dir saves/Qwen3.5-35B-A3B-Base/lora/train_2026-08-14-13-44-45
--bf16 True
--plot_loss True
--trust_remote_code True
--ddp_timeout 180000000
--include_num_input_tokens_seen True
--optim adamw_torch
--adapter_name_or_path saves/Qwen3.5-35B-A3B-Base/lora/train_2026-08-14-88-88-33
--lora_rank 16
--lora_alpha 32
--lora_dropout 0.05
--lora_target all
--pref_beta 0.1
--pref_ftx 0
--pref_loss sigmoid
--freeze_vision_tower True
--freeze_multi_modal_projector True
--image_max_pixels 589824
--image_min_pixels 1024
--video_max_pixels 65536
--video_min_pixels 256
--val_size 0.1
--eval_strategy steps
--eval_steps 100
--per_device_eval_batch_size 1
--deepspeed llamaboard_cache/ds_z3_config.json

Others

No response