CrossEncoder PEFT fine-tuning: confusing TypeError (BatchEncoding not Tensor) with manual get_peft_model; no official add_adapter example
Description
Fine-tuning a CrossEncoder with a PEFT/LoRA adapter currently fails with a confusing error when following the "manual" PEFT pattern, and there is no official example or documentation showing the supported add_adapter path for CrossEncoder.
Minimal reproduction (manual get_peft_model pattern, as found in community notebooks):
from sentence_transformers import CrossEncoder, CrossEncoderTrainer, losses
from sentence_transformers.cross_encoder.evaluation import CrossEncoderClassificationEvaluator
from peft import LoraConfig, get_peft_model, TaskType
model = CrossEncoder("cross-encoder/ms-marco-MiniLM-L6-v2")
# Manual PEFT wrapping, replacing the top-level module
peft_config = LoraConfig(task_type=TaskType.SEQ_CLS, r=16, lora_alpha=32, lora_dropout=0.01,
target_modules=["q_proj", "v_proj"], modules_to_save=["score"])
model.model = get_peft_model(model.model, peft_config)
loss = losses.BinaryCrossEntropyLoss(model)
# ... CrossEncoderTrainer(...).train() -> crashThis raises:
TypeError: embedding(): argument 'indices' (position 2) must be Tensor, not BatchEncodingon the very first training step, deep inside peft/tuners/tuners_utils.py -> transformers/models/.../modeling_*.py -> F.embedding.
Root cause
BaseModel.forward (sentence_transformers/base/model.py) iterates over its submodules and calls each one positionally:
input = module(input, **module_kwargs) # <-- input is a BatchEncoding/dictWhen the top-level module is replaced with a PeftModel (via model.model = get_peft_model(...)), the PEFT forward expects named keyword arguments (input_ids=..., attention_mask=...). It therefore binds the positional BatchEncoding to the first parameter (input_ids) and passes the whole dict into embed_tokens, which fails in F.embedding.
Note: the supported path is model.add_adapter(peft_config). Because PeftAdapterMixin.add_adapter wraps the inner transformers model (Transformer.model) rather than the top-level module, the pipeline stays intact, and Transformer.forward already contains a dedicated isinstance(self.model, PeftModel) branch (sentence_transformers/base/modules/transformer.py, lines ~1222-1233) that calls the base model with **filtered_kwargs. So add_adapter is very likely the correct path — but this is not documented for CrossEncoder anywhere, and the resulting error message when using the manual pattern is extremely misleading.
Proposal
- Documentation/example: Add an official PEFT fine-tuning example for
CrossEncoder(e.g.examples/cross_encoder/training/peft/) analogous toexamples/sentence_transformer/training/peft/. Currently PEFT docs only coverSentenceTransformer. - Better error message (optional): In
BaseModel.forward, detect when a submodule is aPeftModel/PreTrainedModelwhoseforwarddoes not accept a positional dict, and raise a clearValueErrorpointing users tomodel.add_adapter(...)instead of mutatingmodel.model.
I'm happy to contribute the example + README in a PR (linked below).
Source: huggingface/sentence-transformers