使用 rank_pattern 的 add_weighted_adapter 生成的适配器无法重新加载 (linear、ties、dare、magnitude_prune)
作者: amogh-nagri-11创建于 2026年9月14日更新于 2026年9月16日
- Both adapters use the same rank_pattern: combining works, but the result cannot be reloaded for combination_type in ["linear", "ties", "dare_linear", "dare_ties", "magnitude_prune", "svd", "cat"]: torch.manual_seed(0) model = get_peft_model(MLP(), make_config({"lin1": 4}), adapter_name="a") model.add_adapter("b", make_config({"lin1": 4})) kwargs = {} if combination_type in ("linear", "svd", "cat") else {"density": 0.5} model.add_weighted_adapter(["a", "b"], [0.5, 0.5], "merged", combination_type=combination_type, **kwargs) with tempfile.TemporaryDirectory() as tmp_dir: model.save_pretrained(tmp_dir, selected_adapters=["merged"]) try: PeftModel.from_pretrained(MLP(), f"{tmp_dir}/merged") print(f"{combination_type}: reload OK") except RuntimeError as e: print(f"{combination_type}: reload FAILED: {str(e).splitlines()[1].strip()}")
- Same maximum rank, but different ranks for the same module: combining fails immediately torch.manual_seed(0) model = get_peft_model(MLP(), make_config({"lin1": 4}), adapter_name="a") model.add_adapter("b", make_config({})) try: model.add_weighted_adapter(["a", "b"], [0.5, 0.5], "merged", combination_type="linear") except RuntimeError as e: print(f"linear with different per-module ranks: FAILED: {e}") Output:
内容来源: huggingface/peft