Cross-architecture merging: proposal — canonicalization as a preprocessing transform
(#290) lets users attempt cross-architecture merges but does nothing about the two real obstacles: incompatible parameter names and incompatible shapes. I'd like to contribute a path that addresses both, drawn from a working system: we've published merged collectives of up to 9 models across 4 architecture families (Qwen, Llama, Mistral, Phi-lineage) on the Hub, with per-key benchmark evidence — 28.5 min on one A100, streaming, byte-identical output across machines.
The proposal: a preprocessing stage before any merge method runs — (1) per-family canonicalization of parameter names to a shared role-key space, (2) a shape bridge to the anchor's geometry (pad/truncate, optional per-key Procrustes alignment). Exposed as canonicalize: true / bridge: pad|procrustes, it would make the existing methods (linear, task_arithmetic, TIES, SCE, ...) cross-architecture-capable rather than adding a competing method.
Happy to open the first PR with the family detectors + canonical mapping and tests, with the bridge to follow. Questions for maintainers: preferred home for the transform (parameter resolution vs. a method wrapper), and whether mergekit/architecture/ is the right surface for the family detectors.
Source: arcee-ai/mergekit