Save individual routes from a Router-based model as standalone artifacts

Author: ignasgrCreated Mar 29, 2026Updated Sep 11, 2026

Summary

When a SentenceTransformer model uses a Router for asymmetric encoding (separate query and document encoders), there's currently no built-in way to export a single route as a standalone model artifact. I'm wondering whether introducing such functionality would would be useful.

Motivation

In deployment scenarios, the query and document encoders are often deployed independently. For example, the document encoder runs offline during indexing, and the query encoder runs online in a retrieval service (e.g. Elasticsearch). There would be redundancy if loading the full model (both routes) in each service.

The model.save() method already writes each route's modules to separate subdirectories, so the weights are physically separated on disk. However, there's no public API to reconstruct a single route as a standalone SentenceTransformer that can be loaded without any knowledge of Router.

Possible Solution

A method on SentenceTransformer (since that's the natural entry point for saving):

model.save_route("./query_encoder", route="query")
model.save_route("./document_encoder", route="document")

Which would produce standard, fully loadable SentenceTransformer artifacts with no Router wrapper — just the route's modules plus any surrounding modules from the parent model.

Happy to try to implement this if the approach sounds reasonable.

Source: huggingface/sentence-transformers