Docs: deployment guidance for cold starts and offline model loading (scale-to-zero, local_files_only, warmup)

Problem

The docs don't cover what happens to model loading when Sentence Transformers runs on a scale-to-zero platform (Cloud Run, Lambda-style containers, KEDA-scaled pods). Searching docs/ turns up no mention of HF_HUB_OFFLINE, local_files_only, cold starts, or container warmup.

This is complementary to #3839 / #3856, not overlapping: that thread is about a blocking encode() call inside an async def endpoint (request handling). This issue is about the time and failure modes before the first request is served — loading the model itself.

Two things bite people here, and neither is documented:

1. Where cold-start time actually goes. On a scale-to-zero container the first request pays container start + model load + first inference. Without splitting those, you can't tell whether to fix your image, your model, or your platform config.
2. Runtime Hub dependency. By default a fresh container may reach out to the Hub while loading. In restricted or offline environments that fails; in normal environments it adds a variable amount of time that makes cold starts non-deterministic.

What I measured

I ran this on my own deployment (FastAPI + Sentence Transformers, embedding model plus a cross-encoder reranker, scale-to-zero container). Method: 5 paired cold measurements after 15 minutes of idle, comparing a baseline revision against a candidate revision with identical image digest and resources, with no public traffic on either.

	baseline	local-only loading	delta
model load (median)	24.24 s	23.09 s	−4.7%
end-to-end cold (median)	37.68 s	37.78 s	no improvement

The candidate removed the runtime Hub dependency (local-only model loading) and moved the readiness probe so the platform stops routing traffic to an instance that hasn't finished loading.

The part I found most useful to know in advance: breaking the cold start into segments showed that model loading was not the dominant cost in my setup — instance startup and transport were (~27 s of the total). So "make the model load faster" was the wrong lever, and I could only tell that because the segments were measured separately.

I'm reporting the measurement, not a mechanism — if the framing above is wrong for reasons specific to how ST loads models, I'd rather be corrected before writing anything.

Proposed section

Somewhere under a serving/deployment page:

1. What a cold start is made of — container start, model load, first inference. How to time each one, and why an aggregate number hides the lever.
2. Making model loading deterministic — pre-downloading into the image, local_files_only=True, HF_HUB_OFFLINE=1, cache dir placement. Framed as removing a variable network dependency, which matters for reliability and for restricted environments, rather than as a latency fix.
3. Warmup — running a dummy encode() at startup, and ordering it relative to the readiness probe so traffic isn't routed to a not-yet-loaded instance.
4. What did not help in my measurement — so readers don't repeat it: local-only loading alone did not meaningfully change end-to-end cold latency.
5. Trade-offs — image size vs cold start, min-instances cost vs latency.

Offer

Happy to write the PR. Before I do, I'd like a check on two things:

• Is a serving/deployment page the right home for this, or should it go under an existing page?
• Is the local_files_only / HF_HUB_OFFLINE guidance something you want stated in the docs, or do you consider it huggingface_hub territory and out of scope here?