[Bug]: Pause (blocked) and delete are ignored at routing time for wildcard-model deployments — PatternMatchRouter pool is never reconciled

Bug description

Admin pause (the blocked flag, toggled from the UI or PATCH /model/{id}/update) and delete are silently ignored at routing time for deployments whose model_name contains a wildcard (*), for as long as the proxy process keeps running:

• A paused wildcard deployment keeps serving traffic (expected: 403 Model is blocked).
• A deleted wildcard deployment keeps serving traffic (ghost deployment).

Non-wildcard (exact-name) deployments in the same proxy respect both operations immediately. The only way to enforce them for wildcards is docker restart — which then applies retroactively: a pause issued days ago suddenly cuts live traffic at restart time. That makes the pause button a delayed-production-kill-switch that looks like it isn't working, and then blows up later.

Affected versions

• litellm proxy 1.100.0 (docker berriai/litellm:main-stable), STORE_MODEL_IN_DB=true, Postgres backend, single worker.
• Code paths below are from the 1.100.0 shipped package.

Reproduction (fresh proxy, all requests via /v1/chat/completions)

1. Create a fresh wildcard deployment via POST /model/new:

   {
     "model_name": "zzz-test-*",
     "litellm_params": {"model": "hosted_vllm/<any-model>", "api_base": "http://<backend-host>:8011/v1"},
     "model_info": {}
   }
   

   Baseline: zzz-test-9 → 4/4 200 ✅
2. Pause it: PATCH /model/<id>/update {"blocked": true} → 200. GET /model/info now shows model_info.blocked = true for that deployment — the admin view is updated. Routing: zzz-test-9 → 6/6 200, x-litellm-model-id = the paused deployment's id. Expected: 403 litellm.PermissionDeniedError: Model is blocked.
3. Delete it: POST /model/delete {"id": <id>} → 200, deployment disappears from /model/info. Routing: zzz-test-9 → 4/4 200 (ghost still served).
4. Contrast, exact-name group with two deployments (llm-large-auto): pause one deployment → next 6/6 requests route to the other one; pause both → 403 Model is blocked; unblock → immediately served again. The exact-index path honors blocked live; only the wildcard path does not.
5. After docker restart (router rebuilt from DB), a wildcard that was paused long ago suddenly returns 403 Model is blocked — proving the DB state was correct all along and only the in-memory pattern registry was stale.

Root cause (code pointers, 1.100.0)

Wildcard deployments are served from a separate registry PatternMatchRouter (litellm/router_utils/pattern_match_deployments.py), and the main proxy never reconciles it:

1. Router._add_deployment_locked() registers wildcards with append semantics:

   # router.py:8868-8872
   if "*" in deployment.model_name:
       self.pattern_router.add_pattern(deployment.model_name, deployment.to_json(exclude_none=True))
   

   add_pattern just does self.patterns[regex].append(llm_deployment) (pattern_match_deployments.py:62-74) — no dedup, no replace.

2. PatternMatchRouter.remove_deployment(model_id) exists (pattern_match_deployments.py:76-86) but is never called for the main pattern_router. The only call site in router.py (~9087-9091) cleans team_pattern_routers during deletion — the main pool is left untouched on update, upsert, delete and full reload.

3. clear_cache() in model_management_endpoints.py (invoked synchronously by patch_model/update_model, see PR #30367 which measures the full reload at 7-43 s) rebuilds the exact index (model_list / model_id_to_deployment_index_map) via upsert_deployment — whose own "converges on its own" comment covers only those structures. For wildcard deployments the reload merely appends another copy into pattern_router.patterns[regex], while the stale pre-pause copy (and even post-delete copies) remain and keep winning routing hits.

Net effect: the pattern pool is append-only and its entries' model_info.blocked (and existence) are never updated or evicted until process restart.

Secondary cosmetic symptom from the same "no wildcard reconcile" gap: GET /v1/models keeps listing paused wildcard models (get_fully_blocked_model_names() matches model names literally; its own docstring documents the fail-open for wildcard routes).

Impact

• Pause is documented as the admin kill-switch ("Filters out deployments that an admin has paused via LiteLLM_ProxyModelTable.blocked … so paused deployments never serve a request" — _filter_blocked_deployments docstring). For any wildcard route (claude-*, gpt-*, provider catch-alls, etc.) it does nothing until a restart.
• Worse than "not working": on the next restart, all accumulated pauses apply at once and silently cut traffic that has been flowing since the pauses were issued — operators reasonably assume those models are out of service.
• delete for wildcards has the same ghost-serving behavior (previously reported anecdotally as "deleted wildcard still randomly hit, must restart" — same root cause).

Suggested fix

Make the pattern pool's lifecycle symmetric with the exact index:

1. In the wildcard registration branch of _add_deployment_locked (or at the top of upsert_deployment), remove the previous copy before appending: self.pattern_router.remove_deployment(deployment.model_info.id) — makes add_pattern idempotent/replace-per-model-id across reloads. This is the half that fixes pause/update staleness (the core of this report).
2. In delete_deployment (and _delete_deployment used by reload), call self.pattern_router.remove_deployment(model_id) the same way the team pattern pools already do. Note open PR #31763 (June 2026, still unmerged) adds exactly this cleanup for the delete path; even once merged it would not fix symptom 1 (pause), because a paused deployment is never deleted — the append-on-update without replace keeps the stale blocked=false copy (plus one new copy per update) in the pool.

Environment

litellm 1.100.0 · docker berriai/litellm:main-stable · STORE_MODEL_IN_DB=true · Postgres · single worker · routing: default simple-shuffle, plus one group using latency-based routing (bug reproduces on both; the plain wildcard repro above used the default group).