Reliability: per-endpoint TTFB-aware retry budget (kill slow-but-alive free endpoints prematurely)

Summary

The default 45-second retry budget kills slow-but-alive endpoints on every request, wasting the full failover budget before giving them a chance to respond.

Evidence from a single production install (7-day window, 9/15):

Meanwhile 12 requests were exhausted with timeout and the message "retry time budget 45000ms expired with no first byte". A subsequent request to the same model on the same endpoint succeeded at 47s — proving the endpoint was never dead, only slow.

The operator already found a workaround: setting fallback_time_budget_ms=120000. But this is a global sledgehammer — it makes every endpoint wait 120s even when most are fast.

Root cause

getFallbackTimeBudgetMs() returns a single global budget for the entire failover ladder. The loop checks this budget before starting each retry, and aborts in-flight attempts when the budget expires. This works for fast providers (OpenAI, Anthropic, most commercial relays) where a stalled attempt is truly dead. It fails for free-tier endpoints that are alive but throttled — they spend 50-90s warming up, then succeed. The router kills them at 45s, wastes the budget, and the next request repeats the same failure.

Proposed fix: per-endpoint TTFB-aware budget

Each (platform, endpoint_scope) bucket tracks its own observed TTFB distribution (P50, P95 over a sliding 7-day window with 2-day half-life decay, same as the existing reliability bandit). The loop uses this to dynamically widen the budget for slow endpoints:

base = getFallbackTimeBudgetMs()  // 45000ms default
slow = endpointTtfbP95(platform, endpointScope)
if slow > 0:
    effective = max(base, slow + buffer_ms)   // buffer = 10s by default
else:
    effective = base

buffer_ms is tunable (SLOW_ENDPOINT_BUFFER_MS, default 10000). The existing abortInFlight() hedge logic applies unchanged — it just gets a longer timer for slow endpoints.

What stays the same

• Fast endpoints keep their 45s budget — no behavioral change for them
• Budget checking still happens before each retry and mid-attempt via abortInFlight()
• The same environment variable / settings table override is respected as the floor (effective = max(base, calculated))
• Exhaustion messages still name the budget (now per-endpoint-aware)

Configuration

Kill switch: TTFB_BUDGET_DISABLED=1 (same convention as ENDPOINT_HEALTH_DISABLED).

PR plan

Single PR: server/src/lib/fallback-loop.ts + server/src/lib/ttfb-budget.ts (new module) + tests + env docs.

Why this matters for free-tier aggregation

Free endpoints are inherently slower — they run on consumer hardware, share resources with other users, and throttle under load. A one-size-fits-all budget treats "slow" and "dead" identically. The per-endpoint TTFB adaptation lets free-tier endpoints compete on equal footing with commercial providers without forcing operators to set a global 120s budget (which would make fast endpoints unnecessarily slow on exhaustion).

中文备注：此问题 #1218 Gap 2。生产数据来自本机实例（127.0.0.1:31415），错误率约30%，失败请求平均挂7.3s。