Proposal: Preserve resolved API identity in ai-proxy retries (#4396)

Proposal draft: Preserve API identity when ai-proxy retries requests

Status: draft for maintainer review; no fix implementation has begun. Baseline-only investigation has reproduced the defect.

Related report: https://github.com/higress-group/higress/issues/4396

Background

Source analysis on higress-group/higress@faccaad586a3cdc9e85dc7fa39358ff31a6453b2 confirms that retryFailedRequest obtains the resolved ApiName and passes it to sendRetryRequest. However, transformRequestHeadersAndBody does not receive this value and uses ApiNameChatCompletion for all request transformation interfaces and the default body transformation.

For API-specific providers such as Kling, retry reconstruction can therefore select a different path or body transformation from the initial video request. This has now been reproduced through the real proxy-Wasm data path described below.

Baseline runtime reproduction (2026-09-18)

Environment: clean source at the SHA above, Go 1.26.8, actual Envoy 4735dd6b874700fc2bc9a218ce80ba0be759e53f/1.36.4, gateway release v2.2.4 pinned to sha256:3dbd609df5db3fca61653eafe0e2310705e485190c4f8cd02d9aab8f07dcf329. The ai-proxy module was built without product changes using GOOS=wasip1 GOARCH=wasm go build -buildmode=c-shared; module SHA-256: 8491786b399b071833e8711529cf4c1aef99d1efd087d12fe698f1d522b1e812.

Configure a Kling provider in default OpenAI-compatible protocol with apiTokens: [dummy-key-a, dummy-key-b] and retryOnFailure: {enabled: true, maxRetries: 1, retryOnStatus: ["503"], retryTimeout: 5000}. Send POST /v1/videos with JSON {"model":"kling-v1","prompt":"text-retry-1"}. For image-to-video, also include an image fixture field. Both initial routing and internal retry dispatch use a fixed local mock cluster in an isolated Docker internal network; no live provider or credentials are used.

The mock records requests, returns 503 on the first correctly transformed attempt, and accepts a second attempt only if its protocol is correct and its dummy key has rotated. Observed upstream traffic:

Scenario	Initial upstream request	Retry upstream request	Client response
Text-to-video, repeated 3 times	/v1/videos/text2video, model_name=kling-v1; injected 503	/v1/videos, model=kling-v1, no model_name; mock rejects with 422	Original 503
Image-to-video, repeated 3 times	/v1/videos/image2video, model_name=kling-v1; injected 503	/v1/videos, model=kling-v1, no model_name; mock rejects with 422	Original 503
Chat Completions control, repeated 3 times	Correct chat protocol; injected 503	Correct chat protocol with other key; 200	200

No-retry text, image, and chat controls all return 200. All nine retries reach the mock and use the other dummy key. A complete scripted rerun gives the same results: 12 client requests, 21 upstream requests, all baseline-observation assertions pass. The mock's 422 is a diagnostic choice, not a claimed live-provider status. The primary evidence is the actual path/body regression between initial and retry requests.

This is pre-implementation bug investigation, not completion of the future approved baseline/fixed verification TASK. Local raw artifacts and runnable harness have been retained; they have not yet been uploaded as public artifacts. The later implementation verification must publish reproducible baseline/fixed evidence.

Goals

Pass the resolved API identity explicitly from the retry caller through the shared request-conversion helper. Preserve the existing Chat Completions semantics of the independent health-check caller.

Acceptance criteria

• When a non-chat request is retried with another configured token, every applicable request transformation receives the original resolved API name.
• A Kling video-generation retry preserves the expected provider path and body schema.
• Chat Completions retries and generated Chat Completions health checks retain their existing behavior.
• Regression tests cover header, body, combined body/header, and default transformation paths.
• A real proxy-Wasm baseline/fixed comparison uses identical inputs and configuration, with only the built module changed, and records source revisions, module hashes, image digests, assertions, logs, rerun steps, and cleanup evidence.

Scope

This change concerns API-name propagation. It does not redesign retry HTTP methods, retry response streaming, provider-domain overrides, token affinity, or health-check API selection.

Coordinate shared helper changes with PR #4552 (provider domain) and PR #3882 (streaming retry). Search related work again before publishing and before implementation. Credit the analysis in #4396 and confirm whether its reporter is already implementing a fix.

Related Specs Analysis

SPEC-4718001 on this Proposal defines API identity propagation and chat/health-check compatibility. No pre-existing Proposal referencing #4396 was found by the issue-spec search before this Proposal was opened. PR #4552 addresses provider-domain override ordering, and #3882 addresses streaming retry response handling; neither is the API-name propagation change proposed here. Original report #4396 remains the credited source of the issue analysis.

Existing Assumptions Impact

Normal request handling already resolves API-specific behavior. Retry reconstruction must use that same resolved identity rather than assuming every request is Chat Completions. The helper's separate health-check caller intentionally constructs a chat payload and must keep that explicit API choice. No public configuration, retry scheduling, token-selection policy or provider capability mapping changes are proposed.

Contribution process and AI disclosure

Codex materially assisted source analysis and these local planning drafts. The user requested a local fix and asked whether that follows repository policy. Implementation has been deferred because the canonical policy requires explicit maintainer approval of Proposal and Design before implementation, plus authorized TASKs. No approval is inferred from inactivity or an empty assignee field.

issue-spec authentication and workflow validation passed before publication. No existing Proposal referencing #4396 was found. This Proposal requests maintainer review of the confirmed problem and the bounded intended change; approval remains pending. No unresolved product choice was identified in the initial QUESTION discovery pass. SPEC and Design will carry testable requirements and the proposed verification plan; no implementation is authorized by this publication.