Design: Preserve API identity across ai-proxy retry transforms (#4396)

Design draft: Propagate the resolved API name through retry reconstruction

• Proposal Issue: 4718

Status: draft for maintainer review. Proposal and Design approval are pending; no fix implementation has begun. This proposed plan is not self-authorization to implement or to claim fixed-version verification.

Proposal Issue: https://github.com/higress-group/higress/issues/4718 Requirement: SPEC-4718001 (draft on #4718) Related report: https://github.com/higress-group/higress/issues/4396 Analysis baseline: faccaad586a3cdc9e85dc7fa39358ff31a6453b2.

Current Implementation Locations

• plugins/wasm-go/extensions/ai-proxy/main.go:onHttpRequestHeaders/onHttpRequestBody: resolves and stores the request API identity and original request data.
• provider/retry.go:retryFailedRequest/sendRetryRequest: retains the API name, rotates the token, and reconstructs retries.
• provider/failover.go:transformRequestHeadersAndBody/SetApiTokensFailover: shared transformation helper and independent generated health-check caller.
• provider/kling.go:TransformRequestHeaders/TransformRequestBodyHeaders: API-specific video path and model-name conversion.
• Existing regression locations: provider/retry_test.go, provider/kling_test.go, and focused provider tests.

Candidate Plans

The proposed design uses an explicit ApiName argument at the helper boundary. This makes both callers' intent visible and avoids making the synthetic health-check context pretend to represent a real request. Reading API identity implicitly from context would hide that distinction and require a fallback rule. Duplicating the helper into retry-specific and health-check-specific versions would duplicate dispatch and provider-path handling. The explicit argument is the proposed choice for maintainer review; no alternative is being implemented in parallel.

Decisions

1. In plugins/wasm-go/extensions/ai-proxy/provider/failover.go, add an explicit ApiName parameter to transformRequestHeadersAndBody. Forward it to TransformRequestHeaders, TransformRequestBody, TransformRequestBodyHeaders, and defaultTransformRequestBody, preserving the existing interface dispatch order.
2. In provider/retry.go, pass sendRetryRequest's existing resolved apiName into the helper. Recursive retries already retain this argument.
3. In the health-check callback in provider/failover.go, explicitly pass ApiNameChatCompletion. Its generated payload contains messages and its existing behavior intentionally probes Chat Completions; a synthetic health-check context must not be used to infer a video API.
4. Add focused regression coverage in the provider package. Keep API-specific behavior in providers rather than adding Kling-only branches to the generic retry code.

Verification plan proposed

Unit and provider regressions

Cover the header transform and each body-transform dispatch branch with a non-chat API identity. Add a concrete Kling video case for expected path/body conversion and a Chat Completions compatibility case. Check that the health-check call explicitly retains Chat Completions semantics. Include a retry-level assertion so helper-only coverage cannot hide a caller that supplies the wrong identity.

After the approval gate and test implementation, run from the ai-proxy module:

go test ./provider
go test ./...

These commands have not been executed for this change. Native tests alone cannot establish the runtime claim.

Real proxy-Wasm comparison

Adapt docs/developers/wasm-runtime-verification/ to use the built ai-proxy Wasm module and a local mock model upstream. Pin a relevant Higress gateway release by resolved digest. Use the investigation-tested v2.2.4 gateway at sha256:3dbd609df5db3fca61653eafe0e2310705e485190c4f8cd02d9aab8f07dcf329 unless maintainers request a different relevant release. Record actual Envoy version and both module hashes in the approved plan.

Use apiTokens: [dummy-key-a, dummy-key-b] without klingAccessKey/klingSecretKey; this exercises token selection and rotation, not the distinct official AK/SK JWT mode. There is no need for live provider credentials or model calls. Keep the static upstream cluster pinned only to the local mock, including internal retry dispatch. Provider authority may retain its ordinary value; the isolated cluster determines the actual connection destination. Record and verify both initial and retry destination routing, since generateUrl constructs an HTTPS URL and the dispatch uses the route cluster.

For each isolated fixture, reset the mock state. Return a configured retryable status (for example, 503) for the first request regardless of which key is randomly selected. Accept the second request only when it uses the other test key and has the expected provider-native video path and payload. Assert Authorization: Bearer dummy-key-a versus Bearer dummy-key-b on successive attempts.

Use non-streaming video creation as the primary case; include both text-to-video and image-to-video, which were confirmed reachable through the public request path during investigation. Assert exactly one initial request and one retry, correct key rotation, correct upstream protocol, and the expected client response. Add a Chat Completions control case. Do not expand this change to retry GET/method semantics.

Run the same fixtures against the baseline and corrected modules, resetting mock state between runs. The baseline must fail the protocol assertion; the fixed build must pass it. Repeat three times to expose dependence on random initial token selection. Record:

• source SHAs and any uncommitted patch hash;
• both Wasm SHA-256 values and gateway/mock image digests;
• exact build/run/assertion/cleanup commands, environment and configuration;
• incoming requests and upstream observations using dummy credentials only;
• expected/actual assertions, client status/body, gateway and mock logs;
• cleanup proof that test containers, processes, and listener ports are gone.

Execution readiness

The baseline-only mock/client, pinned Envoy configurations, Compose runner, and assertions now exist in the local aggregation workspace under contributions/4396/repro/. A clean scripted baseline run confirmed all six video retry failures and all three successful chat retries, with direct-success controls. The product code patch and fixed-version tests do not exist yet. Before implementation verification, publish the final runnable harness/commands and use the maintainer-approved Design and authorized verification TASK. The approved baseline/fixed comparison must be rerun; these investigation results do not certify a fix.

Impact Scope

The proposed interface change is internal to the provider package and requires both existing call sites to be updated. No public configuration field is introduced. The invariant is that retries use the resolved request API, while generated health checks keep their explicitly selected API.

PRs #4552 and #3882 touch neighboring retry/failover behavior and must be rechecked before implementation. Do not include their independent changes in this patch.

Execution Planning

One bounded implementation TASK owns the API-identity invariant across both callers, the shared helper, and focused provider regression tests; these changes are cohesive and should not be split between writers. A second TASK owns the approved baseline/fixed runtime evidence and depends on the implementation TASK. After maintainer authorization, one non-Coordinator implementation worker will own the bounded code changes; independent read-only review follows. No managed PROCESS or parallel code writers are needed. The only shared touchpoints are the existing retry/failover helper and neighboring active PRs. No genuine unresolved product/design choice was identified in the initial QUESTION discovery pass; maintainer review may add decisions before authorization.

Planning state and policy evidence

On 2026-09-18, the canonical policy blob matched the local file, Git blob 1463b1afed2e82f321718815faa1cbbd1e011004. The policy explicitly requires Proposal and Design approval before implementation. Searches for #4396 found the original report but no linked approved phase issues.

Authenticated GitHub login, queried with GH_TOKEN and GITHUB_TOKEN unset, was Just-CJ. The canonical collaborator-permission query returned HTTP 403, so the maintainer/administrator exception was not established and cannot be used. No product source changes, PRs, or pushes have been made. This draft records pre-implementation investigation; phase publication and explicit maintainer approval are tracked separately.