feat: As a user, I want to route to The Grid through ai-proxy, so that I can consume market-priced inference from APISIX
Description
As a user of ai-proxy, I want to route to The Grid as a first-class provider, so that I can buy market-priced inference through APISIX the same way I route to OpenAI or OpenRouter today.
The Grid is an OpenAI-compatible inference API. Its model ids are market instruments — a task type and a quality tier (text-standard, code-prime, agent-max) or a lab-scoped market (claude-opus-latest, kimi-latest) — rather than fixed model names, and requests are filled by whichever supplier is competitive at the time. /v1/chat/completions and /v1/responses are both served, so it fits the existing openai-chat and openai-responses capabilities without a new protocol.
Adding the provider module itself is routine and mirrors openai.lua. One thing about The Grid does not fit the transport as it stands, and I would like to check the approach before sending it.
The routing redirect
The Grid's Consumption API validates the request, then answers 307 Temporary Redirect with a Location pointing at its routing layer on the same host (api.thegrid.ai/v1/... → api.thegrid.ai/r/v1/...), where inference is actually fulfilled. This is documented and intentional, not an error condition: https://thegrid.ai/docs/api-reference/request-routing-and-redirects
Most clients follow it transparently, so this is invisible to normal SDK users. lua-resty-http does not, so ai-proxy would hand the 307 back downstream. That is worse than it first looks:
ai-rate-limitingmeters zero tokens for every request, because the gateway never sees a response body containingusage.ai-proxy-multihealth checks and instance fallback never trigger, since from the Plugin's point of view every request succeeded.- The
LocationURL is self-authenticating — I confirmed it returns200when replayed with noAuthorizationheader — so the gateway would effectively be converting its configured provider credentials into a bearer URL handed to the caller.
Proposal
Add opt-in redirect following to apisix/plugins/ai-transport/http.lua:
- A provider declares
follow_redirects = true; every existing provider is untouched and keeps today's behaviour. - Exactly one hop,
307/308only.301/302/303are excluded because they permit rewriting the method toGET, which would silently drop the prompt. - Method, body and headers are replayed unchanged; the redirect's own query string replaces the original when present, so
auth.queryproviders stay authenticated otherwise. - Same-origin only. A cross-origin hop would replay the provider credentials in
Authorizationagainst a host named by the response rather than by the configuration, so it is refused and surfaces as a request error.
Following it inside the gateway is what keeps token accounting, retries and fallback working, which is the reason to do this in the transport rather than document it as a caveat.
I have this implemented with tests (same-origin hop followed and the model output returned; cross-origin hop refused; mock endpoints added to t/lib/server.lua that assert the replay preserved method, body, target query and headers) and will open the PR referencing this issue. Happy to split the transport change out into its own PR, or to take a different approach to the redirect, if you would prefer.
Environment
- APISIX version:
master
Source: apache/apisix