#13919·apisix

feat: As a user, I want to route to The Grid through ai-proxy, so that I can consume market-priced inference from APISIX

Author: CastilloLuisCreated Sep 5, 2026Updated Sep 5, 2026

Description

As a user of ai-proxy, I want to route to The Grid as a first-class provider, so that I can buy market-priced inference through APISIX the same way I route to OpenAI or OpenRouter today.

The Grid is an OpenAI-compatible inference API. Its model ids are market instruments — a task type and a quality tier (text-standard, code-prime, agent-max) or a lab-scoped market (claude-opus-latest, kimi-latest) — rather than fixed model names, and requests are filled by whichever supplier is competitive at the time. /v1/chat/completions and /v1/responses are both served, so it fits the existing openai-chat and openai-responses capabilities without a new protocol.

Adding the provider module itself is routine and mirrors openai.lua. One thing about The Grid does not fit the transport as it stands, and I would like to check the approach before sending it.

The routing redirect

The Grid's Consumption API validates the request, then answers 307 Temporary Redirect with a Location pointing at its routing layer on the same host (api.thegrid.ai/v1/...api.thegrid.ai/r/v1/...), where inference is actually fulfilled. This is documented and intentional, not an error condition: https://thegrid.ai/docs/api-reference/request-routing-and-redirects

Most clients follow it transparently, so this is invisible to normal SDK users. lua-resty-http does not, so ai-proxy would hand the 307 back downstream. That is worse than it first looks:

  • ai-rate-limiting meters zero tokens for every request, because the gateway never sees a response body containing usage.
  • ai-proxy-multi health checks and instance fallback never trigger, since from the Plugin's point of view every request succeeded.
  • The Location URL is self-authenticating — I confirmed it returns 200 when replayed with no Authorization header — so the gateway would effectively be converting its configured provider credentials into a bearer URL handed to the caller.

Proposal

Add opt-in redirect following to apisix/plugins/ai-transport/http.lua:

  • A provider declares follow_redirects = true; every existing provider is untouched and keeps today's behaviour.
  • Exactly one hop, 307/308 only. 301/302/303 are excluded because they permit rewriting the method to GET, which would silently drop the prompt.
  • Method, body and headers are replayed unchanged; the redirect's own query string replaces the original when present, so auth.query providers stay authenticated otherwise.
  • Same-origin only. A cross-origin hop would replay the provider credentials in Authorization against a host named by the response rather than by the configuration, so it is refused and surfaces as a request error.

Following it inside the gateway is what keeps token accounting, retries and fallback working, which is the reason to do this in the transport rather than document it as a caveat.

I have this implemented with tests (same-origin hop followed and the model output returned; cross-origin hop refused; mock endpoints added to t/lib/server.lua that assert the replay preserved method, body, target query and headers) and will open the PR referencing this issue. Happy to split the transport change out into its own PR, or to take a different approach to the redirect, if you would prefer.

Environment

  • APISIX version: master