tiered_expr pre-consume reads the whole request body into memory even when the expression never calls param() (defeats the disk cache; retained until settlement)

Deployment source

Repository release / official image (self-hosted)

Your current newapi version

calciumion/new-api:v1.0.0-rc.30 (image tag). The same code path is present on main at 04c64734 (relay/helper/price.go → ResolveIncomingBillingExprRequestInput → readIncomingBillingExprBody → storage.Bytes()); the fix below is written against main.

Submission Checks

• Non-duplicate issue: I have searched existing Issues and confirmed there are no similar issues.
• Read this first: I have fully read the section above, reviewed the docs at https://docs.newapi.ai/ and the project README, and asked AI first, confirming this is not a usage, configuration, or integration question.
• Supported version: I have provided an exact version, commit, or image tag (not latest or unknown) and confirmed that the issue reproduces on an unmodified, supported version from this repository.
• Not a third-party service: I confirm that this issue is not exclusive to a third-party hosting site, relay, API service, or fork that has not been verified against the unmodified repository. Third-party instance issues must be reported to their operator.
• Issue attribution: I have provided evidence that distinguishes the client, new-api, and upstream layers. For relay issues, I compared equivalent redacted requests sent directly upstream and through new-api, confirming that new-api introduces or changes the error rather than merely forwarding the upstream error unchanged.
• Channel and protocol boundary: I confirm that the issue is not caused by a Coding Plan service, reverse-engineered channel, third-party API wrapper, Codex reverse-proxy endpoint, or behavior specific to the Codex API. If first observed through such an interface, I have reproduced it using a standard API protocol supported by this repository.
• Maintainer time: I understand that maintainers have limited time and issues that do not follow this template may be ignored or closed directly.

Issue Description

Actual behavior: In pre-consume, modelPriceHelperTiered (relay/helper/price.go) always calls ResolveIncomingBillingExprRequestInput, which calls readIncomingBillingExprBody → storage.Bytes() and materialises the entire request body into RequestInput.Body, then keeps it on RelayInfo.BillingRequestInput for the whole life of the relay so settlement can reuse it. This happens regardless of whether the billing expression ever reads the body. The only consumer of RequestInput.Body is param(...) (pkg/billingexpr/run.go:92-103); header(...) reads headers, u(...) reads usage, the time functions read the clock. An expression that only uses token variables / len / time bucketing never touches the body, yet a full copy of the body is allocated and retained.

For bodies above the disk-cache threshold this undoes the disk cache: diskStorage.Bytes() is make([]byte, d.size) + io.ReadFull (common/body_storage.go:225), i.e. the bytes that were just moved to disk to save memory are read back into the heap in full. common/gin.go:115-117 already avoids exactly this on the JSON-decode path ("disk-backed JSON: stream-decode directly from the file to avoid materializing the entire payload back into a transient []byte"); the billing path does the opposite, and the copy is not transient.

Impact: Heap growth proportional to request-body size × in-flight requests on tiered_expr models, retained until settlement. Captured heap profile (rc.30, inuse_space total 363.28 MB at a moment when the container's heap inuse was 769 MiB):

150.18 MB (41.3%)  relay.TextHelper → common.Marshal → GeneralOpenAIRequest.MarshalJSON   (outbound body — expected)
 68.37 MB (18.8%)  GetAndValidateRequest → UnmarshalBodyReusable → DecodeJson             (inbound decode — expected)
 66.10 MB (18.2%)  ModelPriceHelper → modelPriceHelperTiered
                     → ResolveIncomingBillingExprRequestInput
                     → readIncomingBillingExprBody → diskStorage.Bytes                    (avoidable)

go tool pprof -peek 'diskStorage\).Bytes' shows readIncomingBillingExprBody as the only caller (100%). On this deployment, none of the 8 configured billing_expr entries use param( or header( (they only use weekday/hour/month/day, p/c/cr/cc/cc1h/img_o, and len), so the 66 MB was read and never looked at.

Frequency: Every request to a tiered_expr model whose body exceeds the disk-cache threshold; on this deployment tiered_expr models are 72.7% of consumption logs over the last 2 days (average prompt 109k–164k tokens, max 794k). Continuous, not intermittent.

Evidence that the issue is in new-api: The allocation site is new-api code (relay/helper/billing_expr_request.go + common/body_storage.go), reached before any upstream call; no client or upstream behaviour is involved. The profile above was taken from the new-api process itself via /debug/pprof/heap.

Billing details: Endpoint POST /v1/chat/completions; models with tiered_expr pricing (e.g. deepseek-flash, deepseek-v4-pro); billing_setting.billing_expr entries use only time functions, token variables and len (no param(/header(). Response usage, consumption log, expected charge and calculation basis: not applicable — charges are correct; the defect is memory retention, not the amount billed.

Steps to Reproduce

1. Configure a tiered_expr price for a model whose expression does not call param(...), e.g. hour >= 9 && hour < 18 ? p * 2 : p (any time-bucketed or token-only expression).
2. Enable the request-body disk cache with a low threshold (Performance settings: disk_cache_enabled=true, disk_cache_threshold_mb=1).
3. Send a POST /v1/chat/completions for that model with a body larger than the threshold (e.g. a 2–3 MB prompt). Any channel type works; the read happens in pre-consume before the upstream call.
4. While the request is in flight, take GET /debug/pprof/heap (with ENABLE_PPROF=true) and inspect with go tool pprof -top -sample_index=inuse_space, or -peek 'diskStorage\).Bytes'.
5. Observe an inuse allocation of exactly the body size attributed to readIncomingBillingExprBody → diskStorage.Bytes, retained until settlement, although the expression never evaluates param(...).

Unit-level reproduction (no traffic needed): call ResolveIncomingBillingExprRequestInput with a BodyStorage implementation that counts Bytes() calls, using a time-only expression — Bytes() is called once. The fix branch adds this as a test (relay/helper/billing_expr_body_test.go).

Expected Result

The body should be read only when the compiled expression actually references param. The information is already available in the same function: modelPriceHelperTiered already calls billingexpr.UsedVarsByHash(exprStr, exprHash)["image_count"] two lines later, and extractUsedVars (pkg/billingexpr/compile.go) records every identifier including function names, so UsedVars(expr)["param"] is exactly the gate needed (verified: time_only → param=false, uses_param → param=true even when param(...) sits inside a ternary that requestRulePatcher rewrites).

A fix with tests is ready on txgo/new-api branch fix/billing-expr-skip-body-read (ad93f178): ResolveIncomingBillingExprRequestInput(c, info, needBody bool) gated by BillingExprNeedsRequestBody(usedVars) (returns true when usedVars == nil, i.e. the expression failed to compile, so failures surface at evaluation instead of becoming a silently missing input); cloneRequestInput gets the same gate so frozen inputs (channel test path) don't copy the body either. go test ./relay/... ./pkg/billingexpr/ ./service/ ./common/ passes (21 packages); mutation check (guard removed, signature kept) turns the new tests red as assertion failures. I will open the PR referencing this issue.

Related Screenshots

Not applicable — server-side memory profile; the relevant pprof excerpt is quoted in the description. Full profile and the capture context are in our downstream tracker and can be attached on request.