[Security] Quota consumption TOCTOU race — user/token quota can be oversold via concurrent requests
[Security] Quota consumption TOCTOU race — user/token quota can be oversold via concurrent requests
Summary
The quota consumption path has a time-of-check-to-time-of-use (TOCTOU) race condition. The
balance sufficiency check and the actual deduction are not atomic, and every deduction is an
unconditional UPDATE ... SET quota = quota - ? with no WHERE quota >= ? guard, no row
lock, and no transaction. By issuing concurrent requests, a user holding a valid API token
can consume more quota than they actually have — quota is oversold and can be driven negative.
For a per-token billing gateway this is direct monetary loss for the operator. It is the same
vulnerability class as the billing race found in the downstream fork new-api (QuantumNous/new-api#6609).
Verified facts (code paths, one-api @ 8df4a26)
1. The check and the deduction are separated in time.
relay/controller/helper.go:68 preConsumeQuota:
helper.go:71— reads the current user quota once viamodel.CacheGetUserQuota.helper.go:76— sufficiency checkuserQuota - preConsumedQuota < 0. This is the TOCTOU check.helper.go:84-87— ifuserQuota > 100*preConsumedQuota,preConsumedQuotais set to0("trusted, no need to pre-consume") and token-level pre-consumption is skipped entirely. In this common path the only deduction happens at the very end of the request.- The real deduction of the user balance only occurs after the upstream LLM call finishes, in
relay/billing/billing.go:23PostConsumeQuota→model/token.go:282PostConsumeTokenQuota. For streaming completions this window spans seconds to minutes.
2. Without Redis (default), nothing atomically decrements the user balance during the request.
model/cache.go:119-125:
func CacheDecreaseUserQuota(id int, quota int64) error {
if !common.RedisEnabled {
return nil // <-- no-op when Redis is not configured
}
err := common.RedisDecrease(fmt.Sprintf("user_quota:%d", id), int64(quota))
return err
}When RedisIsEnabled is false (Redis is optional; REDIS_CONN_STRING unset by default), the
pre-consumption decrement is silently skipped. The only user-balance gate is the single stale
read at helper.go:71, and the balance is only touched at request end.
3. Every deduction is unconditional — there is no WHERE quota >= ? guard.
model/user.go:401-404decreaseUserQuota:DB.Model(&User{}).Where("id = ?", id).Update("quota", gorm.Expr("quota - ?", quota))model/token.go:206-215decreaseTokenQuota:DB.Model(&Token{}).Where("id = ?", id).Updates(map{ "remain_quota": gorm.Expr("remain_quota - ?", quota), ... })
Neither checks the resulting balance, so quota goes negative under concurrent deduction.
4. No concurrency control exists anywhere in the quota path.
A repo-wide search of the quota path (model/token.go, model/user.go, model/cache.go,
relay/billing/billing.go, relay/controller/helper.go) finds zero occurrences of
FOR UPDATE / Locking / ForUpdate / .Transaction / sync.Mutex. The check→deduct is
never serialized.
5. BATCH_UPDATE_ENABLED amplifies the race.
main.go:86 enables config.BatchUpdateEnabled. With it, model/utils.go:38-78 aggregates
deductions in an in-memory map and flushes to the DB only every BATCH_UPDATE_INTERVAL seconds
(default 5, common/config/config.go:113). During that window the DB quota column is stale,
so the helper.go:71 / token.go:225 checks read old values while deductions are pending in
memory — the oversell window grows from "request duration" to "request duration + flush interval".
Reproduction sketch
With a user holding quota Q and a valid token, fire N concurrent /v1/chat/completions
requests each costing ~Q (or simply more concurrent requests than the balance covers). All N
read the same stale balance, all pass the sufficiency check, and all are served. Final balance
is negative (or the operator pays upstream for N - floor(Q/cost) requests that were never
billable). Enabling BATCH_UPDATE_ENABLED=true widens the window and makes reproduction easier.
# N concurrent requests against one user's token; each passes the stale balance check
seq 1 20 | xargs -P 20 -I{} curl -s https://TARGET/v1/chat/completions \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"model":"...","messages":[{"role":"user","content":"..."}]}'Impact
- Financial: quota oversell — the gateway pays upstream providers for usage it cannot bill. Direct loss for every multi-tenant one-api deployment.
- Data integrity:
users.quota/tokens.remain_quotago negative; accounting is corrupted. - Availability: an attacker can drain the operator's upstream budget, denying service to legitimate users.
Prerequisite: a registered account with a self-issued token — i.e. any tenant of a multi-user one-api instance. This is the normal threat model of a billing gateway.
Suggested fix
Make check-and-deduct atomic. Minimal change — guard the deduction and verify affected rows:
// model/user.go — decreaseUserQuota
res := DB.Model(&User{}).
Where("id = ? AND quota >= ?", id, quota).
Update("quota", gorm.Expr("quota - ?", quota))
if res.RowsAffected == 0 {
return errors.New("quota insufficient")
}Apply the same WHERE remain_quota >= ? + RowsAffected check to decreaseTokenQuota, and
wrap the user+token pair in a single DB.Transaction. Alternatively SELECT ... FOR UPDATE
on the user/token row inside a transaction. The pre-consume/post-consume accounting should be
reconciled against the same guarded deduction so the "trusted" path (helper.go:84) cannot
bypass it.
References
- Downstream fork carrying the identical class of bug: QuantumNous/new-api#6609 (billing TOCTOU race).
- Related concurrency gap already acknowledged in this repo: #2397 / #2398 / #2399 (redemption-code race).
Environment
- one-api commit
8df4a2670b98266bd287c698243fff327d9748cf(main, 2025-02-21) - Default config:
BatchUpdateEnabled=false,PreConsumedQuota=500, Redis disabled. - Source-level verification (no third-party deployment was tested).
Source: songquanpeng/one-api