Zero-day retention still reachable after #12745/#12818: TTL guard skipped without cold storage, and maxRetentionTTL unguarded on key tables

Follow-up to #12701 (v2 retention POST writing _retention_days = 0 and expiring log rows on write). Two open PRs address that report — #12745 (reject non-positive TTL at the API layer and in SetTTLV2) and #12818 (guard the TTL DELETE expressions in the cold-storage branch). Reading both against current main, two paths to the same zero-day-retention outcome are left uncovered. Filing separately so they aren't lost if those PRs merge.

Both are code-reading findings against main, not newly observed data loss. Gap 1 is the one that matters in practice.

Gap 1 — the TTL guard never lands on deployments without cold storage

Both MODIFY TTL statements in SetTTLV2 are inside the cold-storage branch (pkg/query-service/app/clickhouseReader/reader.go):

if len(params.ColdStorageVolume) > 0 && coldStorageDuration > 0 {
    ...
    queries = append(queries, fmt.Sprintf(`ALTER TABLE %s ON CLUSTER %s MODIFY TTL toDateTime(timestamp / 1000000000) + toIntervalDay(_retention_days) DELETE, ... TO VOLUME '%s' ...`))
}

With no cold-storage volume configured, SetTTLV2 issues only the MODIFY COLUMN _retention_days UInt16 DEFAULT <multiIf> statements and never issues a MODIFY TTL at all. The table therefore keeps the TTL expression it was created with by the schema migrator — unguarded (SigNoz/signoz-otel-collector, cmd/signozschemamigrator/schema_migrator/v2_squashed_logs_migration.go):

TTL: "toDateTime(timestamp / 1000000000) + toIntervalDay(_retention_days)"                       // logs_v2
TTL: "toDateTime(seen_at_ts_bucket_start) + toIntervalDay(_retention_days) + toIntervalSecond(1800)" // logs_v2_resource

So #12818's if(_retention_days = 0, 30, _retention_days) is applied on exactly the deployments that configure cold storage, and on no others. A zero reaching the column DEFAULT on a hot-storage-only install still expires rows on write, with the same seen_at + 30 minutes amplification on logs_v2_resource.

This also means the guard is not durable even where it does land: it is re-applied per SetTTLV2 call, so a fresh install, a schema re-migration, or any code path that rewrites the TTL without the wrapper drops it again.

Suggested fix: put the if(<col> = 0, <fallback>, <col>) wrapper in the schema migrator's TTL definitions so every table carries it from creation, and keep #12818's version for tables already created. That is a change in signoz-otel-collector, so it can't ride along with #12818 — hence this issue.

Gap 2 — maxRetentionTTL is interpolated unguarded into the key tables

Later in the same function, outside any cold-storage branch:

maxRetentionTTL := params.DefaultTTLDays
for _, rule := range params.TTLConditions {
    maxRetentionTTL = max(maxRetentionTTL, rule.TTLDays)
}

ttlPayload[tableNames[2]] = []string{
    fmt.Sprintf("ALTER TABLE %s ON CLUSTER %s MODIFY TTL timestamp + toIntervalDay(%d) DELETE SETTINGS materialize_ttl_after_modify=0", ...),
}

tableNames[2] and tableNames[3] are the local logs_attribute_keys and logs_resource_keys tables. DefaultTTLDays = 0 with no rules yields toIntervalDay(0) DELETE on both — a literal, not a column reference, so neither #12818's column guard nor a corrected column DEFAULT helps.

#12745's validation does prevent this by rejecting defaultTTLDays <= 0 before SetTTLV2 runs, and #12818's validateTTLConditions change covers the per-rule values. The gap only exists if #12818 merges without #12745 — worth noting because the two PRs read as alternatives and are not.

Suggested fix: clamp at the point of use, e.g. max(maxRetentionTTL, retentionDaysFallback), so the key tables cannot receive a zero-day TTL regardless of what upstream validation allows.

Reproduction (Gap 1)

On a self-hosted install with no cold-storage volume configured:

POST /api/v2/settings/ttl
{"type":"logs","default_ttl_days":30}      # snake_case, per the GET response shape
→ 200

Then:

SHOW CREATE TABLE signoz_logs.logs_v2;
-- _retention_days DEFAULT 0    <- rewritten
-- TTL toDateTime(timestamp / 1000000000) + toIntervalDay(_retention_days)   <- untouched, unguarded

New rows expire on write. With #12818 applied the result is unchanged, because the branch holding the guard never executes.

Version

Originally found on SigNoz v0.137.1 / chart signoz-0.137.1 / otel-collector v0.144.8; the code references above are against current main.