[Bug] Support schema-evolved multi-group Measure aggregation
Summary
Make valid aggregate queries succeed when participating storage groups have different projected schemas. Follow-up to #14066 under #13998.
Size audit and dependencies
Classification: tracking parent, not an executable leaf or automated implementation task.
Boundary: multi-group DistributedPlan.Execute through executeAgg, ReduceRawFrames, and ReducePartialBatches.
Schema alignment and storage-group-aware replica handling are independently testable changes; split them into executable tasks before implementation.
Blocked by: #14066 via BanyanDB PR apache/skywalking-banyandb#1326, to finalize the remaining boundary after legacy removal.
Proposed initial slice
Normalize missing projected, non-grouping tags to null before partial reduction, rather than rejecting the query because partial layouts differ. Do not simply disable structural schema validation.
Concrete oracle
- Old schema:
svc,value; rows(old,2)and(old,3). - New schema:
svc,zone,value; rows(new,west,7)and(new,west,11). - Query:
GROUP BY svc,SUM(value), projectsvc,zone.
Expected unordered output:
old, NULL, 5
new, west, 18Different service keys deliberately avoid ambiguity over representative values of non-grouping tags. Currently differently shaped partial batches can fail with partial ... schema mismatch.
Completion criteria
- The schema-evolution oracle succeeds through the distributed production caller, not only a reducer helper; add a regression that fails on the current structural mismatch.
- Normalization preserves aggregate-state columns, including count sidecars used by MEAN, and rejects genuinely incompatible types with a clear error.
- Replica deduplication preserves storage-group provenance: contributions 5 and 18 for service
apifrom two different storage groups must total 23, even when both groups use shard 0; duplicate replicas of either contribution must not increase that sum. This requires its own focused implementation/test slice. - Each executable child activates the real distributed caller and has independently specified expected results.
Relevant code: pkg/query/vectorized/measure/reduce.go, pkg/query/vectorized/measure/plan/distributed.go; extend distributed-plan and topology-matrix coverage.
go test ./pkg/query/vectorized/measure/...
go test ./test/integration/distributed/query/...Out of scope of the first slice
Missing GroupBy keys: these fail earlier during planning and need a separately specified change, not an assumed reducer-only fix. Also exclude changing aggregate/operator semantics or replacing the ordinary multi-group row merger.
Parent: #13998.
Compatibility and review policy
- Preserve existing on-disk and wire formats and mixed-version compatibility. Any necessary protocol change requires a separately reviewed compatibility contract.
- Use explicit, hand-calculated expected results; do not depend on the removed row engine as the test oracle.
- Human review enforces the no-new-row-path policy. A row-call-site lint guard is not required.
Source: apache/skywalking