[Bug] Shared-data (lake) scan still logs WARNING "failed to find column in schema" for pruned JSON roots — the #63414 fix was only applied to the shared-nothing path
Steps to reproduce
Run a shared-data (cloud-native / lake) cluster. Any 4.x release reproduces; ours is 4.1.3.
Query a lake table with a JSON subfield access path, so that only the subfield is required and the JSON root column is pruned from the scan schema:
CREATE TABLE t (id BIGINT, j JSON) PRIMARY KEY (id) DISTRIBUTED BY HASH(id); INSERT INTO t VALUES (1, parse_json('{"a": {"b": "x"}}')); SELECT id FROM t WHERE get_json_string(j, 'a.b') = 'x';Look at the CN log.
No setup is strictly needed, in fact: StarRocks reproduces this against its own
internal table on any cluster that runs materialized views. The FE task
scheduler reads _statistics_.task_run_history with a subfield path —
RepoExecutor execute SQL | DQL WITH MaxStartRunID AS (
SELECT task_name, cast(history_content_json->'startTaskRunId' as string) start_run_id
FROM _statistics_.task_run_history WHERE ... )— so a cluster with MV refresh tasks warns continuously with no user query involved at all.
Actual behaviour
Two LOG(WARNING) lines per scan, one from each call site:
W20260917 00:40:17.911204 126166930716352 lake_connector.cpp:632] failed to find column in schema: history_content_json
W20260917 00:40:17.911209 126166930716352 lake_connector.cpp:716] failed to find column in schema: history_content_jsonOn our fleet this is ~1.44 M lines/day (~63 k/h), around 57 warning lines per FE query, since each query fans out over tablets and segments. It is the single largest log emitter on our CN nodes.
Expected behaviour
No warning. A pruned JSON root is a normal, expected state — that is exactly the reasoning given when the same warning was removed from the shared-nothing path in #63414:
For a query like
select count(*) from bluesky where get_json_string(data, 'commit.collection') = 'app.bsky.graph.list', the only required column access path isdata.commit.collection, and thedatawould be pruned. so it's a normal case that thedatais not found in the schema.
Root cause
PR #63414 (fixing
#63225, backported to
branch-4.0 as #63458) added ColumnAccessPath::is_root() and guarded the
ROOT-with-children case behind VLOG_ROW — but only in
be/src/exec/pipeline/scan/olap_chunk_source.cpp.
The lake connector carries the same two call sites and was not touched. They are
still LOG(WARNING) in current main:
be/src/connector/lake/lake_connector.cpp:1101— inLakeDataSource::init_column_access_pathsbe/src/connector/lake/lake_connector.cpp:1185— inLakeDataSource::prune_schema_by_access_paths
Checked 2026-09-17 — both warnings are present, unguarded, in every current release line:
| Ref | lake connector | shared-nothing path |
|---|---|---|
3.5.21 (LTS) |
2× LOG(WARNING) |
guarded since #63414 |
4.0.14 |
2× LOG(WARNING) |
guarded |
4.1.4 (latest) |
2× LOG(WARNING) at lake_connector.cpp:635, :719 |
guarded |
main |
2× LOG(WARNING) at lake/lake_connector.cpp:1101, :1185 |
guarded at olap_chunk_source.cpp:513 |
The guard is still present on the shared-nothing path in main, so the pruned-
JSON-root case is evidently still expected rather than something later fixed at
the source.
Both sites continue after logging, so the behaviour is correct — the column is
read whole and results are unaffected. Only the log is wrong.
Proposed fix
Mirror #63414 in the lake connector, e.g. in init_column_access_paths:
} else if (path->is_root() && !path->children().empty()) {
// ROOT path for a JSON field whose root column was pruned:
// sub-paths are still needed, so this is expected.
VLOG_ROW << "Skipping pruned JSON root path: " << root;
} else {
LOG(WARNING) << "failed to find column in schema: " << root;
}and the equivalent guard in prune_schema_by_access_paths.
Happy to open the PR if that is welcome.
Environment
| StarRocks | 4.1.3 (8a8e186) |
| Deployment | shared-data (cloud-native), object storage backend |
| Runtime | Kubernetes, kube-starrocks operator chart 1.11.4 |
| Scale of the symptom | ~1.44 M warning lines/day across 8+ clusters |
Source: StarRocks/starrocks