#79233·starrocks

[Bug] Shared-data (lake) scan still logs WARNING "failed to find column in schema" for pruned JSON roots — the #63414 fix was only applied to the shared-nothing path

Author: ksawerykarwackiCreated Sep 16, 2026Updated Sep 16, 2026

Steps to reproduce

  1. Run a shared-data (cloud-native / lake) cluster. Any 4.x release reproduces; ours is 4.1.3.

  2. Query a lake table with a JSON subfield access path, so that only the subfield is required and the JSON root column is pruned from the scan schema:

    sql
    CREATE TABLE t (id BIGINT, j JSON) PRIMARY KEY (id) DISTRIBUTED BY HASH(id);
    INSERT INTO t VALUES (1, parse_json('{"a": {"b": "x"}}'));
    SELECT id FROM t WHERE get_json_string(j, 'a.b') = 'x';
  3. Look at the CN log.

No setup is strictly needed, in fact: StarRocks reproduces this against its own internal table on any cluster that runs materialized views. The FE task scheduler reads _statistics_.task_run_history with a subfield path —

RepoExecutor execute SQL | DQL WITH MaxStartRunID AS (
    SELECT task_name, cast(history_content_json->'startTaskRunId' as string) start_run_id
    FROM _statistics_.task_run_history WHERE ... )

— so a cluster with MV refresh tasks warns continuously with no user query involved at all.

Actual behaviour

Two LOG(WARNING) lines per scan, one from each call site:

W20260917 00:40:17.911204 126166930716352 lake_connector.cpp:632] failed to find column in schema: history_content_json
W20260917 00:40:17.911209 126166930716352 lake_connector.cpp:716] failed to find column in schema: history_content_json

On our fleet this is ~1.44 M lines/day (~63 k/h), around 57 warning lines per FE query, since each query fans out over tablets and segments. It is the single largest log emitter on our CN nodes.

Expected behaviour

No warning. A pruned JSON root is a normal, expected state — that is exactly the reasoning given when the same warning was removed from the shared-nothing path in #63414:

For a query like select count(*) from bluesky where get_json_string(data, 'commit.collection') = 'app.bsky.graph.list', the only required column access path is data.commit.collection, and the data would be pruned. so it's a normal case that the data is not found in the schema.

Root cause

PR #63414 (fixing #63225, backported to branch-4.0 as #63458) added ColumnAccessPath::is_root() and guarded the ROOT-with-children case behind VLOG_ROW — but only in be/src/exec/pipeline/scan/olap_chunk_source.cpp.

The lake connector carries the same two call sites and was not touched. They are still LOG(WARNING) in current main:

  • be/src/connector/lake/lake_connector.cpp:1101 — in LakeDataSource::init_column_access_paths
  • be/src/connector/lake/lake_connector.cpp:1185 — in LakeDataSource::prune_schema_by_access_paths

Checked 2026-09-17 — both warnings are present, unguarded, in every current release line:

Ref lake connector shared-nothing path
3.5.21 (LTS) LOG(WARNING) guarded since #63414
4.0.14 LOG(WARNING) guarded
4.1.4 (latest) LOG(WARNING) at lake_connector.cpp:635, :719 guarded
main LOG(WARNING) at lake/lake_connector.cpp:1101, :1185 guarded at olap_chunk_source.cpp:513

The guard is still present on the shared-nothing path in main, so the pruned- JSON-root case is evidently still expected rather than something later fixed at the source.

Both sites continue after logging, so the behaviour is correct — the column is read whole and results are unaffected. Only the log is wrong.

Proposed fix

Mirror #63414 in the lake connector, e.g. in init_column_access_paths:

cpp
        } else if (path->is_root() && !path->children().empty()) {
            // ROOT path for a JSON field whose root column was pruned:
            // sub-paths are still needed, so this is expected.
            VLOG_ROW << "Skipping pruned JSON root path: " << root;
        } else {
            LOG(WARNING) << "failed to find column in schema: " << root;
        }

and the equivalent guard in prune_schema_by_access_paths.

Happy to open the PR if that is welcome.

Environment

StarRocks 4.1.3 (8a8e186)
Deployment shared-data (cloud-native), object storage backend
Runtime Kubernetes, kube-starrocks operator chart 1.11.4
Scale of the symptom ~1.44 M warning lines/day across 8+ clusters